Skip to content

Commit

Permalink
Eliminate duplicate section labels
Browse files Browse the repository at this point in the history
  • Loading branch information
szarnyasg committed Sep 18, 2023
1 parent d669973 commit e948db9
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 17 deletions.
30 changes: 15 additions & 15 deletions auditing.tex
Original file line number Diff line number Diff line change
Expand Up @@ -639,7 +639,7 @@ \subsection{Full Disclosure Report}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Auditing Rules for the Business Intelligence Workload}
\label{sec:bi-workload-audit}
\label{sec:auditing-bi-workload-audit}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Expand All @@ -648,19 +648,19 @@ \section{Auditing Rules for the Business Intelligence Workload}
The following section describes the auditing rules specific to the Business Intelligence (BI) workload.

\subsection{Overview}
\label{sec:bi-audit-overview}
\label{sec:auditing-bi-audit-overview}

Implementing the BI workload requires the following key capabilities:

\begin{itemize}
\item Loading the initial snapshot of the social network graph
\item Evaluating the BI read queries (\autoref{sec:bi-reads})
\item Evaluating the BI write operations: inserts (\autoref{sec:bi-insert-operations}) and deletes (\autoref{sec:bi-delete-operations})
\item Performing concurrent reads and writes (\autoref{sec:bi-audit-workflow}) (optional, only allowed if ACID compliance is guaranteed)
\item Evaluating the BI read queries (\autoref{sec:auditing-bi-reads})
\item Evaluating the BI write operations: inserts (\autoref{sec:auditing-bi-insert-operations}) and deletes (\autoref{sec:auditing-bi-delete-operations})
\item Performing concurrent reads and writes (\autoref{sec:auditing-bi-audit-workflow}) (optional, only allowed if ACID compliance is guaranteed)
\end{itemize}

\subsection{Workflow}
\label{sec:bi-audit-workflow}
\label{sec:auditing-bi-audit-workflow}

\begin{figure}[htbp]
\centering
Expand Down Expand Up @@ -720,7 +720,7 @@ \subsection{Runtimes}
Note that a warm-up period is not allowed (unlike the Interactive workload where such a period is required, see \autoref{sec:int-measurement-window}).

\subsection{Scoring Metrics}
\label{sec:bi-scoring-metrics}
\label{sec:auditing-bi-scoring-metrics}

SNBI BI provides four scoring metrics:
the \emph{power score}, the \emph{throughput score}, and their price-adjusted variants,
Expand Down Expand Up @@ -829,10 +829,10 @@ \subsubsection{Throughput Scores}
$$ \textit{throughput@SF/\$} = \textit{throughput@SF} \cdot \frac{\numprint{1000}}{\textit{TCO}} $$

\subsection{Implementation Rules}
\label{sec:bi-implementation-rules}
\label{sec:auditing-bi-implementation-rules}

\subsubsection{Correctness}
\label{sec:bi-correcntess}
\label{sec:auditing-bi-correcntess}

The SUT shall evaluate all operators correctly.
The auditor shall ascertain correctness on the SF10 data set. However, they are allowed to also use data sets of different scale factors, as well as issue custom operations (both reads and writes) to test for the correctness of the implemenation.
Expand All @@ -841,12 +841,12 @@ \subsubsection{Correctness}
The rationale for using this only step is that during concurrent execution of R/W operations in the \emph{throughput test}, it is not possible to guarantee deterministic query results, making validation impossible. Moreover, this step already includes a write batch, therefore the query results indirectly test the correctness of the implementation of write operations.

\subsubsection{Auxiliary Data Structures}
\label{sec:bi-auxiliary-data-structures}
\label{sec:auditing-bi-auxiliary-data-structures}

Using auxiliary data structures (\eg indices, materialized views) is allowed if they are kept in an up-to-date state after each write operation. The full disclosure report should enumerate the auxiliary data structures used by the SUT.

\subsubsection{Query Declarativity}
\label{sec:bi-query-declarativity}
\label{sec:auditing-bi-query-declarativity}

Systems should use a domain-specific query language (\eg Cypher, Gremlin, GSQL) for the implementation, including the read queries and the update operations.
General-purpose programming languages (\eg C, C\texttt{++}, Java, Julia) are not allowed.
Expand All @@ -858,21 +858,21 @@ \subsubsection{Query Declarativity}
In cases when the categorization of the approach used by the SUT's query implementations is uncertain, it is the auditor's responsibility to decide whether the SUT complies with this rule.

\subsubsection{Query Variants}
\label{sec:bi-query-variants}
\label{sec:auditing-bi-query-variants}

Several queries (\eg \queryRefCard{bi-read-14}{BI}{14}) use \variantA and \variantB variants with different sets of input parameters.
The SUT should not receive any hints on which variant it is currently evaluating (\eg Q14\variantA or Q14\variantB).
Moreover, it is not allowed for the query implementations to contain code that aims to detect the query variant used.

\subsection{Scaling}
\label{sec:bi-scaling}
\label{sec:auditing-bi-scaling}

Audited \emph{benchmark runs} of the BI workload shall use SF30 or larger data sets.
The rationale behind this decision is to ensure that there should be a sufficient number of write operations available to guarantee the execution during the duration of the measurement window (see \autoref{fig:bi-batches}).

\subsection{Full Disclosure Report}
\label{sec:bi-fdr}
\label{sec:auditing-bi-fdr}

The \emph{full disclosure report} (FDR) and the \emph{supplementary package} shall contain
the same information as for SNB Interactive (\autoref{sec:int-fdr}),
including, if applicable (\autoref{sec:bi-implementation-rules}), the ACID compliance report (\autoref{sec:acid-compliance}).
including, if applicable (\autoref{sec:auditing-bi-implementation-rules}), the ACID compliance report (\autoref{sec:acid-compliance}).
2 changes: 1 addition & 1 deletion bi.tex
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ \subsubsection{Path Queries}
Moreover, the presence of a path between two nodes \emph{at a given time} does not guarantee that it will always exist during the benchmark execution as deletions can render the endpoints of a path unreachable.

\subsection{Query Variants}
\label{sec:query-variants}
\label{sec:bi-query-variants}

12~queries have a single variant, while 8~queries have two variants, yielding a total of 28~query variants.
As a rule of thumb, variants \variantA are expected to produce a longer runtime
Expand Down
2 changes: 1 addition & 1 deletion interactive-v2.tex
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ \subsection{Parameter curation for path-finding queries}
\end{figure}

\subsection{Query variants}
\label{sec:query-variants}
\label{sec:interactive-v2-query-variants}

The new workload introduces variants for three queries:
\CR[3], \CR[13], and \CR[14].
Expand Down

0 comments on commit e948db9

Please sign in to comment.