-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
3 changed files
with
223 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,25 @@ | ||
\chapter{Benchmark Specification} | ||
\label{sec:benchmark-specification} | ||
|
||
\ldbcsnb is designed to be flexible and to have an affordable entry point. From small single node and in memory systems to large distributed multi-node clusters have its own place in \ldbcsnb. Therefore, the requirements to fulfill for executing \ldbcsnb are limited to pure software requirements to be able to run the tools. While the benchmark specification aims to be portable, the software provided by \ldbcsnb have been developed and tested under Linux-based operating systems. The driver and the clients for the reference implementations were implemented in Java. The generator has two versions: the Hadoop-based one was written in Java, while the Spark-based one is written in a mix of Java and Scala. | ||
\paragraph{Overview} | ||
The LDBC Social Network Benchmark workloads require several components to generate data and updates, produce query substitution parameters, run the benchmark on the system under test, \etc | ||
\autoref{fig:snb-overview} shows a blueprint for the frameworks implementing the \ldbcsnb benchmark workloads. | ||
|
||
\ldbcsnb does not impose the usage of any specific type of system, as it targets systems of different nature and characteristics, from graph databases, graph processing frameworks and RDF systems, to traditional relational database management systems. Consequently, any language or API capable of expressing the proposed queries can be used. Similarly, data can be stored in the most convenient manner the test sponsor may decide, as long as it conforms with the execution rules. Finally, in order to have an official benchmark execution, the results will have to be audited and all the required information disclosed. | ||
\begin{figure}[htbp] | ||
\centering | ||
\includegraphics[scale=\yedscale]{figures/snb-overview} | ||
\caption{ | ||
High-level overview of the frameworks implementing each LDBC Social Network Benchmark workload. | ||
Legend: | ||
\fcolorbox{mydarkyellow}{mylightyellow}{{\scriptsize \textit{\textsf{Software component}}}} | ||
\fcolorbox{mydarkblue}{mylightblue}{{\scriptsize \textsf{Data artifact\vphantom{p}}}} | ||
} | ||
\label{fig:snb-overview} | ||
\end{figure} | ||
|
||
\paragraph*{Portability} | ||
\ldbcsnb is designed to be flexible and to have an affordable entry point. \ldbcsnb aims to accommodate systems from single node in-memory systems to large distributed multi-node clusters. Therefore, the requirements to fulfill for executing \ldbcsnb are limited to pure software requirements to be able to run the tools. | ||
While the benchmark specification aims to be portable, the software provided by \ldbcsnb have been primarily tested on Linux-based operating systems (\eg Ubuntu LTS). The driver and clients for the reference implementations were implemented in Java. The generator has two versions: the Hadoop-based one was written in Java, while the Spark-based one is written in a mix of Java and Scala. | ||
|
||
\paragraph*{Auditable systems} | ||
\ldbcsnb does not impose the usage of any specific type of system, as it targets systems of different nature and characteristics, from graph databases, graph processing frameworks and RDF systems, to traditional relational database management systems. Therefore, data can be stored in the most convenient manner the test sponsor may decide, as long as it conforms with the execution rules. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,202 @@ | ||
<?xml version="1.0" encoding="UTF-8" standalone="no"?> | ||
<graphml xmlns="http://graphml.graphdrawing.org/xmlns" xmlns:java="http://www.yworks.com/xml/yfiles-common/1.0/java" xmlns:sys="http://www.yworks.com/xml/yfiles-common/markup/primitives/2.0" xmlns:x="http://www.yworks.com/xml/yfiles-common/markup/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:y="http://www.yworks.com/xml/graphml" xmlns:yed="http://www.yworks.com/xml/yed/3" xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns http://www.yworks.com/xml/schema/graphml/1.1/ygraphml.xsd"> | ||
<!--Created by yEd 3.21.1--> | ||
<key attr.name="Description" attr.type="string" for="graph" id="d0"/> | ||
<key for="port" id="d1" yfiles.type="portgraphics"/> | ||
<key for="port" id="d2" yfiles.type="portgeometry"/> | ||
<key for="port" id="d3" yfiles.type="portuserdata"/> | ||
<key attr.name="url" attr.type="string" for="node" id="d4"/> | ||
<key attr.name="description" attr.type="string" for="node" id="d5"/> | ||
<key for="node" id="d6" yfiles.type="nodegraphics"/> | ||
<key for="graphml" id="d7" yfiles.type="resources"/> | ||
<key attr.name="url" attr.type="string" for="edge" id="d8"/> | ||
<key attr.name="description" attr.type="string" for="edge" id="d9"/> | ||
<key for="edge" id="d10" yfiles.type="edgegraphics"/> | ||
<graph edgedefault="directed" id="G"> | ||
<data key="d0"/> | ||
<node id="n0"> | ||
<data key="d6"> | ||
<y:ShapeNode> | ||
<y:Geometry height="36.54545304553638" width="475.79952002291355" x="231.43431648305537" y="485.40059034278494"/> | ||
<y:Fill color="#FEE090" transparent="false"/> | ||
<y:BorderStyle color="#FFC329" raised="false" type="line" width="1.0"/> | ||
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Arial" fontSize="12" fontStyle="italic" hasBackgroundColor="false" hasLineColor="false" height="30.8125" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="90.05078125" x="192.87436938645675" xml:space="preserve" y="2.866476522768153">Datagen | ||
(data generator)<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> | ||
<y:Shape type="rectangle"/> | ||
</y:ShapeNode> | ||
</data> | ||
</node> | ||
<node id="n1"> | ||
<data key="d6"> | ||
<y:ShapeNode> | ||
<y:Geometry height="36.54545304553638" width="136.89281412878833" x="562.2668753973949" y="551.9460433883213"/> | ||
<y:Fill color="#E0F3F8" transparent="false"/> | ||
<y:BorderStyle color="#80D6ED" raised="false" type="line" width="1.0"/> | ||
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Arial" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="30.8125" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="73.36328125" x="31.76476643939418" xml:space="preserve" y="2.866476522768153">Factor tables | ||
(statistics)<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> | ||
<y:Shape type="rectangle"/> | ||
</y:ShapeNode> | ||
</data> | ||
</node> | ||
<node id="n2"> | ||
<data key="d6"> | ||
<y:ShapeNode> | ||
<y:Geometry height="36.54545304553638" width="136.8928141287883" x="400.887669430118" y="751.5824025249306"/> | ||
<y:Fill color="#FEE090" transparent="false"/> | ||
<y:BorderStyle color="#FFC329" raised="false" type="line" width="1.0"/> | ||
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Arial" fontSize="12" fontStyle="italic" hasBackgroundColor="false" hasLineColor="false" height="17.40625" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="100.708984375" x="18.09191487689418" xml:space="preserve" y="9.569601522768153">System under test<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> | ||
<y:Shape type="rectangle"/> | ||
</y:ShapeNode> | ||
</data> | ||
</node> | ||
<node id="n3"> | ||
<data key="d6"> | ||
<y:ShapeNode> | ||
<y:Geometry height="36.54545304553638" width="136.8928141287883" x="400.8876694301181" y="685.0369494793941"/> | ||
<y:Fill color="#FEE090" transparent="false"/> | ||
<y:BorderStyle color="#FFC329" raised="false" type="line" width="1.0"/> | ||
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Arial" fontSize="12" fontStyle="italic" hasBackgroundColor="false" hasLineColor="false" height="17.40625" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="98.03125" x="19.43078206439418" xml:space="preserve" y="9.569601522768153">Benchmark driver<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> | ||
<y:Shape type="rectangle"/> | ||
</y:ShapeNode> | ||
</data> | ||
</node> | ||
<node id="n4"> | ||
<data key="d6"> | ||
<y:ShapeNode> | ||
<y:Geometry height="36.54545304553638" width="136.8928141287883" x="562.266875397395" y="618.4914964338577"/> | ||
<y:Fill color="#FEE090" transparent="false"/> | ||
<y:BorderStyle color="#FFC329" raised="false" type="line" width="1.0"/> | ||
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Arial" fontSize="12" fontStyle="italic" hasBackgroundColor="false" hasLineColor="false" height="30.8125" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="121.38671875" x="7.7530476893941795" xml:space="preserve" y="2.866476522768153">Paramgen | ||
(parameter generator)<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> | ||
<y:Shape type="rectangle"/> | ||
</y:ShapeNode> | ||
</data> | ||
</node> | ||
<node id="n5"> | ||
<data key="d6"> | ||
<y:ShapeNode> | ||
<y:Geometry height="36.54545304553638" width="136.89281412878833" x="400.88766943011797" y="551.9460433883213"/> | ||
<y:Fill color="#E0F3F8" transparent="false"/> | ||
<y:BorderStyle color="#80D6ED" raised="false" type="line" width="1.0"/> | ||
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Arial" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="17.40625" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="48.6953125" x="44.09875081439418" xml:space="preserve" y="9.569601522768153">Updates<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> | ||
<y:Shape type="rectangle"/> | ||
</y:ShapeNode> | ||
</data> | ||
</node> | ||
<node id="n6"> | ||
<data key="d6"> | ||
<y:ShapeNode> | ||
<y:Geometry height="36.54545304553638" width="136.89281412878833" x="239.50846346284112" y="551.9460433883213"/> | ||
<y:Fill color="#E0F3F8" transparent="false"/> | ||
<y:BorderStyle color="#80D6ED" raised="false" type="line" width="1.0"/> | ||
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Arial" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="17.40625" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="48.689453125" x="44.10168050189418" xml:space="preserve" y="9.569601522768153">Data set<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> | ||
<y:Shape type="rectangle"/> | ||
</y:ShapeNode> | ||
</data> | ||
</node> | ||
<node id="n7"> | ||
<data key="d6"> | ||
<y:ShapeNode> | ||
<y:Geometry height="36.54545304553638" width="136.89281412878833" x="562.266875397395" y="685.0369494793941"/> | ||
<y:Fill color="#E0F3F8" transparent="false"/> | ||
<y:BorderStyle color="#80D6ED" raised="false" type="line" width="1.0"/> | ||
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Arial" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="17.40625" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="100.703125" x="18.09484456439418" xml:space="preserve" y="9.569601522768153">Query parameters<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> | ||
<y:Shape type="rectangle"/> | ||
</y:ShapeNode> | ||
</data> | ||
</node> | ||
<edge id="e0" source="n3" target="n2"> | ||
<data key="d9"/> | ||
<data key="d10"> | ||
<y:PolyLineEdge> | ||
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/> | ||
<y:LineStyle color="#000000" type="line" width="1.0"/> | ||
<y:Arrows source="none" target="standard"/> | ||
<y:BendStyle smoothed="false"/> | ||
</y:PolyLineEdge> | ||
</data> | ||
</edge> | ||
<edge id="e1" source="n0" target="n1"> | ||
<data key="d9"/> | ||
<data key="d10"> | ||
<y:PolyLineEdge> | ||
<y:Path sx="161.3792059672769" sy="18.296603718256165" tx="0.0" ty="-18.272726522768224"/> | ||
<y:LineStyle color="#000000" type="line" width="1.0"/> | ||
<y:Arrows source="none" target="standard"/> | ||
<y:BendStyle smoothed="false"/> | ||
</y:PolyLineEdge> | ||
</data> | ||
</edge> | ||
<edge id="e2" source="n7" target="n3"> | ||
<data key="d9"/> | ||
<data key="d10"> | ||
<y:PolyLineEdge> | ||
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/> | ||
<y:LineStyle color="#000000" type="line" width="1.0"/> | ||
<y:Arrows source="none" target="standard"/> | ||
<y:BendStyle smoothed="false"/> | ||
</y:PolyLineEdge> | ||
</data> | ||
</edge> | ||
<edge id="e3" source="n4" target="n7"> | ||
<data key="d9"/> | ||
<data key="d10"> | ||
<y:PolyLineEdge> | ||
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/> | ||
<y:LineStyle color="#000000" type="line" width="1.0"/> | ||
<y:Arrows source="none" target="standard"/> | ||
<y:BendStyle smoothed="false"/> | ||
</y:PolyLineEdge> | ||
</data> | ||
</edge> | ||
<edge id="e4" source="n1" target="n4"> | ||
<data key="d9"/> | ||
<data key="d10"> | ||
<y:PolyLineEdge> | ||
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/> | ||
<y:LineStyle color="#000000" type="line" width="1.0"/> | ||
<y:Arrows source="none" target="standard"/> | ||
<y:BendStyle smoothed="false"/> | ||
</y:PolyLineEdge> | ||
</data> | ||
</edge> | ||
<edge id="e5" source="n0" target="n5"> | ||
<data key="d9"/> | ||
<data key="d10"> | ||
<y:PolyLineEdge> | ||
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/> | ||
<y:LineStyle color="#000000" type="line" width="1.0"/> | ||
<y:Arrows source="none" target="standard"/> | ||
<y:BendStyle smoothed="false"/> | ||
</y:PolyLineEdge> | ||
</data> | ||
</edge> | ||
<edge id="e6" source="n0" target="n6"> | ||
<data key="d9"/> | ||
<data key="d10"> | ||
<y:PolyLineEdge> | ||
<y:Path sx="-161.37920596727685" sy="18.296603718256165" tx="0.0" ty="0.0"/> | ||
<y:LineStyle color="#000000" type="line" width="1.0"/> | ||
<y:Arrows source="none" target="standard"/> | ||
<y:BendStyle smoothed="false"/> | ||
</y:PolyLineEdge> | ||
</data> | ||
</edge> | ||
<edge id="e7" source="n6" target="n2"> | ||
<data key="d9"/> | ||
<data key="d10"> | ||
<y:PolyLineEdge> | ||
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"> | ||
<y:Point x="307.9548705272353" y="769.8551290476987"/> | ||
</y:Path> | ||
<y:LineStyle color="#000000" type="line" width="1.0"/> | ||
<y:Arrows source="none" target="standard"/> | ||
<y:BendStyle smoothed="false"/> | ||
</y:PolyLineEdge> | ||
</data> | ||
</edge> | ||
</graph> | ||
<data key="d7"> | ||
<y:Resources/> | ||
</data> | ||
</graphml> |
Binary file not shown.