Skip to content

Commit c96324e

Browse files
mxmo0rhuhnmxmo0rhuhn
committed
Initial publication for NSDI'20
1 parent ea28652 commit c96324e

File tree

158 files changed

+16886
-1
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

158 files changed

+16886
-1
lines changed

‎.dockerignore

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
Dockerfile
2+
.dockerignore
3+
.gitignore
4+
LICENSE
5+
**/README.md
6+
docker-compose.yml
7+
8+
# git folder
9+
.git
10+
11+
# Compiled class file
12+
**/*.class
13+
14+
# Log file
15+
**/*.log
16+
17+
# BlueJ files
18+
*.ctxt
19+
20+
# Mobile Tools for Java (J2ME)
21+
.mtj.tmp/
22+
23+
# Package Files #
24+
**/*.jar
25+
**/*.war
26+
**/*.ear
27+
**/*.zip
28+
**/*.tar.gz
29+
**/*.rar
30+
31+
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
32+
**/hs_err_pid*
33+
**/target/
34+
**/.idea/
35+
**/*.iml

‎.gitignore

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Compiled class file
2+
*.class
3+
4+
# Log file
5+
*.log
6+
7+
# BlueJ files
8+
*.ctxt
9+
10+
# Mobile Tools for Java (J2ME)
11+
.mtj.tmp/
12+
13+
# Package Files #
14+
*.jar
15+
*.war
16+
*.ear
17+
*.zip
18+
*.tar.gz
19+
*.rar
20+
21+
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
22+
hs_err_pid*
23+
24+
target
25+
.idea/
26+
*.iml
27+
28+
benchlog.csv
29+
test.csv

‎AUTHORS

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
- Lukas Burkhalter <[email protected]>
2-
- Max Schrimpf <[email protected]>
2+
- Max Schrimpf <[email protected]>
3+
- Jason Friedman <[email protected]>

‎Dockerfile

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# @Copyright ETH Zürich, see AUTHORS file for more
2+
# Licensed under the Apache License, Version 2.0, see LICENSE file for more details.
3+
4+
# Use a multistep build so we don't have to install the build dependencies (JDK) in the container.
5+
FROM openjdk:11-jdk-buster as build
6+
7+
RUN apt-get update \
8+
&& apt-get install -y maven libssl-dev cmake build-essential
9+
10+
COPY . /build
11+
12+
WORKDIR /build
13+
14+
# Skip tests because of possible race conditions in the cassandra build that might mess up the build
15+
RUN cat /proc/cpuinfo | grep -iq aes && mvn package -DskipTests || mvn package -P \!aesni-native -DskipTests
16+
17+
################################################################################
18+
19+
FROM openjdk:11-jre-buster
20+
21+
RUN apt-get update \
22+
&& apt-get install -y wait-for-it libssl-dev
23+
24+
ENV CLIENT_JAR_NAME "/timecrypt-client-jar-with-dependencies.jar"
25+
ENV TESTBED_JAR_NAME "/timecrypt-testbed-jar-with-dependencies.jar"
26+
ENV SERVER_JAR_NAME "/timecrypt-server-jar-with-dependencies.jar"
27+
28+
COPY docker-start.sh /docker-start.sh
29+
RUN chmod u+x /docker-start.sh
30+
31+
COPY --from=build /build/timecrypt-client/target/$TESTBED_JAR_NAME $TESTBED_JAR_NAME
32+
COPY --from=build /build/timecrypt-client/target/$CLIENT_JAR_NAME $CLIENT_JAR_NAME
33+
COPY --from=build /build/timecrypt-server/target/$SERVER_JAR_NAME $SERVER_JAR_NAME
34+
35+
ENTRYPOINT ["/docker-start.sh"]

‎README.md

+109
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# TimeCrypt
2+
3+
TimeCrypt, a system that provides scalable and real-time analytics over large volumes of encrypted time series data.
4+
In TimeCrypt, data is encrypted end-to-end, and authorized parties can only decrypt and verify queries within their authorized access scope.
5+
6+
TimeCrypt achieves its competitive performance through a careful design of cryptographic primitives tailored for time series data workloads.
7+
Most important it introduces a partially homomorphic-encryption-based access control scheme (HEAC) that allows computations on the encrypted data.
8+
HEAC, in essence is based on a symmetric homomorphic encryption.
9+
However, we improve its performance by a factor of 2x for time series workloads by mapping keys to time and optimizing it for in-range ciphertext aggregations.
10+
11+
With those aggregations the server can pre compute statistical responses over large amounts of data without any knowledge about the plain text in near plain text speed.
12+
Given this the client can then compute even more sophisticated statistical results.
13+
For example a client can calculate the average over a set of data points with just one division if the server already provides the sum and count of the data points.
14+
15+
See [the Paper](https://www.usenix.org/system/files/nsdi20-paper-burkhalter.pdf) for more details.
16+
17+
## Structure of the repository
18+
This repository is split into three parts:
19+
- [**timecrypt-crypto**](timecrypt-crypto/README.md): The cryptographic library for TimeCrypt. It contains of the partially homomorphic-encryption-based access control scheme (HEAC) for different key lengths as well as the key derivation tree implementations.
20+
- [**timecrypt-server**](timecrypt-server/README.md): A prototypical implementation of a TimeCrypt server that stores its data in Cassandra. The server takes care of storing the cunks of raw data as well as the digests of aggregatable meta data.
21+
- [**timecrypt-client**](timecrypt-client/README.md): A example implementation of a client. The client provides a [Java-API](timecrypt-client/src/main/java/ch/ethz/dsg/timecrypt/TimeCryptClient.java) for interacting with TimeCrypt as well as an [interactive (the cli-client)](timecrypt-client/src/main/java/ch/ethz/dsg/timecrypt/CliClient.java) and [non-interactive CLI implementation (the testbed)](timecrypt-client/src/main/java/ch/ethz/dsg/timecrypt/TestBed.java) that simulates a producer like an IoT device.
22+
23+
For more information see the individual README files in the folders.
24+
25+
## Quickstart
26+
The easiest way to get a TimeCrypt server running in no time is to start it with `docker-compose`.
27+
28+
```
29+
docker-compose up --build
30+
```
31+
32+
This will build the project inside a Docker container, create an Docker network for the server and Cassandra and will start both.
33+
34+
Afterwards you have a running TimeCrypt server and just need to connect a client to it.
35+
36+
If you just want to see a Producer in action you can run a testbed client in Docker with:
37+
38+
```
39+
docker run --network=timecrypt-network eth/timecrypt producer
40+
```
41+
42+
For an interactive CLI client session run:
43+
```
44+
docker run --network=timecrypt-network -it eth/timecrypt client
45+
```
46+
47+
The client will automatically create a new key store for the cryptographic material of TimeCrypt. The password for this key store will be taken from the `TIMECRYPT_KEYSTORE_PASSWORD` environment variable. If you want to provide an own password you can do so by
48+
49+
You will afterwards be prompted to create a new profile for storing the confidential data of the streams as well as connection information. For this you can safely use the provided default options.
50+
51+
## Build & run on local system
52+
To build the project without Docker you will need the following prerequisites on your system:
53+
- Maven
54+
- A JDK >= Java version 11
55+
- libssl-dev and cmake for building the OpenSSL support (can be skipped by deactivating the `aes-openssl-native` profile of maven (`-P \!aes-openssl-native`))
56+
57+
The root folder of this repository contains a multi-module build to run it start
58+
```
59+
mvn package
60+
```
61+
it will resolve all dependencies and build the project. Afterwards you can find the following JARs
62+
- **Server:** `timecrypt-server/target/timecrypt-server-jar-with-dependencies.jar`
63+
- **Client:** `timecrypt-client/target/timecrypt-testbed-jar-with-dependencies.jar`
64+
- **Producer / Testbed:** `timecrypt-client/target/timecrypt-client-jar-with-dependencies.jar`
65+
66+
### Server
67+
To start the server you need to provide a connection to a Cassandra database (or run the server with an in memory only data storage by providing `TIMECRYPT_IN_MEMORY=true` as environment variable).
68+
You can provide the Host and port of your Cassandra Server by providing the environment variables `TIMECRYPT_CASSANDRA_HOST` and `TIMECRYPT_CASSANDRA_PORT`. The default values for connecting to the database are host: `127.0.0.1` and port: `9042`. For more configuration options see the [README of the server](timecrypt-server/README.md).
69+
70+
### Client
71+
The client provides an interactive way to create streams, add data points and execute queries on the TimeCrypt server.
72+
73+
On startup the client will ask you to create a Keystore and a profile.
74+
The key store is used for secure storing all TimeCrypt related keys.
75+
The profile will be used to store all private metadata about streams (e.g. their start timestamp or the meaning of their aggregatable meta data (digests)).
76+
It also provides login information for the TimeCrypt server.
77+
78+
During the creation of the profile you can select the host and port of the TimeCrypt server.
79+
80+
### Testbed
81+
The Testbed provides a variety of options for interacting with a TimeCrypt server for an overview invoke it with the `-h` option or see the [README of the client](timecrypt-client/README.md).
82+
83+
## AES-NI
84+
This repository offers native encryption support for the Intel AES - New Instructions(AES-NI) which is hardware-based encryption/decryption that may provide enough acceleration to offset the application performance. However: The build might fail if your system does not support it.
85+
86+
AES-NI is supported from the Intel Westmere processors (mid of 2010 / beginning of 2011) onwards.
87+
To check if you have AES-NI on your Linux-based system run:
88+
89+
```
90+
cat /proc/cpuinfo | grep -iq aes && echo 'AES-NI supported' || echo 'no AES-NI support'
91+
```
92+
93+
On non Linux systems you can install the `cpuid` util search for `aes` in its output as it is advised in the [AES-NI documentation](https://software.intel.com/sites/default/files/m/d/4/1/d/8/AES-NI_Java_Linux_Testing_Configuration_Case_Study.pdf).
94+
95+
If your system does not support AES-NI you can disable it during build with the profile switch (`-P \!aesni-native`) e.g.:
96+
97+
```
98+
mvn package -P \!aesni-native
99+
100+
```
101+
102+
## Development
103+
104+
In order to not confuse your IDE you might want to develop in the individual `timecrypt-*` folders.
105+
To satisfy the dependencies to the crypto library run run:
106+
```
107+
mvn install
108+
```
109+
in the root folder before development.

‎docker-compose.yml

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Copyright (c) 2020 by ETH Zurich, see AUTHORS file for more
2+
# Licensed under the Apache License, Version 2.0, see LICENSE file for more details.
3+
4+
version: "3.5"
5+
6+
services:
7+
cassandra:
8+
container_name: cassandra
9+
image: "cassandra:3.11"
10+
ports:
11+
- "9042:9042"
12+
hostname: casandra
13+
networks:
14+
- tcnw
15+
16+
timecrypt-server:
17+
container_name: timecrypt-server
18+
image: "eth/timecrypt"
19+
ports:
20+
- "15000:15000"
21+
build: .
22+
command: server
23+
depends_on:
24+
- cassandra
25+
environment:
26+
- TIMECRYPT_PORT=15000
27+
- TIMECRYPT_CASSANDRA_HOST=cassandra
28+
- TIMECRYPT_CASSANDRA_PORT=9042
29+
networks:
30+
- tcnw
31+
32+
# interactive tty does not work
33+
# timecrypt-client:
34+
# container_name: timecrypt-client
35+
# build: .
36+
# stdin_open: true
37+
# tty: true
38+
# command: client
39+
# depends_on:
40+
# - timecrypt-server
41+
# links:
42+
# - timecrypt-server
43+
44+
networks:
45+
tcnw:
46+
name: timecrypt-network

‎docker-start.sh

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
#!/bin/bash
2+
3+
set -euo pipefail
4+
5+
# How long to wait for cassandra on server startup
6+
CASSANDRA_TIMEOUT=240
7+
TIMECRYPT_TIMEOUT=240
8+
9+
export TIMECRYPT_CASSANDRA_HOST=${TIMECRYPT_CASSANDRA_HOST:=cassandra}
10+
export TIMECRYPT_CASSANDRA_PORT=${TIMECRYPT_CASSANDRA_PORT:=9042}
11+
12+
export TIMECRYPT_HOST=${TIMECRYPT_HOST:=timecrypt-server}
13+
export TIMECRYPT_PORT=${TIMECRYPT_PORT:=15000}
14+
15+
export TIMECRYPT_KEYSTORE_PASSWORD=${TIMECRYPT_KEYSTORE_PASSWORD:=asdfghjklasdfghjkl}
16+
17+
ACTION="${1:-}"
18+
case "$ACTION" in
19+
server)
20+
# Wait for Cassandra to be available
21+
echo "Starting server - waiting for cassandra"
22+
wait-for-it -t $CASSANDRA_TIMEOUT $TIMECRYPT_CASSANDRA_HOST:$TIMECRYPT_CASSANDRA_PORT
23+
echo "Cassandra up"
24+
25+
java -jar $SERVER_JAR_NAME
26+
;;
27+
bash)
28+
/bin/bash
29+
;;
30+
producer)
31+
echo "Starting producer - waiting for server"
32+
wait-for-it -t $TIMECRYPT_TIMEOUT $TIMECRYPT_HOST:$TIMECRYPT_PORT
33+
echo "Server up"
34+
35+
java -jar $TESTBED_JAR_NAME --verbose "${@:2}"
36+
;;
37+
client)
38+
echo "Starting client - waiting for server"
39+
wait-for-it -t $TIMECRYPT_TIMEOUT $TIMECRYPT_HOST:$TIMECRYPT_PORT
40+
echo "Server up"
41+
42+
java -jar $CLIENT_JAR_NAME "${@:2}"
43+
;;
44+
*)
45+
echo "Action '$ACTION' undefined."
46+
echo ""
47+
echo "Usage: $0 <server|client|bash>"
48+
echo ""
49+
echo "server start the TimeCrypt server"
50+
echo "client start a timecrypt client"
51+
echo "producer start a producer (timecrypt testbed)"
52+
echo "bash starts a bash session"
53+
echo ""
54+
55+
exit 1
56+
esac
57+
58+
exit 0

‎pom.xml

+59
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<!--
3+
~ Copyright (c) 2020 by ETH Zurich, see AUTHORS file for more
4+
~ Licensed under the Apache License, Version 2.0, see LICENSE file for more details.
5+
-->
6+
7+
<project xmlns="http://maven.apache.org/POM/4.0.0"
8+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
9+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
10+
http://maven.apache.org/maven-v4_0_0.xsd">
11+
<modelVersion>4.0.0</modelVersion>
12+
13+
<groupId>ch.ethz.dsg.timecrypt</groupId>
14+
<artifactId>timecrypt-parent</artifactId>
15+
<version>1.0</version>
16+
<packaging>pom</packaging>
17+
<name>Multi Chapter Simple Parent Project</name>
18+
19+
<properties>
20+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
21+
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
22+
23+
<jdk.version>1.8</jdk.version>
24+
25+
<maven.compiler.source>${jdk.version}</maven.compiler.source>
26+
<maven.compiler.target>${jdk.version}</maven.compiler.target>
27+
</properties>
28+
29+
30+
<modules>
31+
<module>timecrypt-crypto</module>
32+
<module>timecrypt-server</module>
33+
<module>timecrypt-client</module>
34+
</modules>
35+
36+
<build>
37+
<pluginManagement>
38+
<plugins>
39+
<plugin>
40+
<groupId>org.apache.maven.plugins</groupId>
41+
<artifactId>maven-compiler-plugin</artifactId>
42+
<version>3.8.1</version>
43+
<configuration>
44+
<skipTests>true</skipTests>
45+
</configuration>
46+
</plugin>
47+
</plugins>
48+
</pluginManagement>
49+
</build>
50+
51+
<dependencies>
52+
<dependency>
53+
<groupId>junit</groupId>
54+
<artifactId>junit</artifactId>
55+
<version>4.10</version>
56+
<scope>test</scope>
57+
</dependency>
58+
</dependencies>
59+
</project>

0 commit comments

Comments
 (0)