Skip to content

Add reproducible builds #233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+100 −3
Open

Add reproducible builds #233

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cb9b9c8
Add makefile command to create reproducible build
SozinM Aug 13, 2025
6243492
Remove unnecessary "unknown" target vendor
alexhulbert Aug 13, 2025
882e2ea
Docker + CI for reproducible builds
alexhulbert Aug 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/workflows/docker_build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,13 @@ jobs:
runner: warp-ubuntu-latest-x64-16x
- target: linux/arm64
runner: warp-ubuntu-latest-arm64-16x
docker_target:
- name: regular
target: rbuilder-runtime
tag_suffix: ""
- name: reproducible
target: rbuilder-reproducible-runtime
tag_suffix: "-reproducible"
steps:
- name: checkout sources
uses: actions/checkout@v4
Expand Down Expand Up @@ -83,6 +90,9 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}
platforms: ${{ matrix.configs.target }}
push: true
tags: ${{ steps.meta.outputs.tags }}
target: ${{ matrix.docker_target.target }}
tags: |
ghcr.io/${{ github.repository }}/op-rbuilder:latest${{ matrix.docker_target.tag_suffix }}
ghcr.io/${{ github.repository }}/op-rbuilder:sha-${{ env.VERSION }}${{ matrix.docker_target.tag_suffix }}
build-args: |
RBUILDER_BIN=op-rbuilder
8 changes: 6 additions & 2 deletions .github/workflows/release.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,12 @@ jobs:
run: |
git config --global --add safe.directory "$(pwd)"
. $HOME/.cargo/env
cargo build --release --features=${{ matrix.features }} --target ${{ matrix.configs.target }} --package op-rbuilder
SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure it will work

RUSTFLAGS="--C target-feature=+crt-static -C link-arg=-static-libgcc -C link-arg=-Wl,--build-id=none -C metadata='' --remap-path-prefix=$(pwd)=." \
CARGO_INCREMENTAL=0 \
LC_ALL=C \
TZ=UTC \
cargo build --release --features=${{ matrix.features }} --locked --target ${{ matrix.configs.target }} --package op-rbuilder
- name: Upload op-rbuilder artifact
uses: actions/upload-artifact@v4
with:
Expand Down
33 changes: 33 additions & 0 deletions .github/workflows/reproducible_verify.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Verify Reproducible Build
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the best idea is to run this on schedule once a week / couple of days for example
This way we will notice if something has broken repro builder, but won't wait for this to pass on every PR, because it will take a LOT of time

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured we were just going to run this manually whenever we have a release that we want to put in a new image. Every 3-4 days def wouldn't hurt though. I'll add that.


on:
workflow_dispatch:

jobs:
verify:
name: Verify reproducible builds
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: Build twice and compare
run: |
export REPRO_FLAGS="--C target-feature=+crt-static -C link-arg=-static-libgcc -C link-arg=-Wl,--build-id=none -C metadata='' --remap-path-prefix=$(pwd)=."
export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
RUSTFLAGS="$REPRO_FLAGS" CARGO_INCREMENTAL=0 LC_ALL=C TZ=UTC \
cargo build --release --locked -p op-rbuilder
mv target/release/op-rbuilder build1
cargo clean
RUSTFLAGS="$REPRO_FLAGS" CARGO_INCREMENTAL=0 LC_ALL=C TZ=UTC \
cargo build --release --locked -p op-rbuilder
mv target/release/op-rbuilder build2
if cmp -s build1 build2; then
echo "✅ Builds are reproducible"
else
echo "❌ Builds differ"
exit 1
fi
21 changes: 21 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,10 +73,31 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
cargo build --release --features="$FEATURES" --package=${RBUILDER_BIN}

#
# Reproducible builder container (deterministic source-date-epoch, no caching, no incremental builds)
#
FROM base AS rbuilder-reproducible
ARG RBUILDER_BIN
ARG FEATURES
WORKDIR /app
COPY . .
RUN SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will force docker to build this 2 times (and one of them would be slow build), let's do it better

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are compiling regular build + reproducible build
Doesn't really make sense, because you could separate them or use a flag to build one or the other.
This now inside the check we are running instead of compiling once we would compile both of the, and probably go from 2min -> 10min

Copy link
Member

@alexhulbert alexhulbert Aug 18, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh I see. That RUN command means that the builds are actually happening during docker build, not docker run. I can just add a --target flag to the PR workflow to force it to build only the non-reproducible. Does that solve the problem?

RUSTFLAGS="--C target-feature=+crt-static -C link-arg=-static-libgcc -C link-arg=-Wl,--build-id=none -C metadata='' --remap-path-prefix=/app=." \
CARGO_INCREMENTAL=0 \
LC_ALL=C \
TZ=UTC \
cargo build --release --locked --features="$FEATURES" --package=${RBUILDER_BIN}

# Runtime container for rbuilder
FROM gcr.io/distroless/cc-debian12 AS rbuilder-runtime
ARG RBUILDER_BIN
WORKDIR /app
COPY --from=rbuilder /app/target/release/${RBUILDER_BIN} /app/rbuilder
ENTRYPOINT ["/app/rbuilder"]

# Reproducible runtime container for rbuilder
FROM gcr.io/distroless/cc-debian12 AS rbuilder-reproducible-runtime
ARG RBUILDER_BIN
WORKDIR /app
COPY --from=rbuilder-reproducible /app/target/release/${RBUILDER_BIN} /app/rbuilder
ENTRYPOINT ["/app/rbuilder"]
29 changes: 29 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,26 @@ GIT_TAG ?= $(shell git describe --tags --abbrev=0)

FEATURES ?=

# Environment variables for reproducible builds
# Initialize RUSTFLAGS
RUST_BUILD_FLAGS =
# Enable static linking to ensure reproducibility across builds
RUST_BUILD_FLAGS += --C target-feature=+crt-static
# Set the linker to use static libgcc to ensure reproducibility across builds
RUST_BUILD_FLAGS += -C link-arg=-static-libgcc
# Remove build ID from the binary to ensure reproducibility across builds
RUST_BUILD_FLAGS += -C link-arg=-Wl,--build-id=none
# Remove metadata hash from symbol names to ensure reproducible builds
RUST_BUILD_FLAGS += -C metadata=''
# Set timestamp from last git commit for reproducible builds
SOURCE_DATE ?= $(shell git log -1 --pretty=%ct)
# Disable incremental compilation to avoid non-deterministic artifacts
CARGO_INCREMENTAL_VAL = 0
# Set C locale for consistent string handling and sorting
LOCALE_VAL = C
# Set UTC timezone for consistent time handling across builds
TZ_VAL = UTC

##@ Help

.PHONY: help
Expand All @@ -31,6 +51,15 @@ build: ## Build (debug version)
op-rbuilder: ## Build op-rbuilder (debug version)
cargo build -p op-rbuilder --bin op-rbuilder --features "$(FEATURES)"

.PHONY: build-reproducible
build-reproducible: ## Build the reth binary into `target` directory with reproducible builds
SOURCE_DATE_EPOCH=$(SOURCE_DATE) \
RUSTFLAGS="${RUST_BUILD_FLAGS} --remap-path-prefix $$(pwd)=." \
CARGO_INCREMENTAL=${CARGO_INCREMENTAL_VAL} \
LC_ALL=${LOCALE_VAL} \
TZ=${TZ_VAL} \
cargo build -p op-rbuilder --bin op-rbuilder --features "$(FEATURES)" --profile "release" --locked --features "$(FEATURES)"

.PHONY: tdx-quote-provider
tdx-quote-provider: ## Build tdx-quote-provider (debug version)
cargo build -p tdx-quote-provider --bin tdx-quote-provider --features "$(FEATURES)"
Expand Down
Loading