Skip to content

Publish the original Experiment 2 run records (#17) - #19

Merged
k2jac9 merged 1 commit into
mainfrom
data/experiment2-runs
Nov 16, 2026
Merged

k2jac9 merged 1 commit into
mainfrom
data/experiment2-runs

Conversation

@IPBin

@IPBin IPBin commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Closes most of #17. The original Experiment 2 run folders were still on my laptop, so this adds them as one archive.

What's in it (data/fleet-witness-judge-runs-2026-11-10.zip, 15.8 MB zipped, about 171 MB unzipped)

  • The colluder-attack runs: DeepSeek and gpt-oss, 0/20/40% colluders, both boards.
  • The honest frontier runs: GPT-6.1 Sol and Gemini 4 Argon.
  • The stopped v1 runs and the pilot.
  • In every run: all votes and attestations, decisions, sealed labels, manifests, and the raw prompts and responses.

One change from the originals. Case 02 contains a fake Stripe-style key, put there on purpose for the models to catch. I replaced it with sk_live_FAKE-KEY-REMOVED so GitHub's secret scanner doesn't block the push. The archive includes the list of changed files and hashes of both versions. Nothing else was changed. I checked that no real API key or personal email is anywhere in it.

What this makes checkable (I ran these against the raw files)

  • The run table: 16,416 calls per model, 0 failed calls, 0 retries, same costs.
  • Abstentions: 225 of 230 hit the length limit; 222 were on safe or forged cases, and 8 were on harmful cases (6 by colluders).
  • Honest DeepSeek approvals of harmful targets: 1 of 3,358 (case 15, 40%, hidden board).
  • gpt-oss gated majority, within the same run: colluders tipped 3 of the 10 harmful releases, and honest reviewers approved the other 7 on their own.

scripts/package_experiment2_runs.py rebuilds the archive. It only reads the local runs. No model calls, no changes to the report.

Adds the recovered Experiment 2 run folders as one 15.8 MB archive: the
colluder-attack populations, honest frontier runs, stopped v1 runs and pilot,
with events, decisions, sealed truth, manifests and raw requests/responses.

The only change from the originals is replacing the synthetic Stripe-style key
from case 02 with a placeholder; original and published hashes are included.
The packaging script reads the local runs and never modifies them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@k2jac9
k2jac9 merged commit 2160a55 into main Nov 16, 2026
24 checks passed
@k2jac9
k2jac9 deleted the data/experiment2-runs branch November 16, 2026 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants