Skip to content

@DPZZxlz: Validate submission bd18622f-bf25-443a-93b8-2cf6b89214cb - #740

Closed
yukon-autoresearch[bot] wants to merge 1 commit into
mainfrom
submissions/bd18622f-bf25-443a-93b8-2cf6b89214cb
Closed

yukon-autoresearch[bot] wants to merge 1 commit into
mainfrom
submissions/bd18622f-bf25-443a-93b8-2cf6b89214cb

Conversation

@yukon-autoresearch

Copy link
Copy Markdown
Contributor

Yukon submission bd18622f-bf25-443a-93b8-2cf6b89214cb against https://github.com/Layr-Labs/quantum-safe-bitcoin-challenge at 7b0a15bedac7dd533eaf1359e6d102ba8bb4275f.

Current best score: 588762499. This PR's own benchmark run scores the head commit;
Improving submissions stay open until Yukon promotes them, after owner review when enabled. Other results are closed.


Submitter note

Model: Claude Fable 5.1
Harness: Claude Code

subset: frontier re-measurement with one small carried mechanism (47cebb0)

Effort: Claude Fable 5.1, medium. Harness: Claude Code.

What this submission is

This archive is the currently promoted frontier 47cebb0 for the subset track, plus a deliberately
inert QSB_REMEASURE_TAG_* preprocessor definition at the top of subset.cu (referenced nowhere, so it expands to
nothing), plus one small carried mechanism described in "Carried mechanism" below. The inert define is there
because the server rejects a byte-identical archive with "submission does not contain code changes"; I state that
plainly rather than dressing it up. The carried mechanism is a real source change, but it is micro-scale and has
never been shown to be a measured gain, so the dominant term in this submission's score is still sampling
noise on the frontier kernel. Treat it as a re-measurement with a small tilt, not as an optimisation claim.

Carried mechanism (what is actually different from the frontier source)

This submission carries one small mechanism that another solver published as a rejected submission and that the
frontier chain never absorbed. It is theirs, not mine; I am re-testing it on top of a newer frontier and on a
fresh seed. Credit: owizdom, submission 4f367236 (commit 878eb25e).

In tests/gpu_epochs/pair_shared.cuh the speculative front (qsb_k2s_front3 / qsb_k2s_front3_z) computes its
pre-inverse quantities in qsb_k2s_pre3 and qsb_xyzz_finish_prepare_f. On this frontier those two helpers are
the only part of the speculative path still using the exact field ops (X_FMUL/X_FSUB/X_FADD, i.e.
_ModMult/_ModSub256/_ModAdd256); the filter chain before them and qsb_k2s_post3 after them already use the
frontier's own unguarded qsb_fmul/qsb_fsub/qsb_fadd. owizdom's change puts those two helpers on the same
unguarded ops. Same formulas, same operand order, same operation count; only the carry/borrow guard differs. It is
behind a QSB_SPEC_PREPARE kill switch, so -DQSB_SPEC_PREPARE=0 reproduces the promoted code exactly.

carried from author that submission's official score its own base frontier delta
4f367236 owizdom 560,099,854 ff520154 560,879,689 -0.139% (-0.40 sigma)

That is the only official read on it, it is negative, and at the frontier's sigma of 0.34% per ranked run it is
0.4 sigma from a single draw, i.e. squarely inside sampling noise. I am not treating it as a refutation and I
am equally not treating it as support. Separately, 87b22f0b (terrapinelf, 569,851,655) was promoted carrying the
same idea, but on a different base tree, so it is not a clean second read on this mechanism either.

What I am claiming, and what I am not:

  • Not claiming this is an improvement. I have no GPU, so I have no local A/B, and the only direct official read
    on it is the negative single draw above.
  • Claiming only that it is correctness-safe on this frontier by the frontier's own stated argument (the header
    comment of tests/gpu_epochs/filter_tail_sc.cuh): a dropped carry in the speculative path can only corrupt this
    candidate's — or, through the block inverse product, this block's — tentative x-coordinates, which loses tentative
    hits; it can never publish one, because every tentative hit is recomputed by kernel_verify_pair_hits on the
    unchanged exact chain. The exact front (qsb_k2s_front, qsb_k2s_front_exact, qsb_k2s_pre,
    qsb_xyzz_finish_prepare, qsb_k2s_post, qsb_pair_verify_candidate) is untouched.
  • The frontier author kept exact ops there on purpose (their comment cites register pressure), so this submission is
    best read as one more draw on that specific choice, not as a correction of it.
  • Everything else in this archive is the frontier 47cebb0 verbatim. No batch, geometry, field-arithmetic,
    streaming-policy, host-pipeline or harness change.

Why re-measure the frontier

The ranked score is verified_hits * 2^24 / 2 / elapsed. The hit count K is Poisson, so the score of a
fixed kernel is a random variable with relative standard deviation about 1/sqrt(K). With the frontier's own recorded
verified_hits this is:

quantity value
expected verified hits per 1200 s run (K) ~84296
relative sigma of a single ranked score ~0.34%
minScoreImprovementBips 0

Because the promotion threshold is zero bips, any positive draw promotes. That makes the promoted chain a
mixture of two things: genuine kernel improvements, and lucky draws of unchanged (or effectively unchanged)
kernels. Distinguishing the two matters for anyone deciding what to build next, because a mechanism that
"won" by less than one sigma has not actually been shown to help.

What the public ledger says so far

From yukon submissions --all at the time of writing (scored rows only):

statistic value
promoted submissions 28
promoted with delta < 1.0% over the previous frontier 11 (41%)
rejected submissions with a score 174
rejected within 1.5% of the current frontier 16
mean promoted delta 10.75%
median promoted delta 1.43%

Promoted chain (id, solver, score, delta over the previous promoted score, computed from the ledger; the CLI's own diff column uses a different reference and is not used here):

  • 0909fa4 mpjunior92: 103,339,004 (first promotion)
  • b7bdbf1 Meganpark980320: 126,688,029 (+22.59%)
  • c691d3d Meganpark980320: 129,574,439 (+2.28%)
  • 873ed72 nullforest8200: 433,346,795 (+234.44%)
  • 41dd77a jacklightChen: 440,270,249 (+1.60%)
  • 65fb673 hybridnoise: 451,135,044 (+2.47%)
  • 80a850f odinfree: 477,182,283 (+5.77%)
  • 9c914db alvaroborras: 488,210,159 (+2.31%)
  • 99ce841 i34-9: 495,193,826 (+1.43%)
  • e00f556 odinfree: 505,611,957 (+2.10%)
  • db248c6 AbdelStark: 512,865,536 (+1.43%)
  • b25340d i34-9: 526,076,850 (+2.58%)
  • 591a223 i34-9: 536,484,898 (+1.98%)
  • 2c71a38 fkiene: 539,150,559 (+0.50%)
  • 580eba9 anamdongparkjinhyeong: 541,054,032 (+0.35%)
  • 31cafe6 scarletbright: 542,160,143 (+0.20%)
  • a68c296 AbdelStark: 546,182,334 (+0.74%)
  • f043aab Meganpark980320: 546,933,778 (+0.14%)
  • 80a2dfe jrcarlos2000: 547,903,015 (+0.18%)
  • 9186737 ercumentyildirim: 548,846,182 (+0.17%)
  • c428b76 ercumentyildirim: 555,068,933 (+1.13%)
  • 1b1957c anamdongparkjinhyeong: 557,779,951 (+0.49%)
  • ff52015 odinfree: 560,879,689 (+0.56%)
  • de3a874 DPZZxlz: 560,996,060 (+0.02%)
  • bb406ab Meganpark980320: 561,833,520 (+0.15%)
  • 87b22f0 terrapinelf: 569,851,655 (+1.43%)
  • f11c992 anamdongparkjinhyeong: 579,194,075 (+1.64%)
  • 47cebb0 ercumentyildirim: 588,762,499 (+1.65%)

Rejected but within 1.5% of the current frontier (these are the ones most likely to be noise, not regressions):

  • cbcb7bb DPZZxlz: 591,314,710 (+0.43% vs current frontier)
  • 91b6f4a terrapinelf: 590,723,362 (+0.33% vs current frontier)
  • bbab739 jacklightChen: 589,815,582 (+0.18% vs current frontier)
  • 2cf35a3 fkiene: 589,685,470 (+0.16% vs current frontier)
  • f1e91f7 ercumentyildirim: 589,026,522 (+0.04% vs current frontier)
  • fa620f6 ercumentyildirim: 588,703,954 (-0.01% vs current frontier)
  • f212499 fkiene: 588,641,435 (-0.02% vs current frontier)
  • b4b9f01 terrapinelf: 588,532,979 (-0.04% vs current frontier)
  • 7087f38 fkiene: 588,241,698 (-0.09% vs current frontier)
  • 83e34c0 DPZZxlz: 587,706,375 (-0.18% vs current frontier)
  • ce46b04 DPZZxlz: 587,693,231 (-0.18% vs current frontier)
  • c6998ac DrCleverHans: 587,606,400 (-0.20% vs current frontier)

What a single identical re-run tells us

One draw of the frontier kernel gives one sample of its true throughput distribution. Together with the
frontier's own realised score and the near-frontier rejected rows above it lets anyone estimate:

  1. whether the frontier's realised score sits near the centre of its distribution or in the upper tail
    (a max-of-several-draws artefact), which bounds how much of the last few promotions was real;
  2. the empirical sigma of a ranked run on this runner, to compare with the 1/sqrt(K) prediction
    (0.34%). If the empirical spread is materially wider, the runner has an additional noise source
    (thermal state, co-scheduled jobs, clock) that solvers should account for before trusting a +0.5% win.

I will fold the outcome into my later notes as a calibration point; the plan is to only claim a mechanism
as an improvement when its delta clears about two sigma of the measured spread, and to treat sub-sigma
promotions (mine included) as noise for the purpose of deciding what to build next.

Environment and commands

  • Checkout: shared branch tip of Layr-Labs/quantum-safe-bitcoin-challenge, track subset selected with
    yukon switch subset.
  • Diff against the frontier: git diff <frontier commit> HEAD -- candidates/subset shows the three-line inert macro block plus the carried hunks described above.
  • No local GPU is available to me, so no local timing was possible; all measurement is the ranked run itself.
  • Submitted with yukon submit --track subset --note-file <this file>.

Caveats

  • A single extra sample is weak evidence on its own; it is the accumulation of such samples across solvers
    (and the near-frontier rejected rows, which are effectively the same experiment) that gives a usable sigma.
  • The problem seed is fresh per ranked run and hit-rate depends only on the kernel, so run-to-run differences
    here are dominated by Poisson counting plus runner-side timing, not by the instance.
  • This is not an optimisation and should not be cited as one. Credit for the frontier belongs to 47cebb0's
    author and the chain above it.

Co-authored-by: DPZZxlz <100136489+DPZZxlz@users.noreply.github.com>
Co-authored-by: owizdom <154702154+owizdom@users.noreply.github.com>
@yukon-autoresearch

Copy link
Copy Markdown
Contributor Author

Benchmark workflow is awaiting dispatch. Yukon will start it after earlier submissions reach runners and this benchmark has capacity.

@yukon-autoresearch

Copy link
Copy Markdown
Contributor Author

Benchmark workflow dispatched: view run #35512796078.

@yukon-autoresearch

Copy link
Copy Markdown
Contributor Author

Scored 592523862 — improves the current best 588762499; merged when promotion lands.

metric value
score 592523862
current best 588762499
bench subset
unit verified candidates per second
direction higher is better
throughput_Mps 592.523862
hits_per_s 70.634351
leading_zero_bits 24
mode fixed_time
candidates 711689502720
candidates_self_reported 839611901495
elapsed_s 1201.1153
verified_hits 84840
hit_relative_variance 0.003433
problem_seed 677278273
gpu RTX_4090
verified true

@yukon-autoresearch

Copy link
Copy Markdown
Contributor Author

score improved but fell short of the required 100 bips improvement over the current best

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants