Skip to content

Clarify experiment bookkeeping in program.md#306

Open
Gabrieliam42 wants to merge 1 commit intokarpathy:masterfrom
Gabrieliam42:fix/program-md-bookkeeping-upstream
Open

Clarify experiment bookkeeping in program.md#306
Gabrieliam42 wants to merge 1 commit intokarpathy:masterfrom
Gabrieliam42:fix/program-md-bookkeeping-upstream

Conversation

@Gabrieliam42
Copy link

Summary

  • clarify that train.py is the only source file to edit
  • explicitly allow results.tsv bookkeeping and run.log updates as experiment artifacts
  • require the results.tsv row to be appended before any reset, checkout, or revert

Why

The previous wording could be read as forbidding the log and results bookkeeping that the experiment loop itself requires. This makes the intended workflow explicit without broadening what source files may be changed.

@elementalcollision
Copy link

Technically, program.md is also a source file to edit, as you can designate which dataset you want to experiment on. I don't necessarily view this as ambiguous as even the agent (in my case Opus and/or Sonnet) understands which files are immutable and which aren't.

@Gabrieliam42
Copy link
Author

Gabrieliam42 commented Mar 17, 2026

All right,
It's just that when I ran it with Codex on VS Code it didn't update results.tsv or run.log because it interpreted that train.py directive as a override restriction, so it ignored the other directives, therefore it didn't work as intended with Codex and GPT-5.4, probably Claude is more permissive. After I modified it as stated it began to work as the project intended.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants