feat: real-time web dashboard for experiment monitoring#334
Open
Death-Incarnate wants to merge 1 commit intokarpathy:masterfrom
Open
feat: real-time web dashboard for experiment monitoring#334Death-Incarnate wants to merge 1 commit intokarpathy:masterfrom
Death-Incarnate wants to merge 1 commit intokarpathy:masterfrom
Conversation
Single-file FastAPI dashboard (dashboard.py) with:
- Overview: KPI cards (best val_bpb, baseline, improvement %, run counts),
val_bpb scatter chart colored by keep/discard/crash, status pie chart,
live GPU stats (util %, VRAM, temperature)
- Live Training: step progress bar, loss + mfu% dual-axis chart (2s poll),
SSE log stream with color-coded output, autoscroll toggle
- Experiments: sortable table, best run highlighted in gold, click commit
hash to see inline train.py diff
- Git History: last 40 commits with click-to-diff
- Controls: launch form (run tag, model picker), stop button, live
hyperparams table parsed from train.py
- Settings: edit program.md in-browser, read-only train.py viewer
Zero frontend deps — pure HTML/CSS/JS served inline from the single Python
file. Configure repo path via REPO_DIR env var, port via PORT env var
(default 7788).
Start with:
cd autoresearch && uv run dashboard.py
Adds fastapi>=0.135.1 and uvicorn>=0.42.0 to pyproject.toml.
Author
|
As context for why this matters right now: Alexey Grigorev just wrote a breakdown of autoresearch that's getting traction — https://alexeyondata.substack.com/p/karpathys-autoresearch-went-viral One of the main things people hit when they clone this repo and start running experiments is zero visibility into what's happening. The training loop is a black box unless you're tailing logs manually. The dashboard fills that gap — live loss curve, GPU stats, log stream, experiment history — all without adding any dependencies to the core research loop. It's opt-in and read-only relative to the experiment state. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A single-file FastAPI dashboard (
dashboard.py) that gives autoresearch a proper UI for monitoring and controlling experiments in real time.Start it
Set
REPO_DIRenv var to point at any autoresearch clone. SetPORTto change from 7788.Tabs
Overview
nvidia-smi)Live Training
Experiments
Git History
Controls
Settings
Implementation notes
REPO_DIRso it works with any clone location