dev: reh3376_dev01 -> main#462
Merged
Merged
Conversation
.claude/ is gitignored (per-developer local config), so the installed skill at .claude/skills/jiminy-governance/SKILL.md is local-only by convention. Commit the reproducible, install-ready copy (jiminy-governance.skill.md) + a README with the one-line install (cp into .claude/skills/) so the skill propagates via the repo. The MCP server it uses is registered in the tracked repo-root .mcp.json. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hooks and the jiminy-governance skill hardcoded session_id="claude-core", so trust/escalation/observations from ALL Claude Code conversations collapsed into one shared MDEMG session. Claude Code already passes a per-conversation session_id on stdin to every hook — the implementation just never used it. Resolver precedence (single rule everywhere): MDEMG_SESSION_ID env (stable- identity escape hatch) > Claude Code stdin session_id (per-conversation default, race-free per hook) > ~/.mdemg/.claude-session (published by SessionStart + UserPromptSubmit for the agent / stdin-less contexts) > claude-core (fallback). Realizes J17's intended per-(session,constraint) isolation. Tracked templates updated (internal/cli/hook_templates/): session-start.sh, prompt-context.sh, post-tool-observe.py, pre-compact.sh + Windows .ps1 variants — every hardcoded claude-core in MDEMG calls replaced with the resolved id; session-start/prompt-context publish the session file. Skill SessionID instruction + handshake steps now resolve <SessionID> instead of claude-core. Live-verified: a hook resolved a stdin session_id and published the session file; post-tool-observe wrote an observation keyed to the per-conversation id in Neo4j (not claude-core). bash -n / py_compile clean; go build + hooks test pass. Note: pre-write-check.py is local-only (no tracked installer template); its fix is applied on this machine but won't propagate via `mdemg hooks install` until it's added to the tracked hooks (follow-up). .claude/* is gitignored, so the live hook copies aren't committed — the tracked templates propagate via install. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The .claude/hooks/ installed copies are tracked (committed before the .claude/* ignore), so apply the same SessionID resolver here as in the embedded templates (prior commit). session-start.sh, prompt-context.sh, post-tool-observe.py, pre-compact.sh now resolve MDEMG_SESSION_ID env > stdin session_id > ~/.mdemg/.claude-session > claude-core. (pre-write-check.py is untracked / local-only — its fix lives on this machine only; tracking it is the follow-up.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The /strict J17 Write/Edit classify gate (pre-write-check.py) was local-only —
no tracked template, so `mdemg hooks install` never installed it and its
SessionID fix wouldn't propagate. Add it as a tracked template
(internal/cli/hook_templates/pre-write-check.py, space_id → {{SPACE_ID}},
runtime URL discovery — no {{MDEMG_URL}} placeholder per the template
convention) and register it in claudeHookFiles() as
{PreToolUse, 8s, "Write|Edit"}. hooks_test expectations updated 5→6.
go build + hooks test + lint clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promote CHANGELOG [Unreleased] → [0.10.1] - 2026-06-08. Adds the jiminy-governance skill + MDEMG MCP registration and the per-conversation SessionID work to the release notes (alongside the already-logged EVENTGRAPH-002, EVENTGRAPH-CLI-001, NOSILENT-001, the docker-PATH fix, and the TSDB schema 22→23→24 bumps). Fresh empty [Unreleased]; comparison link refs updated + backfilled through v0.10.1. The v0.10.1 git tag (triggers release.yml artifact build) + homebrew formula bump are the operator release-cut step on main, post-merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(1) New docs/features/jiminy-governance.md — detailed how-it-works + full file
inventory for the J17 agent-governance system (skill, hooks, SessionID, MCP,
enforcement, runtime state files, install/verify steps).
(2) docs/user/api-reference.md — add the Event Graph Federation section
(POST /v1/eventgraph/{reinforcement,guidance-outcome}-neighborhood) + TOC entry;
these were the only two missing endpoints (audited the full route table).
(3) docs/user/cli-reference.md — add mdemg eventgraph {reinforcement,guidance-
outcome}-neighborhood, model run, watchdog status, migrate context-fingerprint,
data curate/validate/clean; fix the stale `model pull --adapter` description
(MODEL-DIST-002 shipped — no longer "deferred/errors"); update the Command Tree
Summary to match.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Point the parent at the manually-published v0.10.1 homebrew formula (reh3376/homebrew-mdemg@10c1843). The release artifacts published cleanly; the formula update was manual because the CI HOMEBREW_TAP_TOKEN expired (follow-up: rotate the secret so future releases auto-publish). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…Hebbian paths Wire the 3 remaining Hebbian write paths (CoactivateSession, ApplySymbolCoactivation, ApplyNegativeFeedback weaken-only) into the existing reinforcement_events writer via distinct trigger_path values. No schema/writer/ wiring change (V0022 already has trigger_path + signed delta_weight + created_new_edge; writer already injected). Contradict path deferred (CONTRADICTS edges aren't traversed by the federation walk). RETURN-only Cypher edits; Tier-2 asserts unchanged weights. 5 epics, 3 tiers, live Tier-3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s (Epic 1) CoactivateSession (session-internal conversation-observation co-activation, full Hebbian formula) now emits per-pair reinforcement events with trigger_path=coactivate_session. RETURN-only Cypher change: replaced the discarded `count(*)` with the standard 17-field per-pair RETURN (one row per forward edge; reverse is a mirror). Weight SET untouched → update behavior provably unchanged. Mirrors the proven ApplyCoactivation record loop; writer already injected. EXPLAIN-validated (compiles, all RETURN vars in scope, no writes); build + lint clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…_events (Epic 2) SymbolNode-pair co-activation now emits trigger_path=apply_symbol_coactivation rows. Split the weight update out of the ON MATCH clause into a separate SET so the pre-update weight (w) can be captured for prev/new/delta — createdNew (evidence_count=1) keeps a fresh edge at 0.1 and increments matches by +0.05, preserving the original ON-clause weight behavior exactly. eta/surprise/ activation/path_sim are NULL (N/A for symbols); roles default 'symbol_node'. EXPLAIN-validated; build + lint clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rcement_events (Epic 3) The weaken path (existing CO_ACTIVATED_WITH edge weakened by negWeight) now emits trigger_path=apply_negative_feedback rows with a NEGATIVE delta_weight and created_new_edge=false. The FOREACH writes (weaken SET + contradict MERGE) are untouched; only the RETURN changed from aggregated `action,count(*)` to per-pair rows — the Go side counts rows (sum = grouped count, NegativeFeedbackResult preserved) and emits reinforcement events for weaken rows only. prevWeight is captured before the FOREACH SET. Contradict path deliberately not emitted (CONTRADICTS isn't traversed by the federation walk; deferred). EXPLAIN-validated; build + lint clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lly runs
Discovered via EVENTGRAPH-003 live smoke: session co-activation
(CO_ACTIVATED_WITH edges between same-session conversation observations) had
NEVER fired — 0 such edges ever in mdemg-dev across 5495 conversation
observations. Root cause: conversation.NewServiceWithConfig sets
learningService=nil ("set via SetLearningService to avoid circular dependency"),
but SetLearningService had NO caller, so the `if s.learningService != nil` guard
in Observe() always skipped CoactivateSession. The function + its Cypher were
correct (verified by running it directly: 3 pairs, proper Hebbian weights) —
it was just never invoked.
Fix: convSvc.SetLearningService(lea) at construction. Live-verified: 3 distinct
observations in a session now create 6 CO_ACTIVATED_WITH edges + emit
coactivate_session reinforcement events. Standalone fix-commit per the
live-smoke precedent (surprise bugs don't get rolled into the sprint commit).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… close (Epic 4) All four trigger_paths live-verified (apply_coactivation 50, apply_symbol_ coactivation 1000, apply_negative_feedback 1 negative-delta, coactivate_session 4 after the dormancy fix); federation CLI surfaces them. Feature doc updated to all-four-paths + the trigger_path table; CHANGELOG Added (EVENTGRAPH-003) + Fixed (CoactivateSession never-invoked); CLAUDE.md note + correction (CoactivateSession was dead, not "writing via sidecar paths"); verification.md + post.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…alth review (Epic 0) EVENTGRAPH-004 federates the last unfederated Hebbian write — the ApplyNegativeFeedback contradict action — into reinforcement_events (trigger_path=apply_negative_feedback_contradict). Data-decided scope: reuse the existing V0022 sink (zero CONTRADICTS edges exist anywhere; no producer calls /v1/learning/negative-feedback — instrument before the producer arrives, the inverse of the dormancy pattern). Also closes the EVENTGRAPH-003 follow-up: 30h post-fix health review of the revived CoactivateSession path — no tuning needed, textbook session cliques, pre-fix orphans stay as historical record (operator decision). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…inforcement_events (Epic 1) The contradict action (no co-activation edge → MERGE CONTRADICTS) was the last unfederated Hebbian write. The CONTRADICTS MERGE lived inside a FOREACH, where the edge variable is invisible to RETURN — so the original single statement is split into two statements in the SAME ExecuteWrite transaction: (a) weaken (EVENTGRAPH-003 telemetry, RETURN unchanged) and (b) contradict with a per-pair RETURN. Classification is identical: weaken never deletes edges, so contradict's NOT EXISTS sees the same edge set the original OPTIONAL MATCH did. Contradict rows land with trigger_path=apply_negative_feedback_contradict. created_new_edge detected via `c.updated_at IS NULL` (ON MATCH always sets it; ON CREATE never does — invariant pinned by comment). delta_weight is the CONTRADICTS edge's OWN weight delta (+negWeight on create, 0 on re-match); negative-feedback semantics are carried by trigger_path, not the sign. Both statements EXPLAIN-validated against live Neo4j. Tier 1: 2 new parser tests (create/re-match branches); learning suite green; lint clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-match + weaken unchanged (Epic 2) Live against the restarted Epic-1 binary: contradict create row (+0.15, created_new_edge=true), re-match row (delta=0, evidence=2), weaken row byte-equivalent to pre-split behavior (negative delta, floor at 0). Federation CLI surfaces the new trigger_path with no read-side change. UATS learning_negative_feedback 5/5 PASS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…c 3) Feature doc: 5-path trigger_path table + delta-semantics consumer warning (contradict delta is the CONTRADICTS edge's own weight delta — semantics live in trigger_path, not the sign). UATS spec extended: zero-count equals assertions on nonexistent nodes (hash refreshed, 5/5 live). CLAUDE.md architecture note + producer-gap disclosure. Sprint close in post.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Squash-merge workflow leaves a stale merge-base: PR #418's squash (b408bbc) rewrote the same CHANGELOG/CLAUDE.md/service.go regions this branch then extended in EVENTGRAPH-004. Verified before resolving: main == dev01@36377a2 + .github/workflows/codeql.yml exactly (git diff b408bbc 36377a2 shows only codeql.yml), so taking this branch's side of every content conflict is lossless; codeql.yml comes in from main. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Squash merges never advance the dev branch's merge-base, so every sprint touching CHANGELOG.md/CLAUDE.md hit CONFLICTING on its next PR (first bitten: PR #419). New sync-dev-after-merge.yml merges main back into the source *_dev* branch after each merged PR; the GITHUB_TOKEN push triggers no other workflows, so it can never spawn an empty auto-PR (the PR #420 failure mode). Conflicts fail loudly for manual resolution; workflow_dispatch enables manual runs/live testing. auto-pr.yml additionally skips PR creation when branch content is identical to main — guards MANUAL sync pushes, verified against the live repo state (current dev01 ≡ main → empty=true → skip). actionlint clean (untrusted refs passed via env, not inline). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…deep-dive Full-codebase review vs MDEMG's purpose (cognitive substrate / connection layer): 19 map agents (3 vision + 16 subsystem), 3 cross-cutting assessors, synthesizer + adversarial completeness critic (19 revisions applied). Verdict: server-side substrate is mature, but the system is not currently functioning as the assistant's internal dialogue — the per-prompt delivery channel silently no-ops (hook reads .user_prompt, Claude Code sends .prompt), 100% of GENERALIZES edges have NULL weight (22,170/22,170, live-verified), scheduled decay/prune has been a permanent dry-run, RSIC validates 16/17 actions vacuously, and supervision covers 3 of ~14 background loops. Every defect is the same disease: wired-looking seams with no caller, wrong contract, or no reader. 4 phases ≈ 75 days committed: (1) reconnect the loop ends, (2) close the learning loops, (3) survivability + class-ending forcing functions, (4) FT frontier + release hygiene. Top-10 ranked; deferrals explicit. Orchestrator spot-verification annex included (5 claims re-verified live). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…per-prompt channel (Epic 0) Roadmap Q3 Phase 1 rank #1. Audit of all 6 hooks vs the actual Claude Code stdin schemas: prompt-context.sh reads .user_prompt (CC sends .prompt) → channel exits silently on every prompt; post-tool-observe.py reads tool_output (CC sends tool_response) → false "Build/test succeeded" observations with empty output; guidance wrongly coupled to RESULT_COUNT>0; minor pre-compact transcript jq. session-start / pre-bash-check / pre-write-check verified correct. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rompt channel (Epic 1) Claude Code's UserPromptSubmit stdin field is `prompt`; the hook read `.user_prompt`, which is always empty → exit 0 → per-prompt CMS recall, Jiminy guidance, /strict reformulation, the warm trigger, and the retrieve-time Hebbian reinforcement have NEVER fired in any session. Now reads `.prompt // .user_prompt` (legacy fallback kept). Also decouples guidance from recall: the RESULT_COUNT=0 branch no longer exits — it printed its notice then skipped guidance + warm + retrieval reinforcement, coupling independent deliveries. Both copies (live + installer template). Tier 1 simulated stdin: real .prompt payload → first-ever guidance delivery (J17 T1 bootstrap + DICT, 5363 guidance bytes vs 0 forever); legacy fallback works; short/empty/ malformed payloads exit silently (fail-open preserved). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…"succeeded" observations (Epic 2)
Claude Code's PostToolUse stdin field is `tool_response` (string or
object); the hook read `tool_output`, which is always absent → output_str
empty → error indicators never matched → every go build/go test/pytest
Bash call was recorded as "Build/test succeeded" sight-unseen, and real
errors were never observed.
Now reads tool_response (fallback tool_output) via _response_text(),
normalizing string|dict|list (stdout/stderr join). Success classification
requires NON-EMPTY clean output — a silent success records nothing rather
than fabricating; failures land as error observations with real stderr.
Both copies (template regenerated from fixed live, {{SPACE_ID}}
placeholder preserved, verified identical modulo placeholder). Tier 1
against real CMS: failing build → error obs with stderr; passing →
progress; empty → no record.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ine shape (Epic 3)
Transcript lines are {type, message:{content:[{type, text|name, ...}]}};
the old top-level `.content` read always yielded empty, so pre-compaction
snapshots never carried recent-activity context. New jq walks
.message.content[] extracting .text/.name. Verified against this
session's real transcript (old: nothing; new: real activity). Both
copies, placeholders preserved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…act pin + close (Epics 4-5) Live in the real session: first-ever guidance delivery (J17 T1 bootstrap + DICT, 5363 bytes vs 0 forever); real failing build → error observation with actual compiler output in CMS. PostToolUse success-only firing documented as a limitation. Hook stdin contract pinned in CLAUDE.md. Drift + clique-semantics findings logged for HOOKSYNC-001 / Phase 2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… reh3376_dev01
Lead sprint of the training-integrity remediation (operator-sequenced eval-integrity-first). Makes the adapter promotion gate trustworthy: clean leak-free 17-task holdout (rebuild — valid_clean is stale/wrong shape), leak-audit preflight, GGUF serving (not the OOM-prone MLX side-server), zero-call hard-fail, and baseline-as-recomputed-report (not a frozen 0.8338 constant). Discloses the sequencing constraint: the honest baseline number is re-run at the close of the reward sprint (it must use the fixed reward), so this sprint stamps a provisional one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ted tasks' production rows (Epic 1) The clean-eval builder + benchmark matcher strict-hash-matched ANY spec with a pinned system_prompt_hash, ignoring dynamic_prompt. But ape.reflect's prompt is built at init() by interpolating AllowedLLMActions into the text — its hash drifts every time the action enum changes (RSIC has added actions repeatedly). Result: ape.reflect's 71,033 production rows (under hash ef068ab) were invisible to a filter pinned to 39b2bc (0 matches) — the single largest training target had its entire corpus silently excluded from eval AND curation. Fix: Spec now carries dynamic_prompt; the matcher (run_benchmark) and build_clean_eval match dynamic/enum-templated prompts by task_name, not by the moving hash. ape.reflect's spec corrected to dynamic_prompt=true (was false despite being enum-templated). Proven end-to-end: clean-eval coverage 10/17 → 12/17 (ape.reflect + hidden.reclassify recovered, 240 rows). 90 neural tests pass. Residual: 5 tasks (consulting.synthesis, guardrail.evaluate, metalearn.generalize, retrieval.rerank_nli, summarize.generate) have ~0 production rows — genuinely uncaptured, a separate capture-gap issue. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(Epic: gate-set) The promotion gate evaled on the 99%-leaked valid_golden. Re-pointed benchmark_phase10.yaml + the UBENCH spec to the rebuilt leak-free valid_clean (240 rows / 12 tasks with real current-prompt production data; 5 capture-gap tasks excluded as uncoverable — operator-approved honest-12 over fake-17). Re-pinned config.sha256 + golden_holdout sha256/expected_rows(240)/expected_tasks(12). ubench lint 100%. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A benchmark run that made zero successful calls — or where no task contributed a score (weight_used==0) — reported aggregate 0.0, indistinguishable from a genuine low score. This is the false-0.0 class behind FT-CLASSIFY-002's four silent failures (port rot, model 404s, llama-server answering /v1/models before load). run() now computes specs_with_zero_successful_calls + a status field; a degenerate run is status=error and main() exits nonzero (the gate/CI/operator rejects it instead of mistaking a broken run for a model judgment). Pin tests on _count_zero_call_specs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e summarize.generate (Epic: capture-gap) The capture-gap investigation found summarize.generate (default-ON, LLM_SUMMARY_ENABLED=true) runs in the `mdemg ingest` subprocess where SetDefaultRecorder was NEVER called (only `mdemg serve` wired it) — so every summary LLM call was made and silently dropped, losing the task's entire production corpus (the EMBED-WIRE recorder-gap class, via a process boundary). Now wires a short-lived buffered LLM recorder in runIngest before summarize.New (which captures defaultRecorder at construction), guarded by LLM_INTERACTION_LOGGING, with flush+close on exit. summarize.generate production rows will now accumulate → coverable by the clean eval over time (13th task). The other 4 zero-row tasks (consulting.synthesis, guardrail.evaluate, metalearn.generalize, retrieval.rerank_nli) are correctly GATED-OFF (SYNTHESIS/GUARDRAIL/METALEARN/RERANK_ENABLED all default-false) — expected zeros, no fix needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
make test-eval-leak runs audit_eval_leakage.py over valid_clean vs the SFT training sources, exits nonzero on ANY overlap — the gate must re-verify the eval is disjoint from training (the valid_golden 99%-leak class; build-time filtering isn't a substitute as the corpus grows). The script existed but had zero callers. Verified: valid_clean CLEAN, 0/240 overlap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… reh3376_dev01
…clean file:line
The dynamic_prompt=true change is correct, but I'd also appended an
annotation to system_prompt_source — the ULTS --verify-hashes runner
parses that field AS the file:line to hash, so the annotation broke it
('source file not found'), failing the merge-blocking ULTS gate.
Restored the clean 'internal/ape/llm_reflector.go:74 (rendered with
AllowedLLMActions enum)'; dynamic_prompt=true retained. ape.reflect now
11/11 PASS locally. (UBTS --output + UOBS metric-not-found in the same
CI run are pre-existing continue-on-error soft-fails, not from this PR.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2nd training-integrity remediation sprint. The distill gate (x9_distill_capture_v2.py:360: kept = mean(reward_vector) >= 0.8, global) drops spec-correct-but-terse answers because coverage_score/ explanation_quality/coherence_score reward length over correctness — gutting ape.reflect (largest target) + summarize + synthesize, then balanced_sampler amplifies the verbose-skew. Principle: inclusion selects for CORRECTNESS not length. Fix: length-neutral correctness rewards + per-task inclusion thresholds + a forcing-function test (each of the 12 covered tasks' known-correct golden rows clear their gate) + distribution check. Closes with the eval-integrity-deferred GGUF serving + honest baseline recompute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…c 1) The distill inclusion gate (mean(reward_vector) >= 0.8) selected for LENGTH, not correctness — the corpus-skew mechanism behind 3 discarded retrains. Four reward functions used length/count ladders that dropped spec-correct-but-terse answers below the gate and rewarded verbosity upward: - coverage_score: <20 words→0.4 / <50→0.7 / then rising → now substantive content scores 0.9 flat (length-neutral); empty→0.0, pure-repetition→0.3. - explanation_quality: <20 words→0.6 cliff → now substantive→0.9. - coherence_score: required >=2 sentences + 10 words → now any coherent non-repetitive response→0.9; pure repetition→0.4. - insight_count: rewarded bullet COUNT (>=5→1.0) → now >=1 genuine insight→0.9 (no upward count reward; stops bullet-spam, stops dropping single-insight reflections to 0.5 — ape.reflect, the largest target). Verified: terse-correct now clears the 0.8 gate; verbosity/bullet-count no longer rewarded above concise; varied detailed content unaffected (0.9); empty/repetition still rejected. Tests rewritten to pin the new semantics (78 pass). Subtler keyword-bag functions (specificity/actionability) left for the continuation — they reward content signals, not raw length. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ndings (Epic 2-3)
Epic 2: --reward-threshold-map JSON ({"task": float}) overrides the global
--reward-threshold per task in x9_distill_capture_v2.py, so tasks whose reward
arrays have a different natural ceiling can gate at the right bar. Records the
per-task gate in each row + the manifest. Live-verified end-to-end: real
OpenAI + TSDB run with {"consulting.classify": 0.6} applied the override
(3/3 captured, manifest per_task reward_threshold=0.6).
Epic 3 (live Tier 3, docs/development/reward-correctness-001/live_findings.md):
scored REAL production llm_interactions at the 0.8 gate, old vs new rewards.
Validated Epic 1: hidden.summarize recovered 69/72 real concise summaries the
old length ladder dropped. Surfaced three larger correctness issues the
length fix does NOT close (the real dominant suppressors for the big tasks):
1. ape.reflect (54k, largest target): json_valid mean 0.133 — ~87% of recent
responses TRUNCATED mid-JSON (prompt ~5800 + ~3000 output > 8192 per-slot
KV bound). Production serving/capture defect; gate correctly rejects.
Recommended own-sprint follow-up (raise output budget, re-capture).
2. jiminy.evaluate: explanation_quality=0.0 on correct {violations,warnings}
responses — wrong reward for the schema (no top-level explanation key).
Reward-array fix, operator-gated (changes a ULTS array + re-grades).
3. jiminy.synthesize: keyword-bag follow_rate/specificity just below gate —
the deferred Epic 1 continuation.
Also fixed 2 pre-existing lint nits in the touched file (F541, E741).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes REWARD-CORRECTNESS-001 at Epic 1+2+3. Epic 4 (baseline recompute) explicitly deferred behind the ape.reflect truncation fix per operator sequencing — recomputing over a known-truncated corpus would bake in the corruption. Next sprint: ape.reflect truncation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root-caused the ape.reflect ~87% truncation (largest training target) to a structurally-unbounded prompt: live-measured 7489 tokens (Current Assessment ~3895 + 5-cycle history ~2693), leaving only ~700 of the 8192 per-slot KV budget for output — 191/200 invalid responses cluster at 490-520 tokens_out, truncating mid-JSON at the ceiling. Compression already on; not a max_tokens cap. Plan: bound the prompt to a configurable token budget (gate verbose TSDB dataset fields, cap history cycles, final drop-oldest guard) so output always has ~4000-token headroom, with an optional serving-slot increase as the safety margin. Lever A (structural prompt budget) + Lever B (KV slot) proposed, picked at execution. Tier 3 proof: fresh ape.reflect json_valid recovers 0.13 → ~1.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ut (Epic 1-2) Implements the structural fix for the ape.reflect ~87% mid-JSON cutoff. The prompt was unbounded (live 7489 tok), leaving only ~700 of the 8192 per-slot KV budget for output, so the largest training target's responses were cut off mid-array. buildUserPrompt now enforces a token budget: - gate the verbose TSDB dataset fields (LLMPerformance x17 / Retrieval / Embedding / TrainingReadiness, ~3895 of 7489 prompt tokens) behind RSIC_LLM_REFLECT_INCLUDE_DATASETS (default false); scalar health metrics the detectors use are always kept; - cap history cycles via RSIC_LLM_REFLECT_HISTORY_CYCLES (default 3, was hardcoded 5); - final budget guard (RSIC_LLM_REFLECT_PROMPT_BUDGET_TOKENS default 3500, 0 disables): drops history oldest-first, then trims the assessment tail, logging loudly what was dropped (never silent). estimateTokens calibrated to the measured 2.3 chars/tok ratio, slightly conservative. 3 config fields (range-validated, no hardcoding) wired config -> LLMReflectorConfig -> server.go. 6 Tier-1 tests: dataset gating, history cap, drop-history-under-budget, trim-assessment-under-budget, under-budget-unchanged, estimator. Full ape suite + lint + config scanner (687/687) clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…md (Epic 5) Live Tier 3 result documented: 3/3 fresh post-restart ape.reflect rows valid JSON (100%, up from ~13%), tokens_in ~2575 (from ~7489). Corrected the stale CLAUDE.md ape.reflect prompt-size figure (~5800 -> ~7489 live-measured) and added the per-slot KV "prompt+output share the budget" guidance. Closes APE-PROMPT-BUDGET-001. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner
APE-PROMPT-BUDGET-001 — bound the ape.reflect prompt so output can't truncateThird sprint of the training-integrity remediation (after EVAL-INTEGRITY-001 #460, REWARD-CORRECTNESS-001 #461). Fixes the issue REWARD-CORRECTNESS-001's live run surfaced: ape.reflect — the largest LLM training target (54k rows) — was emitting ~87% truncated, invalid-JSON responses. Root cause (live-measured)The assembled prompt had grown to 7489 tokens (Current Assessment ~3895, dominated by verbose TSDB dataset fields; 5-cycle history ~2693). The llama-server per-slot KV bound is Fix
3 range-validated config fields (no hardcoding), 6 Tier-1 tests. Live Tier 3 (real stack — required)Restarted the server with the fix; the 3 fresh post-restart ape.reflect rows:
Lever B (serving slot) — considered, not appliedRaising the per-slot KV bound ( Carried forwardThe historical truncated rows stay in TSDB (forward-only fix; correctly Files
|
… import shift The Epic 1-2 import (log/slog) + struct fields shifted llmReflectSystemPrompt from line 74 to 80. The ULTS hash verifier reads from line-2 and grabs the first backtick string; at the stale :74 the search region now included the `"` backtick in `quoted[i] = `"` + a + `"`` → wrong hash. The system prompt TEXT is unchanged (hash 39b2bc… still matches at :80). Updated system_prompt_source :74 → :80. Local glob verify: ape.reflect 11/11 PASS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
reh3376
approved these changes
Jun 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Development branch changes from
reh3376_dev01.Commits
mdemg modelCLI + pluggable Fetcher interfaceAuto-generated PR from reh3376_dev01 push