Release v0.8.0 — verify-integrity + remove deprecated --data-audit / evaluation.staging_ttl_days#65
Conversation
…udit / evaluation.staging_ttl_days MINOR release (`__version__` 0.7.0 → 0.8.0, `__api_version__` 1.0.0 → 1.1.0). Added (feature + public API): - `forgelm verify-integrity MODEL_DIR` subcommand + `forgelm.verify_integrity()` / `VerifyIntegrityResult` stable library API (Article 15 model-integrity manifest verification). Added to `forgelm.__all__` → MINOR `__api_version__` bump (snapshot regenerated as tests/_data/api_signatures_1.1.0.json). Removed (deprecated in v0.7.0, removal due this minor per the one-minor cadence): - `evaluation.staging_ttl_days` config field (alias). `retention.staging_ttl_days` is now the only form; `EvaluationConfig` is `extra="forbid"` so the legacy key raises a validation error. Removed the reconciliation/alias-forward machinery. - `forgelm --data-audit PATH` CLI flag. Use the `forgelm audit PATH` subcommand. Removed the legacy dispatch path + the now-unemitted `cli.legacy_flag_invoked` audit event (dropped from the catalog). Release mechanics: - CHANGELOG [Unreleased] → [0.8.0] (Added/Changed/Deprecated/Removed/Fixed). - pyproject `version = "0.8.0"`; site version literals re-derived (update_site_version). - Notebook pins bumped 0.7.0 → 0.8.0. - Deprecation-behaviour tests replaced with removal-assertion tests; docs (EN+TR) updated to past-tense removal; historical roadmap/design records untouched. Verified: ruff, full pytest (2856 passed), 12 doc/CI guards, dry-run, fresh-install `forgelm --version` = 0.8.0 — all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @cemililik, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughThis PR performs the ForgeLM v0.8.0 release: it hard-removes the deprecated ChangesForgeLM v0.8.0 Release
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request prepares the codebase for the v0.8.0 release by removing the deprecated forgelm --data-audit CLI flag and the evaluation.staging_ttl_days configuration field, while introducing a new model-integrity verification subcommand and API. Feedback on the changes suggests cleaning up an outdated comment regarding --data-audit in forgelm/cli/_dispatch.py and updating a tag reference from v0.7.0 to v0.8.0 in a notebook comment to keep it in sync with the code.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| if getattr(args, "data_audit", None): | ||
| _dispatch_legacy_data_audit(args) | ||
|
|
||
| # (The legacy ``forgelm --data-audit PATH`` flag was removed in v0.8.0; |
There was a problem hiding this comment.
The comment on line 216 (# --data-audit operates on a JSONL file/directory only — no config needed.) is now outdated and misleading because the --data-audit flag has been completely removed. Since line 216 is a context line, it cannot be directly modified here, but it should be cleaned up or removed in a separate commit to keep the documentation accurate.
| "# matches the wheel above — `main` drifts as features land and would otherwise\n", | ||
| "# reference fields the installed Pydantic schema rejects.\n", | ||
| "!wget -q https://raw.githubusercontent.com/HodeTech/ForgeLM/v0.7.0/config_template.yaml\n", | ||
| "!wget -q https://raw.githubusercontent.com/HodeTech/ForgeLM/v0.8.0/config_template.yaml\n", |
There was a problem hiding this comment.
The comment on line 40 still references the v0.7.0 tag, but the command on line 43 has been updated to download from the v0.8.0 tag. Please update the comment on line 40 to reference v0.8.0 as well to keep them in sync.
…udit comment (PR#65 review) Two PR #65 review findings: - notebooks/*.ipynb: the 'Pinned to v0.7.0' / 'from the v0.7.0 tag' prose comments still named v0.7.0 while the actual pins/URLs were already bumped to v0.8.0 — synced the comments (11 notebooks). - forgelm/cli/_dispatch.py: removed the stale '# --data-audit operates on a JSONL file/directory only — no config needed.' leftover comment (the flag was removed in v0.8.0; the adjacent removal note stays). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
CHANGELOG.md (1)
2151-2151:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winKeep the release boundary and public contract in sync.
The release notes and the features page still reference the pre-v0.8.0 surface in two places. Please update both so readers don’t pick up the wrong diff or the wrong exit-code contract.
♻️ Proposed fixes
-[Unreleased]: https://github.com/HodeTech/ForgeLM/compare/v0.7.0...HEAD +[Unreleased]: https://github.com/HodeTech/ForgeLM/compare/v0.8.0...HEAD- <p data-i18n-html="features.eval.exit.body">0 / 1 / 2 / 3 / 4 — a public contract for CI/CD.</p> + <p data-i18n-html="features.eval.exit.body">0 / 1 / 2 / 3 / 4 / 5 — a public contract for CI/CD.</p>🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@CHANGELOG.md` at line 2151, Update the version references in both CHANGELOG.md and site/features.html to keep the release boundary consistent across both documents. In CHANGELOG.md at line 2151, update the Unreleased comparison URL to reference v0.8.0 instead of v0.7.0, changing the comparison endpoint from v0.7.0...HEAD to v0.8.0...HEAD. In site/features.html at line 239, update any pre-v0.8.0 surface references to reflect the post-v0.8.0 API contract and exit-code behavior so that both files document the same release boundary and public contract.notebooks/data_curation.ipynb (1)
404-412: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winMissing Python version pinning in two notebook metadata blocks — codifies inconsistency with release standards.
Both notebooks now have their
forgelmpackage versions pinned to v0.8.0, but they lack Python version specifications in theirlanguage_infometadata sections. The coding guideline explicitly requires all notebooks to pin both kernel and package versions for reproducibility. Three other notebooks in this release (dpo_alignment, galore_memory_optimization, grpo_reasoning) correctly specify"version": "3.11.0". Since we're updating these notebooks as part of the v0.8.0 release, both should be brought into compliance to ensure reproducible environments across all training notebooks.
notebooks/data_curation.ipynb#L404-L412: Add"version": "3.11.0"tolanguage_infonotebooks/ingestion_playground.ipynb#L415-L423: Add"version": "3.11.0"tolanguage_info🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@notebooks/data_curation.ipynb` around lines 404 - 412, Both notebooks are missing Python version specifications in their language_info metadata blocks. In notebooks/data_curation.ipynb at lines 404-412, add a new line with "version": "3.11.0" inside the language_info metadata object. Apply the identical change to notebooks/ingestion_playground.ipynb at lines 415-423 by adding "version": "3.11.0" to its language_info metadata block. Ensure both additions follow the same formatting pattern as the existing metadata properties to maintain consistency across the notebook files.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@notebooks/kto_binary_feedback.ipynb`:
- Around line 23-24: The pip install command on line 24 installs bitsandbytes
without pinning it to a specific version, which violates the reproducibility
requirement for notebooks. Add a version constraint to the bitsandbytes package
in the same pip install command (for example, using ==X.Y.Z syntax) to match the
exact version pinning already applied to forgelm[qlora]==0.8.0. This ensures the
notebook will produce consistent results across different runs and environments.
In `@notebooks/multi_dataset.ipynb`:
- Around line 23-24: The pip install command on line 24 leaves the bitsandbytes
package unversioned, violating the reproducibility requirement for notebooks.
Add an explicit version constraint to bitsandbytes in the pip install command
(following the same pattern as forgelm[qlora]==0.8.0) to ensure the notebook
environment is reproducible across different execution runs.
In `@notebooks/post_training_workflow.ipynb`:
- Line 39: The pip install command on line 39 pins the forgelm package version
but leaves bitsandbytes unpinned, violating reproducibility requirements. Modify
the pip install statement to add an exact version pin for bitsandbytes following
the same pattern as forgelm (e.g., bitsandbytes==X.X.X where X.X.X is an
appropriate stable version), ensuring all packages in the notebook have explicit
version pins.
In `@notebooks/safety_evaluation.ipynb`:
- Around line 25-26: The bitsandbytes package in the pip install command is
unpinned while forgelm[qlora] is pinned to version 0.8.0, violating the
reproducibility requirement for notebooks. Pin bitsandbytes to a specific
version by adding a version constraint in the same format as
forgelm[qlora]==0.8.0 (e.g., bitsandbytes==X.X.X) in the pip install statement
to ensure consistent environment setup across notebook runs.
In `@notebooks/synthetic_data_training.ipynb`:
- Around line 28-29: The pip install command on line 29 leaves the bitsandbytes
package unpinned while forgelm[qlora] is pinned to version 0.8.0. Add an exact
version pin to bitsandbytes (e.g., bitsandbytes==X.Y.Z) in the same pip install
statement to comply with the notebook reproducibility requirement that all
packages must have pinned versions. Check the appropriate bitsandbytes version
that is compatible with forgelm 0.8.0 and add it to the install command
alongside the already-pinned forgelm package.
---
Outside diff comments:
In `@CHANGELOG.md`:
- Line 2151: Update the version references in both CHANGELOG.md and
site/features.html to keep the release boundary consistent across both
documents. In CHANGELOG.md at line 2151, update the Unreleased comparison URL to
reference v0.8.0 instead of v0.7.0, changing the comparison endpoint from
v0.7.0...HEAD to v0.8.0...HEAD. In site/features.html at line 239, update any
pre-v0.8.0 surface references to reflect the post-v0.8.0 API contract and
exit-code behavior so that both files document the same release boundary and
public contract.
In `@notebooks/data_curation.ipynb`:
- Around line 404-412: Both notebooks are missing Python version specifications
in their language_info metadata blocks. In notebooks/data_curation.ipynb at
lines 404-412, add a new line with "version": "3.11.0" inside the language_info
metadata object. Apply the identical change to
notebooks/ingestion_playground.ipynb at lines 415-423 by adding "version":
"3.11.0" to its language_info metadata block. Ensure both additions follow the
same formatting pattern as the existing metadata properties to maintain
consistency across the notebook files.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b319a590-6990-480b-9345-15050f850f45
📒 Files selected for processing (49)
CHANGELOG.mdconfig_template.yamldocs/guides/data_audit-tr.mddocs/guides/data_audit.mddocs/guides/gdpr_erasure-tr.mddocs/guides/gdpr_erasure.mddocs/qms/sop_data_management-tr.mddocs/qms/sop_data_management.mddocs/reference/audit_event_catalog-tr.mddocs/reference/audit_event_catalog.mddocs/reference/configuration-tr.mddocs/reference/configuration.mddocs/reference/soc2_trust_criteria_mapping-tr.mddocs/reference/soc2_trust_criteria_mapping.mddocs/reference/usage-tr.mddocs/reference/usage.mddocs/usermanuals/en/reference/cli.mddocs/usermanuals/tr/reference/cli.mdforgelm/_version.pyforgelm/cli/_dispatch.pyforgelm/cli/_parser.pyforgelm/cli/subcommands/_audit.pyforgelm/compliance.pyforgelm/config.pynotebooks/data_curation.ipynbnotebooks/dpo_alignment.ipynbnotebooks/galore_memory_optimization.ipynbnotebooks/grpo_reasoning.ipynbnotebooks/ingestion_playground.ipynbnotebooks/kto_binary_feedback.ipynbnotebooks/multi_dataset.ipynbnotebooks/post_training_workflow.ipynbnotebooks/quickstart_sft.ipynbnotebooks/safety_evaluation.ipynbnotebooks/synthetic_data_training.ipynbpyproject.tomlsite/compliance.htmlsite/contact.htmlsite/features.htmlsite/guide.htmlsite/index.htmlsite/js/translations.jssite/privacy.htmlsite/quickstart.htmlsite/terms.htmltests/_data/api_signatures_1.1.0.jsontests/test_cli_subcommands.pytests/test_config.pytests/test_gdpr_erasure.py
💤 Files with no reviewable changes (7)
- docs/reference/audit_event_catalog.md
- docs/reference/usage-tr.md
- docs/reference/audit_event_catalog-tr.md
- docs/guides/data_audit-tr.md
- docs/reference/usage.md
- docs/guides/data_audit.md
- tests/test_gdpr_erasure.py
| "# Pinned to v0.8.0; bump on each release\n", | ||
| "!pip install -q --no-cache-dir 'forgelm[qlora]==0.8.0' bitsandbytes\n", |
There was a problem hiding this comment.
Pin bitsandbytes to an exact version for reproducible notebook runs.
Line 24 still installs bitsandbytes without a version, so resolves can drift over time and break reproducibility.
Suggested patch
-!pip install -q --no-cache-dir 'forgelm[qlora]==0.8.0' bitsandbytes
+!pip install -q --no-cache-dir 'forgelm[qlora]==0.8.0' 'bitsandbytes==0.43.3'As per coding guidelines, "**/*.ipynb: Every *.ipynb notebook must pin kernel and package versions for reproducibility".
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "# Pinned to v0.8.0; bump on each release\n", | |
| "!pip install -q --no-cache-dir 'forgelm[qlora]==0.8.0' bitsandbytes\n", | |
| "# Pinned to v0.8.0; bump on each release\n", | |
| "!pip install -q --no-cache-dir 'forgelm[qlora]==0.8.0' 'bitsandbytes==0.43.3'\n", |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@notebooks/kto_binary_feedback.ipynb` around lines 23 - 24, The pip install
command on line 24 installs bitsandbytes without pinning it to a specific
version, which violates the reproducibility requirement for notebooks. Add a
version constraint to the bitsandbytes package in the same pip install command
(for example, using ==X.Y.Z syntax) to match the exact version pinning already
applied to forgelm[qlora]==0.8.0. This ensures the notebook will produce
consistent results across different runs and environments.
Source: Coding guidelines
| "# Pinned to v0.8.0; bump on each release\n", | ||
| "!pip install -q --no-cache-dir 'forgelm[qlora]==0.8.0' bitsandbytes\n", |
There was a problem hiding this comment.
Pin bitsandbytes explicitly in the install command.
Line 24 leaves bitsandbytes unversioned, which makes notebook environments non-reproducible across runs.
As per coding guidelines, "**/*.ipynb: Every *.ipynb notebook must pin kernel and package versions for reproducibility".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@notebooks/multi_dataset.ipynb` around lines 23 - 24, The pip install command
on line 24 leaves the bitsandbytes package unversioned, violating the
reproducibility requirement for notebooks. Add an explicit version constraint to
bitsandbytes in the pip install command (following the same pattern as
forgelm[qlora]==0.8.0) to ensure the notebook environment is reproducible across
different execution runs.
Source: Coding guidelines
| "# Install ForgeLM (skip on Colab if quickstart_sft already ran in this session)\n", | ||
| "!pip install -q --no-cache-dir 'forgelm==0.7.0' bitsandbytes\n", | ||
| "# Download config_template.yaml from the v0.7.0 tag (NOT main) so the YAML\n", | ||
| "!pip install -q --no-cache-dir 'forgelm==0.8.0' bitsandbytes\n", |
There was a problem hiding this comment.
Avoid unpinned bitsandbytes in notebook bootstrap.
Line 39 installs bitsandbytes without an exact version; this conflicts with reproducibility requirements for notebooks.
As per coding guidelines, "**/*.ipynb: Every *.ipynb notebook must pin kernel and package versions for reproducibility".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@notebooks/post_training_workflow.ipynb` at line 39, The pip install command
on line 39 pins the forgelm package version but leaves bitsandbytes unpinned,
violating reproducibility requirements. Modify the pip install statement to add
an exact version pin for bitsandbytes following the same pattern as forgelm
(e.g., bitsandbytes==X.X.X where X.X.X is an appropriate stable version),
ensuring all packages in the notebook have explicit version pins.
Source: Coding guidelines
| "# Pinned to v0.8.0; bump on each release\n", | ||
| "!pip install -q --no-cache-dir 'forgelm[qlora]==0.8.0' bitsandbytes\n", |
There was a problem hiding this comment.
Pin bitsandbytes to preserve reproducible notebook setup.
Line 26 keeps bitsandbytes unpinned; environment resolution can change between runs.
As per coding guidelines, "**/*.ipynb: Every *.ipynb notebook must pin kernel and package versions for reproducibility".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@notebooks/safety_evaluation.ipynb` around lines 25 - 26, The bitsandbytes
package in the pip install command is unpinned while forgelm[qlora] is pinned to
version 0.8.0, violating the reproducibility requirement for notebooks. Pin
bitsandbytes to a specific version by adding a version constraint in the same
format as forgelm[qlora]==0.8.0 (e.g., bitsandbytes==X.X.X) in the pip install
statement to ensure consistent environment setup across notebook runs.
Source: Coding guidelines
| "# Pinned to v0.8.0; bump on each release\n", | ||
| "!pip install -q --no-cache-dir 'forgelm[qlora]==0.8.0' bitsandbytes\n", |
There was a problem hiding this comment.
Pin bitsandbytes with an exact version in this changed install cell.
Line 29 installs an unpinned package, which violates notebook reproducibility requirements.
As per coding guidelines, "**/*.ipynb: Every *.ipynb notebook must pin kernel and package versions for reproducibility".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@notebooks/synthetic_data_training.ipynb` around lines 28 - 29, The pip
install command on line 29 leaves the bitsandbytes package unpinned while
forgelm[qlora] is pinned to version 0.8.0. Add an exact version pin to
bitsandbytes (e.g., bitsandbytes==X.Y.Z) in the same pip install statement to
comply with the notebook reproducibility requirement that all packages must have
pinned versions. Check the appropriate bitsandbytes version that is compatible
with forgelm 0.8.0 and add it to the install command alongside the
already-pinned forgelm package.
Source: Coding guidelines
…etadata (PR#65 review) Three still-valid PR #65 review findings (bitsandbytes-pin finding skipped — the project convention + check_notebook_pins's own accepted-pin example install 'forgelm[qlora]==X bitsandbytes' with bitsandbytes pulled in pinned via the qlora extra): - CHANGELOG.md: bump the [Unreleased] compare link to v0.8.0...HEAD and add the [0.8.0]: v0.7.0...v0.8.0 link. - site exit-code claim: '0 / 1 / 2 / 3 / 4' was missing code 5 (wizard-cancelled) — corrected to '0 / 1 / 2 / 3 / 4 / 5' across features.html + all 6 translations.js locales, matching forgelm/cli/_exit_codes.py. - notebooks/{data_curation,ingestion_playground}.ipynb: added the missing language_info.version (3.11.0) to match the other tutorial notebooks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 participant
Release v0.8.0 (MINOR)
__version__0.7.0 → 0.8.0 ·__api_version__1.0.0 → 1.1.0Why MINOR (not patch)
A new CLI subcommand (
verify-integrity) and new stable public symbols (verify_integrity,VerifyIntegrityResult) shipped since v0.7.0 — perdocs/standards/release.mdthat maps to a MINOR +__api_version__bump, not a patch.Added
forgelm verify-integrity MODEL_DIR+forgelm.verify_integrity()/VerifyIntegrityResult— re-hashes a model dir against its Article 15model_integrity.jsonmanifest (exit 0/1/2,--output-format json). Added toforgelm.__all__→ snapshot regenerated (tests/_data/api_signatures_1.1.0.json).merge.ties_trim_fraction/dare_drop_rate/dare_seed) +synthetic.sanity_failure_rate(already in[Unreleased]).Removed (deprecated in v0.7.0 → removed this minor, per the one-minor cadence)
evaluation.staging_ttl_daysfield — use canonicalretention.staging_ttl_days; legacy key now raises a validation error (extra="forbid"). Reconciliation/alias machinery deleted.forgelm --data-audit PATHflag — use theforgelm audit PATHsubcommand. Legacy dispatch + the now-unemittedcli.legacy_flag_invokedaudit event removed (dropped from catalog).Release mechanics
[Unreleased]→[0.8.0] — 2026-06-16; site version literals re-derived; notebook pins 0.7.0 → 0.8.0.Verification
Full pytest 2856 passed; ruff; 12 doc/CI guards (bilingual, anchor, cli-help, yaml-snippets, audit-catalog, tr-links, usermanual-self-contained, numerical-claims, notebook-pins, wizard-defaults, site-version, no-analysis);
forgelm --config config_template.yaml --dry-run; fresh-installforgelm --version= 0.8.0 — all green.🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes: v0.8.0
New Features
Changed
Removed
forgelm --data-auditflag; useforgelm auditinstead.evaluation.staging_ttl_days; useretention.staging_ttl_days.API Updates
1.1.0.Documentation