refactor(toml): replace smol-toml with @rainbowatcher/toml-edit-js #277

WitMiao · 2025-12-31T09:42:18Z

User description

Description

This PR migrates the TOML parsing library from smol-toml to @rainbowatcher/toml-edit-js, enabling format-preserving TOML editing capabilities.

Why This Change?

The new library (toml-edit-js) wraps the Rust toml_edit crate via WASM, providing fine-grained TOML modifications while preserving:

User comments
Manual formatting
Unmanaged configuration fields

This is critical for ZCF's configuration management, where we need to update specific fields without destroying user customizations.

Key Changes

New utility module: src/utils/toml-edit.ts with format-preserving editing functions
Incremental editing: zcf-config.ts now uses batchEditToml() when file exists
Dependency swap: Replaced smol-toml with @rainbowatcher/toml-edit-js
Comprehensive tests: Added 244 lines of test coverage for new functionality

Files Modified

package.json - Dependency update
pnpm-workspace.yaml - Catalog update
pnpm-lock.yaml - Lockfile update
src/utils/toml-edit.ts - New TOML editing utility module
src/utils/zcf-config.ts - Integrated incremental editing
src/utils/code-tools/codex.ts - Updated import
src/utils/code-tools/CLAUDE.md - Updated documentation
tests/unit/utils/toml-edit.test.ts - New comprehensive tests
tests/unit/utils/zcf-config.test.ts - Updated mocks
tests/unit/utils/code-tools/codex.test.ts - Updated import
tests/unit/utils/code-tools/toml-parser-refactor.test.ts - Updated descriptions

Type of Change

Testing

All existing tests pass
New tests added for toml-edit.ts module
Typecheck passes
Linting passes

Checklist

Self-review completed
Documentation updated
No new warnings introduced
Code follows project guidelines

Additional Information

Branch: refactor/taplo-toml → main
Commits: 1
Files Changed: 11 (+583, -66)

Commit History

ba4934f refactor(toml): replace smol-toml with @rainbowatcher/toml-edit-js

🤖 Generated with /zcf-pr command

PR Type

Enhancement, Tests

Description

Replace smol-toml with @rainbowatcher/toml-edit-js for format-preserving TOML editing
Add new toml-edit.ts utility module with parsing, stringifying, and incremental editing functions
Implement incremental TOML editing in zcf-config.ts to preserve user comments and formatting
Add 244 lines of comprehensive test coverage for new TOML editing functionality

Diagram Walkthrough

flowchart LR
  A["smol-toml<br/>Basic parsing"] -->|"Replace with"| B["@rainbowatcher/toml-edit-js<br/>WASM-based toml_edit"]
  B -->|"Enables"| C["Format-preserving<br/>editing"]
  C -->|"Preserves"| D["Comments &<br/>Formatting"]
  C -->|"Preserves"| E["User<br/>customizations"]
  F["New toml-edit.ts<br/>utility module"] -->|"Provides"| G["parseToml<br/>stringifyToml<br/>editToml<br/>batchEditToml"]
  G -->|"Used by"| H["zcf-config.ts<br/>Incremental updates"]
  H -->|"Fallback to"| I["Full stringify<br/>if edit fails"]

File Walkthrough

Relevant files

Enhancement

3 files

toml-edit.ts `New format-preserving TOML editing utility module`	+180/-0
zcf-config.ts `Implement incremental TOML editing with format preservation`	+68/-6
codex.ts `Update import from smol-toml to toml-edit module`	+12/-1

Documentation

1 files

CLAUDE.md `Update documentation for new TOML parsing approach`	+18/-4

Tests

4 files

toml-edit.test.ts `Add comprehensive tests for format-preserving TOML editing`	+244/-0
zcf-config.test.ts `Update mocks and tests for new toml-edit integration`	+45/-38
codex.test.ts `Update import reference in codex test file`	+1/-1
toml-parser-refactor.test.ts `Update test descriptions for new TOML library`	+2/-2

Dependencies

2 files

package.json `Replace smol-toml with @rainbowatcher/toml-edit-js dependency`	+1/-1
pnpm-lock.yaml `Update lockfile with new dependency and remove old one`	+11/-12

Configuration changes

1 files

pnpm-workspace.yaml `Update catalog with new TOML library version`	+1/-1

Note

Addresses MCP config corruption (Issue #259) by isolating TOML mutations and preserving user formatting/comments.

Adopts @rainbowatcher/toml-edit-js and adds src/utils/toml-edit.ts for format‑preserving parse/stringify/edit APIs
Introduces codex-toml-updater.ts to update only model, model_provider, model_providers.*, and mcp_servers.* without cross‑contamination (SSE url services remain untouched)
Refactors Codex flows (codex.ts, codex-configure.ts, codex-provider-manager.ts, features.ts) to replace writeCodexConfig with targeted updates (add/edit/delete providers, batch MCP updates, provider switching)
Fixes top‑level TOML field handling in zcf-config.ts to safely update version/lastUpdated outside sections; uses incremental batchEditToml when possible
Updates docs (CLAUDE.md) for new TOML APIs and replaces dependency smol-toml → @rainbowatcher/toml-edit-js (package/workspace/lockfile)
Extensive test updates/additions validating incremental edits, MCP preservation, Windows env handling, and edge cases

^{Written by Cursor Bugbot for commit 45cdb0b. This will update automatically on new commits. Configure here.}

qodo-free-for-open-source-projects · 2025-12-31T09:42:47Z

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢	No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
⚪	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: TOML Input Validation: The `parseToml`, `editToml`, and `batchEditToml` functions accept external TOML content without explicit validation or sanitization before passing to the underlying WASM library Referred Code export function parseToml<T = Record<string, unknown>>(content: string): T { ensureTomlInitSync() return rawParse(content) as T } /** * Stringify JavaScript object to TOML string * * @param data - JavaScript object to stringify * @returns TOML string * * @example * ```typescript * const toml = stringifyToml({ section: { key: 'value' } }) * // [section] * // key = "value" * ``` */ export function stringifyToml(data: Record<string, unknown>): string { ensureTomlInitSync() return rawStringify(data) ... (clipped 33 lines) Learn more about managing compliance generic rules or creating your own custom rules
Update

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

qodo-free-for-open-source-projects · 2025-12-31T09:44:08Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category Suggestion Impact

Possible issue

✅ ~~Ensure top-level TOML fields are updated~~

Suggestion Impact:

The suggestion identified the critical issue that top-level fields (version, lastUpdated) cannot be updated incrementally using batchEditToml. The commit implements a solution to this problem by creating a new function updateTopLevelTomlFields that manually handles these fields using string operations. While the implementation differs from the suggested approach (the commit doesn't check for changes and do a full rewrite, but instead implements targeted string manipulation), it addresses the core issue of ensuring top-level fields are properly updated.

code diff:

  * Update top-level TOML fields (version, lastUpdated) in content string
  * Since editToml only supports nested paths with dots, we handle top-level
  * fields manually using string operations to preserve formatting.
  *
  * This function:
- * - Updates existing top-level fields if they exist
+ * - Updates existing top-level fields if they exist (only in top-level area)
  * - Adds missing top-level fields at the beginning of the file
  * - Preserves comments and formatting
+ * - Does NOT modify fields inside [sections]
  */
 function updateTopLevelTomlFields(content: string, version: string, lastUpdated: string): string {
-  let result = content
-
-  // Update or add version field
-  // Match version field at the start of a line (allowing leading whitespace)
-  const versionRegex = /^(\s*)version\s*=\s*["'][^"']*["']/m
-  const versionMatch = result.match(versionRegex)
+  // Find the first [section] to determine top-level boundary
+  // This ensures we only operate on true top-level fields, not section fields
+  const firstSectionMatch = content.match(/^\[/m)
+  const topLevelEnd = firstSectionMatch?.index ?? content.length
+
+  // Split content into top-level area and rest (sections)
+  let topLevel = content.slice(0, topLevelEnd)
+  const rest = content.slice(topLevelEnd)
+
+  // Update or add version field in top-level area only
+  // Match version field at the start of a line (no indentation for top-level)
+  const versionRegex = /^version\s*=\s*["'][^"']*["'][ \t]*$/m
+  const versionMatch = topLevel.match(versionRegex)
   if (versionMatch) {
-    // Update existing version, preserving indentation
-    const indent = versionMatch[1] || ''
-    result = result.replace(versionRegex, `${indent}version = "${version}"`)
+    // Update existing version
+    topLevel = topLevel.replace(versionRegex, `version = "${version}"`)
   }
   else {
-    // Add version at the beginning (before first section or at start)
-    // Find the first non-comment, non-blank line or first section
-    const firstContentMatch = result.match(/^(\s*)(?!#|\[)/m)
-    if (firstContentMatch) {
-      // Insert before first content line, preserving any leading comments
-      const insertPos = firstContentMatch.index || 0
-      result = `${result.slice(0, insertPos)}version = "${version}"\n${result.slice(insertPos)}`
-    }
-    else {
-      // File is empty or only has comments/sections, add at the very beginning
-      result = `version = "${version}"\n${result}`
-    }
-  }
-
-  // Update or add lastUpdated field
-  // Match lastUpdated field at the start of a line (allowing leading whitespace)
-  const lastUpdatedRegex = /^(\s*)lastUpdated\s*=\s*["'][^"']*["']/m
-  const lastUpdatedMatch = result.match(lastUpdatedRegex)
+    // Add version at the beginning of top-level area (after comments)
+    topLevel = insertAtTopLevelStart(topLevel, `version = "${version}"`)
+  }
+
+  // Update or add lastUpdated field in top-level area only
+  const lastUpdatedRegex = /^lastUpdated\s*=\s*["'][^"']*["'][ \t]*$/m
+  const lastUpdatedMatch = topLevel.match(lastUpdatedRegex)
   if (lastUpdatedMatch) {
-    // Update existing lastUpdated, preserving indentation
-    const indent = lastUpdatedMatch[1] || ''
-    result = result.replace(lastUpdatedRegex, `${indent}lastUpdated = "${lastUpdated}"`)
+    // Update existing lastUpdated
+    topLevel = topLevel.replace(lastUpdatedRegex, `lastUpdated = "${lastUpdated}"`)
   }
   else {
-    // Add lastUpdated after version (or at beginning if version doesn't exist)
-    const versionLineMatch = result.match(/^(\s*)version\s*=\s*["'][^"']*["']/m)
-    if (versionLineMatch) {
-      // Find the end of the version line (including newline)
-      const versionEnd = (versionLineMatch.index || 0) + versionLineMatch[0].length
-      // Check if there's already a newline after version
-      const hasNewline = result[versionEnd] === '\n'
-      const newline = hasNewline ? '' : '\n'
-      result = `${result.slice(0, versionEnd)}${newline}lastUpdated = "${lastUpdated}"\n${result.slice(versionEnd)}`
-    }
-    else {
-      // No version field, add lastUpdated at the beginning
-      const firstContentMatch = result.match(/^(\s*)(?!#|\[)/m)
-      if (firstContentMatch) {
-        const insertPos = firstContentMatch.index || 0
-        result = `${result.slice(0, insertPos)}lastUpdated = "${lastUpdated}"\n${result.slice(insertPos)}`
-      }
-      else {
-        result = `lastUpdated = "${lastUpdated}"\n${result}`
-      }
-    }
-  }
-
-  return result
+    // Add lastUpdated after version field
+    topLevel = insertAfterVersionField(topLevel, `lastUpdated = "${lastUpdated}"`)
+  }
+
+  // Ensure there's a newline between top-level fields and first section
+  if (rest.length > 0 && !topLevel.endsWith('\n\n') && !topLevel.endsWith('\n')) {
+    topLevel += '\n'
+  }
+
+  return topLevel + rest
 }

Modify writeTomlConfig to perform a full rewrite if top-level fields like
version or lastUpdated change, preventing stale configuration data.

src/utils/zcf-config.ts [78-146]

 function writeTomlConfig(configPath: string, config: ZcfTomlConfig): void {
   try {
     // Ensure parent directory exists
     const configDir = dirname(configPath)
     ensureDir(configDir)
 
     // Check if file exists for incremental editing
     if (exists(configPath)) {
       const existingContent = readFile(configPath)
+      const existingConfig = readTomlConfig(configPath);
 
-      // Build edits for section fields only (editToml requires nested paths with dots)
-      // Top-level fields like 'version' and 'lastUpdated' cannot be edited incrementally
+      // If top-level fields have changed, a full rewrite is necessary
+      // because batchEditToml only supports nested fields.
+      if (!existingConfig || existingConfig.version !== config.version || existingConfig.lastUpdated !== config.lastUpdated) {
+        const tomlContent = stringifyToml(config as unknown as Record<string, unknown>);
+        writeFile(configPath, tomlContent);
+        return;
+      }
+
+      // Build edits for section fields only
       const edits: Array<[string, unknown]> = [
         // General section
         ['general.preferredLang', config.general.preferredLang],
         ['general.currentTool', config.general.currentTool],
       ]
 
       // Optional general fields
       if (config.general.templateLang !== undefined) {
         edits.push(['general.templateLang', config.general.templateLang])
       }
       if (config.general.aiOutputLang !== undefined) {
         edits.push(['general.aiOutputLang', config.general.aiOutputLang])
       }
 
       // Claude Code section
       edits.push(
         ['claudeCode.enabled', config.claudeCode.enabled],
         ['claudeCode.outputStyles', config.claudeCode.outputStyles],
         ['claudeCode.defaultOutputStyle', config.claudeCode.defaultOutputStyle],
         ['claudeCode.installType', config.claudeCode.installType],
         ['claudeCode.currentProfile', config.claudeCode.currentProfile],
         ['claudeCode.profiles', config.claudeCode.profiles],
       )
 
       // Optional Claude Code fields
       if (config.claudeCode.version !== undefined) {
         edits.push(['claudeCode.version', config.claudeCode.version])
       }
 
       // Codex section
       edits.push(
         ['codex.enabled', config.codex.enabled],
         ['codex.systemPromptStyle', config.codex.systemPromptStyle],
       )
 
       try {
         // Apply incremental edits preserving user customizations
         const updatedContent = batchEditToml(existingContent, edits)
         writeFile(configPath, updatedContent)
       }
       catch {
         // Fall back to full stringify if incremental editing fails
         const tomlContent = stringifyToml(config as unknown as Record<string, unknown>)
         writeFile(configPath, tomlContent)
       }
     }
     else {
       // Create new file with full configuration
       const tomlContent = stringifyToml(config as unknown as Record<string, unknown>)
       writeFile(configPath, tomlContent)
     }
   }
   catch {
     // Silently fail if cannot write config - user's system may have permission issues
     // The app should still work without saved preferences
   }
 }

[Suggestion processed]

Suggestion importance[1-10]: 9

__

Why: This suggestion correctly identifies a critical bug where top-level fields like version are not updated, leading to stale configuration, and provides a correct fix.

High

Update

cursor

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

src/utils/zcf-config.ts

codecov · 2025-12-31T09:54:01Z

Codecov Report

❌ Patch coverage is 88.68421% with 43 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/utils/code-tools/codex-toml-updater.ts	73.20%	41 Missing ⚠️
src/utils/zcf-config.ts	98.11%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

- add new toml-edit utility module with format-preserving editing - implement incremental TOML editing in zcf-config to preserve user customizations - replace smol-toml imports across the codebase - add comprehensive tests for toml-edit functionality - update documentation for new TOML parsing approach

src/utils/zcf-config.ts

- add section boundary detection to prevent modifying fields inside sections - fix newline handling when inserting lastUpdated after version field - add insertAtTopLevelStart helper for proper comment preservation - add insertAfterVersionField helper for correct field positioning - add comprehensive test cases for edge scenarios (PR #277 bugs) Refs: #277

src/utils/zcf-config.ts

…comments - Update regex patterns in insertAfterVersionField and updateTopLevelTomlFields functions to handle inline comments for version and lastUpdated fields. - Add tests to ensure correct handling of TOML fields with inline comments and trailing spaces. - Improve overall robustness of TOML configuration handling. Refs: #277

src/utils/zcf-config.ts

…on corruption - Introduced a new utility module `codex-toml-updater` for precise modifications to TOML files, ensuring that changes to API configurations do not affect MCP configurations. - Updated multiple functions across the codebase to utilize targeted updates, preserving existing MCP services and other configurations. - Added a new markdown file documenting the issue and solution for reference. - Enhanced tests to validate the integrity of configurations after updates. Refs: #259

- Removed the writeCodexConfig function to streamline configuration handling. - Updated tests to replace calls to writeCodexConfig with batchUpdateCodexMcpServices, ensuring proper functionality and integration. - Enhanced mocking in tests to reflect the removal of writeCodexConfig, maintaining test coverage for configuration operations. Refs: #259

…cf-config - Ensure content is always inserted on a new line after the version field in the insertAfterVersionField function. - Update the writeTomlConfig function to conditionally include optional Claude Code fields only if they are defined, preventing unnecessary edits and potential issues with batch editing. Refs: #259

- add section boundary detection to prevent modifying fields inside sections - fix newline handling when inserting lastUpdated after version field - add insertAtTopLevelStart helper for proper comment preservation - add insertAfterVersionField helper for correct field positioning - add comprehensive test cases for edge scenarios (PR #277 bugs) Refs: #277

…comments - Update regex patterns in insertAfterVersionField and updateTopLevelTomlFields functions to handle inline comments for version and lastUpdated fields. - Add tests to ensure correct handling of TOML fields with inline comments and trailing spaces. - Improve overall robustness of TOML configuration handling. Refs: #277

qodo-free-for-open-source-projects bot added the Review effort 3/5 label Dec 31, 2025

WitMiao linked an issue Dec 31, 2025 that may be closed by this pull request

配置codex api后，mcp配置被修改 #259

Closed

cursor bot reviewed Dec 31, 2025

View reviewed changes

src/utils/zcf-config.ts Show resolved Hide resolved

WitMiao force-pushed the refactor/taplo-toml branch from ba4934f to 8dca0d0 Compare January 7, 2026 02:49

cursor bot reviewed Jan 7, 2026

View reviewed changes

src/utils/zcf-config.ts Outdated Show resolved Hide resolved

src/utils/zcf-config.ts Outdated Show resolved Hide resolved

src/utils/zcf-config.ts Outdated Show resolved Hide resolved

WitMiao added 2 commits January 8, 2026 23:56

docs: move legacy plan markdown files to .zcf/plan/history

b46b336

cursor bot reviewed Jan 8, 2026

View reviewed changes

src/utils/zcf-config.ts Outdated Show resolved Hide resolved

cursor bot reviewed Jan 9, 2026

View reviewed changes

src/utils/zcf-config.ts Outdated Show resolved Hide resolved

src/utils/zcf-config.ts Outdated Show resolved Hide resolved

WitMiao added 3 commits January 11, 2026 22:22

WitMiao merged commit 60b4453 into main Jan 12, 2026
5 of 6 checks passed

WitMiao deleted the refactor/taplo-toml branch January 12, 2026 04:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(toml): replace smol-toml with @rainbowatcher/toml-edit-js #277

refactor(toml): replace smol-toml with @rainbowatcher/toml-edit-js #277

Uh oh!

WitMiao commented Dec 31, 2025 •

edited by cursor bot

Loading

Uh oh!

qodo-free-for-open-source-projects bot commented Dec 31, 2025 •

edited

Loading

Uh oh!

qodo-free-for-open-source-projects bot commented Dec 31, 2025 •

edited

Loading

Uh oh!

cursor bot left a comment

Uh oh!

Uh oh!

codecov bot commented Dec 31, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

refactor(toml): replace smol-toml with @rainbowatcher/toml-edit-js #277

refactor(toml): replace smol-toml with @rainbowatcher/toml-edit-js #277

Uh oh!

Conversation

WitMiao commented Dec 31, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User description

Description

Why This Change?

Key Changes

Files Modified

Type of Change

Testing

Checklist

Additional Information

Commit History

PR Type

Description

Diagram Walkthrough

File Walkthrough

Uh oh!

qodo-free-for-open-source-projects bot commented Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Compliance Guide 🔍

Uh oh!

qodo-free-for-open-source-projects bot commented Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Code Suggestions ✨

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

This PR is being reviewed by Cursor Bugbot

Uh oh!

Uh oh!

codecov bot commented Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

WitMiao commented Dec 31, 2025 •

edited by cursor bot

Loading

qodo-free-for-open-source-projects bot commented Dec 31, 2025 •

edited

Loading

qodo-free-for-open-source-projects bot commented Dec 31, 2025 •

edited

Loading

codecov bot commented Dec 31, 2025 •

edited

Loading