feat(scanner): add scanner.opencodeDbPaths to settings.json#409
Merged
feat(scanner): add scanner.opencodeDbPaths to settings.json#409
Conversation
Adds a persistent, declarative counterpart to TOKSCALE_EXTRA_DIRS for the
OpenCode scanner. Users who launch opencode with `OPENCODE_DB` pointing at
a path outside `~/.local/share/opencode` (the upstream env var supports
absolute paths) can now point tokscale at those dbs in
`~/.config/tokscale/settings.json`:
{
"scanner": {
"opencodeDbPaths": [
"/custom/location/opencode.db",
"/another/location/opencode-stable.db"
]
}
}
Implementation shape:
- `tokscale_core::scanner::ScannerSettings` — new public, Default,
serde-camelCase struct. Lives in core so any non-CLI consumer of the
scanner sees the same config surface.
- `scan_all_clients_with_scanner_settings` — new scanner entry point that
runs auto-discovery then merges user-configured paths into
`ScanResult.opencode_dbs` via `merge_user_opencode_db_paths`, which
canonicalizes for dedup, silently skips non-existent paths (stale
config must never break a scan), and rejects WAL/SHM sidecars through
the same `is_opencode_db_filename` check used in #406.
- `ScannerSettings` threaded through `ReportOptions` and
`LocalParseOptions` (both now derive `Default`), and through the
private `parse_all_messages_with_pricing_with_env_strategy` helper.
- CLI `Settings` struct gets a `scanner: ScannerSettings` field with
`#[serde(default)]` so older settings.json files without the key still
load cleanly. `load_scanner_settings()` is the single helper every
entry point calls to populate the options structs.
Why add it here instead of in a brand-new config file: tokscale already
has `~/.config/tokscale/settings.json` for TUI prefs, so `scanner` is
just another top-level section. Avoids introducing a second config file
to manage and keeps the CLI-side loader to a one-line helper.
Constraint: must reuse the same filename validation as auto-discovery so
sidecars (.db-wal, .db-shm, .db-journal) can never be passed in by
accident — they would immediately fail on rusqlite::Connection::open.
Constraint: unknown/malformed/missing user paths must degrade gracefully,
never erroring — the config outlives any single opencode install.
Rejected: separate `scanner.toml` | two config files is worse for
discoverability than one `settings.json` with sections.
Rejected: env var `TOKSCALE_OPENCODE_DB_PATHS` | env vars are hard to
discover and don't survive shell sessions; users asked explicitly for
a persistent config knob.
Rejected: TUI-only setting | the scanner runs from CLI-headless paths
too (headless, server, wrapped), and threading a TUI struct into
tokscale-core would invert the dependency direction.
Confidence: high
Scope-risk: moderate (touches every entry point that builds
ReportOptions/LocalParseOptions, but the change at each site is a
one-line field addition)
Directive: when adding new top-level scanner knobs, add them to
`ScannerSettings` in core — do NOT put scanner config in the CLI
`Settings` struct directly, or headless callers will lose access.
Not-tested: concurrent scanner_settings updates during a running scan
(not a supported scenario — settings are snapshotted at call time)
… hermeticize data loader tests parse_local_clients was the one reporting path that still called scan_all_clients_with_env_strategy, which silently discarded options.scanner_settings. Every other path (parse_all_messages_with_pricing_with_env_strategy, get_model_report, get_monthly_report, generate_graph_with_loaded_pricing, parse_local_unified_messages_resolved) already passes scanner_settings, so 'tokscale clients' and the wrapped agents view were the only code paths that never honored scanner.opencodeDbPaths from settings.json. Route parse_local_clients through scan_all_clients_with_scanner_settings to match, then pin the behavior with a regression test that builds an external opencode.db and asserts it only shows up when the caller passes it via ScannerSettings.opencode_db_paths. tui/data/mod.rs::DataLoader::load also reads ~/.config/tokscale/settings.json via load_scanner_settings(), and the same file's unit tests go through DataLoader::load(), so any developer with scanner.opencodeDbPaths configured sees machine-dependent test results. Replace the three call sites with a small cfg(test)-gated helper that returns ScannerSettings::default() under cfg(test) and delegates to load_scanner_settings() in production, plus a test that asserts the helper stays hermetic.
…is enabled scan_all_clients_with_scanner_settings used to call merge_user_opencode_db_paths unconditionally after the inner scan, which bypassed the existing enabled.contains(&ClientId::OpenCode) guard. A request like 'tokscale --claude' would still pull in user-pinned OpenCode dbs from scanner.opencodeDbPaths, parse the SQLite rows, and inflate parse_local_clients counts even though message-level filters later dropped the rows from the returned message vector. Beyond the counts inconsistency, this also wasted SQLite parsing work on every non-opencode scan for users with the new config. Move the merge into scan_all_clients_with_env_strategy_inner, right inside the existing 'if enabled.contains(&ClientId::OpenCode)' block, so the merge is naturally scoped to opencode-enabled scans without duplicating the (non-obvious) 'synthetic enables all' enable logic. scan_all_clients_with_scanner_settings becomes a thin pass-through wrapper that forwards scanner_settings to the inner function. Add two regression tests: - scanner: test_scan_all_clients_with_scanner_settings_respects_opencode_client_filter exercises the four canonical filter shapes (claude, opencode, synthetic, empty) against a temp home with both an auto-discoverable opencode.db and a user-configured external db, asserting opencode_dbs is empty under --claude and populated for the other three. - lib (end-to-end): test_parse_local_clients_claude_filter_ignores_scanner_settings_opencode_db_paths builds a real Claude JSONL session plus an external opencode SQLite pinned via ScannerSettings.opencode_db_paths and asserts a --clients claude run yields counts.OpenCode == 0, counts.Claude == 1, and zero opencode rows in the returned messages.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
4 tasks
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #406.
Follow-up to the discussion under #387: adds a persistent, declarative counterpart to the
TOKSCALE_EXTRA_DIRSenv var so users who launch opencode withOPENCODE_DBpointing at a path outside~/.local/share/opencodecan pin those dbs in~/.config/tokscale/settings.json.{ "scanner": { "opencodeDbPaths": [ "/custom/location/opencode.db", "/another/location/opencode-stable.db" ] } }Why this shape
Tokscale already has
~/.config/tokscale/settings.jsonfor TUI preferences, soscanneris just another top-level section — no second config file to manage. The struct lives intokscale-core(not just the CLI) so headless consumers of the scanner get the same config surface.Changes
tokscale-core::scannerScannerSettingsstruct withopencode_db_paths: Vec<PathBuf>, serde camelCase,#[serde(default)]at both struct and field level so older/partial settings.json files deserialize cleanly.scan_all_clients_with_scanner_settings(home_dir, clients, use_env_roots, settings)entry point. Runs auto-discovery then merges user-configured paths viamerge_user_opencode_db_paths, which:is_opencode_db_filenamecheck introduced in fix(opencode): track all channel-suffixed dbs #406.scan_all_clients_with_env_strategyandscan_all_clientsbecome thin wrappers that passScannerSettings::default(), preserving backward compatibility.tokscale-core::libReportOptionsandLocalParseOptionsboth grow ascanner_settings: scanner::ScannerSettingsfield and deriveDefault.parse_all_messages_with_pricing_with_env_strategytakes&ScannerSettingsand passes it to the new scanner entry point. All four internal call sites (model report, monthly report, graph, local-graph) thread it through from their enclosing options.tokscale-cli::tui::settingsSettingsstruct gets ascanner: ScannerSettingsfield with#[serde(default)].load_scanner_settings()helper — the single function every CLI entry point calls to populateLocalParseOptions/ReportOptions.CLI call sites wired (one-line field addition each):
commands/wrapped.rs(× 2)main.rsrun_models_report / run_monthly_report / run_clients_command / run_graph_command / run_submit_commandtui/data/mod.rs(× 3)Test plan
cargo test --workspace— 886 tests pass (322 + 81 + 480 + 3 + 0 doctest), 2 ignored (unrelated, pre-existing)cargo clippy -p tokscale-core --lib --tests -- -D warnings— cleantokscale-coretests (inscanner::tests):test_merge_user_opencode_db_paths_picks_up_path_outside_xdg— the flagship OPENCODE_DB=/abs/path use casetest_merge_user_opencode_db_paths_skips_nonexistent_and_sidecars— graceful degradation for stale config + sidecar rejectiontest_merge_user_opencode_db_paths_dedups_against_auto_discovered— canonicalized deduptest_scanner_settings_deserialize_from_json_camel_case—opencodeDbPathsfield contract + empty-object formtest_scan_all_clients_with_scanner_settings_merges_user_path— end-to-end throughscan_all_clients_with_scanner_settingswith both an auto-discovered db and a user-configured outside-XDG dbtokscale-clitests (intui::settings::tests):settings_load_backfills_scanner_when_missing_from_json— older settings.json without the key still loadssettings_load_reads_scanner_opencode_db_paths— paths deserialize from the expected shapesettings_accepts_empty_scanner_object—"scanner": {}is a valid no-opsettings_round_trips_scanner_section_through_json— save → load preserves paths verbatimFollow-ups (out of scope)
scanner.opencodeDbPathsinteractively. Right now users have to hand-edit the JSON file.:memory:in-process opencode dbs — still no file to scan.ScannerSettingsschema for other clients' override paths (currently only opencode has a use case).Summary by cubic
Adds persistent scanner config via
scanner.opencodeDbPathsin~/.config/tokscale/settings.jsonso users can pin OpenCode SQLite DBs outside the default data dir; paths merge with auto-discovery, dedupe by canonical path, skip missing, ignore SQLite sidecars, are honored across all report paths (includingparse_local_clientsand the wrapped UI), and are only merged when the OpenCode client is enabled so--clients claudedoesn’t import OpenCode DBs.New Features
tokscale-core::scanner::ScannerSettings(camelCase,Default) withopencodeDbPaths.scan_all_clients_with_scanner_settingsmerges user paths with discovery; dedupes, skips missing, rejects WAL/SHM/journal viais_opencode_db_filename.ReportOptionsandLocalParseOptionscarryscanner_settings(Default); wrappers pass defaults for backward compatibility.Settingsaddsscanner;load_scanner_settings()threads config into all command entry points; README documents the setting.Bug Fixes
parse_local_clientsnow passesscanner_settings, sotokscale clientsand wrapped views honoropencodeDbPaths.opencodeDbPathswhen OpenCode is enabled, preventing unrelated scans (e.g.--clients claude) from importing OpenCode DBs or inflating counts.tui::DataLoaderuses acfg(test)helper to ignore real settings in tests, keeping unit tests hermetic.Written for commit 0e1a79d. Summary will update on new commits.