Skip to content

fix(analysis): discover external types with visibility, attributes, and generics - #83

Merged
thwbh merged 1 commit into
developfrom
82-external-crate-false-negatives
Sep 17, 2026
Merged

thwbh merged 1 commit into
developfrom
82-external-crate-false-negatives

Conversation

@thwbh

@thwbh thwbh commented Sep 17, 2026

Copy link
Copy Markdown
Owner

The external-crate lookup introduced in #78 used a raw substring match (content.contains("struct X") / content.contains("enum X")), which produced both false negatives and false positives:

  • false negatives for multi-line declarations and any form a stricter keyword-anchored regex would miss
  • false positives where a longer identifier (e.g. ExternalFooBar) matched a search for a shorter one (e.g. ExternalFoo)

Replace the substring heuristic with a syn-based AST match on the item identifier. A cheap contains(type_name) pre-filter still skips the vast majority of registry files before paying for a parse; candidate files are then parsed with syn::parse_file and checked for a struct/enum item whose ident exactly equals the requested type name. This transparently handles pub, pub(crate), #[derive(...)] attributes, generics, and multi-line declarations, and recurses into inline mod blocks.

Files that fail to parse (macro-heavy/generated registry sources) are skipped instead of aborting the walk.

Adds 9 regression tests covering pub(crate) visibility, derive attributes, generics, multi-line declarations, pub enum with attributes, nested modules, prefix false-positive avoidance, missing types, and unparseable files. Tests run #[serial] since they share the CARGO_HOME env var.

…nd generics (#82)

The external-crate lookup introduced in #78 used a raw substring match
(content.contains("struct X") / content.contains("enum X")), which produced
both false negatives and false positives:

- false negatives for multi-line declarations and any form a stricter
  keyword-anchored regex would miss
- false positives where a longer identifier (e.g. ExternalFooBar) matched a
  search for a shorter one (e.g. ExternalFoo)

Replace the substring heuristic with a syn-based AST match on the item
identifier. A cheap contains(type_name) pre-filter still skips the vast
majority of registry files before paying for a parse; candidate files are
then parsed with syn::parse_file and checked for a struct/enum item whose
ident exactly equals the requested type name. This transparently handles
pub, pub(crate), #[derive(...)] attributes, generics, and multi-line
declarations, and recurses into inline mod blocks.

Files that fail to parse (macro-heavy/generated registry sources) are
skipped instead of aborting the walk.

Adds 9 regression tests covering pub(crate) visibility, derive attributes,
generics, multi-line declarations, pub enum with attributes, nested
modules, prefix false-positive avoidance, missing types, and unparseable
files. Tests run #[serial] since they share the CARGO_HOME env var.
@chrome-grid

chrome-grid Bot commented Sep 17, 2026

Copy link
Copy Markdown

🟪🔷 netrunner :: intrusion report

░▒▓ N E T R U N N E R ▓▒░
   🟪 GRID STATUS: JACKED IN   |   netrunner-review: pass

CI passed cleanly. The PR replaces a fragile substring heuristic with an AST-based search for external types, correctly handling visibility, attributes, generics, and multi-line declarations. All new tests pass, including regressions for issue #82.


📡 NETWATCH · gate checks

➖ automerge — skipped
✅ Test Suite — success
✅ Code Coverage — success
✅ Validate Examples — success

🟢 VERDICT // JACKED IN — deck's clean, netrunner-review gate open.

"Your chrome’s polished, choom—no ICE in sight. Arasaka’s gonna hate this one."
— netrunner 🟪🔷

verdict: APPROVE · model: mistral-medium-2508

@thwbh
thwbh merged commit ee1f27c into develop Sep 17, 2026
5 checks passed
@thwbh
thwbh deleted the 82-external-crate-false-negatives branch September 17, 2026 10:22
thwbh added a commit that referenced this pull request Sep 17, 2026
* chore(deps): bump actions/cache from 5 to 6 (#63)

Bumps [actions/cache](https://github.com/actions/cache) from 5 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@v5...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump actions/setup-node from 6 to 7 (#65)

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 6 to 7.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](actions/setup-node@v6...v7)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): update syn requirement from 2.0.106 to 3.0.2 (#66)

Updates the requirements on [syn](https://github.com/dtolnay/syn) to permit the latest version.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](dtolnay/syn@2.0.106...3.0.2)

---
updated-dependencies:
- dependency-name: syn
  dependency-version: 3.0.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump taiki-e/install-action (#69)

Bumps the github-actions-updates group with 1 update: [taiki-e/install-action](https://github.com/taiki-e/install-action).


Updates `taiki-e/install-action` from 2 to 2.85.3
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/install-action@v2...v2.85.3)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.85.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump taiki-e/install-action (#70)

Bumps the github-actions-updates group with 1 update: [taiki-e/install-action](https://github.com/taiki-e/install-action).


Updates `taiki-e/install-action` from 2.85.3 to 2.85.8
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/install-action@v2.85.3...v2.85.8)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.85.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump taiki-e/install-action (#71)

Bumps the github-actions-updates group with 1 update: [taiki-e/install-action](https://github.com/taiki-e/install-action).


Updates `taiki-e/install-action` from 2.85.8 to 2.85.11
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/install-action@v2.85.8...v2.85.11)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.85.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump taiki-e/install-action (#72)

Bumps the github-actions-updates group with 1 update: [taiki-e/install-action](https://github.com/taiki-e/install-action).


Updates `taiki-e/install-action` from 2.85.11 to 2.86.3
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/install-action@v2.85.11...v2.86.3)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.86.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump taiki-e/install-action (#75)

Bumps the github-actions-updates group with 1 update: [taiki-e/install-action](https://github.com/taiki-e/install-action).


Updates `taiki-e/install-action` from 2.86.3 to 2.86.7
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/install-action@v2.86.3...v2.86.7)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.86.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump taiki-e/install-action (#79)

Bumps the github-actions-updates group with 1 update: [taiki-e/install-action](https://github.com/taiki-e/install-action).


Updates `taiki-e/install-action` from 2.86.7 to 2.87.3
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/install-action@v2.86.7...v2.87.3)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.87.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump taiki-e/install-action (#80)

Bumps the github-actions-updates group with 1 update: [taiki-e/install-action](https://github.com/taiki-e/install-action).


Updates `taiki-e/install-action` from 2.87.3 to 2.87.8
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/install-action@v2.87.3...v2.87.8)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.87.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* External type lookup (#78)

* chore(release): prevent dirty work tree on publication

* Add External crate type lookup and nix dev enviroment.

Extend the analysis module to search the Cargo registry for
type definitions in external crates, cache discovered paths, and parse
the
files. Add tests covering the external‑type lookup feature.

* Improving the logic for validating internal types that are part of a
type from an external crate

---------

Co-authored-by: Stefan Poindl <stefan@tohuwabohu.io>

* fix(analysis): discover external types with visibility, attributes, and generics (#82) (#83)

The external-crate lookup introduced in #78 used a raw substring match
(content.contains("struct X") / content.contains("enum X")), which produced
both false negatives and false positives:

- false negatives for multi-line declarations and any form a stricter
  keyword-anchored regex would miss
- false positives where a longer identifier (e.g. ExternalFooBar) matched a
  search for a shorter one (e.g. ExternalFoo)

Replace the substring heuristic with a syn-based AST match on the item
identifier. A cheap contains(type_name) pre-filter still skips the vast
majority of registry files before paying for a parse; candidate files are
then parsed with syn::parse_file and checked for a struct/enum item whose
ident exactly equals the requested type name. This transparently handles
pub, pub(crate), #[derive(...)] attributes, generics, and multi-line
declarations, and recurses into inline mod blocks.

Files that fail to parse (macro-heavy/generated registry sources) are
skipped instead of aborting the walk.

Adds 9 regression tests covering pub(crate) visibility, derive attributes,
generics, multi-line declarations, pub enum with attributes, nested
modules, prefix false-positive avoidance, missing types, and unparseable
files. Tests run #[serial] since they share the CARGO_HOME env var.

* fix(analysis): make unresolved external types observable; fix CARGO_HOME fallback (#84) (#88)

The external-crate lookup (#78, hardened in #82) silently returned None when a
referenced type could not be found, producing quietly-incomplete bindings
with no indication that anything was wrong (the original #77 symptom). The
registry-root resolution also used format!("{}/.cargo", h), which is wrong on
Windows.

Resolve_types_lazily now records every type name it could not resolve
(neither in-project nor via the Cargo registry) on the analyzer and exposes
them via CommandAnalyzer::unresolved_types(). analyze_project_with_verbose
prints a non-fatal warning to stderr listing the unresolved names and
pointing at #84, so CLI and build.rs users see it even without --verbose.
Generation continues as before to preserve backward-compatible output.

The CARGO_HOME fallback now uses PathBuf::join (not string formatting) and
additionally honors USERPROFILE when HOME is unset, fixing the Windows path.

Vendored-dependency (.cargo/config.toml source.*.replace-with) and
cargo-metadata-based discovery are intentionally out of scope here; they are
tracked separately in #84's suggested-direction list and depend on this
reportability groundwork.

Adds 5 tests: fresh-analyzer invariant, unresolved-with-empty-registry,
unresolved-with-absent-registry-dir, resolved-in-project-type-not-reported,
and USERPROFILE-fallback-locates-type. Tests are #[serial] (shared CARGO_HOME).

* perf(analysis): persist external-crate type lookups in .typecache (#87) (#89)

The external-crate registry walk (#78/#82) was invoked on every reference to a
type within a single analyze() pass, with no result caching, so missing types
re-walked the Cargo registry DFS from scratch each time (#87). The in-process
memo added in the first cut of this change only helped within one process
invocation — on the next cargo build / tauri-typegen generate, the walk ran
again from a cold cache, which defeats the point for warm builds.

Persist the lookup index in .typecache so it survives across runs, matching the
existing generation-cache mechanism (that's what .typecache was added for).

Changes:
- GenerationCache carries a new external_type_index: HashMap<String,
  Option<PathBuf>> (type name -> defining file, or None for known-missing).
  Bumped CURRENT_VERSION to 3; the field is #[serde(default)] so older cache
  files deserialize cleanly and are then invalidated by the version check.
- The index is derived data and is intentionally excluded from combined_hash,
  so registry churn does not force regeneration; invalidation rides the
  command/struct/event/config hashes as before.
- CommandAnalyzer exposes seed_external_type_cache() and
  external_type_lookup_cache() so the cache can be loaded into the analyzer
  before analysis and read back out after generation.
- Both consumers (CLI and build.rs) load the previous .typecache and seed the
  analyzer before analyze_project*, then save a fresh cache (including the
  index) after successful generation.
- find_external_type_path_cached remains the memoizing lookup the resolver
  uses; seeded entries short-circuit the walk, unseeded types fall back to it
  and are memoized in turn.

Correctness: the index is a hint, not authoritative — a stale positive entry
pointing at a file that no longer exists is handled by the downstream
parse_and_cache_file failing gracefully. A stale negative entry is an accepted
trade-off matching .typecache's existing source-stability assumption: when the
user's code changes such that new external types are referenced, the command
hash changes, .typecache is regenerated, and the index is rebuilt.

Tests: 3 new in generation_cache (index stored/exposed, excluded from
combined_hash, round-trips through save/load) and 3 new in analysis (seeded
positive skips walk, seeded negative skips walk, unseeded falls back to walk
and memoizes). All #[serial] (shared CARGO_HOME).

* 0.5.3

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: GarandPLG <89148639+GarandPLG@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant