[Feature]: Reuse loader transform results across persistent module-graph invalidation

[matthewdavis-oai]

What problem does this feature solve?

Rspack's persistent cache currently stores a NormalModule (including its post-loader source) with the module graph. This makes a valid warm graph very fast, but a graph-level invalidation also discards otherwise reusable loader transforms. Large applications then repeat expensive JS/TS transforms and Rust-to-JavaScript loader callbacks even when the source and loader pipeline are unchanged.

This is particularly visible in large monorepos, where a legitimate resolver, bundling, peer-context, or build-dependency change can invalidate the graph while nearly all source transforms remain reusable. In one real application the two persistent module graphs and source-map scopes occupy about 8.1 GiB / 11,250 files; the equivalent transform-only cache used by its Vite path is about 166 MiB / 7,595 entries. The latter survives ordinary graph/configuration churn, but implements thousands of JavaScript/filesystem lookups and therefore loses some of Rspack's native-performance advantage.

The requested feature is an independently versioned, packed loader-transform result cache that can be consulted before JS loaders run. It should allow a new/invalidated module graph to reuse unchanged transforms while still resolving, parsing, linking, optimizing, and emitting a fresh graph normally.

There is already interest in reusable loader transforms in discussion #11965, which focuses on portable/remote cache storage. This request is narrower: decouple the transform artifact and its invalidation identity from the complete module graph, with local packed storage as a useful first step.

The current source appears to have a natural boundary:

• NormalModule::build invokes run_loaders, creates the transformed source/map, and then parses it to discover dependencies (source, source).
• The build-module-graph phase precedes finishModules, seal, chunk-graph construction, optimization, and code generation (source).
• The existing before_loaders hook cannot bail out with a cached result, and a non-builtin loader yields through the JS-loader scheduler (source). A native lookup before run_loaders would avoid those callbacks on a hit.

This is distinct from making transforms faster: the aim is to avoid re-running any unchanged JS/TS loader pipeline after a graph-level miss.

What does the proposed API of configuration look like?

The exact API is flexible, but an opt-in shape could be:

export default {
  cache: {
    type: 'persistent',
    version: graphVersion,
    loaderTransforms: {
      // Independent of the module-graph cache version.
      version: 'source-transforms-v1',
      storage: {
        type: 'filesystem',
        directory: '.cache/rspack-loader-transforms',
      },
      // An initial implementation could explicitly limit eligibility to
      // known-pure JS/TS rules or loaders.
      include: [/\.[cm]?[jt]sx?$/],
    },
  },
};

An API attached to module.rules or loader options would also work. The important properties are:

1. Independent identity. Changing cache.version, unrelated build dependencies, or graph/bundling configuration can invalidate the graph without discarding valid transform entries. A transform-pipeline/version change must invalidate those entries.
2. Native lookup and packed storage. A hit occurs before run_loaders/yield_to_js, avoiding a JS callback and one-file-per-module filesystem contention. Reads/writes should be bounded, atomic, and compactable.
3. Correct cache key. Include resource content and request/query/fragment, ordered resolved loader identities/options/versions, source-map mode, relevant target/environment/configuration, and an explicit transform version.
4. Correct side-effect replay. Restore transformed content/map and the loader result metadata needed by parsing and snapshots (file/context/missing/build dependencies, diagnostics, parse metadata/additional data, emitted assets as applicable), and always bypass entries for cacheable(false) or unsupported/impure loaders.
5. Observable behavior. Persistent-cache debug logs/stats should distinguish transform hits, misses, bypasses, and graph recovery so adopters can verify correctness and benefit.

A minimal acceptance/reproduction case could use an intentionally expensive JS loader that counts calls:

1. Compile N JS modules with graph version A; expect N loader calls and populate the transform store.
2. Restart with graph version B but identical source/loader configuration; expect a fresh graph and zero loader calls.
3. Edit one source file; expect one loader call.
4. Change the loader option/version; expect the relevant transforms to miss.
5. Exercise cacheable(false) and a loader-added dependency; verify no stale output.

Would the maintainers be open to a separately recoverable loader-transform occasion/store at this boundary, or is there an existing/planned cache API that already addresses this use case?

[chenjiahan]

This makes a lot of sense. I’ve wanted a built-in "cache-loader" in Rspack for a while, especially for slower loaders like babel-loader and less-loader. This would be very useful.