Skip to content

Add fast-path for simple filters and key lookups - #437

Open
AdamMagued wants to merge 1 commit into
antonmedv:masterfrom
AdamMagued:fix-issue-371
Open

AdamMagued wants to merge 1 commit into
antonmedv:masterfrom
AdamMagued:fix-issue-371

Conversation

@AdamMagued

Copy link
Copy Markdown

Fixes #371

Summary

This change introduces an optimized fast-path evaluation path in internal/engine for simple filters and property lookups, bypassing JS engine instantiation and serialization overhead when processing large streams (such as NDJSON) or slurped arrays.

Changes

  1. Filter Fast-Path:

    • Parses common filter shorthands (?.property == value, ===, !=, !==, <, <=, >, >=).
    • Supports ternary skip patterns (x.property == value ? x : skip).
    • Supports .includes() for strings and arrays.
    • Preserves JavaScript abstract and strict equality semantics including numeric coercions.
    • For streams, evaluates filters in place and discards non-matching objects without allocating JS runtime values.
    • For arrays and slurped inputs, builds filtered arrays directly from matching node subtrees.
  2. Lookup Fast-Path:

    • Parses direct property chains (.field, .nested.field, .[index]).
    • Directly extracts matching child nodes via path traversal.
    • Validates node tokens against lenient parser errors to preserve error reporting parity.
  3. Fallback:

    • Any query containing JavaScript operations not handled by the fast path transparently delegates to the Goja runtime.

Verification

  • All existing tests in the test suite pass cleanly (go test ./...).
  • Added unit tests in internal/engine/fastpath_test.go covering stream filtering, slurped filtering, loose coercion, comparisons, lookups, and edge cases.
  • Benchmark results demonstrate a 6x to 7x latency reduction for simple filtering and lookup workloads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add fast-path for simple filters

1 participant