feat(ast-grep): Add experimental ast-grep Wasm plugin

[kdy1]

I'm trying to see if we can support ast-grep using a Wasm plugin so a user can modify AST using ast-grep, instead of building a plugin

[changeset-bot]

⚠️ No Changeset found

Latest commit: 5d6dfa6

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[kdy1]

cc @HerringtonDarkholme for visibility

[HerringtonDarkholme]

Hi Donny!

For some background, ast-grep now is relying on tree-sitter as the parser. So using it with SWC will have to include another parser. Also, the tree-sitter cannot be compiled as WASM using Rust. It has to be compiled with emscripten.

However, if ast-grep can be changed to be parser independent, say, providing a trait for AST node/Parser, it is possible to compile it with SWC to a standalone WASM.

It will take quite a lot of effort, but it is not impossible! I can do some experiments in the coming weekend and give you a rough estimate of how much time it will take.

[HerringtonDarkholme]

Hi Donny, I'm working on ast-grep's parser abstraction on this branch now. ast-grep/ast-grep#1940
The PR is still WIP but it is promising that ast-grep can be independent of tree-sitter.

[HerringtonDarkholme]

Hi Donny, now ast-grep is fully independent of tree-sitter parser now. See https://github.com/ast-grep/ast-grep/blob/290b31e6e44a9891f99d243b57c4ae1bdbaa340f/crates/core/Cargo.toml#L20

However, integrating swc still needs significant change in swc plugin system.

1. First, ast-grep needs implementing two core traits: SgNode and Doc. https://github.com/ast-grep/ast-grep/blob/290b31e6e44a9891f99d243b57c4ae1bdbaa340f/crates/core/src/source.rs#L28-L67. For now swc's AST has many structs/enums that need changes to implement that trait. If swc can provide a "type-less" AST view, it can be easier to implement

2. Second, swc's plugin is based on Visitor or Fold. But ast-grep's Mather or Replacer only process one node at a time. It is possible to use find_all and replace_all on the root program. Or use swc's Visitor to match ast-grep rule against all nodes. The difference is about who will traverse the AST: the first approach uses ast-grep and the second approach uses swc.

3. swc's plugin transform code by returning new AST node from Visitor/Transformation methods. But ast-grep's change is based on string. This also needs change in swc.

[HerringtonDarkholme]

This is an example of using oxc as parser. ast-grep/ast-grep#1970

oxc is used because it has an untyped AST which makes the integration faster. But the general idea is three steps:

1. Implement Langauge trait https://github.com/ast-grep/ast-grep/blob/26cffdd127b7cf659a14f1a030e971c520709a81/crates/oxc/src/binding.rs#L183 Language trait is used to parse Pattern and mapping numeric kind_id to human readable kind string and vice versa.
2. implement the Doc trait https://github.com/ast-grep/ast-grep/blob/26cffdd127b7cf659a14f1a030e971c520709a81/crates/oxc/src/binding.rs#L502-L516 Doc trait is used to maintain the source code representation and parsing.
3. implement Tree node traversal methods in SgNode https://github.com/ast-grep/ast-grep/blob/26cffdd127b7cf659a14f1a030e971c520709a81/crates/oxc/src/binding.rs#L405