Initial implementation of AST for CQL

[antvaset]

• Parsing CQL source files into ASTs modeled using CQL language constructs (distinct from ELM).
• Implementation of AST walker for read-only traversal of the CQL AST.
• WIP implementation of ELM emitter that transforms CQL AST to ELM.
• Showing the parse tree and AST in the playground.
• KMP for SystemModelInfoProvider.
• New FilesToStringsTask Gradle task with codegen enabling file contents to be inlined at compile time and used as strings (useful for e.g. inlining system-modelinfo.xml for SystemModelInfoProvider-in-JS).
• New LoadTestResourcesTask Gradle task with multiplatform codegen enabling access to test resources in commonTest.
• Limiting multiplatform test execution to JVM and JS-in-Node.js

[github-actions]

Formatting check succeeded!

[codecov]

Codecov Report

❌ Patch coverage is 0% with 112 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.61%. Comparing base (61ec335) to head (f25f667).

Files with missing lines	Patch %	Lines
...tlin/org/cqframework/cql/cql2elm/ast/ElmEmitter.kt	0.00%	96 Missing ⚠️
...framework/cql/cql2elm/frontend/CompilerFrontend.kt	0.00%	16 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #1640      +/-   ##
============================================
- Coverage     65.88%   65.61%   -0.27%     
  Complexity     1645     1645              
============================================
  Files           477      479       +2     
  Lines         27505    27617     +112     
  Branches       5471     5495      +24     
============================================
  Hits          18121    18121              
- Misses         7084     7196     +112     
  Partials       2300     2300              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[cmoesel]

I don't have any comments on the code (haven't looked at it), but I'm curious -- what's the motivation/vision for this CQL AST?

[JPercival]

I don't have any comments on the code (haven't looked at it), but I'm curious -- what's the motivation/vision for this CQL AST?

The ultimate goal is to split the compiler into a frontend (parsing, AST generation, type inference, some types of optimization like inlining) and a backend (ELM generation). The motivations for that are multi-fold.

1. The current Cql2ElmVisitor is difficult to maintain. By splitting the functionality in to phases we expect it to be easier to evolve going forward.
   • In particular, ELM generation can be "dumb" given a hypothetical fully type-checked and validated AST structure. The ELM generation itself is simple transform -- it's all the scope book-keeping, library resolution, type checking, etc that come from the CQL side that make it complex.
   • Similarly, intermingling ELM generation makes it more difficult to implement type inference.
   • As a side effect of the above, both the parsing and the ELM generation are more easily unit tested in isolation. See the preliminary unit tests in the PR. This will allow us to shrink the test deck while improving code coverage, there by improving the quality of the codebase overall.
2. CQL has begun to evolve independently of the ELM. An intermediate representation allows us to change one side of the equation without affecting the other.
   • This will allow us to more easily support various revisions of CQL while still generating the same ELM.
   • For example, all the CQL 2.0 changes obviate the need for ModelInfo entirely. We could internally "rewrite" using statements from CQL 1.3-1.5 to be inlined type definitions. This allows us to have "one" way manage types in the CQL compiler that we need to maintain, while being transparent to both the CQL users and the ELM consumers
3. It allows better IDE tools to be built.
   • Currently, the VS Code plugin is driven off of the ELM graph. Certain syntax subtleties like on or before end of are lost, and therefore we can't effectively support autocomplete of that statement. Hooking in "earlier" in the compilation process allows us to assist with syntax, not just semantics.
   • Similarly, it may be faster without need to go through the full ELM generation.
4. It allows the CQL compiler to target things beside ELM.
   • For example, it could emit FHIR-on-SQL statements based on the AST, or .Net bytecode, or whatever.

[cmoesel]

The ultimate goal is to split the compiler into a frontend (parsing, AST generation, type inference, some types of optimization like inlining) and a backend (ELM generation). The motivations for that are multi-fold...

Thanks for additional insight. That's helpful. It's been a while since I've written any significant CQL-to-ELM code (probably nearing a decade). I didn't realize the ELM model itself caused so much complexity -- I assumed it was mainly the type inference, function resolution, and other various CQL magic, but it sounds like that stuff is just shifting to the AST generator now. That said, ELM was based on Health e-Decisions, so I shouldn't be surprised that conforming to that schema might introduce additional complications.

There's not much to argue with here: better separation of concerns, better testing, better consistency across versions, better IDE tools, and easier translation into other target formats. I like it.

One small note: I wanted to see what the AST looks like, so I built it and ran the CQL playground. In the process of doing that, I found that the CQL-to-ELM UI README needs to be updated. It currently says to browse to http://localhost:3000/cql-to-elm-ui, but that should be http://localhost:3000/clinical_quality_language/playground now.

[antvaset]

Hi @cmoesel, sorry the README got outdated after we deployed the UI to cqframework.org. Now it's fixed.

[JPercival]

There's not much to argue with here... I like it.

Is that "I like the idea" or "I like the AST implementation?"

[cmoesel]

There's not much to argue with here... I like it.

Is that "I like the idea" or "I like the AST implementation?"

I like the idea. I haven't looked at the implementation closely enough to comment on that. Is there any documentation that lists all of the possible AST nodes and their relationships?

[sonarqubecloud]

Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

[JPercival]

@cmoesel - Docs updated. A few key ones:

Src/java/cql/README.md

Src/java/docs/DECISIONS.md

Src/java/cql/AST.md

[sonarqubecloud]

Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud