Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(developer): use ABNF to validate LDML transform #13236

Merged
merged 14 commits into from
Feb 21, 2025

Conversation

srl295
Copy link
Member

@srl295 srl295 commented Feb 13, 2025

  • copy abnf from draft CLDR 47
  • also, output keyman_core_ldml.sh for scripts
  • generates dependency-free .js files ( abnf -> peggy -> .js ) for use in parsing

Fixes: #13175

@keymanapp-test-bot skip

@srl295 srl295 self-assigned this Feb 13, 2025
@keymanapp-test-bot keymanapp-test-bot bot added the user-test-missing User tests have not yet been defined for the PR label Feb 13, 2025
@keymanapp-test-bot keymanapp-test-bot bot added this to the B18S1 milestone Feb 13, 2025
@keymanapp-test-bot keymanapp-test-bot bot removed the user-test-missing User tests have not yet been defined for the PR label Feb 13, 2025
@srl295 srl295 marked this pull request as draft February 13, 2025 22:48
- update fetch-latest-cldr.sh - so next time, abnf will get updated.
- note, this is abnf from 47 but we’re not doing the full 47 import yet because it's not released.

Fixes: feat(developer): utilize CLDR ABNF rules to validate transforms #13175
- convert each .abnf to a .pegjs file

Fixes: #13175
- just convert over all abnf (as we do with copying imports)
- hard coded CLDR version number for now

Fixes: #13175
@srl295 srl295 force-pushed the feat/developer/13175-abnf-on-compileepic-ldml branch from ebc7bb8 to 2e842bc Compare February 14, 2025 19:12
@github-actions github-actions bot added common/web/ core/ Keyman Core and removed core/ Keyman Core common/web/ labels Feb 14, 2025
@srl295 srl295 marked this pull request as ready for review February 19, 2025 19:38
@srl295
Copy link
Member Author

srl295 commented Feb 19, 2025

Okay, I think I've fixed the build issues, should be ready for review.

- late breaking update from CLDR, update with in-progress fixes to escaping chars slated for v47

Fixes: #13175
@srl295
Copy link
Member Author

srl295 commented Feb 19, 2025

pulled in latest ABNF which supports \+, \* etc

@github-actions github-actions bot added common/web/ core/ Keyman Core and removed core/ Keyman Core common/web/ labels Feb 19, 2025

<transforms type="simple">
<transformGroup>
<transform from="(abc){1,}" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment on what you are testing here?

@@ -27,24 +27,44 @@ builder_describe "Keyman kmc Keyboard Compiler module" \
"--dry-run,-n don't actually publish, just dry run"

builder_describe_outputs \
configure /node_modules \
configure /developer/src/kmc-ldml/src/util/abnf/46/transform-from-required.js \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we originally have /node_modules so configure step will run if it's not present?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can there be multiple lines with the same target?
anyway- without this line, configure didn't re-run because node_modules was already present.

Copy link
Member

@mcdurdin mcdurdin Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The /node_modules is the default for node projects if there are no local targets -- ensures npm ci is called if it is missing. If there is a local target (in this case transform-from-required.js) then you need to use that instead.

At this point, we only support a single output per target (we could support more, but we start to drift into being a full build system with out-of-date checks, and that's not really our goal with build.sh -- that is the responsibility of other build systems: gradle, meson, xcodebuild, delphi, etc).

The intent of this is to avoid unnecessary configure and build steps for dependencies when the build is already in a good state. If the build is not in a good state, you can:

  1. git clean -fdx "$KEYMAN_ROOT"; ./build.sh (the big hammer but also deletes local config yeuch), or
  2. $dep/build.sh clean clean action
  3. ./build.sh --force-deps (ignore outputs and reconfigure and rebuild each dep)

See also discussion at builder_describe_outputs and build script parameters - dependencies section

(Also, for best results, generally the referenced output should be the last thing generated in the action -- so if the action fails half way it will re-run automatically.)

@github-actions github-actions bot added common/web/ core/ Keyman Core and removed core/ Keyman Core common/web/ labels Feb 20, 2025
@github-actions github-actions bot added common/web/ core/ Keyman Core and removed core/ Keyman Core common/web/ labels Feb 20, 2025
Copy link
Member

@mcdurdin mcdurdin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@srl295 srl295 merged commit 38b1fec into master Feb 21, 2025
28 checks passed
@srl295 srl295 deleted the feat/developer/13175-abnf-on-compileepic-ldml branch February 21, 2025 05:46
@keyman-server
Copy link
Collaborator

Changes in this pull request will be available for download in Keyman version 19.0.8-alpha

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

feat(developer): utilize CLDR ABNF rules to validate transforms
4 participants