feat: Add schema validation and data combination script #1
Conversation
|
I started to put some changes up. This is not ready for review yet. I will update this later @captainbrosset |
jcscottiii
changed the title
jcscottiii
force-pushed
the
jcscottiii/schemas-lint
branch
from
575e95a to
b95a191
Compare
|
There are actual lint errors in the data currently: $ npm run lint:schemas
> lint:schemas
> node lint-schemas.js
Linting mapping files against schemas...
Linting /usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/wpt.json...
/usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/wpt.json is valid.
Error: Feature ID "viewport-segments" does not exist in web-features.
Error: Feature ID "viewport" does not exist in web-features.
Error: Feature ID "mixin" does not exist in web-features.
Linting /usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/state-of-surveys.json...
/usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/state-of-surveys.json is valid.
Linting /usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/standards-positions.json...
/usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/standards-positions.json is valid.
Linting /usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/mdn-docs.json...
/usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/mdn-docs.json is valid.
Error: Feature ID "gravity-sensor" does not exist in web-features.
Linting /usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/interop.json...
/usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/interop.json is valid.
Linting /usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/chrome-use-counters.json...
/usr/local/google/home/jamescscott/code/final/web-features-mappings/mappings/chrome-use-counters.json is valid.
Linting failed with errors. |
|
@captainbrosset This is ready for review. cc: @foolip |
There was a problem hiding this comment.
This looks great to me at a high level. If others want to dig in deeper into the technical details, then great. But I won't, both because I'm not super familiar with schemas and validation, but also because this early in the life of the repo, I don't think we need to block this, and can instead always go back and iterate quickly on these files if needed.
Very interesting. I honestly don't know where these unknown IDs come from. Maybe they were created at some point in the web-features repo and then removed, but that seems unlikely. |
It's weird because WPT does have a feature called So this ID definitely existed at some point. |
|
IDs in WPT aren't validated and might not exist in web-features. This one did show up in an audit previously: I've sent web-platform-dx/web-features#3285 to make it exist! |
|
@foolip cool! Maybe we can add this repo (web-features-mappings) to the list of data sources that the consumer audit checks! |
This commit introduces two main features to improve data quality and ease of use: Schema Validation: A schemas.json file is added to define a strict schema for each of the mapping files. A new lint-schemas.js script validates the mapping files against these schemas. It also checks that all feature IDs exist in the web-features package. Combined Data: A new combine-mappings.js script aggregates data from all individual mapping files into a single web-features-mappings.combined.json file. This provides a single, convenient file for data consumers. The package.json has been updated with lint:schemas and combine scripts to run these new tools. The README and .gitignore have also been updated accordingly.
jcscottiii
force-pushed
the
jcscottiii/schemas-lint
branch
from
b95a191 to
766c75d
Compare
|
I rebased these changes and it should merge cleanly now. I don't have access to merge this. Regarding fixing the lint errors, I filed #3 |
This commit introduces two main features to improve data quality and ease of use:
Schema Validation:
A schemas.json file is added to define a strict schema for each of the mapping files.
A new lint-schemas.js script validates the mapping files against these schemas. It also checks that all feature IDs exist in the web-features package.
Combined Data:
A new combine-mappings.js script aggregates data from all individual mapping files into a single
web-features-mappings.combined.jsonfile. This provides a single, convenient file for data consumers.The package.json has been updated with lint:schemas and combine scripts to run these new tools. The README and .gitignore have also been updated accordingly.