Validate module spec against JSON schema#7094
Conversation
Add formal JSON Schema validation for `nextflow module validate`, backed by the upstream Nextflow module schema. Schema validation runs between structure and Nextflow-specific spec checks, and overlapping hand-coded checks (name/description required, per-param type/description required, TODO type placeholder) are removed in favour of the schema as single source of truth. A `--schema` flag accepts a remote URL, a `file:` URI, or a local path to override the default schema location; load failures abort with a clear error. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
There was a problem hiding this comment.
In #7056, we added the $schema in meta.yml, but it is not taken into account in this PR as it is just considering using the DEFAULT_SCHEMA_URL or the URL provided by the user. Not a big deal now because both will be the same value, but thinking in the future, we could have meta.yml files pointing to a version which is not the default.
I was initially thinking the implementation on the PR is wrong, and we should use the schema in $schema. However, this could also point to a wrong schema and it will break the validation. @pditommaso have you considered it? What should we do in the case $schema is not the same as the DEFAULT_SCHEMA_URL. Should the validation fail, or should we ignore/warn and always validate through the DEFAULT_SCHEMA_URL (or the user specified one)?
…sMap [ci fast] Drop the duplicated literal so the default schema URL has a single source of truth, per review feedback on #7094. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Replace the hardcoded SpecVersion.VersionFlag.V202012 with SpecVersionDetector.detect, so the validator follows whatever draft the schema declares via $schema and aborts with a clear message if the draft is missing or unsupported. Per review feedback on #7094. Decompose validate() into self-contained helpers (parseSchema, detectSpecVersion, buildSchema, loadMeta), each with its own contextual error handling. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
|
Allow me to review again this, not fully sure on some details |
|
Marking as draft until you review |
- Use case-insensitive scheme check for http/https/file: locations - Set User-Agent on the GitHub raw fetch to reduce rate-limit risk - Rename CLI flag from --schema to -schema per Nextflow convention Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
|
Ok, made a few minor changes |
3 participants
Summary
nextflow module validatenow formally validatesmeta.ymlagainst the upstream module JSON schema (nextflow-io/schemas/main/module/v1/schema.json) usingcom.networknt:json-schema-validator.name/descriptionrequired, per-paramtype/descriptionrequired,TODO_TYPEplaceholder) are removed — the schema is the single source of truth for those rules. Checks the current schema does not cover stay inModuleSpec.validate():version/licenserequired, semver pattern,namespace/nameregex, and theTODO_DESCRIPTIONplaceholder.--schema <url-or-path>flag onnextflow module validateto override the default schema location. Accepts anhttp(s)://URL, afile:URI, or a local file path.--schemaas the escape hatch.A follow-up PR against
nextflow-io/schemaswill tighten the schema (requireversion/license, add the semver and namespace/name patterns, rejectTODO_DESCRIPTION) so the remaining hand-coded checks can also be removed.Test plan
./gradlew :nextflow:test --tests 'nextflow.module.ModuleSpecTest' --tests 'nextflow.module.ModuleSpecFactoryTest' --tests 'nextflow.module.ModuleSchemaValidatorTest' --tests 'nextflow.cli.module.CmdModuleValidateTest'— all greenModuleSchemaValidatorTest: pass, missing required field, invalid enum value,file:URI schema, hard-fail on missing schemaCmdModuleValidateTestrewritten to use a hermetic local schema fixture (no network in CI) and to split the missing-fields case into schema-level vs. nextflow-only scenariosnextflow module validate <module>against a real modulenextflow module validate <module> --schema ./local-schema.json