feat(myopencre): strengthen backend CSV import validation with row-level errors and actionable feedback #694
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please review this PR after the above PR has been merged.
📝 Note for reviewers
This PR intentionally limits its scope to backend-only validation improvements in the following file:
application/web/web_main.pyapplication/frontend/src/pages/MyOpenCRE/MyOpenCRE.tsxNo frontend files, APIs, import logic, or data models were modified.
Any additional file diffs shown by GitHub are inherited from stacked branch history and are not part of this change.
Summary
This PR strengthens backend CSV import validation for MyOpenCRE by making validation
errors precise, structured, and actionable.
Earlier backend work (e.g. #683 / #684) introduced basic validation checks, but several
important failure modes still resulted in ambiguous, late, or unstructured errors.
This PR does not replace those validations — it extends and hardens them so that
common CSV mistakes are caught earlier and reported clearly.
The result is safer imports and significantly better feedback when a CSV is invalid,
without changing import behavior.
What is different from previous backend validation
Previous backend validation focused on whether an import could proceed.
This PR focuses on how failures are reported when an import cannot proceed.
Specifically, this PR adds:
All existing validation logic remains intact — this PR makes failures easier to
understand and fix.
Background
Before this PR, CSV validation had several gaps:
These issues made it difficult for users to correct CSV files even when they were
close to valid.
What changed
✅ Row-level validation with column context
CRE 0)✅ Explicit CRE format validation
CRE values are validated to ensure they follow:
Errors include a concrete example:
616-305|Development processes for security
✅ Graceful handling of invalid CRE IDs
InvalidCREIDException✅ Schema robustness
✅ Frontend compatibility
Scope
Test evidence
Invalid CRE format
Response
{ "row": 2, "column": "CRE 0", "code": "INVALID_CRE_FORMAT", "message": "Invalid CRE entry format", "example": "616-305|Development processes for security" }Screenshot
Invalid CRE ID
Response
{ "code": "INVALID_CRE_ID", "message": "CRE ID 'CRE-999' does not fit pattern '\\d\\d\\d-\\d\\d\\d'", "example": "616-305|Development processes for security" }Screenshot
Extra columns detected
Response
{ "type": "SCHEMA_ERROR", "message": "Row 2 has more columns than header. Please ensure the CSV matches the exported template." }Screenshot
Valid import (unchanged behavior)
Response
{ "status": "success", "import_type": "created", "new_cres": ["616-305"], "new_standards": 1 }Screenshot