[BUGFIX] Support standing up a FileDataContext on a read-only filesystem - #12000
Open
joshua-stauffer wants to merge 4 commits into
Open
[BUGFIX] Support standing up a FileDataContext on a read-only filesystem#12000joshua-stauffer wants to merge 4 commits into
joshua-stauffer wants to merge 4 commits into
Conversation
FileDataContext previously re-scaffolded a project whenever the gitignored uncommitted/* directories or a config-variables file were absent, even though a committed great_expectations.yml already made the project usable. On a fresh version-control checkout (where uncommitted/* is intentionally untracked) or a config that resolves values purely via environment-variable interpolation, this false-negatived and triggered a destructive re-scaffold. Redefine is_project_scaffolded to hinge on the presence of a committed great_expectations.yml alone, so a version-controlled project is recognized as already set up without requiring gitignored runtime-output directories or a separate config-variables file to exist on disk.
✅ Deploy Preview for niobium-lead-7998 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
TupleFilesystemStoreBackend.__init__ eagerly creates its base directory for convenience. On a read-only filesystem this aborts construction even though the directory isn't needed to read, and even though _set/_move already self-create their own leaf directory before writing. Tolerate only the two read-only-shaped failures (a read-only directory, or a read-only mount) at construction time and defer to first write; re-raise every other OSError so a genuine fault on a writable filesystem still fails fast exactly as before.
Prove end to end that standing up a FileDataContext against a fully-scaffolded, version-controlled project succeeds on a genuinely read-only filesystem: initialization, resolving a configured datasource whose config uses environment-variable interpolation, loading a suite, building a validator, and running a validation to an in-memory result. The fixture emulates a fresh version-control checkout by deleting the gitignored uncommitted/ subtree before making the project read-only -- without that deletion the suite would pass whether or not the underlying fixes are present, since a fully scaffolded project never exercises either failure mode. A skip-clean probe keeps the suite green in environments where read-only permissions cannot be enforced (root, or a mode-ignoring filesystem). A store-backed write against the read-only context still fails with a clear error rather than being silently swallowed, and an empty project directory on a writable filesystem still scaffolds a full project as before.
joshua-stauffer
marked this pull request as ready for review
July 24, 2026 23:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Standing up a file-backed
DataContext(gx.get_context(mode="file")orproject_root_dir=...) against a fully-scaffolded, version-controlled projecton a read-only filesystem currently crashes during initialization.
The root cause is that the "is this project already scaffolded" check treats
the gitignored
uncommitted/*runtime-output directories (and a separateconfig-variables file that also lives under
uncommitted/) as requiredsignals. On a fresh version-control checkout those directories are absent by
design, so the check false-negatives and triggers a full destructive
re-scaffold — which then fails outright on a read-only filesystem.
This PR redefines that check to hinge on the presence of a committed
great_expectations.ymlalone. A project carrying a valid, committed configis recognized as already set up without requiring gitignored runtime
directories to exist on disk, so a clean checkout (and a config that resolves
values purely via environment-variable interpolation) is no longer
mistaken for an unscaffolded project.
First-use scaffolding for a genuinely new/empty project directory is
unchanged.
Test plan
pytest tests/data_context/test_data_context.py tests/data_context/test_get_data_context.py -v— 55 passed, 5 skipped (pre-existing, unrelated)ruff check/ruff format --checkclean on changed files