You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I want to validate that a text column's values match a given strftime date/time format (e.g. "%Y-%m-%d" or "%Y-%m-%dT%H:%M:%S%z") — a common check on date columns that arrive as strings before parsing. Users have asked for exactly this and reported that it seemed to disappear from the docs/Gallery: see Discourse: "Does GX support date format check?" ("Can we have it added in the newer versions?") and issue #11603.
ExpectColumnValuesToMatchStrftimeFormatalready exists and is exported from great_expectations.expectations, but it is not in the Expectation Gallery, has no generated schema, carries none of the support metadata, and its backend coverage is incomplete — so it's effectively invisible and its supported surface is undefined.
Proposed capability
Bring ExpectColumnValuesToMatchStrftimeFormat up to the supported-core bar: Gallery-published (schema + metadata), a correct docstring, and a defined, tested backend matrix.
Current backend state: the backing metric implements Pandas and Spark but has no SQL (SqlAlchemy) implementation. So the target matrix requires a decision (see Additional context): either add a SQL implementation, or ship Pandas + Spark as the supported set with SQL explicitly declared out of scope, documented as a dialect limitation.
Implementation Guide
Source class: great_expectations/expectations/core/expect_column_values_to_match_strftime_format.py. Backing metric: great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py (pandas + Spark; no SqlAlchemy branch). Reference template for the supported bar: great_expectations/expectations/core/expect_column_values_to_match_regex.py + its metric + its schema great_expectations/expectations/core/schemas/ExpectColumnValuesToMatchRegex.json.
Decision to make first: SQL support
Generic strftime-format validation in SQL isn't straightforward — strftime format tokens do not map cleanly onto the format models of Snowflake, PostgreSQL, BigQuery, etc., and each dialect's date-parse/TRY_CAST behavior differs. There is precedent for shipping supported-but-not-on-every-dialect: ExpectColumnValuesToMatchRegex raises NotImplementedError on dialects without regex support and omits SQL Server + Snowflake from its SUPPORTED_DATA_SOURCES.
Two acceptable outcomes — pick one and make it the definition of done:
(A) Ship Pandas + Spark supported, SQL declared out. Lowest risk. SUPPORTED_DATA_SOURCES = Pandas + Spark only; document the SQL limitation in the docstring. This unblocks the Gallery and the two working engines immediately.
(B) Add a SQL implementation. Implement a _sqlalchemy branch (per-dialect date-format validation via TRY_CAST/TO_DATE/TRY_TO_TIMESTAMP, raising NotImplementedError on dialects that can't express it), and add only the dialects you actually validate to SUPPORTED_DATA_SOURCES. This is the hardest single piece of work here — scope it deliberately; it's fine to land (A) first and pursue (B) as a follow-up.
Definition of done (both outcomes)
Docstring fix (bug): the current docstring is a plain string containing the literal token {FAILURE_SEVERITY_DESCRIPTION}, which is never interpolated and renders raw. Convert to an __doc__ = f"""...""" f-string (like the regex exemplar) and bring it to the standard Gallery format (short description, Args, Supported Data Sources, Data Quality Issues, Example Data, passing + failing Code Examples).
Support metadata: add module constants EXPECTATION_SHORT_DESCRIPTION, DATA_QUALITY_ISSUES ([DataQualityIssues.VALIDITY.value]), SUPPORTED_DATA_SOURCES (matching your chosen matrix); a library_metadata ClassVar ("maturity": "production", has_full_test_suite: True, manually_reviewed_code: True, with _library_metadata alias); and a Config.schema_extra that stamps them into the schema. Copy the exact shape from the regex exemplar.
Schema: add the class to the supported_expectations list in tasks.py, run invoke schemas --sync, and commit great_expectations/expectations/core/schemas/ExpectColumnValuesToMatchStrftimeFormat.json. (CI's test_schemas_updated enforces it.)
Renderers: at least a prescriptive renderer if missing.
Integration tests at tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_match_strftime_format.py covering the full target matrix via @parameterize_batch_for_data_sources. The current test file exists but is Spark-only — extend it to cover pandas (and SQL, if outcome B), with passing and failing cases, including a timezone-aware format (%z), which was a real regression area (issue Validating datetime strings with timezone produces an error #9203).
Testing note for community contributors. Pandas, sqlite, spark, postgres, and mysql run locally (some via Docker Compose); BigQuery, Snowflake, Databricks, and Redshift run only in maintainer CI with credentials. Write tests against the full target matrix, run what you can locally, and CI covers the cloud dialects. Run a single module locally by marker from the repo root, e.g. pytest tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_match_strftime_format.py -m unit (pandas), or -m sqlite / -m spark (see AGENTS.md for marker usage).
Use case
I want to validate that a text column's values match a given
strftimedate/time format (e.g."%Y-%m-%d"or"%Y-%m-%dT%H:%M:%S%z") — a common check on date columns that arrive as strings before parsing. Users have asked for exactly this and reported that it seemed to disappear from the docs/Gallery: see Discourse: "Does GX support date format check?" ("Can we have it added in the newer versions?") and issue #11603.ExpectColumnValuesToMatchStrftimeFormatalready exists and is exported fromgreat_expectations.expectations, but it is not in the Expectation Gallery, has no generated schema, carries none of the support metadata, and its backend coverage is incomplete — so it's effectively invisible and its supported surface is undefined.Proposed capability
Bring
ExpectColumnValuesToMatchStrftimeFormatup to the supported-core bar: Gallery-published (schema + metadata), a correct docstring, and a defined, tested backend matrix.Current backend state: the backing metric implements Pandas and Spark but has no SQL (SqlAlchemy) implementation. So the target matrix requires a decision (see Additional context): either add a SQL implementation, or ship Pandas + Spark as the supported set with SQL explicitly declared out of scope, documented as a dialect limitation.
Implementation Guide
Source class:
great_expectations/expectations/core/expect_column_values_to_match_strftime_format.py. Backing metric:great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py(pandas + Spark; no SqlAlchemy branch). Reference template for the supported bar:great_expectations/expectations/core/expect_column_values_to_match_regex.py+ its metric + its schemagreat_expectations/expectations/core/schemas/ExpectColumnValuesToMatchRegex.json.Decision to make first: SQL support
Generic
strftime-format validation in SQL isn't straightforward —strftimeformat tokens do not map cleanly onto the format models of Snowflake, PostgreSQL, BigQuery, etc., and each dialect's date-parse/TRY_CASTbehavior differs. There is precedent for shipping supported-but-not-on-every-dialect:ExpectColumnValuesToMatchRegexraisesNotImplementedErroron dialects without regex support and omits SQL Server + Snowflake from itsSUPPORTED_DATA_SOURCES.Two acceptable outcomes — pick one and make it the definition of done:
SUPPORTED_DATA_SOURCES= Pandas + Spark only; document the SQL limitation in the docstring. This unblocks the Gallery and the two working engines immediately._sqlalchemybranch (per-dialect date-format validation viaTRY_CAST/TO_DATE/TRY_TO_TIMESTAMP, raisingNotImplementedErroron dialects that can't express it), and add only the dialects you actually validate toSUPPORTED_DATA_SOURCES. This is the hardest single piece of work here — scope it deliberately; it's fine to land (A) first and pursue (B) as a follow-up.Definition of done (both outcomes)
{FAILURE_SEVERITY_DESCRIPTION}, which is never interpolated and renders raw. Convert to an__doc__ = f"""..."""f-string (like the regex exemplar) and bring it to the standard Gallery format (short description, Args, Supported Data Sources, Data Quality Issues, Example Data, passing + failing Code Examples).EXPECTATION_SHORT_DESCRIPTION,DATA_QUALITY_ISSUES([DataQualityIssues.VALIDITY.value]),SUPPORTED_DATA_SOURCES(matching your chosen matrix); alibrary_metadataClassVar ("maturity": "production",has_full_test_suite: True,manually_reviewed_code: True, with_library_metadataalias); and aConfig.schema_extrathat stamps them into the schema. Copy the exact shape from the regex exemplar.supported_expectationslist intasks.py, runinvoke schemas --sync, and commitgreat_expectations/expectations/core/schemas/ExpectColumnValuesToMatchStrftimeFormat.json. (CI'stest_schemas_updatedenforces it.)tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_match_strftime_format.pycovering the full target matrix via@parameterize_batch_for_data_sources. The current test file exists but is Spark-only — extend it to cover pandas (and SQL, if outcome B), with passing and failing cases, including a timezone-aware format (%z), which was a real regression area (issue Validating datetime strings with timezone produces an error #9203).Testing note for community contributors. Pandas, sqlite, spark, postgres, and mysql run locally (some via Docker Compose); BigQuery, Snowflake, Databricks, and Redshift run only in maintainer CI with credentials. Write tests against the full target matrix, run what you can locally, and CI covers the cloud dialects. Run a single module locally by marker from the repo root, e.g.
pytest tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_match_strftime_format.py -m unit(pandas), or-m sqlite/-m spark(seeAGENTS.mdfor marker usage).