Skip to content

Commit

Permalink
Add an example for the target data validator.
Browse files Browse the repository at this point in the history
  • Loading branch information
holdenk committed Sep 8, 2023
1 parent 072f216 commit 1c8aab2
Showing 1 changed file with 30 additions and 0 deletions.
30 changes: 30 additions & 0 deletions target-validator/ex.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
detailedErrors: true
# We might have a large number of errors so just show the first 5
numErrorsToReport: 5

email:
smtpHost: smtp.example.com
subject: Data Validation Summary
from: [email protected]
to:
- [email protected]

tables:
- db: uk_data
table: adult
# Columns that taken together uniquely specifies each row (think of groupBy)
keyColumns:
- CompanyNumber
- EmployerId
- CompanyLinkToGPGInfo
- ResponsiblePerson
# Used to filter
condition: MaleBonusPercent >= FemaleBonusPercent
checks:
# We expect at least 500 records
- type: rowCount
minNumRows: 500
# We don't expect more than 1% not companies in the dataset.
- type: nullCheck
column: CompanyNumber
threshold: 0.01

0 comments on commit 1c8aab2

Please sign in to comment.