DATATEAM-463

[knockknockyoo]

Feature(data workflow): configure automated data promotion workflow

Summary

• Configure automated data promotion from QA → Stage → Production
• Integrate check-promotion-upper deployment for multi-stage validation
• Update promotion schedule to weekly (Saturday 12:00 PM EST)

Changes

• Updated prefect_c1_prod.yaml with enhanced check-promotion-upper description
• Modified .github/workflows/auto_data_promotion_c1_qa_to_stage_prod.yml:
  • Integrated check-promotion-upper for pre and post-promotion checks

Dependencies

• check-promotion-upper deployment must be deployed in Prefect
• config/sync_status.yml tracking for model detection

DATATEAM-463

the auto_data_promotion workflow

1. Automate the data promotion from CloudOne QA to CloudOne Stage and Production
2. add the validation steps and send the message to the Slack channel.

[Copilot]

Pull request overview

This PR adds an automated data promotion workflow from CloudOne QA to Stage and Production environments, along with the necessary Prefect deployment configuration and sync status tracking.

Changes:

• Added a new GitHub Actions workflow (.github/workflows/auto_data_promotion_c1_qa_to_stage_prod.yml) that runs weekly on Saturday at 3:00 AM EST to detect updated models, validate QA readiness, export QA data to S3, import to Stage and Prod, verify each promotion step, and send Slack notifications.
• Added the check-promotion-upper Prefect deployment in prefect_c1_prod.yaml and added the missing work_pool block for the existing update-c1-upper deployment.
• Added last_promoted_sha_stage_prod tracking key to config/sync_status.yml for tracking the last promoted commit in the QA → Stage → Prod workflow.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
.github/workflows/auto_data_promotion_c1_qa_to_stage_prod.yml	New workflow automating QA → Stage → Prod promotion with model detection, validation, S3 export/import, and Slack notifications
prefect_c1_prod.yaml	Added check-promotion-upper deployment and work_pool for update-c1-upper
config/sync_status.yml	Added last_promoted_sha_stage_prod tracking field

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[David-YuWei]

@knockknockyoo let's shift the scheduled time from 3am to 12pm on Saturday. Thanks!

[knockknockyoo]

@knockknockyoo let's shift the scheduled time from 3am to 12pm on Saturday. Thanks!

@David-YuWei

1. I have re-scheduled time from 3am to 12pm on Saturday for stage/prod. Also rescheduled 3am for dev/qa.
2. Included the prune job to remove the pre-release.
3. Updated the log message on stage/prod running on prefect.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[David-YuWei]

PR looks good to me!