Delete rolled-up daily archives from database and S3

[Copilot]

Daily archive records that have been rolled up into monthlies and had their source data deleted serve no purpose. They accumulate in the archives_archive table indefinitely, and their S3 files consume storage space.

Implementation

• Added DeleteRolledUpDailyArchives() to purge daily archives where rollup_id IS NOT NULL AND deleted_on IS NOT NULL
• Queries archives first to retrieve their S3 locations before deletion
• Deletes S3 files for archives that were uploaded (when Location is set)
• Uses single batch DELETE for database records for efficiency
• Called in ArchiveOrg() after source record deletion completes
• Added DeleteS3File() helper function in s3.go for S3 object deletion
• Added test coverage for the deletion workflow

Example

// Deletes all daily archives that have been rolled up and had records deleted
// Also deletes their S3 files if they were uploaded
deletedCount, err := DeleteRolledUpDailyArchives(ctx, rt, org, MessageType)

The function identifies and removes daily archive metadata records and their S3 files while preserving the monthly rollups that contain the actual archived data. S3 deletion errors are logged but don't block database cleanup to ensure consistency.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Delete dailies once rolled up into monthlies</issue_title>
<issue_description>These are currently kept around but don't serve any purpose. See TODO in ArchiveOrg</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Delete dailies once rolled up into monthlies #132

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.03%. Comparing base (ddc57d6) to head (9203da2).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #141   +/-   ##
=======================================
  Coverage   58.03%   58.03%           
=======================================
  Files           7        7           
  Lines         908      908           
=======================================
  Hits          527      527           
  Misses        276      276           
  Partials      105      105           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[rowanseymour]

@copilot needs to delete any uploaded files from S3 for the archive rows being deleted. If Location is set then it has the bucket:key of the uploaded file.

[Copilot]

@copilot needs to delete any uploaded files from S3 for the archive rows being deleted. If Location is set then it has the bucket:key of the uploaded file.

Added S3 file deletion in commit 0b83789. The function now:

1. Queries archives to get their Location values
2. Deletes S3 files for archives with Location set (bucket:key format)
3. Deletes database records

S3 deletion errors are logged but don't block the database cleanup to ensure consistency.

[rowanseymour]

Repalced by #143