Skip to content

Persist the nightly claims-reverify report to S3 for DWH ingestion (low priority) #20965

Description

@CamSoper

Priority: low — backlog item from the 2026-08-18 DWH ingestion audit; do after content-review ingestion is healthy again (pulumi/data#949).

Problem

The nightly claims-reverify lane (claims-reverify.yml) produces .claims-reverify-report.json — the verdict trail for every volatile entity it re-checks, including the demotion fields added in #20851 (demoted_from per entity, n_demoted in meta). Today it survives only as a GitHub Actions artifact with 90-day retention (#20893 fixed the upload).

Only its effects reach S3: stale_claims markers written into ledger/<slug>.json and the reverify signal in health/state.json. The report itself — the accuracy-regression signal pulumi/data#921 wanted ("verdict mix by model is how we'd measure a model swap's effect") — has no durable home, so entities that re-verify fresh or inconclusive, and the demotion trail, are invisible to the warehouse.

Proposed work

  1. Upload each run's report to the content-review ledger bucket under an immutable date-partitioned prefix, e.g. reverify/runs/<YYYY-MM-DD>.json (the report already carries schema_version: 1 and checked_at).
  2. Shape is already warehouse-friendly: {schema_version, checked_at, entities: [{entity_key, verdict, demoted_from, confidence, evidence, source, route, error, pages: [{path, slug}]}], meta: {n_snapshots, n_entities, n_due, n_checked, n_stale, n_fresh, n_inconclusive, n_demoted, skipped?}}.
  3. The bucket's DWH read policy already covers all prefixes — once objects exist, file a follow-up ingestion request in pulumi/data (it should join to the claims index on entity_key).

Metadata

Metadata

Assignees

Labels

area/automationIssues relating to automation (mostly GHA workflows) and automatically generated contentkind/task

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions