Skip to content

Warn when a shared_secret_grants target still holds the generated placeholder password #421

Description

@justin808

Summary

When a shared_secret_grants target secret still holds the placeholder password that cpflow generate ships in templates/postgres.yml, every review-app deploy fails deep in the release phase with a bare Postgres authentication error. Nothing in the grant, policy, binding, or template is wrong, so the failure is very hard to attribute.

This is a request for an early, actionable diagnostic — not a bug report. cpflow behaved correctly throughout the investigation described below.

What happened downstream

In shakacode/react-on-rails-starter-tanstack, review apps share one staging Postgres via:

shared_secret_grants:
  - name: database
    secret_name: <app>-staging-pg
    policy_name: <app>-staging-database-policy

<app>-staging-pg was created by cpflow setup-app from the generated postgres.yml template, whose committed default is:

data:
  password: the_password #Replace this with a real password
  username: <app>

That placeholder was never replaced. The persistent staging app was independently hand-patched to carry the real password as an inline literal, so staging kept working while every review app failed — which made the failure look intermittent or external.

Every deploy failed identically:

PG::ConnectionBad: FATAL:  password authentication failed for user "<app>"
ERROR: Failed to run release script.  → exit code 64

Why it took so long to attribute

  1. All config inspection passes. Grant, placeholder substitution, policy target, and binding are all correct — only the stored value is wrong, and that is invisible to config review.

  2. The reveal binding is applied and released correctly. Polling every 10s during a real deploy: EMPTYreveal -> <app>-review-pr-205-identity at 02:36:27ZEMPTY at 02:44:42Z. The release script failed at 02:40:13Z, inside that window. cpflow does exactly what its docs describe.

  3. Verifying the credential the obvious way gives a false pass. The stock postgres image ships:

    host  all  all  127.0.0.1/32   trust
    host  all  all  all            scram-sha-256
    

    so psql -h 127.0.0.1 inside the container authenticates with any password. Only a non-loopback test exercises scram-sha-256. This trap is easy to fall into when debugging a shared-database setup.

Several rounds of investigation concluded "external authorization boundary, no repo-owned remediation" before the real cause was found.

Suggested improvement

Warn when a shared_secret_grants target secret still contains the generator's placeholder value — ideally during setup-app / deploy-image, before the release phase runs:

WARNING: shared_secret_grants entry 'database' targets secret '<app>-staging-pg',
whose 'password' field still contains the generated placeholder 'the_password'.
Review apps will fail database authentication until this is replaced.

A literal-value check against the shipped placeholder is cheap and needs no database connectivity. Optional extras:

  • Mention the loopback-trust pitfall in docs/secrets-and-env-values.md, so operators verifying a shared credential test over the non-loopback path.
  • Note in the shared-secret docs that reusing an app-scoped <app>-pg secret as a shared grant target inherits the generator placeholder.

Environment

  • cpflow 5.2.0, reusable workflow @v5.2.0
  • Downstream repo: shakacode/react-on-rails-starter-tanstack (issue Add timeout for runner jobs #194)
  • Control Plane shared Postgres in a separate GVC, inboundAllowType: same-org

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions