[tmm]

A kept zbugs-swift backend (ZBUGS_KEEP=1) now starts Postgres again when it stops. Set ZBUGS_PG_START to a command that starts it, for example docker compose -f apps/zbugs/docker/docker-compose.yml start. The backend runs it once Postgres has missed three checks 10 seconds apart, and again after each further three. This changes zbugs-swift only, so there's no changeset.

Cause

On tmm's Mac, zbugs was down from Nov 9 at 17:18 EST until about 13:33 the next day.

• Postgres was stopped: both Postgres containers got a fast shutdown request, which is what docker stop sends to the postgres image. That happened at 22:18:49 UTC on Nov 9, and again at 05:34:02 UTC on Nov 10, 8 s after restart: always had started them. Nothing restarted them after the second stop. Docker's restart: always only restarts a stopped container when Docker itself restarts, and Docker didn't restart until someone ran docker start by hand.
• zero-cache never exited: it logged 19,156 ECONNREFUSED and 57P03 retries, from the change-streamer, lag-reporter and reaper, with backoff capped at 10 s. So the restart-on-exit from fix(zbugs-swift): restart zero-cache in a kept backend when it exits #120 never ran. That restart only covers sleep, when zero-cache gives up on connect timeouts and exits with code 14.
• zero-cache needs no change: it catches up by itself once Postgres answers. It did so after OrbStack's 17:16 self-update, which kept Postgres down for 12 s, and again at 00:33:57.

We don't know who sent the stops. No Amp thread on the Mac ran Docker commands at either time, and tmm's shell history has none. The fix doesn't depend on that.

Reproduction and checks

These ran in an orb with a local Postgres on 6434, server.ts with ZBUGS_KEEP=1, and pg_ctl -m fast stop standing in for docker stop.

• On main: zero-cache retried every 10 s and never exited, and fix(zbugs-swift): restart zero-cache in a kept backend when it exits #120's restart didn't run. Once Postgres was started again, replication went back to OK with no restart, and zbugs' BackendTests (log in, create an issue and a comment) passed.
• With this change, ZBUGS_PG_START set to pg_ctl … start:
  • A 10 s outage, like the OrbStack update, didn't run the command, and zero-cache recovered by itself.
  • A sustained stop ran the command after about 26 s. Postgres answered, replication went back to OK, and BackendTests passed.
  • A failing start command, with the data directory moved away, was logged as ZBUGS_PG_START failed: …. It was retried after three more misses and succeeded once the directory was back.
• oxfmt --check and oxlint --type-aware pass on the changed files.
• CI's zbugs backend runs without ZBUGS_KEEP, so it's unaffected.

After merge

On tmm's Mac, ~/.zbugs-swift/start.sh gets ZBUGS_PG_START='docker compose -f apps/zbugs/docker/docker-compose.yml start'. Compose names the project docker after the file's directory, which is the project those two containers belong to. start only starts existing containers; it never recreates them.