Skip to content

Commit 962bfe8

Browse files
JohnMcLearclaude
andauthored
feat(updater): tier 4 — autonomous update in maintenance window (#7607) (#7753)
* docs(updater): plan tier 4 — autonomous update in maintenance window (#7607) Maps PR 4 of the auto-update design spec (§"Tier 4 — autonomous") to concrete files, tasks, and verification steps. Subsequent commits scaffold against this plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(updater): MaintenanceWindow module — wall-clock window math for tier 4 Pure module: parseWindow, inWindow, nextWindowStart. Supports tz=local|utc and cross-midnight ranges. Used by upcoming Scheduler + UpdatePolicy changes. 22 vitest unit tests cover format validation, same-day + cross-midnight boundaries, and host-local vs UTC clock comparisons. DST handling is absorbed by JS Date constructor's wall-clock normalization (documented in the file header). Refs #7607 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(updater): tier 4 backend — window-gated UpdatePolicy + Scheduler Wires MaintenanceWindow into the existing tier 3 backend so autonomous updates only fire while `now` is inside `updates.maintenanceWindow`. UpdatePolicy - new optional `maintenanceWindow` input - canAutonomous flips on only for git+tier=autonomous+parse-valid window - new reasons `maintenance-window-missing` / `maintenance-window-invalid` - rollback-failed still wins over window denial Scheduler - decideSchedule snaps scheduledFor forward to nextWindowStart when canAutonomous + grace lands outside the window - decideTriggerApply returns a new `{action: 'defer'}` when canAutonomous + fire-time is outside the window; carries nextStart for the runner - canAutonomous=false preserves Tier 3 behavior unchanged index.ts wires settings.updates.maintenanceWindow through both passes and re-arms the timer on defer. Status endpoint surface (nextWindowOpensAt) + admin UI picker land in a follow-up commit. Settings adds `maintenanceWindow: {start, end, tz} | null`, defaulting to null. settings.json.template / settings.json.docker document the shape. Tests - 22 vitest cases for MaintenanceWindow already cover the math - 4 new UpdatePolicy cases for the window outcomes - 6 new Scheduler cases for tier-4 schedule/trigger paths - Full backend-new suite: 629 passed (35 files) Refs #7607 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(updater): tier 4 admin UI — window status, deferred subtitle, banner GET /admin/update/status now returns: - `maintenanceWindow`: the parsed window object (admin sessions only) - `nextWindowOpensAt`: ISO of the next window opening when tier=autonomous UpdatePage - new "Maintenance window" section when tier=autonomous, shows current window summary + next opens at, or "Not configured" when unset - scheduled panel now appends a "deferred until <iso>" line when the backend has snapped scheduledFor to the next window opening UpdateBanner - new variant when tier=autonomous and policy.reason is `maintenance-window-missing` or `maintenance-window-invalid`, linking to /admin/update i18n - 8 new keys under `update.banner.*`, `update.page.policy.*`, `update.page.scheduled.*`, `update.window.*` (en.json only; translations follow via the usual locale workflow) Interactive picker is intentionally deferred — admins edit `updates.maintenanceWindow` via the parsed JSONC settings editor (#7709). A follow-up commit may add a thin write-through component if the JSONC round-trip turns out to be too rough for typical operators. Refs #7607 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(updater): tier 4 — window gate, DST notes, runbook §12 (#7607) CHANGELOG: flip Tier 4 from "designed, not yet implemented" to current. Document maintenanceWindow shape, snap-forward, defer-at-fire, and the two missing/invalid policy reasons. doc/admin/updates.md: new "Tier 4 — autonomous in a maintenance window" section with config example, policy gating, DST/timezone notes, admin UI behavior. runbook: §12 walks a disposable VM through missing-window, malformed, outside-window deferral, fire-at-opening, and window-closes-mid-grace. Adds five sign-off checklist items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(updater): tier 4 window-boundary integration (#7607) Mocha integration covering the four scenarios called out in the spec §"Tier 4 — autonomous": - outside-window: decideSchedule snaps scheduledFor forward to the next opening and the snapped value round-trips through saveState - inside-window at fire-time: decideTriggerApply returns fire - window-closes-mid-grace: decideTriggerApply returns defer with nextStart at the next opening; persisted state moves forward - cancel during deferred-grace: state returns to idle, and the next decideSchedule pass re-emits a schedule snapped to the next opening All 4 cases passing locally under tsx mocha. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(updater): real SMTP via nodemailer (mail.* settings) (#7607) Replaces the (would send email) stub introduced in PR #7601 with a nodemailer-backed transport. The dependency is lazy-imported so installs that don't set mail.host pay no runtime cost. Settings additions - new top-level mail block: host, port, secure, from, auth (user/pass) - mail.host=null keeps the legacy log-only behaviour; the Notifier still updates dedupe state so we don't re-evaluate every tick - settings.json.template documents the shape inline - settings.json.docker reads MAIL_HOST / MAIL_FROM / MAIL_PORT / MAIL_SECURE from env so operators can configure via container env Transport - lazy import('nodemailer') on first send - transport cached by host; settings reload picks up new host without needing a restart - send errors are swallowed (logged warn) so a transient SMTP failure can never poison the surrounding updater state machine - successful sends log at info; legacy "(would send email)" path remains the visible signal when mail is disabled Refs #7607 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(updater): preflight checks target tag's engines.node (#7607) Before mutating the working tree, runPreflight now reads the target tag's package.json via `git show <tag>:package.json` and verifies that process.versions.node satisfies its engines.node range. Failures land at preflight-failed cleanly (no rollback needed — nothing has changed yet). Motivation: a release that bumps the Node floor used to either fail mid-`pnpm install` (which then rolls back successfully) or restart on the new build and crash in the boot path (which then rolls back via the health-check timer). Both paths recover, but they burn a drain + restart cycle on a condition we can reject upfront. Implementation - new PreflightReason `node-engine-mismatch` - new dep `readTargetEnginesNode(tag)` — runs the git-show as a child process with stdio captured to a string; missing tag / missing file / malformed JSON / missing engines.node all resolve to null (treated as "no constraint, pass") - uses existing semver dep with includePrerelease: true - new PreflightInput field `currentNodeVersion`; threaded from process.versions.node in both wirings (scheduler + manual apply) - check runs *after* signature verification so we trust the package.json - PreflightResult carries an optional `detail` string; applyPipeline appends it to the lastResult.reason so the admin UI shows e.g. "node-engine-mismatch: target requires Node >=26.0.0, running 25.0.0" Tests: 6 new vitest cases (no engines.node, satisfies, fails below floor, caret range, loose-spaced range, ordering after signature). Full backend-new: 635 passed (was 629). Refs #7607 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(updater): email admin on auto-rollback / preflight-failed (#7607) Before this commit, only the terminal rollback-failed state emailed the admin. Auto-recovered failures (rolled-back-install-failed, rolled-back- build-failed, rolled-back-health-check, rolled-back-crash-loop) and pre- flight-failed surfaced only via the /admin/update banner — so a 3am autonomous update that failed because of, say, a Node engine bump would roll back silently and stay invisible until the admin next logged in. Notifier - new EmailKinds: 'update-preflight-failed', 'update-rolled-back', 'update-rollback-failed' - new pure decideOutcomeEmail(input) → {toSend, newState} - dedupe key `<outcome>:<targetTag>` in EmailSendLog.lastFailureKey: same outcome on same tag emits one email per cycle (kills retry-loop spam); a different outcome or different tag resets the key - rollback-failed always fires (terminal — overrides dedupe) - state.ts validator + loadState backfill the new field for legacy state files (Tier 1/2/3 installs upgrading in place) Wiring - new index.ts helper notifyApplyFailure() loads state, runs the pure notifier, sends (via the nodemailer-backed sendEmailViaSmtp from the previous commit), persists the new dedupe key — all best-effort - schedulerTriggerApply: fires on applyUpdate returning preflight-failed or rolled-back - /admin/update/apply HTTP handler: same - boot path in expressCreateServer: if state.lastResult is a failure outcome we haven't already emailed about, fire then. Covers: - health-check timeout rollback (timer expired between boots) - crash-loop forced rollback caught on a later boot - preflight-failed where the process didn't get to email before exit - unacknowledged rollback-failed terminal Tests - 8 new vitest cases for decideOutcomeEmail (adminEmail=null, each outcome's content, dedupe by tag, dedupe by outcome, rollback-failed bypass) - Full backend-new suite: 643 passed (was 635) Refs #7607 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(updater): address Qodo review on tier 4 - UpdatePage: only show "deferred until" subtitle when scheduledFor actually matches nextWindowOpensAt. The previous `scheduledFor > now + 60s` heuristic misfired during a normal in-window 15-min grace period. - applyPipeline: return the enriched preflight reason (`reason: detail`) instead of only `pf.reason`, so /admin/update/apply 409 bodies and failure-notify emails preserve diagnostics like the Node engine mismatch detail. - updater/index: key the cached nodemailer transport on the full set of SMTP options (host + port + secure + auth) so runtime changes to port/credentials via reloadSettings() invalidate the cache. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent dbd4662 commit 962bfe8

32 files changed

Lines changed: 1678 additions & 50 deletions

CHANGELOG.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,9 @@ Both clients hit the **stable 3.x API surface**, so server operators don't need
2929
- **Self-update subsystem — Tier 3 (auto with grace window).**
3030
- On a git install, set `updates.tier: "auto"` to have new releases applied automatically after `preApplyGraceMinutes`. During the grace window, `/admin/update` shows a live countdown plus Cancel and Apply now buttons. Schedules are persisted to `var/update-state.json`, so an Etherpad restart during the grace window rehydrates the timer instead of losing the schedule. A new release tag detected mid-grace re-arms the timer; if `adminEmail` is set, a one-shot `grace-start` notification fires per scheduled tag (issue #7607).
3131
- The terminal `rollback-failed` state continues to disable auto/autonomous attempts globally until acknowledged; manual click stays available because an admin click *is* the intervention the terminal state requires.
32-
- Tier 4 (autonomous in a maintenance window) remains designed but unimplemented and will land in a subsequent release.
32+
- **Self-update subsystem — Tier 4 (autonomous in a maintenance window).**
33+
- Set `updates.tier: "autonomous"` together with `updates.maintenanceWindow: {"start":"HH:MM","end":"HH:MM","tz":"local"|"utc"}` to constrain autonomous updates to a nightly window. The scheduler snaps `scheduledFor` forward to the next window opening when grace would otherwise land outside the window, and defers the fire when the window has closed by the timer callback. Cross-midnight windows (`end < start`) are supported; DST transitions are absorbed by the host's wall-clock arithmetic.
34+
- A missing or malformed window degrades the policy to Tier 3 with an explicit `policy.reason` of `maintenance-window-missing` / `maintenance-window-invalid`; an admin banner surfaces the misconfiguration so autonomous behavior is not silently disabled. Closes #7607.
3335
- **Privacy — drop swagger-ui telemetry, document phone-homes, add opt-outs.**
3436
- Dropped `swagger-ui-express` because upstream injects a Scarf analytics pixel that cannot be disabled at install or runtime (see [swagger-api/swagger-ui#10573](https://github.com/swagger-api/swagger-ui/issues/10573)). `/api-docs` now serves a vendored copy of [Scalar](https://github.com/scalar/scalar) (MIT) configured with `withDefaultFonts: false` and `telemetry: false` so no outbound calls are made.
3537
- New `privacy.updateCheck` (default `true`) — set to `false` to disable the hourly `UpdateCheck.ts` request to `${updateServer}/info.json`.

admin/src/components/UpdateBanner.tsx

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,23 @@ export const UpdateBanner = () => {
5252
);
5353
}
5454

55+
// Tier 4: tier is autonomous but the maintenance window isn't usable.
56+
// Surface that before the generic "update available" banner so the admin
57+
// knows the autonomous behavior is sitting idle.
58+
const policyReason = updateStatus.policy?.reason;
59+
if (updateStatus.tier === 'autonomous'
60+
&& (policyReason === 'maintenance-window-missing'
61+
|| policyReason === 'maintenance-window-invalid')) {
62+
return (
63+
<div className="update-banner update-banner-window" role="status">
64+
<strong>
65+
<Trans i18nKey={`update.banner.${policyReason}`}/>
66+
</strong>{' '}
67+
<Link to="/update">{t('update.banner.cta')}</Link>
68+
</div>
69+
);
70+
}
71+
5572
// Tier 3: scheduled update — show countdown banner instead of the plain
5673
// "update available" one.
5774
if (updateStatus.execution?.status === 'scheduled') {

admin/src/pages/UpdatePage.tsx

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,54 @@ export const UpdatePage = () => {
192192
values={{tag: scheduled.targetTag, remaining: fmtRemaining(remainingMs)}}
193193
/>
194194
</p>
195+
{/* Tier 4: only surface the deferral subtitle when `scheduledFor`
196+
was actually snapped forward to the next window opening. The
197+
backend keeps `scheduledFor = now + grace` whenever that lands
198+
inside the window, so we can't use a fixed time-distance
199+
heuristic (a normal 15-min grace would falsely match). Instead,
200+
compare against `nextWindowOpensAt` with a small tolerance — the
201+
two are computed seconds apart at request time, so an exact-ish
202+
match is the only safe signal that the schedule was deferred. */}
203+
{us.tier === 'autonomous' && us.nextWindowOpensAt
204+
&& Math.abs(new Date(scheduled.scheduledFor).getTime()
205+
- new Date(us.nextWindowOpensAt).getTime()) < 60 * 1000 && (
206+
<p className="update-scheduled-deferred">
207+
<Trans
208+
i18nKey="update.page.scheduled.deferred_until"
209+
values={{at: us.nextWindowOpensAt}}
210+
/>
211+
</p>
212+
)}
213+
</section>
214+
)}
215+
216+
{us.tier === 'autonomous' && (
217+
<section className="update-maintenance-window">
218+
<h2><Trans i18nKey="update.window.title"/></h2>
219+
{us.maintenanceWindow ? (
220+
<>
221+
<p>
222+
<Trans
223+
i18nKey="update.window.summary"
224+
values={{
225+
start: us.maintenanceWindow.start,
226+
end: us.maintenanceWindow.end,
227+
tz: us.maintenanceWindow.tz,
228+
}}
229+
/>
230+
</p>
231+
{us.nextWindowOpensAt && (
232+
<p>
233+
<Trans
234+
i18nKey="update.window.next_opens_at"
235+
values={{at: us.nextWindowOpensAt}}
236+
/>
237+
</p>
238+
)}
239+
</>
240+
) : (
241+
<p><Trans i18nKey="update.window.unset"/></p>
242+
)}
195243
</section>
196244
)}
197245

admin/src/store/store.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,12 @@ export type LastResult = null | {
2525
at: string;
2626
};
2727

28+
export interface MaintenanceWindow {
29+
start: string;
30+
end: string;
31+
tz: 'local' | 'utc';
32+
}
33+
2834
export interface UpdateStatusPayload {
2935
currentVersion: string;
3036
latest: null | {
@@ -44,6 +50,9 @@ export interface UpdateStatusPayload {
4450
execution: Execution;
4551
lastResult: LastResult;
4652
lockHeld: boolean;
53+
// Tier 4 additions:
54+
maintenanceWindow: MaintenanceWindow | null;
55+
nextWindowOpensAt: string | null;
4756
}
4857

4958
type ToastState = {

doc/admin/updates.md

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Etherpad ships with a built-in update subsystem.
55
- **Tier 1 (notify)** — default. A banner appears in the admin UI when a new release is available, and pad users see a discreet badge if the running version is severely outdated or flagged as vulnerable. No execution.
66
- **Tier 2 (manual click)** — admins on a git install can click "Apply update" at `/admin/update`. Etherpad drains active sessions, runs `git fetch / checkout / pnpm install / pnpm run build:ui`, and exits with code 75 so a process supervisor restarts it on the new version. Auto-rolls back on failure.
77
- **Tier 3 (auto with grace window)** — opt-in. On a git install, a newly detected release transitions execution state to `scheduled` and is applied after `preApplyGraceMinutes`. During the grace window, `/admin/update` shows a live countdown plus Cancel and Apply now buttons; an admin email (if `adminEmail` is set) fires once per scheduled tag.
8-
- **Tier 4 (autonomous in maintenance window)**designed, not yet implemented.
8+
- **Tier 4 (autonomous in maintenance window)**opt-in. Tier 3 + `updates.maintenanceWindow` is required; the scheduler only fires while the wall clock is inside the configured window. Updates detected outside the window queue for the next opening.
99

1010
## Settings
1111

@@ -192,3 +192,43 @@ The right way to give docker admins an in-product Apply button is to delegate to
192192
- **Deploy webhook.** New setting `updates.dockerWebhook`. When set, the Apply button on a docker install POSTs to the configured URL and trusts the orchestrator (Render / Railway / Fly / Portainer / Coolify / GitHub Actions — they all expose redeploy webhooks) to do the actual pull-and-recreate.
193193

194194
Direct Docker-socket access (mount `/var/run/docker.sock` into the container) is **out of scope** — anyone who escapes the Etherpad process via that socket gets root on the host. Admins who want fully autonomous docker updates should run [Watchtower](https://containrrr.dev/watchtower/) alongside Etherpad rather than bake equivalent privilege into Etherpad itself.
195+
196+
## Tier 4 — autonomous in a maintenance window
197+
198+
Tier 4 layers a wall-clock window on top of Tier 3 so autonomous updates only run while it is safe to drain sessions (typically nightly).
199+
200+
To enable, on a git install:
201+
202+
```jsonc
203+
{
204+
"updates": {
205+
"tier": "autonomous",
206+
"preApplyGraceMinutes": 15,
207+
"maintenanceWindow": { "start": "03:00", "end": "05:00", "tz": "local" }
208+
}
209+
}
210+
```
211+
212+
`start` and `end` are 24-hour `HH:MM` wall-clock times in the configured `tz` (`"local"` or `"utc"`). `end` is exclusive; `end < start` denotes a cross-midnight window (`22:00–02:00` runs from 22:00 through 01:59).
213+
214+
### How the window gate works
215+
216+
1. `evaluatePolicy` returns `canAutonomous: true` only when the install is `git`, tier is `"autonomous"`, no terminal `rollback-failed` is set, and `updates.maintenanceWindow` is set and parse-valid. Missing/malformed windows return `canAutonomous: false` with `policy.reason` equal to `maintenance-window-missing` / `maintenance-window-invalid`, and the rest of the policy degrades to Tier 3 (`canAuto: true`). An admin banner surfaces the misconfiguration so the autonomous behavior is never silently disabled.
217+
2. When the scheduler picks up a new release while `canAutonomous: true`, it computes `scheduledFor = now + preApplyGraceMinutes`. If that timestamp falls **outside** the window, it is snapped forward to the **next opening** of the window.
218+
3. When the timer fires, the scheduler re-checks the clock. If the window has already closed (long grace, clock skew, host suspend), the fire is **deferred**: `var/update-state.json` is updated with a new `scheduledFor` pointing at the next opening, the timer is re-armed, and the actual apply runs at the next valid moment.
219+
220+
### DST and timezone notes
221+
222+
- `tz: "utc"` is recommended for hosts running across DST boundaries — the window is interpreted against the same wall clock every day of the year.
223+
- `tz: "local"` follows the host's local time. On DST spring-forward days, a window starting at a non-existent local time (e.g. `02:30` in `America/New_York` on the second Sunday of March) silently lands at the next valid wall-clock minute via the host JS `Date` constructor's normalization. On fall-back days, the first occurrence of the wall-clock start time is used.
224+
- Cross-midnight windows (`end < start`) span at most 24 hours; longer "windows" should be split into two settings, e.g. by running Tier 3 instead.
225+
226+
### Admin UI
227+
228+
`/admin/update` shows a "Maintenance window" section when `updates.tier == "autonomous"`:
229+
230+
- Configured: summary `HH:MM–HH:MM (tz)` plus "Next window opens at …".
231+
- Not configured: a clear "Not configured" message and a top-of-page banner that links back to the page.
232+
- During a deferred-grace schedule, the scheduled panel shows both the countdown to `scheduledFor` and an explanatory "Outside maintenance window. Update will start when the window opens at …" line.
233+
234+
Admins edit `updates.maintenanceWindow` via the parsed JSONC settings editor at `/admin/settings`. Saving an invalid shape is caught at boot — the warning is logged via the `updater` log4js category and the policy downgrades to Tier 3.

0 commit comments

Comments
 (0)