chore(pr): add PR template with "New runtime I/O" + online/offline test split - #89
Open
antkawam wants to merge 2 commits into
Open
chore(pr): add PR template with "New runtime I/O" + online/offline test split#89antkawam wants to merge 2 commits into
antkawam wants to merge 2 commits into
Conversation
….9.1) PR #84 (per-endpoint per-model AIP overrides, v1.9.0) shipped with two gaps in the CDK task role policy: 1. `bedrock:GetInferenceProfile` action was never granted. The action is required by the new health-loop union (resolves every AIP override per endpoint), the auto-migration on startup (introspects legacy `inference_profile_arn` ARNs to derive the foundation model), and the compat shim on `POST /admin/endpoints` (same path). 2. The `bedrock:InvokeModel*` resource scope only included system inference profiles (`inference-profile/*anthropic.claude*`), not user-created AIPs (`application-inference-profile/*`). So even with GetInferenceProfile granted, dispatching a request through an AIP override would fail with AccessDeniedException at InvokeModel time. Symptom on prod: adding any AIP override caused that endpoint's health check to fail (GetInferenceProfile → AccessDenied), the endpoint got marked unhealthy, and team requests routed to it fell back to the default routing path — the override silently never applied. The deployed prod task role was inspected via `aws iam get-role-policy` to confirm. docs/endpoints.md already lists GetInferenceProfile in the required-permissions table from PR #84; the public docs were correct, the CDK was the gap. Patch bumps the gateway and CLI to 1.9.1.
…st split PR #84 (v1.9.0) shipped a CDK IAM gap that made AIP overrides silently no-op — request returned 200, dispatch fell back to CRI. The bug class: a change introduces new runtime I/O (AWS API call, env var, network reach, schema) that integration tests can't see because the boundary is mocked AND the failure mode at the API surface is indistinguishable from happy path. This PR template forces every change to enumerate its new runtime I/O and split the test plan into offline (mocks/CI) vs online (deployed environment with recorded evidence). The reviewer gate is: if the doc adds a "Required permissions" entry but `infra/stack.ts` doesn't grant it (or vice versa), block. Companion changes (local, in `.claude/`): - `.claude/CLAUDE.md` — Release Evidence Gate section - `.claude/skills/spec/SKILL.md` — `[online]` vs `[offline]` criteria - `.claude/skills/deploy/SKILL.md` — evidence-capture step on --staging, gate on --prod-release - `.claude/deploys/` — evidence ledger directory + format README
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.github/pull_request_template.mdwith mandatory "New runtime I/O" section and offline/online test-plan split..claude/-side discipline changes (Release Evidence Gate in.claude/CLAUDE.md,[online]/[offline]criteria in/spec, evidence-capture step in/deploy).New runtime I/O
Pure documentation / process change.
Why
PR #84 (v1.9.0) shipped a CDK IAM gap that made AIP overrides silently no-op. The bug class: change introduces a new runtime I/O the IaC/config doesn't grant; integration tests mock the boundary; the failure mode is a happy 200 response. Three release gates (CI, staging, prod) passed it through because none observed the new I/O against the deployed environment.
This template forces every PR to enumerate the runtime I/O it introduces and to split tests into offline (mocks) vs online (deployed environment, recorded evidence). The doc-IaC contract becomes a reviewer gate: a permission listed in
docs/but not granted ininfra/stack.tsis an automatic block.Test plan
Offline:
.github/pull_request_template.mdgh pr create(the file path is the documented convention)Online: