chore: Update summarize-test-suite skill to scope code-regression verdicts

[marcabreracast]

Description

Refines the summarize-test-suite skill (.agents/skills/summarize-test-suite/SKILL.md) so nightly summaries stop mislabeling recurring backend failures as code regressions while still catching genuine provider and API regressions.

Changes:

• Category 1 (code regression) no longer fires on symptom alone. Attribute-value mismatches, unexpected plan diffs, and INVALID_ATTRIBUTE are red only when they do not match a known-benign 3b signature or a category 4 propagation-lag mismatch.
• Adds sub-bucket 3b, Backend job / verify failures, enumerating three known benign signatures: LDAP CA-cert verify reporting FAILED, the acceptance-test sample-dataset load job reaching FAILED, and stale cluster/monitoring messages (Cannot validate cluster compatibility due to stale monitoring data, OPERATION_INVALID_SHARDS_NO_PRIMARY).
• Adds a symptom vs cause discipline rule: novel backend error patterns (a new error code, a wide break, or a previously passing suite now failing) stay category 1 (red) until proven benign, so API-side regressions remain visible to on-call.
• 3b failures are reported under the existing API errors count line, which the monthly summary parser already recognizes, so the monthly script is unchanged.

Observed in the Sep 9 and Sep 10 nightly Test Suite runs, where LDAP verify, online archive / search index / collection restore sample-data seed, and advanced cluster stale-monitoring failures were misclassified as code regressions.

Link to any related issue(s):

Type of change:

• Bug fix (non-breaking change which fixes an issue). Please, add the "bug" label to the PR.
• New feature (non-breaking change which adds functionality). Please, add the "enhancement" label to the PR. A migration guide must be created or updated if the new feature will go in a major version.
• Breaking change (fix or feature that would cause existing functionality to not work as expected). Please, add the "breaking change" label to the PR. A migration guide must be created or updated.
• This change requires a documentation update
• Documentation fix/enhancement

Required Checklist:

• I have signed the MongoDB CLA
• I have read the contributing guides
• I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
• I have added tests that prove my fix is effective or that my feature works per HashiCorp requirements
• I have added any necessary documentation (if appropriate)
• I have run make fix and verified my code
• If changes include deprecations or removals I have added appropriate changelog entries.
• If changes include removal or addition of 3rd party GitHub actions, I updated our internal document. Reach out to the APIx Integration slack channel to get access to the internal document.

Further comments

[augmentcode]

🤖 Augment PR Summary

Summary: Refines the nightly test-suite skill so known recurring backend job and verification failures are separated from provider/API regressions.
Changes: Adds API-error sub-bucket 3b, symptom-versus-cause guidance, and API-error template output while preserving the monthly parser's recognized category label.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 2 suggestions posted.

Comment augment review to trigger a new review at any time.

[Copilot]

🟢 Approval recommended

The change is documentation-only and low risk, with only a minor punctuation fix suggested in the updated emoji legend.

Pull request overview

Refines the internal .agents/skills/summarize-test-suite guidance to reduce false “code regression” (red) verdicts for recurring Atlas backend/verification failures, while keeping genuinely novel provider/API regressions classified as red for on-call visibility.

Changes:

• Tightens category 1 (code regression) classification so attribute/plan/INVALID_ATTRIBUTE symptoms are only red when they don’t match known-benign signatures or propagation-lag cases.
• Adds category 3b (“Backend job / verify failures”) with explicit recurring signatures and guidance to report them under the existing API errors count line.
• Updates discipline rules and Slack output templates to reflect the new 3b sub-bucket and symptom-vs-cause classification.

File summaries

File	Description
.agents/skills/summarize-test-suite/SKILL.md	Updates failure categorization rules and output templates to prevent recurring backend failures from being misclassified as code regressions.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[EspenAlbert]

Thank you for improving this.
One thing. Would it be better to have all the different known test instabilities in a single place? This way it is easier to add remove and we don't need exceptions spread out?