test: add read-after-write verification to BDD scenarios

[millerjp]

Problem

Many BDD scenarios that perform write operations (create, update, delete) only check the HTTP response status code and/or response body. They do not follow up with a read (GET) to verify the change was actually persisted to the database.

This means a bug where the API returns 200 but fails to persist the change (e.g., transaction rollback, storage error swallowed, cache-only write) would pass all tests.

Scope

Across 832 write-operation scenarios in 197 feature files (out of 3,069 total scenarios):

• 218 scenarios (26%) have proper read-after-write verification (GET + field assertions)
• 374 scenarios (45%) have partial verification (GET without field assertions, or field assertions on the write response only)
• 240 scenarios (29%) have NO read-after-write verification at all

614 scenarios (74%) need improvement.

---

Gap Analysis by Area

REST API — High-Gap Areas

Admin Users (admin_users.feature) — 53% gap

Pattern	Count	Examples
GOOD (write + GET + assert)	3	"List users includes created users", "Get user by ID"
PARTIAL (response-only check)	5	"Create a user as admin" — asserts response fields, no subsequent GET
GAP (no read-back)	9	"Delete user" — 204 response only, no GET to verify deletion persisted

What's needed: After POST /admin/users, follow with GET /admin/users/{id} and assert all fields (username, email, role, enabled). After DELETE, follow with GET and assert 404. After PUT (update role, disable), follow with GET and assert changed fields.

Admin API Keys (admin_apikeys.feature) — 61% gap

Pattern	Count	Examples
GOOD	3	"List API keys", "Get API key by ID", "Authenticate with created key"
PARTIAL	4	"Create an API key" — response checked, no independent GET
GAP	11	"Rotate API key" — no verification old key is revoked; "Delete API key" — 204 only

What's needed: After create, GET the key and assert all fields. After rotate, GET and verify new key prefix + old key disabled. After revoke, GET and verify enabled: false. After delete, GET and assert 404.

DEK/KEK Registry (dek_registry.feature) — ~55% gap

Pattern	Count	Examples
GOOD	15	"Create KEK with all fields" + GET, "Undelete KEK" + GET
PARTIAL	5	"Create KEK with minimal fields" — no GET to verify defaults persisted
GAP	25-30	"Update KEK" — PUT response checked, no subsequent GET; "Soft-delete KEK" — 204 only

What's needed: Every KEK/DEK create MUST be followed by GET verifying all fields including defaults. Every update MUST be followed by GET asserting changed fields. Every delete MUST be followed by GET verifying 404 (or ?deleted=true for soft-delete visibility). Every undelete MUST verify the entity is accessible again via GET.

Exporters (exporters.feature) — ~75% gap

Pattern	Count	Examples
GOOD	3	"List exporters after creating several"
PARTIAL	2	"Create exporter with all fields" — response checked, no GET
GAP	18-20	All create variants (minimal, contextType, filters), update, delete, pause/resume/reset

What's needed: After create, GET the exporter and assert all fields (name, contextType, subjects, config). After pause/resume/reset, GET status and assert state changed. After delete, GET and assert 404.

Compare-and-Set (compare_and_set.feature) — 77% gap

Pattern	Count	Examples
GOOD	3	"Sequential CAS registration" — GET /versions verifies [1,2,3]
GAP	10	Most CAS success scenarios only check response, no GET to verify version was actually persisted

What's needed: After each CAS registration, GET the version back and verify the schema content + version number + metadata.

REST API — Lower-Gap Areas (still need fixes)

Area	File	GOOD	GAP	Notes
Config/Mode	configuration.feature, mode_management.feature	20	6	Generally well-tested; gaps in error-adjacent scenarios
Deletions	deletion*.feature	18	4	Good pattern overall; some soft-deletes don't verify ?deleted=true
Schema Registration	schema_registration.feature	4	3	First-registration scenarios lack GET
Schema Import	schema_import.feature	4	0	Excellent — all imports verified via GET
Account Self-Service	account_self_service.feature	2	1	Password change verified via re-auth; minor gap
Metadata/RuleSets	metadata_rulesets.feature	4	1	Good; minor gap in identity test

MCP Tool Scenarios

File	GOOD	PARTIAL	GAP	Key gaps
mcp_schema_write.feature	0	2	2	delete_subject and delete_version — no read to confirm deletion
mcp_admin.feature	7	0	2	change_password — no auth verification; rotate_apikey — no state check
mcp_exporter.feature	3	1	3	delete_exporter, reset_exporter, pause/resume — no state verification
mcp_dek.feature	5	0	1	delete_dek — no list to verify absence
mcp_config.feature	6	0	0	Excellent
mcp_encryption_lifecycle.feature	8	0	0	Excellent
mcp_kms_e2e.feature	15	0	0	Excellent — gold standard

Context Features — No gaps

All 14 context feature files + 2 MCP context files have comprehensive write → read → verify patterns across 153+ scenarios. No action needed.

---

Acceptance Criteria

Pattern to follow

Every write operation scenario MUST follow this pattern:

Create:

When I POST "/resource" with body '{"name": "test"}'
Then the response status should be 200
# READ-BACK VERIFICATION:
When I GET "/resource/{id}"
Then the response status should be 200
And the response field "name" should be "test"
And the response field "enabled" should be "true"   # verify defaults too

Update:

When I PUT "/resource/{id}" with body '{"name": "updated"}'
Then the response status should be 200
# READ-BACK VERIFICATION:
When I GET "/resource/{id}"
Then the response status should be 200
And the response field "name" should be "updated"

Delete:

When I DELETE "/resource/{id}"
Then the response status should be 204
# READ-BACK VERIFICATION:
When I GET "/resource/{id}"
Then the response status should be 404

Soft-delete:

When I DELETE "/resource/{id}"
Then the response status should be 200
# READ-BACK VERIFICATION — absent from normal list:
When I GET "/resources"
Then the response should not contain "test"
# READ-BACK VERIFICATION — visible with deleted flag:
When I GET "/resources?deleted=true"
Then the response should contain "test"

Specific deliverables

• Admin Users (admin_users.feature): Add GET verification after all create, update, delete operations (~9 scenarios)
• Admin API Keys (admin_apikeys.feature): Add GET verification after create, update, delete, revoke, rotate (~11 scenarios)
• DEK/KEK Registry (dek_registry.feature): Add GET verification after all CRUD operations (~25 scenarios)
• Exporters (exporters.feature): Add GET verification after create, update, delete, pause/resume/reset (~18 scenarios)
• Compare-and-Set (compare_and_set.feature): Add GET verification after CAS registrations (~10 scenarios)
• Schema Registration (schema_registration.feature): Add GET verification for first-registration scenarios (~3 scenarios)
• Deletions (deletion*.feature): Add ?deleted=true verification for soft-delete scenarios (~4 scenarios)
• MCP Schema Write (mcp_schema_write.feature): Add read tools after delete tools (~2 scenarios)
• MCP Admin (mcp_admin.feature): Add verification for password change and key rotation (~2 scenarios)
• MCP Exporter (mcp_exporter.feature): Add state verification after delete/pause/resume/reset (~3 scenarios)
• MCP DEK (mcp_dek.feature): Add list verification after DEK delete (~1 scenario)

What to assert on read-back

Don't just check status 200 — assert the actual field values:

• Users: username, email, role, enabled
• API Keys: name, role, enabled, keyPrefix (after rotate: new prefix)
• KEK: name, kmsType, kmsKeyId, doc, shared, deleted
• DEK: subject, algorithm, version, encryptedKeyMaterial
• Exporters: name, contextType, subjects, config, status (after pause/resume)
• Schemas: schemaType, schema content, version, id
• Config/Mode: compatibility level, mode value

---

Estimated scope

240 scenarios need read-after-write verification added (currently have none).
374 scenarios need verification strengthened (have partial coverage — either a GET without field assertions, or assertions only on the write response).

Full gap breakdown by file (top 20)

File	Gaps	Partial	Write scenarios	Notes
auth_basic.feature	19	4	23	Auth writes, no read-back
auth_ldap.feature	19	3	22	Auth writes, no read-back
auth_jwt.feature	18	2	21	Auth writes, no read-back
auth_oidc.feature	18	2	21	Auth writes, no read-back
contexts_global_config.feature	18	0	22	Config inheritance chains
auth_mtls.feature	13	2	16	Auth writes, no read-back
audit_rest.feature	12	0	12	Audit-focused, writes have zero read-back
reserved_fields.feature	10	1	11	Reserved field handling
auth_flows.feature	9	2	11	Cross-auth-method flows
ruleset_validation.feature	9	15	30	RuleSet validation (mostly partial)
advanced_features.feature	6	6	15	Mixed feature tests
dek_registry.feature	6	41	72	Heavy partial — 41 scenarios need field asserts
contexts_config_mode.feature	5	0	9	Config/mode per context
exporters.feature	5	21	31	Exporter CRUD (21 partial)
import_mode_comprehensive.feature	5	37	43	Import scenarios (37 partial)
rest_schema_analysis.feature	5	9	14	Analysis endpoints
rest_subject_diff_evolve.feature	5	7	12	Schema diff/evolution
compare_and_set.feature	4	7	14	CAS registration
mode_exhaustive.feature	4	7	13	Mode enforcement
rest_schema_search.feature	4	12	16	Schema search

Gold standard examples to follow

• mcp_kms_e2e.feature — 15/15 scenarios have full write→read→assert patterns
• schema_import.feature — 4/4 imports verified via GET with content assertion
• contexts_isolation.feature — 13/13 scenarios verify both write persistence and cross-context isolation
• delete_global_config.feature — every set→delete→get chain fully verified

[millerjp]

Review findings — not implemented

Verified against feature/mcp:

No systematic "read-after-write" verification pattern exists in the BDD step definitions or feature files. While many tests register a schema and then read it back ad-hoc, there is no:

• Reusable step definition like "the registered schema should be retrievable"
• Systematic verification that every write operation's result can be read back
• Consistency verification between the write response and subsequent read

Some tests do this implicitly (e.g., auth_ldap.feature registers then reads), but it's not a deliberate verification pattern applied consistently.

Issue remains valid and open.