security: implement group 5 - monitoring and logging infrastructure

[rshade]

Monitoring & Logging Infrastructure

Reference: security-groups/group-5-monitoring-logging.md

Tasks

1. Centralized log storage — KMS-encrypted GCS bucket with lifecycle policies (7-year retention for compliance)
2. Log sinks — route security events from Cloud SQL, GKE, Compute, and IAM to centralized storage and BigQuery
3. Prometheus/Grafana deployment (optional) — monitoring stack with security dashboards
4. Security alerting rules — alerts for error rates, PSA violations, unauthorized API access, database connection failures
5. Real-time log analysis — Cloud Function + Pub/Sub pipeline for critical security events

Important Notes

• All credentials (Grafana admin password, SMTP config, Slack webhooks, email addresses) must use config.requireSecret() or config.require() — never hardcode
• Alert names should reference Pod Security Admission, not deprecated PodSecurityPolicy

Acceptance Criteria

• Security logs centralized and encrypted
• Log retention policies comply with requirements
• Monitoring and alerting operational
• No hardcoded credentials in configuration