Skip to content

REQUIREMENTS.md

Latest commit

 

History

History
259 lines (171 loc) · 5.92 KB

REQUIREMENTS.md

File metadata and controls

259 lines (171 loc) · 5.92 KB
Raw

KMS-Backed Encryption Extension for PostgreSQL

Requirements Document (MVP + Roadmap)

1. Overview

1.1 Purpose

This document defines functional and non-functional requirements for a PostgreSQL extension that provides modern, KMS-backed column-level encryption. The implementation uses:

  • A Rust cryptographic core,
  • Exposed to PostgreSQL via a thin C shim,
  • Allowing compatibility with PostgreSQL versions beyond what pgx currently supports.

The goal is to make secure, authenticated column encryption simple and maintainable for enterprise use cases.

1.2 Scope

In-scope (overall project):

  • Column-level encryption & decryption UDFs
  • Envelope encryption with external KMS
  • Key rotation with versioned crypto schemes
  • Catalog tables for metadata
  • A standalone Rust crypto engine invoked through a C shim

Out-of-scope:

  • Transparent tablespace/WAL encryption
  • Replacing PostgreSQL authentication or RLS
  • Graphical user interfaces
  • Implementing new KMS systems

2. Goals and Non-Goals

2.1 Primary Goals

  • Simple adoption via SQL functions.
  • Strong security posture (modern, authenticated encryption; no DIY crypto).
  • Safe and secure-by-default.
  • Versioned crypto schemes for long-term maintenance.
  • Future-proof PostgreSQL version compatibility via C shim.
  • Easy local testing using a KMS backend that runs on a laptop.

2.2 Non-Goals

  • Fully transparent TDE.
  • Acting as a full enterprise DAM/DLP product.
  • Full IAM/RBAC replacement.
  • Zero-configuration magic encryption.

3. Stakeholders and Use Cases

3.1 Stakeholders

  • Application developers
  • DBAs and SREs
  • Security & compliance teams

3.2 Representative Use Cases

  • Encrypting PII (email, phone numbers, SSNs) at rest
  • Using Vault/KMS keys instead of in-app key management
  • Rotating keys or algorithms without breaking applications
  • Supporting multi-application or multi-tenant deployments

4. System Context and Constraints

4.1 Target Environment

  • PostgreSQL versions 13–18
  • Linux (primary), other OS best-effort
  • Shared library extension

4.2 Tech Stack

  • Rust library (cdylib or staticlib): crypto, KMS integration, envelope logic
  • C shim: PostgreSQL-facing wrapper (SQL-callable functions)
  • Build: Cargo + PGXS/CMake

4.3 MVP KMS Backend

HashiCorp Vault Transit Engine

Chosen because:

  • Popular in enterprises
  • Easy to run locally (Docker or dev mode)
  • Clean encrypt/decrypt API
  • Compatible with envelope encryption workflows

KMS integration is abstracted to allow AWS KMS, GCP KMS, Azure Key Vault later.

5. MVP Requirements

5.1 Functional Requirements (MVP)

F1. Basic Encryption/Decryption UDFs

Provide SQL functions:

  • encrypt_column(plaintext anyelement, key_alias text) RETURNS bytea
  • decrypt_column(ciphertext bytea) RETURNS anyelement

Requirements:

  • Support baseline types: text, varchar, bytea, int4, int8, numeric
  • Always use AEAD (AES-256-GCM or XChaCha20-Poly1305)
  • Include AEAD AAD: table, column, optional tenant/app ID

F2. Metadata Catalog

Create schema pgenvelope (name TBD):

Tables:

pgenvelope_schemes

  • scheme_id
  • name
  • algorithm
  • kms_key_alias
  • encrypted_dek
  • active
  • created_at

pgenvelope_config

  • KMS type (vault)
  • Endpoint
  • Credentials
  • Global settings

F3. Envelope Encryption Model

  • Data Encryption Key (DEK) encrypted with KMS, stored locally

  • Ciphertexts embed:

    • scheme ID
    • nonce
    • ciphertext

  • DEKs cached in memory with TTL to avoid KMS round-trips

F4. Key Rotation (MVP)

  • Ability to create new scheme versions
  • Mark active/inactive
  • Manually re-encrypt a column:
    • pgenvelope_reencrypt_column(table_name, column_name, target_scheme_id)

F5. KMS Integration (Vault Transit)

  • VaultKmsClient implements trait KmsClient
  • Token-based auth from config
  • Errors must distinguish transient vs permanent

F6. SQL & Operational Interface

Core SQL API:

  • encrypt_column
  • decrypt_column
  • pgenvelope_create_scheme(...)
  • pgenvelope_activate_scheme(...)
  • pgenvelope_deactivate_scheme(...)
  • pgenvelope_show_schemes()
  • pgenvelope_test_kms()
  • pgenvelope_reencrypt_column(...)

Permissions:

  • Only superuser or role pgenvelope_admin may manage config/schemes
  • Apps can be granted encrypt/decrypt rights

F7. Error Handling

  • Clear error on tag mismatch, invalid ciphertext, unknown scheme
  • No logging of plaintext or keys
  • Configurable logging level (MVP subset ok)

F8. C Shim / FFI Requirements

  • PostgreSQL wrappers defined in C
  • Rust functions exposed via stable ABI
  • Memory always allocated in PostgreSQL contexts (palloc)
  • ABI version check must be performed

F9. Non-Functional Requirements

  • Rust crypto crates must be modern and maintained
  • Minimal unsafe and only at FFI boundary
  • Performance should not regress typical OLTP workloads excessively
  • Integration tests using local Vault instance
  • Documented threat model

6. Roadmap (Post-MVP)

6.1 Additional KMS Backends

  • AWS KMS
  • GCP KMS
  • Azure Key Vault
  • PKCS#11 / HSM

6.2 Deterministic & Searchable Encryption

  • Explicit deterministic mode for equality search
  • Hash-based indexing helpers

6.3 Transparent Types / Domains

  • encrypted_text, encrypted_int, etc.
  • Provides near-transparent I/O

6.4 Policy Integration

  • RLS-aware decrypt-or-mask functions
  • Column policies defining who may decrypt what

6.5 Advanced Rotation

  • Background jobs
  • Incremental re-encryption with progress tracking
  • Resumable jobs

6.6 Multi-Tenant Key Isolation

  • Per-tenant key aliases
  • Tenant-bound AEAD AAD

6.7 Audit/Compliance

  • Audit tables for scheme changes and reencrypt actions
  • Compliance reporting views

6.8 Hardening / Certification

  • FIPS-mode builds
  • Side-channel hardening
  • Strict configuration modes

6.9 CLI Tools

  • Environment bootstrap
  • Offline migration
  • Configuration validation