forked from postgres/postgres
-
Notifications
You must be signed in to change notification settings - Fork 14
Reorganize Architecture chapter #557
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Andriciuc
wants to merge
16
commits into
TDE_REL_17_STABLE
Choose a base branch
from
improve-architecture-chapter-docs
base: TDE_REL_17_STABLE
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
7b4d602
Reorganize Architecture chapter
Andriciuc 4624505
Merge branch 'TDE_REL_17_STABLE' into improve-architecture-chapter-docs
Andriciuc f36ea4a
added information from architecture.md to each subtopic
Andriciuc a026cf5
modify intro to fit new structure
Andriciuc 0c24350
Merge branch 'TDE_REL_17_STABLE' into improve-architecture-chapter-docs
Andriciuc f3beb20
remove warning of WAL beta from wal_encryption variable
Andriciuc a80bf9b
update overview.md with updated customizable support and extension ex…
Andriciuc feafa58
move usage guide to Technical reference outside of Architecture
Andriciuc 97281ce
add intro paragraph for Usage guide and replace Architecture card wit…
Andriciuc ff1300e
update overview and internals with clearer text
Andriciuc 0e300c5
Merge branch 'TDE_REL_17_STABLE' into improve-architecture-chapter-docs
Andriciuc 77f9fa9
Merge branch 'TDE_REL_17_STABLE' into improve-architecture-chapter-docs
Andriciuc dab1384
update text for clarity in SMGR API and WAL encryption
Andriciuc d4ceca5
Merge branch 'TDE_REL_17_STABLE' into improve-architecture-chapter-docs
Andriciuc d548385
updates based on feedback
Andriciuc 4cf7172
Update key-prov-management.md
Andriciuc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
36 changes: 34 additions & 2 deletions
36
contrib/pg_tde/documentation/docs/advanced-topics/tech-reference.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,39 @@ | ||
# Overview of technical capabilities | ||
# Technical reference overview | ||
|
||
This section covers the internal components and tools that power `pg_tde`. | ||
|
||
Use it to understand how encryption is implemented, fine-tune a configuration, leverage advanced CLI tools and functions for diagnostics and customization. | ||
|
||
[Architecture](../architecture/architecture.md){.md-button} [GUC Variables](../variables.md){.md-button} [Functions](../functions.md){.md-button} | ||
<div data-grid markdown><div data-banner markdown> | ||
|
||
### :material-playlist-check: Usage guide { .title } | ||
|
||
Step-by-step instructions for using `pg_tde`. Set up the extension, configure key providers, create and rotate keys, manage permissions, and encrypt tables. | ||
|
||
[Follow the usage guide :material-arrow-right:](usage-guide.md){.md-button} | ||
|
||
</div><div data-banner markdown> | ||
|
||
### :material-database-sync: Streaming replication { .title } | ||
|
||
Learn how to configure PostgreSQL streaming replication with `pg_tde` using the `tde_heap` access method. Covers primary and standby setup, key management requirements, and validation steps. | ||
|
||
[Set up replication with `tde_heap` :material-arrow-right:](../replication.md){.md-button} | ||
|
||
</div><div data-banner markdown> | ||
|
||
### :material-tune: GUC Variables { .title } | ||
|
||
Configure how `pg_tde` behaves with PostgreSQL. Control WAL encryption, enforce encryption for new tables, and manage global provider inheritance. Includes scope levels, defaults, and permission requirements. | ||
|
||
[Configure GUC variables :material-arrow-right:](../variables.md){.md-button} | ||
|
||
</div><div data-banner markdown> | ||
|
||
### :material-function-variant: Functions { .title } | ||
|
||
Use built-in functions to manage key providers, create and rotate principal keys, and verify encryption status. Includes commands for Vault, KMIP, and local providers, plus utilities to inspect or validate keys. | ||
|
||
[Browse available functions :material-arrow-right:→](../functions.md){.md-button} | ||
|
||
</div></div> |
176 changes: 176 additions & 0 deletions
176
contrib/pg_tde/documentation/docs/advanced-topics/usage-guide.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,176 @@ | ||
# Usage guide | ||
|
||
This guide shows how to use `pg_tde` in practice. It walks you through setting up `pg_tde`, adding and managing key providers, creating and rotating keys, setting permissions, and encrypting tables. | ||
|
||
Use this section as a step-by-step reference when deploying or operating `pg_tde` in your PostgreSQL environment. | ||
|
||
## Set up pg_tde | ||
|
||
To get started with `pg_tde`, follow these steps: | ||
|
||
* Add `pg_tde` to the `shared_preload_libraries` in `postgresql.conf` as this is required for the SMGR extensions | ||
* Execute `CREATE EXTENSION pg_tde` in the databases where they want to use encryption | ||
* Optionally, enable `pg_tde.wal_encrypt` in `postgresql.conf` to encrypt WAL writes | ||
* Optionally, disable `pg_tde.inherit_global_providers` in `postgresql.conf` (it is enabled by default) | ||
|
||
## Add providers | ||
|
||
You can add keyring providers to either the global or database specific scope. | ||
|
||
If `pg_tde.inherit_global_providers` is `on`, global providers are visible for all databases, and can be used. | ||
If `pg_tde.inherit_global_providers` is `off`, global providers are only used for WAL encryption. | ||
|
||
To add a global provider: | ||
|
||
```sql | ||
pg_tde_add_global_key_provider_<TYPE>('provider_name', ... details ...) | ||
``` | ||
|
||
To add a database specific provider: | ||
|
||
```sql | ||
pg_tde_add_database_key_provider_<TYPE>('provider_name', ... details ...) | ||
``` | ||
|
||
## Change providers | ||
|
||
To change a value of a global provider: | ||
|
||
```sql | ||
pg_tde_change_global_key_provider_<TYPE>('provider_name', ... details ...) | ||
``` | ||
|
||
To change a value of a database specific provider: | ||
|
||
```sql | ||
pg_tde_change_database_key_provider_<TYPE>('provider_name', ... details ...) | ||
``` | ||
|
||
These functions also allow changing the type of a provider but **do not** migrate any data. They are expected to be used during infrastructure migration, for example when the address of a server changes. | ||
|
||
## Change providers from the command line | ||
|
||
To change a provider from a command line, `pg_tde` provides the `pg_tde_change_key_provider` command line tool. | ||
|
||
This tool work similarly to the above functions, with the following syntax: | ||
|
||
```sh | ||
pg_tde_change_key_provider <dbOid> <providerType> ... details ... | ||
``` | ||
|
||
!!! note | ||
Since this tool is expected to be offline, it bypasses all permission checks. This is also the reason why it requires a `dbOid` instead of a name, as it has no way to process the catalog and look up names. | ||
|
||
This tool does not validate any parameters. | ||
|
||
## Delete providers | ||
|
||
Providers can be deleted by using the following functions: | ||
|
||
```sql | ||
pg_tde_delete_database_key_provider(provider_name) | ||
pg_tde_delete_global_key_provider(provider_name) | ||
``` | ||
|
||
For database specific providers, the function first checks if the provider is used or not, and the provider is only deleted if it's not used. | ||
|
||
For global providers, the function checks if the provider is used anywhere, WAL or any specific database, and returns an error if it is. | ||
|
||
## List/query providers | ||
|
||
`pg_tde` provides 2 functions to show providers: | ||
|
||
* `pg_tde_list_all_database_key_providers()` | ||
* `pg_tde_list_all_global_key_providers()` | ||
|
||
These functions return a list of provider names, type and configuration. | ||
|
||
## Provider permissions | ||
|
||
`pg_tde` implements access control based on execution rights on the administration functions. | ||
|
||
For keys and providers administration, it provides two pair of functions: | ||
|
||
```sql | ||
pg_tde_GRANT_database_key_management_TO_role | ||
pg_tde_REVOKE_database_key_management_FROM_role | ||
``` | ||
|
||
## Create and rotate keys | ||
|
||
Principal keys can be created using the following functions: | ||
|
||
```sql | ||
pg_tde_create_key_using_(global/database)_key_provider('key-name', 'provider-name') | ||
``` | ||
|
||
Principal keys can be used or rotated using the following functions: | ||
|
||
```sql | ||
pg_tde_set_key_using_(global/database)_key_provider('key-name', 'provider-name') | ||
pg_tde_set_server_key_using_(global/database)_key_provider('key-name', 'provider-name') | ||
pg_tde_set_default_key_using_(global/database)_key_provider('key-name', 'provider-name') | ||
``` | ||
|
||
## Default principal key | ||
|
||
With `pg_tde.inherit_global_key_providers`, it is also possible to set up a default global principal key, which will be used by any database which has the `pg_tde` extension enabled, but doesn't have a database specific principal key configured using `pg_tde_set_key_using_(global/database)_key_provider`. | ||
|
||
With this feature, it is possible for the entire database server to easily use the same principal key for all databases, completely disabling multi-tenancy. | ||
|
||
### Manage a default key | ||
|
||
You can manage a default key with the following functions: | ||
|
||
* `pg_tde_create_key_using_global_key_provider('key-name','provider-name')` | ||
* `pg_tde_set_default_key_using_global_key_provider('key-name','provider-name')` | ||
* `pg_tde_delete_default_key()` | ||
|
||
!!! note | ||
`pg_tde_delete_default_key()` is only possible if there's no database currently using the default principal key. | ||
Changing the default principal key will rotate the encryption of internal keys for all databases using the current default principal key. | ||
|
||
### Delete a key | ||
|
||
The `pg_tde_delete_key()` function unsets the principal key for the current database. If the current database has any encrypted tables, and there isn’t a default principal key configured, it reports an error instead. If there are encrypted tables, but there’s also a default principal key, internal keys will be encrypted with the default key. | ||
|
||
!!! note | ||
WAL keys **cannot** be unset, as server keys are managed separately. | ||
|
||
## Current key details | ||
|
||
`pg_tde_key_info()` returns the name of the current principal key, and the provider it uses. | ||
|
||
`pg_tde_server_key_info()` does the same for the server key. | ||
|
||
`pg_tde_default_key_info()` does the same for the default key. | ||
|
||
`pg_tde_verify_key()` checks that the key provider is accessible, that the current principal key can be downloaded from it, and that it is the same as the current key stored in memory - if any of these fail, it reports an appropriate error. | ||
|
||
## Key permissions | ||
|
||
Users with management permissions to a specific database `(pg_tde_(grant/revoke)_(global/databse)_key_management_(to/from)_role)` can change the keys for the database, and use the current key functions. This includes creating keys using global providers, if `pg_tde.inherit_global_providers` is enabled. | ||
|
||
Also the `pg_tde_(grant/revoke)_database_key_management_to_role` function deals with only the specific permission for the above function: it allows a user to change the key for the database, but not to modify the provider configuration. | ||
|
||
## Create an encrypted table | ||
|
||
To create an encrypted table, use the following command: | ||
|
||
```sql | ||
CREATE TABLE t1(a INT) USING tde_heap; | ||
``` | ||
|
||
## Alter an encrypted table | ||
|
||
To alter or modify an encrypted table, use the following command: | ||
|
||
```sql | ||
ALTER TABLE t1 SET ACCESS METHOD tde_heap; | ||
``` | ||
|
||
## Change the pg_tde.inherit_global_keys setting | ||
|
||
It is possible to use `pg_tde` with `inherit_global_keys = on`, refer to the global keys or keyrings in databases, and then change this setting to `off`. | ||
|
||
In this case, existing references to global providers or the global default principal key keep working as before, but new references to the global scope cannot be made. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.