Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/styles/config/vocabularies/Percona/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,5 @@ S3-compatible
uncomment
CAs
Ceph
Alibaba
Alibaba Cloud
30 changes: 15 additions & 15 deletions docs/details/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ etc. as the `mongo` shell or `mongodump` command does.

The `pbm-agent` processes should connect to their localhost `mongod` with a standalone type of connection.

```{.bash data-prompt="$"}
```bash
pbm-agent --mongodb-uri "mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin"
```

Alternatively:

```{.bash data-prompt="$"}
```bash
export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin"
pbm-agent
```
Expand All @@ -32,15 +32,15 @@ etc. as the `mongo` shell or `mongodump` command does.

=== "The `pbm` CLI connection string"

```{.bash data-prompt="$"}
```bash
pbm status --mongodb-uri "mongodb://pbmuser:secretpwd@mongocsvr1:27017,mongocsvr2:27017,mongocsvr3:27017/?replicaSet=configrs&authSource=admin"
```

Alternatively:

```{.bash data-prompt="$"}
```bash
export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@mongocsvr1:27017,mongocsvr2:27017,mongocsvr3:27017/?replicaSet=configrs&authSource=admin"
$ pbm status
pbm status
```

Replace the `pbmuser:secretpwd` with the credentials of [the user who owns the pbm process](../install/configure-authentication.md#create-the-pbm-user)
Expand Down Expand Up @@ -86,28 +86,28 @@ Specify new values in MongoDB connection URI string as follows:

=== "The pbm-agent connection string"

```
```bash
pbm-agent --mongodb-uri "mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&readConcernLevel=local&w=1"
```

Alternatively:

```
```bash
export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@localhost:27017/?authSource=admin&readConcernLevel=local&w=1"
pbm-agent
```

=== "The `pbm` CLI connection string"

```{.bash data-prompt="$"}
```bash
pbm status --mongodb-uri "mongodb://pbmuser:secretpwd@mongocsvr1:27017,mongocsvr2:27017,mongocsvr3:27017/?replicaSet=configrs&authSource=admin&readConcernLevel=local&w=1"
```

Alternatively:

```{.bash data-prompt="$"}
```bash
export PBM_MONGODB_URI="mongodb://pbmuser:secretpwd@mongocsvr1:27017,mongocsvr2:27017,mongocsvr3:27017/?replicaSet=configrs&authSource=admin&readConcernLevel=local&w=1"
$ pbm status
pbm status
```

Supported values are:
Expand All @@ -131,14 +131,14 @@ For [Kerberos authentication :octicons-link-external-16:](https://docs.percona.c

Specify the following string for MongoDB connection URI:

```
```bash
PBM_MONGODB_URI="mongodb://<username>%40<KERBEROS_REALM>@<hostname>:27018/?authMechanism=GSSAPI&authSource=%24external&replSetName=xxxx"
```

Note that you must first obtain the ticket for the `pbm` user with the `kinit` command before you start the **pbm-agent**:

```{.bash data-prompt="$"}
$ sudo -u {USER} kinit pbm
```bash
sudo -u {USER} kinit pbm
```

Note that the `{USER}` is the user that you will run the `pbm-agent` process.
Expand All @@ -147,7 +147,7 @@ Note that the `{USER}` is the user that you will run the `pbm-agent` process.

For [authentication and authorization via Native LDAP :octicons-link-external-16:](https://docs.percona.com/percona-server-for-mongodb/latest/authorization.html#authentication-and-authorization-with-direct-binding-to-ldap), you only create roles for LDAP groups in MongoDB as the users are stored and managed on the LDAP server. However, you still define the `$external` database as your authentication source:

```
```bash
PBM_MONGODB_URI="mongodb://<user>:<password>@<hostname>:27017/?authMechanism=PLAIN&authSource=%24external&replSetName=xxxx"
```

Expand All @@ -170,7 +170,7 @@ When using [AWS IAM authentication :octicons-link-external-16:](https://docs.per

The MongoDB connection URI string then looks like the following:

```
```bash
PBM_MONGODB_URI="mongodb://<aws_access_key_id>:<aws_secret_access_key>@<hostname>:27017/?authMechanism=MONGODB-AWS&authSource=%24external&replSetName=xxxx"
```

Expand Down
20 changes: 10 additions & 10 deletions docs/details/azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,32 +26,32 @@ For either method you need a storage account.
1. Install the [Azure CLI :octicons-link-external-16:](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). After the installation, the `az` is available for you.
2. Sign in to Azure CLI:

```{.bash data-prompt="$"}
$ az login
```bash
az login
```

3. Create a Resource group if it's not created for you:

```{.bash data-prompt="$"}
$ az group create --name <your-resource-group> --location <your-location>
```bash
az group create --name <your-resource-group> --location <your-location>
```

For the list of available locations, run:

```{.bash data-prompt="$"}
$ az account list-locations
```bash
az account list-locations
```

4. Create a storage account:

```{.bash data-prompt="$"}
$ az storage account create --name <storage-account-name> --resource-group <your-resource-group> --location <your-location> --sku Standard_LRS
```bash
az storage account create --name <storage-account-name> --resource-group <your-resource-group> --location <your-location> --sku Standard_LRS
```

4. Create a blob container:

```{.bash data-prompt="$"}
$ az storage container create --account-name <storage-account-name> --name <your-container> --public-access off
```bash
az storage container create --account-name <storage-account-name> --name <your-container> --public-access off
```

??? example "Expected output"
Expand Down
8 changes: 4 additions & 4 deletions docs/details/gcs.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,14 @@ To use GCS, you need the following:

2. Create a bucket

```{.bash data-prompt="$"}
$ gcloud storage buckets create my-gcs-bucket --location=US
```bash
gcloud storage buckets create my-gcs-bucket --location=US
```

3. Verify the bucket creation

```{.bash data-prompt="$"}
$ gcloud storage buckets list
```bash
gcloud storage buckets list
```

After the bucket is created, apply the proper [permissions for PBM to use the bucket](storage-configuration.md#permissions-setup).
Expand Down
24 changes: 12 additions & 12 deletions docs/details/minio.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,20 @@ This document provides an overview of MinIO as the closest S3-compatible storage

2. Configure the `mc` command line tool with a MinIO Server

```{.bash data-prompt="$"}
$ mc alias set myminio http://127.0.0.1:9000 MINIO_ACCESS_KEY MINIO_SECRET_KEY
```bash
mc alias set myminio http://127.0.0.1:9000 MINIO_ACCESS_KEY MINIO_SECRET_KEY
```

3. Create a bucket

```{.bash data-prompt="$"}
$ mc mb myminio/my-minio-bucket
```bash
mc mb myminio/my-minio-bucket
```

4. Verify the bucket creation

```{.bash data-prompt="$"}
$ mc ls myminio
```bash
mc ls myminio
```

After the bucket is created, apply the proper [permissions for PBM to use the bucket](storage-configuration.md#permissions-setup).
Expand Down Expand Up @@ -105,25 +105,25 @@ Let's assume that your custom CA certificate is at `/etc/ssl/minio-ca.crt` path

2. Set the `SSL_CERT_FILE` environment variable to that file's path on each host where `pbm-agent` and PBM CLI are running:

```{.bash data-prompt="$"}
$ export SSL_CERT_FILE=/etc/ssl/minio-ca.crt
```bash
export SSL_CERT_FILE=/etc/ssl/minio-ca.crt
```

If this variable isn't set, PBM uses the system root certificates.

3. Restart `pbm-agent`:

```{.bash data-prompt="$"}
$ sudo systemctl start pbm-agent
```bash
sudo systemctl start pbm-agent
```

4. Verify that your custom certificate is recognized. Check PBM logs for successful storage access.


Alternatively, you can turn off the TLS verification of the S3 storage in Percona Backup for MongoDB configuration:

```{.bash data-prompt="$"}
$ pbm config --set storage.minio.insecureSkipTLSVerify=True
```bash
pbm config --set storage.minio.insecureSkipTLSVerify=True
```

!!! warning
Expand Down
12 changes: 6 additions & 6 deletions docs/details/oss.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,24 +25,24 @@ You can create a bucket via the [Alibaba Cloud Management Console :octicons-link
1. [Install](https://www.alibabacloud.com/help/en/oss/developer-reference/install-ossutil2#DAS) and configure the Alibaba Cloud OSS client. After the installation, the `ossutil` command line tool is available for you.
2. Specify the region:

```{.bash data-prompt="$"}
$ ossutil config
```bash
ossutil config
```

Press Enter until you see the prompt `Please enter Region [cn-hangzhou]:` and specify the desired region.

3. Create a bucket:

```{.bash data-prompt="$"}
$ ossutil mb oss://your-bucket-name
```bash
ossutil mb oss://your-bucket-name
```

Replace `your-bucket-name` with the desired name for your bucket.

4. Verify that the bucket is created:

```{.bash data-prompt="$"}
$ ossutil ls
```bash
ossutil ls
```

After you created a bucket, apply the [necessary permissions](storage-configuration.md#permissions-setup) for the user identified by the access credentials you plan to use with PBM.
Expand Down
63 changes: 59 additions & 4 deletions docs/details/s3-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

[Configuration example :material-arrow-down:](#configuration-example){.md-button}


## Storage bucket creation

To create a bucket, do the following.
Expand All @@ -17,14 +18,14 @@

2. Create an S3 bucket

```{.bash data-prompt="$"}
$ aws s3api create-bucket --bucket my-s3-bucket --region us-east-1
```bash
aws s3api create-bucket --bucket my-s3-bucket --region us-east-1
```

3. Verify the bucket creation

```{.bash data-prompt="$"}
$ aws s3 ls
```bash
aws s3 ls
```

After the bucket is created, apply the proper [permissions for PBM to use the bucket](storage-configuration.md#permissions-setup).
Expand Down Expand Up @@ -149,5 +150,59 @@

This upload retry increases the chances of data upload completion in cases of unstable connection.

<<<<<<< HEAD

Check warning on line 153 in docs/details/s3-storage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/details/s3-storage.md#L153

[Google.Headings] '<<<<<<< HEAD' should use sentence-style capitalization.
Raw output
{"message": "[Google.Headings] '\u003c\u003c\u003c\u003c\u003c\u003c\u003c HEAD' should use sentence-style capitalization.", "location": {"path": "docs/details/s3-storage.md", "range": {"start": {"line": 153, "column": 1}}}, "severity": "WARNING"}
=======
## Data upload to storage with self-signed TLS certificates

Percona Backup for MongoDB supports data upload to S3-compatible storage service over HTTPS with a self-signed or a private CA certificate. This feature is especially important when you use services like MinIO, Ceph, or internal S3 gateways that don't use certificates signed by public Certificate Authorities (CAs).

Providing a whole chain of certificates is recommended to ensure the connection is legit. The `SSL_CERT_FILE` environment variable specifies the path to a custom certificate chain file in PEM-format that PBM uses to validate TLS/SSL connection.

### Usage example

Let's assume that your custom CA certificate is at `/etc/ssl/minio-ca.crt` path and your S3 endpoint is `https://minio.internal.local:9000`. To use self-issued TLS certificates, do the following:

Check warning on line 163 in docs/details/s3-storage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/details/s3-storage.md#L163

[Google.We] Try to avoid using first-person plural like 'Let's'.
Raw output
{"message": "[Google.We] Try to avoid using first-person plural like 'Let's'.", "location": {"path": "docs/details/s3-storage.md", "range": {"start": {"line": 163, "column": 1}}}, "severity": "WARNING"}

1. Ensure the cert file is in PEM format. Use the following command to check it:

```bash
cat /etc/ssl/minio-ca.crt
```

??? example "Sample output"


```{text .no-copy}
-----BEGIN CERTIFICATE-----
MIIC+TCCAeGgAwIBAgIJANH3WljB...
-----END CERTIFICATE-----
```

2. Set the `SSL_CERT_FILE` environment variable to that file's path on each host where `pbm-agent` and PBM CLI are running:

Check warning on line 180 in docs/details/s3-storage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/details/s3-storage.md#L180

[Google.WordList] Use 'command-line tool' instead of 'CLI'.
Raw output
{"message": "[Google.WordList] Use 'command-line tool' instead of 'CLI'.", "location": {"path": "docs/details/s3-storage.md", "range": {"start": {"line": 180, "column": 108}}}, "severity": "WARNING"}

```bash
export SSL_CERT_FILE=/etc/ssl/minio-ca.crt
```

If this variable isn't set, PBM uses the system root certificates.

3. Restart `pbm-agent`:

```bash
sudo systemctl start pbm-agent
```

4. Verify that your custom certificate is recognized. Check PBM logs for successful S3 access.


Alternatively, you can disable the TLS verification of the S3 storage in Percona Backup for MongoDB configuration:

Check warning on line 197 in docs/details/s3-storage.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/details/s3-storage.md#L197

[Google.WordList] Use 'turn off' or 'off' instead of 'disable'.
Raw output
{"message": "[Google.WordList] Use 'turn off' or 'off' instead of 'disable'.", "location": {"path": "docs/details/s3-storage.md", "range": {"start": {"line": 197, "column": 24}}}, "severity": "WARNING"}

```bash
pbm config --set storage.s3.insecureSkipTLSVerify=True
```

!!! warning

Use this option with caution as it might leave a hole for man-in-the-middle attacks.
>>>>>>> Removed prompt sign from code blocks


Loading