Skip to content

Commit

Permalink
docs
Browse files Browse the repository at this point in the history
Signed-off-by: Henrique Goncalves <[email protected]>
  • Loading branch information
kamushadenes committed Jan 30, 2025
1 parent ef7035f commit 1a5d8e3
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 17 deletions.
42 changes: 28 additions & 14 deletions docs/deploy_k8s.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ This guide assumes that you already have a running kubernetes cluster. If you're
The deployment will use a Helm chart which is a package for Kubernetes applications, acting like a blueprint or template that defines the resources needed to deploy an application on a Kubernetes cluster. Charts are stored in `helm-repo`.

To add a remote helm repo:

1. Run: `helm pull oci://ghcr.io/doublecloud/transfer-helm/transfer`. It will pull latest **transfer** helm chart locally

2. After adding the repo, perform the repo indexing process by running `helm repo update`.


### 2. Create a Namespace for Transfer

While it is not strictly necessary to isolate the **transfer** installation into its own namespace, it is good practice and recommended as a part of the installation.
Expand Down Expand Up @@ -61,12 +61,10 @@ transferSpec:
type: INCREMENT_ONLY # type of transfer, one of: INCREMENT_ONLY, SNAPSHOT_ONLY, SNAPSHOT_AND_INCREMENT
src:
type: source_type # for example: pg, s3, kafka ...
params:
... # source type params, all params can be founded in `model_source.go` for provider folder
params: ... # source type params, all params can be founded in `model_source.go` for provider folder
dst:
type: target_type # for example: s3, ch, kafka ...
params:
... # target type params, all params can be founded in `model_destination.go` for provider folder
params: ... # target type params, all params can be founded in `model_destination.go` for provider folder
```
### 4. Installing Transfer
Expand Down Expand Up @@ -96,9 +94,9 @@ By default, transfer pods will expose metrics on `9091` ports, and add a default

```yaml
annotations:
prometheus.io/path: /metrics
prometheus.io/port: '9091'
prometheus.io/scrape: 'true'
prometheus.io/path: /metrics
prometheus.io/port: "9091"
prometheus.io/scrape: "true"
```
This will expose metrics scrapper for k8s scrapper. You can re-use our grafana dashboard template from Grafana template [here](../assets/grafana.tmpl.json).
Expand All @@ -109,28 +107,41 @@ This template will generate for you something like this:
Be caution: in template you must replace **<Your-Prometheus-source-ID>** before deployement to grafana.
#### Add a prefix to metrics
To add a prefix to metrics, use the `--metrics-prefix` flag. For example, to add a prefix of `transfer` to all metrics, use the following command:

```
trcli --metrics-prefix transfer ...
```
### 6. Secrets management
For secrets management we recommend to use env-vars in paar with secret operator, for example [Hashicorp Vault](https://developer.hashicorp.com/vault/docs/platform/k8s/injector/examples)
#### specify secrets in transferSpec
Use ENV-vars substitute for src and dst params, you may specify
##### Hashicorp Vault
[Hashicorp Vault](https://developer.hashicorp.com/vault/docs/platform/k8s/injector/examples)
```yaml
transferSpec:
dst:
type: ch
params:
Password: "${FOO}"
dst:
type: ch
params:
Password: "${FOO}"
env:
FOO: "/vault/secret"
```

##### K8s secret

[k8s secret](https://kubernetes.io/docs/concepts/configuration/secret/)

```yaml
secret_env:
- env_name: FOO
Expand All @@ -139,18 +150,21 @@ secret_env:
```
##### ENV-vars
Don't recommend to use this way, but you may specify env-vars directly in values.yaml
```yaml
env:
FOO: "secret"
```
#### After load
#### After load
After load this transfer yaml would be:
```yaml
dst:
type: ch
params:
Password: "secret"
Password: "secret"
```
14 changes: 11 additions & 3 deletions docs/integrations/connect-prometheus-to-transfer.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ By default, transfer pods will expose metrics on `9091` ports, and add a default

```yaml
annotations:
prometheus.io/path: /metrics
prometheus.io/port: '9091'
prometheus.io/scrape: 'true'
prometheus.io/path: /metrics
prometheus.io/port: "9091"
prometheus.io/scrape: "true"
```
This will expose metrics scrapper for k8s scrapper. You can re-use our grafana dashboard template from Grafana template [here](https://github.com/doublecloud/transfer/blob/main/assets/grafana.tmpl.json).
Expand All @@ -21,3 +21,11 @@ This template will generate for you something like this:
![demo_grafana_dashboard.png](../_assets/demo_grafana_dashboard.png)
Be caution: in template you must replace **<Your-Prometheus-source-ID>** before deployement to grafana.
## Add a prefix to metrics
To add a prefix to metrics, use the `--metrics-prefix` flag. For example, to add a prefix of `transfer` to all metrics, use the following command:

```
trcli --metrics-prefix transfer ...
```

0 comments on commit 1a5d8e3

Please sign in to comment.