Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ Identity requires a database to store information about resource authorization a
:::note
There are no default values for the variables above. See
[supported environments](/reference/supported-environments.md#camunda-platform-8-self-managed) for a list of
supported databases.
supported databases. You may also review the [RDBMS support policy](/self-managed/concepts/rdbms-support-policy.md).
:::

### Running Identity on Amazon Aurora PostgreSQL
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: schema-and-migration
title: Schema and data migration
description: "The orchestration cluster stores data in Elasticsearch and provides tools to manage schema and migrations."
description: "The orchestration cluster stores data with secondary storage and provides tools to manage schema and migrations."
---

The orchestration cluster persists runtime and task data in Elasticsearch. On first startup, all required indices and templates are automatically created.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ You can configure secondary storage using Helm charts, Docker Compose, or manual

Camunda uses the `data.secondary-storage` configuration to define which database backend supports advanced web applications and APIs.

:::note
For the latest list of supported relational databases and versions, see the
[RDBMS version support policy](/self-managed/concepts/rdbms-support-policy.md).
:::

<Tabs groupId="configuration" defaultValue="helm" queryString values={[
{label: 'Helm', value: 'helm' },
{label: 'Docker Compose', value: 'docker-compose' },
Expand All @@ -36,7 +41,7 @@ data:
password:
```

You can switch to Elasticsearch by using:
To configure Elasticsearch instead:

```yaml
data:
Expand All @@ -46,7 +51,7 @@ data:
url: http://elasticsearch:9200/
```

To explicitly disable secondary storage (for example, when running only the Zeebe engine), you can set:
To explicitly disable secondary storage (for example, when running only the Zeebe engine), set:

```yaml
global:
Expand All @@ -56,6 +61,7 @@ global:
When this flag is set, all secondary-storage-dependent components are automatically disabled.

</TabItem>

<TabItem value="docker-compose">

If you’re using Docker Compose, configure your environment variables within the relevant service definition:
Expand All @@ -72,8 +78,8 @@ For Elasticsearch:

```yaml
environment:
- CAMUNDA_DATA_SECONDSTORAGE_TYPE=elasticsearch
- CAMUNDA_DATA_SECONDSTORAGE_ELASTICSEARCH_URL=http://elasticsearch:9200
- CAMUNDA_DATA_SECONDARYSTORAGE_TYPE=elasticsearch
- CAMUNDA_DATA_SECONDARYSTORAGE_ELASTICSEARCH_URL=http://elasticsearch:9200
```

To disable secondary storage:
Expand All @@ -84,6 +90,7 @@ environment:
```

</TabItem>

<TabItem value="manual">

In Self-Managed or Camunda 8 Run deployments, you can also configure storage directly in the `application.yaml` file:
Expand All @@ -109,19 +116,21 @@ data:
```

</TabItem>

</Tabs>

## Choosing a storage backend

| Scenario | Recommended backend | Reason |
| :---------------------------------------- | :---------------------------------------------- | :------------------------------------------------------------ |
| Local testing or Camunda 8 Run quickstart | H2 | Fast, lightweight, and runs entirely in memory or file-based. |
| Production workloads | Elasticsearch or supported RDBMS | Scalable and persistent; designed for concurrent queries. |
| Debugging and troubleshooting | H2 or PostgreSQL (for debugging and inspection) | Easier to inspect and visualize data. |
| Scenario | Recommended backend | Reason |
| :---------------------------------------- | :--------------------------------- | :------------------------------------------------------------ |
| Local testing or Camunda 8 Run quickstart | H2 | Fast, lightweight, and runs entirely in memory or file-based. |
| Production workloads | Elasticsearch or a supported RDBMS | Scalable and persistent; designed for concurrent queries. |
| Debugging and troubleshooting | H2 or PostgreSQL | Easier to inspect and visualize data. |

:::note
H2 is suitable for testing and local development only.
For production use, Operate and Tasklist require a persistent secondary storage backend such as an RDBMS or Elasticsearch.
For production use, Operate and Tasklist require a persistent secondary storage backend such as a supported RDBMS or Elasticsearch.
Consult the [RDBMS version support policy](/self-managed/concepts/rdbms-support-policy.md) when choosing a relational database.
:::

## Run without secondary storage
Expand All @@ -134,4 +143,4 @@ In this mode:
- The Zeebe engine and primary storage remain active for process execution.
- This configuration is best suited for local development or minimal-resource environments.

See [Run without secondary storage](./no-secondary-storage.md) for configuration examples and limitations.
See [run without secondary storage](./no-secondary-storage.md) for configuration examples and limitations.
14 changes: 8 additions & 6 deletions docs/self-managed/concepts/secondary-storage/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,14 @@ Secondary storage is not a duplicate of primary data. It represents the historic

### Supported storage options

Camunda supports the following secondary storage backends, depending on the version and use case:

| Database type | Availability | Use case |
| :------------------------------ | :------------------- | :------------------------------------------------------------------- |
| Elasticsearch/OpenSearch | General availability | Default for most production installations. |
| RDBMS (H2, PostgreSQL, MariaDB) | 8.9-alpha1+ | Lightweight local testing (H2) or future enterprise-grade RDBMS use. |
Camunda supports multiple secondary storage backends.
For the latest list of supported database versions, see the
[RDBMS version support policy](/self-managed/concepts/rdbms-support-policy.md).

| Database type | Availability | Use case |
| :----------------------- | :------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Elasticsearch/OpenSearch | General availability | Default for most production installations. |
| RDBMS | 8.9-alpha1+ | Supports relational databases for secondary storage. See the [RDBMS support policy](/self-managed/concepts/rdbms-support-policy.md) for supported vendors and versions. |

:::info OpenSearch support
Camunda 8 supports both [Amazon OpenSearch](https://aws.amazon.com/opensearch-service) and the open-source [OpenSearch](https://opensearch.org/) distribution.
Expand Down
229 changes: 229 additions & 0 deletions docs/self-managed/deployment/helm/configure/database/rdbms.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
---
id: rdbms
sidebar_label: Configure RDBMS
title: Configure RDBMS in Helm chart
description: "Configure an external relational database (RDBMS) as secondary storage for Camunda 8 Self-Managed using the Helm chart."
---

Camunda 8 Self-Managed supports using an external relational database (RDBMS) as the Orchestration Cluster’s secondary storage instead of Elasticsearch or OpenSearch.

## Prerequisites

Provide a supported relational database that is reachable by the Camunda components.

See the [RDBMS support policy](/self-managed/concepts/rdbms-support-policy.md) for the complete list of supported databases and versions.

Ensure that:

- Your network allows traffic from Camunda pods to the database.
- Required JDBC parameters (SSL/TLS, authentication, failover) are configured as needed.
- The database user has permissions to create and modify schema objects if `autoDDL` is enabled.

## Configuration

### Parameters

| Parameter | Type | Default | Description |
| -------------------------------------------------------------------------------------------------- | ----------------- | ------- | ---------------------------------------------------------------------------------- |
| `orchestration.data.secondaryStorage.type` | string | `""` | Sets the secondary storage type. Must be `rdbms` when using a relational database. |
| `orchestration.data.secondaryStorage.rdbms.url` | string | `""` | JDBC connection URL for the external database. |
| `orchestration.data.secondaryStorage.rdbms.username` | string | `""` | Username used to authenticate to the database. |
| `orchestration.data.secondaryStorage.rdbms.secret.existingSecret` | string | `""` | Name of an existing Kubernetes secret containing the password. |
| `orchestration.data.secondaryStorage.rdbms.secret.existingSecretKey` | string | `""` | Key within the Kubernetes secret that stores the password. |
| `orchestration.data.secondaryStorage.rdbms.secret.inlineSecret` | string | `""` | Inline password value. Use only for testing environments. |
| `orchestration.data.secondaryStorage.rdbms.flushInterval` | ISO-8601 duration | `""` | How often the exporter flushes events to the database. |
| `orchestration.data.secondaryStorage.rdbms.autoDDL` | boolean | `true` | Enables Liquibase-powered auto-creation of database schema. |
| `orchestration.data.secondaryStorage.rdbms.prefix` | string | `""` | Table name prefix added to all RDBMS tables. |
| `orchestration.data.secondaryStorage.rdbms.queueSize` | integer | `1000` | Size of the exporter queue. Larger values buffer more events. |
| `orchestration.data.secondaryStorage.rdbms.queueMemoryLimit` | integer | `20` | Memory limit (MB) for the exporter queue. |
| `orchestration.data.secondaryStorage.rdbms.history.defaultHistoryTTL` | ISO-8601 duration | `""` | Default TTL for historic process data. |
| `orchestration.data.secondaryStorage.rdbms.history.defaultBatchOperationHistoryTTL` | ISO-8601 duration | `""` | TTL for batch operation history. |
| `orchestration.data.secondaryStorage.rdbms.history.batchOperationCancelProcessInstanceHistoryTTL` | ISO-8601 duration | `""` | TTL for cancel-process-instance history data. |
| `orchestration.data.secondaryStorage.rdbms.history.batchOperationMigrateProcessInstanceHistoryTTL` | ISO-8601 duration | `""` | TTL for migrate-process-instance history. |
| `orchestration.data.secondaryStorage.rdbms.history.batchOperationModifyProcessInstanceHistoryTTL` | ISO-8601 duration | `""` | TTL for modify-process-instance history. |
| `orchestration.data.secondaryStorage.rdbms.history.batchOperationResolveIncidentHistoryTTL` | ISO-8601 duration | `""` | TTL for resolve-incident history. |
| `orchestration.data.secondaryStorage.rdbms.history.minHistoryCleanupInterval` | ISO-8601 duration | `""` | Minimum interval for periodic history cleanup. |
| `orchestration.data.secondaryStorage.rdbms.history.maxHistoryCleanupInterval` | ISO-8601 duration | `""` | Maximum interval for periodic history cleanup. |
| `orchestration.data.secondaryStorage.rdbms.history.historyCleanupBatchSize` | integer | `1000` | Batch size when deleting history data. |
| `orchestration.data.secondaryStorage.rdbms.history.usageMetricsCleanup` | string | `""` | Cleanup configuration for usage metrics. |
| `orchestration.data.secondaryStorage.rdbms.history.processCache.maxSize` | string | `""` | Cache size for historic process definitions. |
| `orchestration.data.secondaryStorage.rdbms.history.batchOperationCache.maxSize` | integer | `10000` | Cache size for batch operation history. |
| `orchestration.data.secondaryStorage.rdbms.history.connectionPool.maximumPoolSize` | integer | `""` | Maximum number of JDBC connections. |
| `orchestration.data.secondaryStorage.rdbms.history.connectionPool.minimumIdle` | integer | `""` | Minimum number of idle JDBC connections. |
| `orchestration.data.secondaryStorage.rdbms.history.connectionPool.idleTimeout` | ISO-8601 duration | `""` | Maximum time a connection may remain idle. |
| `orchestration.data.secondaryStorage.rdbms.history.connectionPool.maxLifetime` | ISO-8601 duration | `""` | Maximum lifetime of a JDBC connection. |
| `orchestration.data.secondaryStorage.rdbms.history.connectionPool.connectionTimeout` | ISO-8601 duration | `""` | Timeout for acquiring a connection from the pool. |

### Example usage

:::warning Important
Operate does not support RDBMS until **Camunda 8.9.0-alpha3**.
:::

```yaml
orchestration:
exporters:
camunda:
enabled: false
rdbms:
enabled: true
data:
secondaryStorage:
type: rdbms
rdbms:
url: jdbc:postgresql://hostname:5432/camunda
username: camunda
secret:
existingSecret: camunda-db-secret
existingSecretKey: password
```

## Loading JDBC drivers into pods

Some databases—such as Oracle—require JDBC drivers that cannot be included in the Camunda image due to licensing restrictions. You must provide these drivers at runtime using one of the following approaches:

1. **Init container**: Download JDBC drivers at pod startup.
2. **Custom Docker image**: Build an image that bundles the driver.
3. **Mounted volume**: Mount a persistent volume or secret that contains the driver.

### Option 1: Using an init container

:::note
This example uses `/driver-lib`, which the Orchestration Cluster automatically adds to the classpath. If you use a different directory, additional override configuration may be required (command, entrypoint).
:::

```yaml
orchestration:
exporters:
camunda:
enabled: false
rdbms:
enabled: true
data:
secondaryStorage:
type: rdbms
rdbms:
url: jdbc:oracle:thin:@//hostname:1521/FREEPDB1
username: myuser
secret:
inlineSecret: mypassword
extraVolumeMounts:
- name: jdbcdrivers
mountPath: /driver-lib
extraVolumes:
- name: jdbcdrivers
emptyDir: {}
initContainers:
- name: fetch-jdbc-drivers
image: alpine:3.19
imagePullPolicy: Always
command:
- sh
- -c
- >
wget https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc11/23.9.0.25.07/ojdbc11-23.9.0.25.07.jar
-O /driver-lib/ojdbc.jar
volumeMounts:
- name: jdbcdrivers
mountPath: /driver-lib
securityContext:
runAsUser: 1001
```

### Option 2: Using a custom Docker image

:::warning Important
This approach has not yet been validated in production.
:::

```dockerfile
FROM camunda/camunda-platform:8.8.0
ADD ojdbc8.jar /driver-lib/ojdbc8.jar
```

```sh
docker build -t internal-registry/orchestration:8.8.0 .
docker push internal-registry/orchestration:8.8.0
```

To use this custom image:

```yaml
orchestration:
exporters:
camunda:
enabled: false
rdbms:
enabled: true
image:
repository: internal-registry/orchestration
tag: 8.8.0
data:
secondaryStorage:
type: rdbms
rdbms:
url: jdbc:oracle:thin:@//hostname:1521/FREEPDB1
username: myuser
secret:
inlineSecret: mypassword
```

### Option 3: Mounting a JDBC driver from a volume

:::warning Important
Mounting an `emptyDir volume` does not persist across pod restarts. Use a ConfigMap, PersistentVolume, or custom image for production.
:::

```yaml
orchestration:
exporters:
camunda:
enabled: false
rdbms:
enabled: true
data:
secondaryStorage:
type: rdbms
rdbms:
url: jdbc:oracle:thin:@//hostname:1521/FREEPDB1
username: myuser
secret:
inlineSecret: mypassword
extraVolumeMounts:
- name: jdbcdrivers
mountPath: /driver-lib
extraVolumes:
- name: jdbcdrivers
```

Copy the driver manually to the pod:

```sh
kubectl cp /path/to/ojdbc8.jar <pod-name>:/driver-lib/ojdbc8.jar
```

## Verifying connectivity

To confirm that the Orchestration Cluster is successfully writing to the database:

1. Verify that tables were created (for example, `\dt` or `\d` in PostgreSQL).
2. Deploy a process model and start a process instance using Web Modeler.
3. Query the database directly:

```sql
SELECT * FROM process_instances;
```

4. Review logs for messages such as:

```
io.camunda.application.commons.rdbms.MyBatisConfiguration - Initializing Liquibase for RDBMS with global table trimmedPrefix ''.
...

io.camunda.exporter.rdbms.RdbmsExporter - [RDBMS Exporter[] RdbmsExporter created with Configuration: flushInterval=PT0.5S, queueSize=1000
io.camunda.exporter.rdbms.RdbmsExporter - [RDBMS Exporter[] Exporter opened with last exported position 3318
...
org.springframework.web.servlet.DispatcherServlet - Completed initialization in 0 ms
```

If the flush interval is long or the queue size is large, exported data may take several seconds to appear in the database.
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ username: `postgres`
password: `examplePassword`
```

- **Supported versions:**: Check the [supported environments](/reference/supported-environments.md) page to confirm which PostgreSQL versions are supported.
- **Supported versions:**: Check the [supported environments](/reference/supported-environments.md) and [RDBMS support policy](/self-managed/concepts/rdbms-support-policy.md) pages to confirm which PostgreSQL versions are supported.
- **Database setup:** Ensure the required databases exist in your PostgreSQL instance. For this guide, create the following databases:

```SQL
Expand Down
Loading
Loading