Skip to content

add confluent cloud #4039

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 11, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
sidebar_label: 'Kafka Connector Sink on Confluent Cloud'
sidebar_position: 2
slug: /integrations/kafka/cloud/confluent/custom-connector-cloud
description: 'Guide to using the fully managed ClickHouse Connector Sinkon Confluent Cloud'
title: 'Integrating Confluent Cloud with ClickHouse'
keywords: ['Kafka', 'Confluent Cloud']
---

import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx';
import Image from '@theme/IdealImage';

# Integrating Confluent Cloud with ClickHouse

<div class='vimeo-container'>
<iframe src="//www.youtube.com/embed/SQAiPVbd3gg"
width="640"
height="360"
frameborder="0"
allow="autoplay;
fullscreen;
picture-in-picture"
allowfullscreen>
</iframe>
</div>

## Prerequisites {#prerequisites}
We assume you are familiar with:
* [ClickHouse Connector Sink](../kafka-clickhouse-connect-sink.md)
* Confluent Cloud

## The official Kafka connector from ClickHouse with Confluent Cloud {#the-official-kafka-connector-from-clickhouse-with-confluent-cloud}

### Installing on Confluent Cloud {#installing-on-confluent-cloud}
This is meant to be a quick guide to get you started with the ClickHouse Sink Connector on Confluent Cloud.
For more details, please refer to the [official Confluent documentation](https://docs.confluent.io/cloud/current/connectors/bring-your-connector/custom-connector-qs.html#uploading-and-launching-the-connector).

#### Create a Topic {#create-a-topic}
Creating a topic on Confluent Cloud is fairly simple, and there are detailed instructions [here](https://docs.confluent.io/cloud/current/client-apps/topics/manage.html).

#### Important notes {#important-notes}

* The Kafka topic name must be the same as the ClickHouse table name. The way to tweak this is by using a transformer (for example [`ExtractTopic`](https://docs.confluent.io/platform/current/connect/transforms/extracttopic.html)).
* More partitions does not always mean more performance - see our upcoming guide for more details and performance tips.

#### Gather your connection details {#gather-your-connection-details}
<ConnectionDetails />


#### Install Connector {#install-connector}
Install the fully managed ClickHouse Sink Connector on Confluent Cloud following the [official documentation](https://docs.confluent.io/cloud/current/connectors/cc-clickhouse-sink-connector/cc-clickhouse-sink.html).


#### Configure the Connector {#configure-the-connector}
During the configuration of the ClickHouse Sink Connector, you will need to provide the following details:
- hostname of your ClickHouse server
- port of your ClickHouse server (default is 8443)
- username and password for your ClickHouse server
- database name in ClickHouse where the data will be written
- topic name in Kafka that will be used to write data to ClickHouse

The Confluent Cloud UI supports advanced configuration options to adjust poll intervals, batch sizes, and other parameters to optimize performance.

#### Known limitations {#known-limitations}
* See the list of [Connectors limitations in the official docs](https://docs.confluent.io/cloud/current/connectors/cc-clickhouse-sink-connector/cc-clickhouse-sink.html#limitations)
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
sidebar_label: 'Kafka Connector Sink on Confluent Platform'
sidebar_position: 2
sidebar_position: 3
slug: /integrations/kafka/cloud/confluent/custom-connector
description: 'Using ClickHouse Connector Sink with Kafka Connect and ClickHouse'
title: 'Integrating Confluent Cloud with ClickHouse'
Expand All @@ -10,7 +10,7 @@ import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.md
import Image from '@theme/IdealImage';
import AddCustomConnectorPlugin from '@site/static/images/integrations/data-ingestion/kafka/confluent/AddCustomConnectorPlugin.png';

# Integrating Confluent Cloud with ClickHouse
# Integrating Confluent platform with ClickHouse

<div class='vimeo-container'>
<iframe src="//www.youtube.com/embed/SQAiPVbd3gg"
Expand All @@ -27,16 +27,16 @@ import AddCustomConnectorPlugin from '@site/static/images/integrations/data-inge
## Prerequisites {#prerequisites}
We assume you are familiar with:
* [ClickHouse Connector Sink](../kafka-clickhouse-connect-sink.md)
* Confluent Cloud and [Custom Connectors](https://docs.confluent.io/cloud/current/connectors/bring-your-connector/overview.html).
* Confluent Platform and [Custom Connectors](https://docs.confluent.io/cloud/current/connectors/bring-your-connector/overview.html).

## The official Kafka connector from ClickHouse with Confluent Cloud {#the-official-kafka-connector-from-clickhouse-with-confluent-cloud}
## The official Kafka connector from ClickHouse with Confluent Platform {#the-official-kafka-connector-from-clickhouse-with-confluent-platform}

### Installing on Confluent Cloud {#installing-on-confluent-cloud}
This is meant to be a quick guide to get you started with the ClickHouse Sink Connector on Confluent Cloud.
### Installing on Confluent platform {#installing-on-confluent-platform}
This is meant to be a quick guide to get you started with the ClickHouse Sink Connector on Confluent Platform.
For more details, please refer to the [official Confluent documentation](https://docs.confluent.io/cloud/current/connectors/bring-your-connector/custom-connector-qs.html#uploading-and-launching-the-connector).

#### Create a Topic {#create-a-topic}
Creating a topic on Confluent Cloud is fairly simple, and there are detailed instructions [here](https://docs.confluent.io/cloud/current/client-apps/topics/manage.html).
Creating a topic on Confluent Platform is fairly simple, and there are detailed instructions [here](https://docs.confluent.io/cloud/current/client-apps/topics/manage.html).

#### Important Notes {#important-notes}

Expand Down
3 changes: 2 additions & 1 deletion docs/integrations/data-ingestion/kafka/confluent/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,6 @@ title: 'Integrating Confluent Cloud with ClickHouse'

Confluent platform provides two options to integration with ClickHouse

* [ClickHouse Connect Sink on Confluent Cloud](./custom-connector.md) using the custom connectors feature
* [ClickHouse Connect Sink on Confluent Cloud](./confluent-cloud.md)
* [ClickHouse Connect Sink on Confluent Platform](./custom-connector.md) using the custom connectors feature
* [HTTP Sink Connector for Confluent Platform](./kafka-connect-http.md) that integrates Apache Kafka with an API via HTTP or HTTPS
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
sidebar_label: 'HTTP Sink Connector for Confluent Platform'
sidebar_position: 3
sidebar_position: 4
slug: /integrations/kafka/cloud/confluent/http
description: 'Using HTTP Connector Sink with Kafka Connect and ClickHouse'
title: 'Confluent HTTP Sink Connector'
Expand Down
1 change: 1 addition & 0 deletions sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -814,6 +814,7 @@ const sidebars = {
items: [
"integrations/data-ingestion/kafka/index",
"integrations/data-ingestion/kafka/kafka-clickhouse-connect-sink",
"integrations/data-ingestion/kafka/confluent/confluent-cloud",
"integrations/data-ingestion/kafka/confluent/custom-connector",
"integrations/data-ingestion/kafka/msk/index",
"integrations/data-ingestion/kafka/kafka-vector",
Expand Down