Does CDC topic support Kafka API? #37852
Unanswered
zhangruiddn
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi, I want to set up a ingestion pipeline streaming CDC changes of YDB into ClickHouse. ClickHouse has Kafka table engine that acts as a Kafka client (with client lib: https://github.com/confluentinc/librdkafka). Based on https://ydb.tech/docs/en/reference/kafka-api/examples?version=v25.3, YDB topics should support Kafka client. I tested this on regular YDB topics and it works. However, for CDC topics (created by YDB when adding changefeed), the Kafka client throws error like
% ERROR: Topic users/feed1 [0] error: Failed to fetch committed offset for group "my-group" topic users/feed1 [0]: Broker: Unknown topic or partitionIs this a known limitation for CDC topics, or I miss some settings here? Thank you for your help!
Beta Was this translation helpful? Give feedback.
All reactions