Skip to content

Commit e03bc50

Browse files
authored
Merge pull request #5746 from ClickHouse/md/clickpipes-kinesis-compression-docs
Add compression documentation for Kinesis ClickPipes
2 parents aefeb1b + d075c9e commit e03bc50

1 file changed

Lines changed: 18 additions & 1 deletion

File tree

docs/integrations/data-ingestion/clickpipes/kinesis/01_overview.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: 'Seamlessly connect your Amazon Kinesis data sources to ClickHouse
44
slug: /integrations/clickpipes/kinesis
55
title: 'Integrating Amazon Kinesis with ClickHouse Cloud'
66
doc_type: 'guide'
7-
keywords: ['clickpipes', 'kinesis', 'streaming', 'aws', 'data ingestion']
7+
keywords: ['clickpipes', 'kinesis', 'streaming', 'aws', 'data ingestion', 'compression', 'gzip', 'zstd', 'lz4', 'snappy']
88
integration:
99
- support_level: 'core'
1010
- category: 'clickpipes'
@@ -92,6 +92,23 @@ You have familiarized yourself with the [ClickPipes intro](../index.md) and setu
9292
The supported formats are:
9393
- [JSON](/interfaces/formats/JSON)
9494

95+
## Compression {#compression}
96+
97+
ClickPipes for Kinesis automatically detects and decompresses compressed records. Unlike Kafka, where the client library handles decompression transparently, Kinesis delivers raw bytes — ClickPipes handles this for you with no configuration required.
98+
99+
The following compression codecs are supported:
100+
101+
- **gzip**
102+
- **zstd**
103+
- **lz4**
104+
- **snappy** (framed format)
105+
106+
Compression is detected automatically via magic bytes in each record. If no known compression signature is found, the record is treated as uncompressed. The detected compression type is also surfaced during schema inference, so the sample data preview in the UI will correctly show the decompressed data.
107+
108+
:::note
109+
Auto-detection is safe for text-based formats like JSON and CSV, as printable ASCII characters will never collide with compression magic bytes.
110+
:::
111+
95112
## Supported data types {#supported-data-types}
96113

97114
### Standard types support {#standard-types-support}

0 commit comments

Comments
 (0)