Skip to content

Commit 498ac65

Browse files
committed
Minor edits
1 parent b602e48 commit 498ac65

File tree

1 file changed

+10
-14
lines changed

1 file changed

+10
-14
lines changed

data-transforms/go/iceberg-fanout/README.adoc

Lines changed: 10 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -74,13 +74,13 @@ Consider this approach for use cases like:
7474
└─────┬───────┬────────┬───────────┘
7575
│ │ │
7676
▼ ▼ ▼
77-
┌─────┴───────┴────────┴──────┐
78-
│ Iceberg-Enabled Topics │
79-
│ • orders │
80-
│ • inventory │
81-
│ • customers │
82-
│ (mode: value_schema_id_prefix) │
83-
└─────────┬────────────────────┘
77+
┌─────┴───────┴────────┴───────────
78+
│ Iceberg-Enabled Topics
79+
│ • orders
80+
│ • inventory
81+
│ • customers
82+
│ (mode: value_schema_id_prefix)
83+
└─────────┬────────────────────────
8484
│ Redpanda:
8585
│ • Validates against Avro Schema
8686
│ • Writes to Iceberg tables in Tiered Storage
@@ -191,13 +191,11 @@ During deployment, the transform registers three Avro schemas with Schema Regist
191191
* `inventory-value` - Inventory message schema (product_id, quantity, timestamp)
192192
* `customers-value` - Customer message schema (customer_id, name, timestamp)
193193
+
194-
The `CreateSchema` method is idempotent, so redeploying the transform is safe and won't create duplicate schemas.
195-
+
196194
For each routed message, the transform:
197195
+
198-
1. Converts JSON data to Avro format using the goavro library
199-
2. Adds Schema Registry wire format header: `[0x00][schema_id (4 bytes BE)][avro bytes]`
200-
3. Routes the encoded message to the appropriate output topic
196+
. Converts JSON data to Avro format using the goavro library
197+
. Adds Schema Registry wire format header: `[0x00][schema_id (4 bytes BE)][avro bytes]`
198+
. Routes the encoded message to the appropriate output topic
201199

202200
. Verify the transform is deployed:
203201
+
@@ -248,8 +246,6 @@ customers-value 1 3 AVRO
248246
inventory-value 1 2 AVRO
249247
orders-value 1 1 AVRO
250248
----
251-
+
252-
TIP: The Iceberg-enabled output topics use `value_schema_id_prefix` mode. Messages include Schema Registry wire format (magic byte + schema ID + Avro payload), which Redpanda uses to validate against the registered Avro schemas before writing to Iceberg tables.
253249

254250
=== Build and run the producer
255251

0 commit comments

Comments
 (0)