You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/integrate/etl/iceberg-risingwave.md
+25-15Lines changed: 25 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,9 +1,12 @@
1
1
(iceberg-risingwave)=
2
+
2
3
# Stream processing from Iceberg tables to CrateDB using RisingWave
3
4
4
-
[RisingWave] is a stream processing platform that allows configuring data sources, views on that data, and destinations where results are materialized.
5
+
[RisingWave] is a stream processing platform that allows configuring data
6
+
sources, views on that data, and destinations where results are materialized.
5
7
6
-
This guide aims to show you an example with data coming from an Iceberg table and aggregations materialized in real-time in CrateDB.
8
+
This guide aims to show you an example with data coming from an Iceberg table
9
+
and aggregations materialized in real-time in CrateDB.
quay.io/minio/minio server /data --console-address ":9001"
19
22
```
20
23
21
-
And let's create a bucket called `warehouse`, for this point a browser to http://localhost:9001 , login with `minioadmin` / `minioadmin` , and click on "Create bucket", enter "warehouse", and click again on "Create bucket".
24
+
And let's create a bucket called `warehouse`, for this point a browser to
25
+
http://localhost:9001 , login with `minioadmin` / `minioadmin` , and click on
26
+
"Create bucket", enter "warehouse", and click again on "Create bucket".
On the first console we will use [PyIceberg] and [IPython] to create an Iceberg table, and later we will add data and see how aggregations materialize in CrateDB in real-time.
42
+
On the first console we will use [PyIceberg] and [IPython] to create an Iceberg
43
+
table, and later we will add data and see how aggregations materialize in
44
+
CrateDB in real-time.
38
45
39
-
On the second console we will do the RisingWave and CrateDB setups, and we will leave a Python script running for the streaming of changes.
46
+
On the second console we will do the RisingWave and CrateDB setups, and we will
47
+
leave a Python script running for the streaming of changes.
40
48
41
49
And on the 3rd console we will review how data appears in CrateDB.
42
50
43
51
## Creating an Iceberg table
44
52
45
-
Let's start on the first console.
46
-
We use a Python script to create an Iceberg table on the bucket we created earlier on Minio, and as we want to keep things simple, we will use an ephemeral in-memory catalog.
53
+
Let's start on the first console. We use a Python script to create an Iceberg
54
+
table on the bucket we created earlier on Minio, and as we want to keep things
55
+
simple, we will use an ephemeral in-memory catalog.
0 commit comments