Read Apache Hudi data on Onehouse.ai using StarRocks (WIP)

[alberttwong]

Prerequisites

For this tutorial you need to:

• Have a MySQL client.
• A StarRocks or CelerData database cluster.
• An account on Onehouse.ai
• Import data into Onehouse.ai
• Create a StarRocks External Catalog and query the data.
• Query the data on Onehouse.ai

Have a MySQL client

This is out of scope for the tutorial.

A StarRocks or CelerData database cluster

This is out of scope for the tutorial.

An account on Onehouse.ai

This is out of scope for the tutorial.

Create a StarRocks External Catalog and query the data.

drop catalog hudi_catalog_glue;
CREATE EXTERNAL CATALOG hudi_catalog_glue PROPERTIES (
  "type" = "hudi",
  "hive.metastore.type" = "glue",
  "aws.glue.use_instance_profile" = "false",
  "aws.glue.region" = "us-west-2",
  "aws.glue.access_key" = "XXXX",
  "aws.glue.secret_key" = "YYYYY",
  "aws.s3.use_instance_profile" = "false",
  "aws.s3.region" = "us-west-2",
  "aws.s3.access_key" = "XXXXXXX",
  "aws.s3.secret_key" = "YYYYYY"
);
SHOW CATALOGS;
SHOW DATABASES from hudi_catalog_glue;
set CATALOG hudi_catalog_glue;
use nyctaxi_onehouse;
show tables;
select * from taxi_green_rt;