diff --git a/README.md b/README.md index 41c292e..2de88bc 100644 --- a/README.md +++ b/README.md @@ -69,6 +69,23 @@ make debezium.register.customers; make debezium.register.products; ``` +### Running Flink Applications + +Flink supports Java 11 but uses Java 8 due to its SQL (Hive) dependency. +The Flink SQL Application within this project is written in Kotlin for SQL Readability. + + +You can run it as an Application in IDEA. (it is not a Kotlin Application) +For Flink Application, the required dependencies are already included within the Production Docker Image or EMR cluster. +Therefore, they are set as 'Provided' dependencies in the Maven project, so to run them locally, +you can include the `Add dependencies with "provided" scope to classpath"` IDEA option as shown in the screenshot below. + +After running the Local Flink Application, you can access the Flink Job Manager UI from localhost:8081. +- [(Flink Job Manager UI of Local Application) http://localhost:8081/](http://localhost:8081/) + +![idea](./docs/images/idea.png) + + ## DBT Starter kit ```bash @@ -98,3 +115,17 @@ $ SELECT * FROM iceberg.staging.stg_regions LIMIT 10; make airflow.shell; airflow dags backfill dag_dbt --local --reset-dagruns -s 2022-09-02 -e 2022-09-03; ``` + +## Screenshots + +### Flink Job Manager UI +![flink](./docs/images/flink.png) + +### Kafka UI +![kafka](./docs/images/kafka.png) + +### Minio UI +![minio](./docs/images/minio.png) + +### Running Local Flink Application in IDEA +![kafka](./docs/images/application.png) diff --git a/docs/application.png b/docs/application.png new file mode 100644 index 0000000..92a529b Binary files /dev/null and b/docs/application.png differ diff --git a/docs/flink.png b/docs/flink.png new file mode 100644 index 0000000..6b95404 Binary files /dev/null and b/docs/flink.png differ diff --git a/docs/idea.png b/docs/idea.png new file mode 100644 index 0000000..5f4164e Binary files /dev/null and b/docs/idea.png differ diff --git a/docs/kafka.png b/docs/kafka.png new file mode 100644 index 0000000..2726499 Binary files /dev/null and b/docs/kafka.png differ diff --git a/docs/minio.png b/docs/minio.png new file mode 100644 index 0000000..799811b Binary files /dev/null and b/docs/minio.png differ