Does kyuubi support spark sql, flink sql, hive sql on k8s?

[kissycn]

Does kyuubi support spark sql, flink sql, hive sql on k8s?

[pan3793]

That depends on

1. whether the computing engine itself supports running on K8s
2. whether Kyuubi supports launching the computing engine in K8s mode

Specific to the engines you listed

• Spark
• Flink (2 is not implemented)
• Hive (neither 1 nor 2 is sufficient)

[kissycn]

First, thanks for your reply my bro
2、For flink, does the community have a plan for Kyuubi supports launching the computing engine in K8s mode?
I don’t know the implementation details of kyuubi, so I don’t know whether the implementation difficulty of flink sql on k8s is high or not. Is it feasible for me or other community members to contribute to the difficulty?
3、For hive, hive supports spark as the execution engine, so theoretically hive sql on spark can run on k8s'

[pan3793]

@link3280 may have ideas for 2

for 3, the engine of Kyuubi is almost a HiveServer2 instead of MR/Spark/Tez

[pan3793]

@iodone I remember that HiveServer on YARN/K8s cases mentioned in your previous share, do you have more details for that?

[link3280]

@kissycn

2、For flink, does the community have a plan for Kyuubi supports launching the computing engine in K8s mode?

I think so. We're doing it step by step.

Is it feasible for me or other community members to contribute to the difficulty?

Definitely, contributions are welcome. You could start by supporting kubernetes-session mode in Flink local engine if you are interested.

[kissycn]

ok i'll try it