Kyuubi k8s but engines onprem #5471
Replies: 4 comments 75 replies
-
|
Maybe yes (depends on what "already deployed engines onprem" means), at least for Spark. One of the typical deployments is, copying the Spark client and configurations into your Kyuubi server image, and Kyuubi server could submit your Spark application to your onprem cluster, e.g. Hadoop YARN. 
|
Beta Was this translation helpful? Give feedback.
-
|
This is how I want to deploy kyuubi:
Is it clear? May be I need to provide more details. |
Beta Was this translation helpful? Give feedback.
-
|
You're right. Our spark config param spark.driver.port is not set. It means that any port can be taken by driver. I'll try to place spark.driver.port=10100 to spark and kyuubi configs. |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
I've opened range of ports 20000-65000 and now it's connecting without error but I can't see any select statement results as curl output. |
Beta Was this translation helpful? Give feedback.
-
|
Looks like it's working. I can drop tables and select from them but without any output. |
Beta Was this translation helpful? Give feedback.
-
|
I've found the solution. Thank you for advices. It's ok now. |
Beta Was this translation helpful? Give feedback.
-
Not true. I have big expirience working with hive, spark, flink on prem and cloud, but kyuubi is new framework for me. I deployed it on prem and after digging docs could not find possible solution for my case. I understand than any question here seems newbie for you. I can't change it) |
Beta Was this translation helpful? Give feedback.
-
|
I have faced problem like this #4436, but leading slash is already removed. |
Beta Was this translation helpful? Give feedback.
-
|
does directly accessing the Kyuubi server work? if yes, check your zonde data |
Beta Was this translation helpful? Give feedback.
-
|
Yes, it does. I have connected to pod and tried to start kyuubi servce by myself, but got error: Ingress for web UI is also available. We have on prem kyuubi installation and it has the same error when I'm trying to connect via beeline terminal Onprem kyuubi can accept JDBC connection via 10009 port, but k8s installation not. I've tried port 443 but not succeed. |
Beta Was this translation helpful? Give feedback.
-
|
I forgot to use ssl and jks. Now beeline terminal works perfectly. |
Beta Was this translation helpful? Give feedback.
-
|
Unfortunatelly, k8s installations still is unavailabel throught JDBC. I tried to forward port 10009 on my pc but it's unavailable too. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @pan3793! Can you help me out with debugging kyuubi k8s ha? We have connected our 3 pods k8s deployment with 3 onprem zookeeper installation and enabled ha in kyuubi configuration but ingress still changes randomly pod to deal with API requests. We've enabled kyuubi.frontend.connection.url.use.hostname=true, but servers line look strage.
|
Beta Was this translation helpful? Give feedback.
-
|
I hope this docs is helpful |
Beta Was this translation helpful? Give feedback.
-
|
In short, if 10.233.29.108:10009 works for you, simplify forwarding the external TCP traffic via ingress to that endpoint |
Beta Was this translation helpful? Give feedback.
-
|
@pan3793 it's done already, but jdbc connection through port is unreachable.
|
Beta Was this translation helpful? Give feedback.
-
|
HTTP/HTTPS protocol does not work, I have emphasized the TCP protocol several times. |
Beta Was this translation helpful? Give feedback.
-
|
Okay, now I got it. |
Beta Was this translation helpful? Give feedback.
-
Hi all.
Is it possible to deploy kyuubi with kubernetes in stateless mode to use already deployed engines onprem (hive, spark, flink)? I didn't find straight answer in docs.
Beta Was this translation helpful? Give feedback.
All reactions