-
Notifications
You must be signed in to change notification settings - Fork 914
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Improvement] Define a general Helm chart configuration approach #6123
Comments
@pan3793 @zwangsheng @chgl Could you please have a look at it? |
Looks good to me! |
maybe just one minor detail: make sure all changed configs contribute to the checksum annotation for the statefulset so kyuubi is automatically restarted: https://github.com/apache/kyuubi/blob/master/charts/kyuubi/templates/kyuubi-statefulset.yaml#L41 |
@chgl I'm afraid it is impossible to calculate and keep updated checksum for entities in |
@dnskr , you're right of course! I hadn't considered the |
quick question, is restarting kyuubi when spark default config updated a must? |
@sudohainguyen Very good question. In the current implementation there is no Kyuubi auto restart on Spark config changes, but I assume it might be needed to stop running Spark immediately on some config changes. |
yeah I think it depends on how users expect it |
Created draft implementation #6521 |
Suggested approach and the draft implementation has an unsolved problem described below. Some properties affect both helm chart and the application. For instance, Ideally, it would be great to move such properties outside the default Any ideas? |
@dnskr Thanks for continuously working on this area. Carry my idea from the Slack channel - overriding configurations via |
@pan3793 I created improvement proposal based on your suggestion: Provide Kyuubi configuration properties with cli options. |
Code of Conduct
Search before asking
What would you like to be improved?
There is no clear common approach at the moment on how to configure Kyuubi deployment if the Helm chart is used.
I would like to discuss requirements, limitations and different options to choose one approach to follow and support it in the Kyuubi Helm chart configuration. The problem has been mentioned and discussed in multiple issues and PRs, so the idea is to collect all opinions in one place and make the decision.
Configuration system of the Apache Kyuubi
The configuration system allows to configure values using the following options (ordered from low to high prio):
Runtime options
JDBC Connection URLs
andSET Statements
Can be skipped in the discussion, because they used only when Kyuubi is up and running.
Runtime option
Environment variables
Configured by
{{ .Values.env }}
and{{ .Values.envFrom }}
value properties.The Helm chart users can specify environment variables to provide necessary configuration values with low effort if needed. The properties also allow to use provided (existing)
ConfigMaps
andSecrets
as the sources of environment variables, for instance:Static options
Represented by configuration files which should be located in each Kyuubi container in specific paths.
In general case, the easiest way to provide files into Kubernetes pod (container) is to mount
ConfigMap
orSecret
to a specific path.[IN PROGRESS] Requirements
Note: this section is in progress and subject to discuss.
ConfigMaps
under the hood from value properties ofvalue.yaml
file.Secrets
should never be created and managed by Helm chart because of security consideration!ConfigMaps
andSecrets
by resource name as a reference.ConfigMaps
andSecrets
with priority order.Multiple
ConfigMaps
andSecrets
might have key duplicates, so the implementation should clearly resolve the collision by merging keys in priority order.ConfigMaps
managed by the chart withConfigMaps
andSecrets
provided by user with priority order.The issue with key duplicates should be clearly resolved.
ConfigMaps
managed by the chart should have the lowest prio.value.yaml
file.Some configuration files might be huge and complex, so the idea is to prevent identation issues in
values.yaml
file.ConfigMaps
andSecrets
from one or many configuration files.Users might have a lot of xml, properties and other files, so the idea is to help users to create
ConfigMap
andSecret
resources in a simple way.How should we improve?
[IN PROGRESS] Approach
Note: this section is in progress and subject to discuss.
values.yaml
by system like Kyuubi, Hadoop, Spark, Trino etc.files
property to specify various files.Users can define files with any file name. Each entity within
files
property used as a key/value pair in the correspondingConfigMap
.from
property to specify list of existingConfigMaps
andSecrets
to be mounted to the configuration path of Kyuubi container.The implementation idea is to use Projected Volumes with
core/v1/SecretProjection
andcore/v1/ConfigMapProjection
entities.Also it will allow to merge
ConfigMap
created fromfiles
property with the entities fromfrom
property.xxxConfDir
property toxxxConf
property.ConfigMap
from file or directory, see Kubernetes docs .Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: