[wip] Add new chart and docs

[rsarm]

This can be tested with a local helm repo

Launch the local helm repo

mkdir build/helm/helm-repo
cd build/helm/helm-repo
python -m http.server 8000

Create the charts in the repo

cd build/helm

helm_repo_dir=helm-repo

helm repo remove firecrestv2

helm dependency update firecrest
cp firecrest/charts/*tgz $helm_repo_dir

cd $helm_repo_dir
helm repo index .

 # `python -m http.server` must be running in $helm_repo_dir
helm repo add firecrestv2 http://localhost:8000/ 
helm repo update

When this is ready, you can do helm search repo firecrestv2

# NAME                 	CHART VERSION	APP VERSION                 	DESCRIPTION
firecrestv2/firecrest	2.2.1        	2.2.1      A Helm chart for deploying FirecREST
firecrestv2/keycloak 	1.0.0        	26.0.7     A Helm chart for deploying a Keycloak identity ...
firecrestv2/minio    	1.0.0        	2022       A Helm chart for deploying a MinIO object stora...
firecrestv2/secrets  	1.0.0        	1.0.0      A Helm chart for deploying a MinIO object stora...
firecrestv2/slurm    	1.0.0        	24.11.3    A Helm chart for deploying a Slurm cluster in a...

Deploy the demo environment

To deploy the demo environment, begin by installing the necessary subcharts that provide these services. These include:

• Keycloak: for authentication and identity management.

• MinIO: as an S3-compatible storage backend.

• Slurm: the Slurm cluster that FirecREST will target.

The firecrestv2/secrets chart is used to deploy a Kubernetes Secret containing user SSH keys.
These keys are mounted into the pod where FirecREST runs, allowing it to authenticate with the underlying HPC system.

Make sure all the deployments are dene within a same Kubernetes namespace (in this example, fcv2-demo).
If it doesn’t exist yet, it will be created with the first command.

helm install keycloak  firecrestv2/keycloak -nfcv2-demo -f values-keycloak.yaml --create-namespace
helm install minio     firecrestv2/minio    -nfcv2-demo -f values-minio.yaml
helm install slurm     firecrestv2/slurm    -nfcv2-demo
helm install secrets   firecrestv2/secrets  -nfcv2-demo

the values files are

• values-keycloak.yaml

keycloak:
  url: keycloak-helm.iram-tds.tds.cscs.ch

• values-minio.yaml

host: minio-demov2.iram-tds.tds.cscs.ch

• values-firecrest.yaml.txt

Once these components are deployed, FirecREST itself can be installed

helm install firecrest firecrestv2/firecrest -nfcv2-demo -f values-firecrest.yaml

At this point, all the core services should be up and running.

You can verify the deployment by checking the status of the pods in the namespace:

kubectl get pods -n fcv2-demo

helm delete keycloak  -nfcv2-demo
helm delete minio     -nfcv2-demo
helm delete slurm     -nfcv2-demo
helm delete secrets   -nfcv2-demo
helm delete firecrest -nfcv2-demo

[theely]

Hi Rafael,
This is great work! I tried to set up the environment locally, and it was mostly working.
I have some feedback:

1. Given that we are already providing multiple environments (demo image, docker compose, helm prod deployment), this additional helm repo might confuse users. We should be careful with naming. If this set of helm charts is meant to run a local firecrest environment for evaluation or dev purposes I suggest renaming the root folder to: build/helm/firecrest-dev-env or something similar.
2. Could we push this concept further and provide templated value files? It would be awesome if one could start the whole environment with a single command. Similar to what openFGA is providing:

helm install openfga openfga/openfga \
  --set datastore.engine=postgres \
  --set datastore.uri="postgres://postgres:[email protected]:5432/postgres?sslmode=disable" \
  --set postgresql.enabled=true \
  --set postgresql.auth.postgresPassword=password \
  --set postgresql.auth.database=postgres

Link to openfga setup page: https://artifacthub.io/packages/helm/openfga/openfga

[rsarm]

Why "mostly working"? Something didn't work right?

It already should work as you describe. I haven't tried it, but I think you can pass the values.yaml or pass the parameters independently with the cli using --set.

[theely]

The doc in the PR is not super linear, I got confused. First the chart repo needs to be created and only then the http server should be launched inside the same folder.

Regarding the point 2) I would recommend extending the main Chart file and including all required dependencies, like this:
https://github.com/openfga/helm-charts/blob/main/charts/openfga/Chart.yaml
Also, I would include a value file that contains the default values that can be overwritten via the --set option. like this:
https://github.com/openfga/helm-charts/blob/main/charts/openfga/values.yaml

The end goal is to provide users with a single command to load all containers at once:

helm install firecrest firecrestv2/firecrest-dev-env
 --set scheduler=slurm \
 --set s3.enabled=true \
 --set s3.host=minio-demov2.iram-tds.tds.cscs.ch \
 --set keycloak.url= keycloak-helm.iram-tds.tds.cscs.ch \

And a single one to uninstall all
helm uninstall firecrest

[jpdorsch]

Hi @rsarm and @theely ,

I will close this PR as it was replaced by #121 based on this PR.

Thanks for the work done, it was really helpful.

Cheers!