[bitnami/airflow] Allow customizing the empty-dir volume parameters

[NeonSludge]

Description of the change

This PR makes it possible to customize the properties of the currently hardcoded empty-dir volume used to hold temporary Airflow data (writeable venv, DAGs fetched from git repositories, temporary files, etc.). In addition to a regular emptyDir volume (current behavior) it also provides a choice of a memory-backed emptyDir and a generic ephemeral volume backed by a PVC.

The PR also adds a couple of currently missing empty-dir mounts for the Airflow local settings file (airflow_local_settings.py).

Benefits

Using emptyDir volumes to store what might be a non-negligible amount of data is sometimes not desirable. This change keeps the old behavior by default but provides more options for ephemeral data storage.

Possible drawbacks

Applicable issues

Additional information

Checklist

• Chart version bumped in Chart.yaml according to semver. This is not necessary when the changes only affect README.md files.
• Variables are documented in the values.yaml and added to the README.md using readme-generator-for-helm
• Title of the pull request follows this pattern [bitnami/<name_of_the_chart>] Descriptive title
• All commits signed off and in agreement of Developer Certificate of Origin (DCO)

[jotamartos]

Thanks for your contribution. Can you take a look at my suggestions?

[NeonSludge]

Force-pushed a version with all the suggested changes.

[jotamartos]

Sorry for not asking this before, there are 2 changes that are not related to the emptyDir. Could you please clarify why did you add them? I suggest you create a separate PR for them as it's a separate issue.

[jotamartos]

One final question, why did you include this change? This is not related to the emptyDir volume.

[NeonSludge]

This mount is necessary to pass the Airflow local settings file to all components that might need it.

For example, it can be used to change the SQLAlchemy connection parameters which are expected to be passed as a dictionary. Here is an example of doing this via the bitnami/airflow chart values:

localSettings: |-
  my_args = {
      "keepalives": 1,
      "keepalives_idle": 240,
      "keepalives_interval": 15,
      "keepalives_count": 8,
      "sslmode": "disable"
  }

  my_async_args = {
      "ssl": False
  }
overrideConfiguration:
  database:
    sql_alchemy_connect_args: airflow_local_settings.my_args
    sql_alchemy_connect_args_async: airflow_local_settings.my_async_args

The problem is that this mount is currently missing from the wait-for-db-migrations init containers and from the default worker pod template. At the same time the overrideConfiguration value is being correctly passed everywhere and this leads to errors when it has references to the missing local settings file.

The mount references the same emptyDir volume so I would argue it is related but I can certainly split this into a second PR if you deem it necessary.