Operator doesn't wait for user secrets before provisioning users

[amrut-asm]

Expected Behavior

The operator should wait for user secrets (specified within tenant.uesrs) to be created before provisioning the users. It shouldn't just not create any users in case the expected secrets are missing.

Current Behavior

Operator doesn't wait for user secrets before provisioning users

Your Environment

• Version used (minio-operator): minio-operator Helm Chart (version: 7.1.1)
• Environment name and version (e.g. kubernetes v1.17.2): Talos 1.9.6 (Kubernetes v1.32.0)
• Operating System and version (uname -a): Talos Linux

[ramondeklein]

We expect the user secrets to be present during tenant creation. We don't want to hold provisioning of the tenant, because of the user secrets are not present. Why can't you create the user secrets before creating the tenant?

[amrut-asm]

We expect the user secrets to be present during tenant creation. We don't want to hold provisioning of the tenant, because of the user secrets are not present. Why can't you create the user secrets before creating the tenant?

Because there might be a delay between the creation of the secret and the provisioning of the tenant (maybe the secret is created after the tenant using Flux or something similar). If tenant.users is ignored when the secret isn't present then I'm not exactly sure why we even proceed with the tenant provisioning.

Also, what about the case when a new user secret is added to tenant.users? The operator currently does not create new users (added afterwards) because user provisioning happens just once (unlike buckets). What's the reason behind this behavior?

[ramondeklein]

Provisioning buckets and users during cluster creation wasn't the best concept in MinIO operator. Typically, Kubernetes operators try to keep objects synchronized at all times. Creating buckets and users via the tenant seems to be a comfortable solution, but it's hard to make it work correctly. In hindsight, we shouldn't have added this. We just can't remove it anymore, because that would break current setups.

We thought of creating MinIO User and Bucket CRDs that would be synchronized with the actual MinIO objects. But that's a pain too, because these objects can be modified using mc too and that would render them useless. That's why we didn't proceed with that plan.

Most of our customers just use mc to create users once provisioning is complete. There is a community Terraform provider -that seems to be well maintained-, so you may want to try that. We only provide the mc tool (that can also be run inside a pod) to provision the cluster.

[alekc]

This should be explicitely stated in helm charts. People use operator and assume that it... operates. The fact that user field is completely unused once the cluster has been deployed (or there is a race condition) is completely unexpected.