Helm support `registries.conf` for OCI configuration/auth info #391

gjenkins8 · 2025-02-27T15:29:53Z

No description provided.

mtrmac

Just some drive-by notes.

mtrmac · 2025-03-03T22:36:18Z

hips/hip-XXXX.md

+```toml
+# registries.conf
+[[registry]]
+prefix = "oci.example.com"
+```


In registries.conf, it’s not necessary to add a no-op entry for a registry. (In fact it’s currently somewhat annoying to do, although that would probably be best fixed in the parser package.)

mtrmac · 2025-03-03T22:39:50Z

hips/hip-XXXX.md

+```
+
+```json
+# auth.json


For better or worse, auth.json is intentionally placed into tmpfs filesystems cleared on reboot. (The tools may read other locations in the home directory, but they, without specific instruction otherwise, don’t update them.)

That might be a fairly disruptive change to users of “login”, so it probably should be discussed as such.

hips/hip-XXXX.md

FeynmanZhou · 2025-03-05T01:14:31Z

hips/hip-XXXX.md

+
+Including support for repository prefixes (allowing different credentials for different prefixes), registry mirrors, and registry "aliasing". Features Helm would like to introduce. But is currenly blocked by a lack of mechanism to store detail
+
+(in particular, the existing registry configuration Helm uses, Docker’s `$HOME/.docker/config.json`, etc do not support registry aliases nor prefixes)


Is do not support registry aliases nor prefixes the only limitation of Docker’s $HOME/.docker/config.json? I would expect that this proposal could clarify more on the problems and limitations here

@FeynmanZhou are there other limitations you want to highlight?

I'm not sure what limitations @FeynmanZhou is referring to, but one limitation this may or may not address is namespaced auth. Like with Harbor, you might have one credential for oci://localhost/production and another for oci://localhost/development and config.json as I recall only supports auth at the domain level.

FeynmanZhou · 2025-03-05T01:16:34Z

hips/hip-XXXX.md

+## Specification
+
+Helm will utilize the `registries.conf` specification when determinging OCI registry information (authentication credentials, etc):
+<https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md>


You may want to reference a versioned link instead of main branch in case the link is broken in the main branch

mattfarina · 2025-03-13T13:15:23Z

hips/hip-XXXX.md

+
+## Open issues
+
+- Support for registies.conf in ORAS


Since we use ORAS for this, do we need to get agreement from them before this moves forward?

Hi @mattfarina ,

@sabre1041 brought up this proposal to the ORAS community meeting on Mar 5. There is also a GitHub issue tracked in oras-go oras-project/oras-go#918.

As this is a dependency for ORAS from Helm v4, what's the expected timeline to getregisties.conf supported in oras-go?

hips/hip-XXXX.md

scottrigby

I like the idea of supporting this in Helm.

We may want to note in this HIP that registry aliases would help with airgapped environments—an important use case for Helm users—where an in-cluster registry would replace external registries defined as the storage for chart dependencies.

There are existing means in Helm to overwrite a registry for container images in airgap—many charts make images/oci registries configurable, but even if they do not, this can be done with a post-renderer. However, there is currently no way to do this with the repository specified by chart dependencies without manually manipulating the Chart.yaml.

scottrigby · 2025-03-22T15:51:23Z

Note that the Podman tools that standardize registries.conf have been accepted as a CNCF Sandbox project: cncf/sandbox#309

scottrigby · 2025-03-22T15:51:58Z

Also note this related issue: helm/helm#13615

Signed-off-by: George Jenkins <[email protected]>

sabre1041 · 2025-11-04T01:34:11Z

hips/hip-XXXX.md

+---
+hip: 9999
+title: "registries.conf support for OCI registry management"
+authors: [ "George Jenkins <[email protected]>", "Andy Block <[email protected]>" ]


Suggested change

authors: [ "George Jenkins <[email protected]>", "Andy Block <andy@redhat.com>" ]

authors: [ "George Jenkins <[email protected]>", "Andrew Block <andy.block@gmail.com>" ]

sabre1041 · 2025-11-04T01:35:36Z

hips/hip-XXXX.md

+
+- support for repository prefixes (allowing different credentials for different prefixes)
+- registry mirrors
+- registry "aliasing"—features


Add:

allowing/denying registries

sabre1041 · 2025-11-04T01:46:49Z

hips/hip-XXXX.md

+- registry mirrors
+- registry "aliasing"—features
+
+Helm would like to introduce features depending on these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.


Suggested change

Helm would like to introduce features depending on these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.

Helm would like to introduce features leaning upon these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.

sabre1041 · 2025-11-04T01:50:57Z

hips/hip-XXXX.md

+- registry "aliasing"—features
+
+Helm would like to introduce features depending on these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.
+With `registries.conf` having been standardized by other container ecosystem CLI tools (podman, buildah, skopeo, etc.).


Should we also mention that these are also CNCF hosted tools/projects

sabre1041 · 2025-11-04T01:55:25Z

hips/hip-XXXX.md

+
+`registries.conf` was picked as being a format intended for consumption beyond the Docker application container ecosystem.
+
+ORAS (the library Helm uses for supporting OCI functionality) has planned support for `registries.conf` client-side OCI registry management.


Should we include reference(s) to the ORAS specific features?

sabre1041 · 2025-11-04T04:21:32Z

hips/hip-XXXX.md

+Helm will utilize the `registries.conf` specification when determining OCI registry information (authentication credentials, etc.):
+<https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md>
+
+`registries.conf` will be preferred either when a `registries.conf` file already exists on the user's system, or when Helm supports and a user utilizes functionality that cannot be supported by Docker's configuration file in the future.


Should we indicate where we will search for entries?

sabre1041 · 2025-11-04T04:41:05Z

hips/hip-XXXX.md

+}
+```
+
+Helm will use ORAS v3 for updating (and reading) `registries.conf` (TODO: link to ORAS v3 `registries.conf` implementation).


Should we call out the fact that ORAS v3 development has yet to begin as of this date

I wouldn't characterize it as hasn't begun. An oras-go v2 branch was cut and there are several PRs out for v3. Nothing has merged though.

sabre1041 · 2025-11-04T04:49:40Z

hips/hip-XXXX.md

+
+To account for existing configuration in Docker’s registry configuration, Helm will prefer `registries.conf` when resolving OCI registries (including credentials), and fall back to the existing storage mechanism if `registries.conf` does not contain an entry for the required OCI registry (including if `registries.conf` does not exist).
+
+Helm must expect (and even encourage) users to utilize other tooling to manage `registries.conf`.


Or at least provide some form of guidance

Not quite following?

Do you mean guidance on using other tools to manage? (if so, I think this is a detail we can leave for docs)

sabre1041 · 2025-11-04T04:50:52Z

hips/hip-XXXX.md

+
+1. A corrupt `registries.conf` will cause an error for existing workflows
+2. An invalid or incompatible with Helm `registries.conf` entry for the given OCI registry will cause the user's workflow to fail
+3. Helm’s preference for `registries.conf` will break users who assume credentials are stored in Docker’s registry configuration


Do we want to include an explicit option to opt out of the feature and fall back. That way, we are covered for all scenarios

TerryHowe · 2025-11-04T11:47:43Z

hips/hip-XXXX.md

+    "auth": "Zm9vOmJhcgo="
+  }
+}
+```


Do you want to mention policy.json around here since that is used for allow/deny?

I guess along these lines, hopefully certs.d would be supported. That could be out of the scope for this document though.

right, I want to focus this HIP on initial registries.conf/auth.json support (ie. feature parity and switch over from Docker conf).

And leave extending to support future functionality to either:

ORAS improvements

Explicit HIPs for additional Helm behavior changes

TerryHowe · 2025-11-04T11:49:19Z

hips/hip-XXXX.md

+}
+```
+
+Helm will use ORAS v3 for updating (and reading) `registries.conf` (TODO: link to ORAS v3 `registries.conf` implementation).


I wouldn't characterize it as hasn't begun. An oras-go v2 branch was cut and there are several PRs out for v3. Nothing has merged though.

TerryHowe · 2025-11-04T11:53:52Z

hips/hip-XXXX.md

+Helm's fallback to Docker's registry configuration ensures the vast majority of existing user workflows remain the same.
+
+However, there are three potential incompatibility scenarios:
+


I think there is another possibility here where the user doesn't have a registries.conf but has a auth.json My understanding is the registries.conf is not required.

yeah, when I originally wrote this, I thought registries.conf was always paired with auth.json.

https://github.com/helm/community/pull/391/files#r1978339008 mentions the same

I have (significantly) reworked the language to include auth.json. Strictly, auth.json is all that is needed to replace docker/config.json. But the overall motivation is to be able to utilize features implemented by registries.conf.

Signed-off-by: George Jenkins <[email protected]>

gjenkins8 · 2025-11-07T22:18:11Z

hips/hip-XXXX.md

+
+`auth.json`:
+
+1. Location specified in environment variable `REGISTRY_AUTH_FILE` if set


need to consider that ${XDG_RUNTIME_DIR} is tmpfs on linux by default

https://github.com/helm/community/pull/391/files#r1978342074

Signed-off-by: George Jenkins <[email protected]>

pull-request-size bot added the size/L label Feb 27, 2025

gjenkins8 force-pushed the gjenkins/registries.conf_support branch from 99c6d56 to cea4a1d Compare February 27, 2025 15:54

mtrmac reviewed Mar 3, 2025

View reviewed changes

sabre1041 reviewed Mar 5, 2025

View reviewed changes

hips/hip-XXXX.md Outdated Show resolved Hide resolved

FeynmanZhou reviewed Mar 5, 2025

View reviewed changes

mattfarina reviewed Mar 13, 2025

View reviewed changes

scottrigby reviewed Mar 16, 2025

View reviewed changes

scottrigby mentioned this pull request Mar 22, 2025

Configure helm to use a oci registry mirror/proxy without setting HTTP(S)_PROXY helm/helm#13615

Closed

scottrigby mentioned this pull request May 23, 2025

Fix 3.18.0 regression: registry login with scheme helm/helm#30887

Merged

3 tasks

blakepettersson mentioned this pull request Jul 23, 2025

Improved perfromance helm execution argoproj/argo-cd#23402

Open

SimonTheLeg mentioned this pull request Aug 11, 2025

Full automatic mirroring of helm charts and images used by KKP kubermatic/kubermatic#14885

Open

work

1b89550

Signed-off-by: George Jenkins <[email protected]>

gjenkins8 force-pushed the gjenkins/registries.conf_support branch from 07c67e8 to 1f6c412 Compare November 3, 2025 22:47

gjenkins8 changed the title ~~H4HIP: Helm support registries.conf for OCI configuration/auth info~~ Helm support registries.conf for OCI configuration/auth info Nov 3, 2025

rework

6335ac4

Signed-off-by: George Jenkins <[email protected]>

gjenkins8 force-pushed the gjenkins/registries.conf_support branch from 1f6c412 to 6335ac4 Compare November 3, 2025 22:59

gjenkins8 marked this pull request as ready for review November 3, 2025 23:00

sabre1041 reviewed Nov 4, 2025

View reviewed changes

TerryHowe reviewed Nov 4, 2025

View reviewed changes

update

7588b79

Signed-off-by: George Jenkins <[email protected]>

gjenkins8 commented Nov 7, 2025

View reviewed changes

gjenkins8 added 3 commits November 7, 2025 15:13

features

850727a

Signed-off-by: George Jenkins <[email protected]>

cleanup

b96cb0f

Signed-off-by: George Jenkins <[email protected]>

Remove HELM_OCI_AUTH_CONFIG_PREFERENCE

b00f334

Signed-off-by: George Jenkins <[email protected]>


		Including support for repository prefixes (allowing different credentials for different prefixes), registry mirrors, and registry "aliasing". Features Helm would like to introduce. But is currenly blocked by a lack of mechanism to store detail

		(in particular, the existing registry configuration Helm uses, Docker’s `$HOME/.docker/config.json`, etc do not support registry aliases nor prefixes)

	authors: [ "George Jenkins <[email protected]>", "Andy Block <andy@redhat.com>" ]
	authors: [ "George Jenkins <[email protected]>", "Andrew Block <andy.block@gmail.com>" ]

	Helm would like to introduce features depending on these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.
	Helm would like to introduce features leaning upon these functionalities. Supporting `registries.conf` would enable them without Helm having to create or implement its own mechanisms.


		`registries.conf` was picked as being a format intended for consumption beyond the Docker application container ecosystem.

		ORAS (the library Helm uses for supporting OCI functionality) has planned support for `registries.conf` client-side OCI registry management.


		To account for existing configuration in Docker’s registry configuration, Helm will prefer `registries.conf` when resolving OCI registries (including credentials), and fall back to the existing storage mechanism if `registries.conf` does not contain an entry for the required OCI registry (including if `registries.conf` does not exist).

		Helm must expect (and even encourage) users to utilize other tooling to manage `registries.conf`.

		Helm's fallback to Docker's registry configuration ensures the vast majority of existing user workflows remain the same.

		However, there are three potential incompatibility scenarios:


		`auth.json`:

		1. Location specified in environment variable `REGISTRY_AUTH_FILE` if set

Helm support registries.conf for OCI configuration/auth info #391

Are you sure you want to change the base?

Helm support registries.conf for OCI configuration/auth info #391

Conversation

gjenkins8 commented Feb 27, 2025

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FeynmanZhou Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

scottrigby left a comment

Choose a reason for hiding this comment

Uh oh!

scottrigby commented Mar 22, 2025

Uh oh!

scottrigby commented Mar 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gjenkins8 Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Helm support `registries.conf` for OCI configuration/auth info #391

Helm support `registries.conf` for OCI configuration/auth info #391

FeynmanZhou Mar 13, 2025 •

edited

Loading

gjenkins8 Nov 7, 2025 •

edited

Loading