[Bug]: images tagged by their own digest are hidden from metadb, search and the UI

[zaldre]

zot version

main @ b689b06 (also present in v2.1.20; the rule is unchanged since #1359, May 2023)

Describe the bug

An ordinary image whose tag is its own digest — repo:sha256-<64 hex> — is
silently excluded from metadb, and therefore from the search extension and the
UI, while remaining fully served over the distribution API.

This is the shape mirroring tools write when the source reference was
digest-pinned: oc-mirror renders repo@sha256:x as repo:sha256-x. On a
disconnected mirror seeded that way, whole repositories can be invisible in the
UI despite being present and pullable.

The cause is IsReferrersTag:

func IsReferrersTag(tag string) bool {
	referrersTagRule := regexp.MustCompile(`sha256\-[A-Za-z0-9]*$`)

	return referrersTagRule.MatchString(tag)
}

Two separate problems:

1. The rule is unanchored and over-permissive. It matches any tag merely
   ending in something digest-shaped, and [A-Za-z0-9]* accepts suffixes no
   digest can produce. notsha256-<hex>, v1-sha256-<hex> and sha256-zzz are
   all treated as referrers tags today.

2. Anchoring alone does not fix the reported case. sha256-<64 hex> is a
   genuine referrers shape, so a strict ^sha256-[a-f0-9]{64}$ still excludes
   it. Shape cannot distinguish a referrers fallback tag from an image tagged by
   digest, because both have exactly the same shape.

The two are distinguishable, just not by shape. Per the referrers tag
schema, the fallback tag encodes the subject's digest, so the index
it tags is by construction a different manifest from the one named. A tag that
names its own target therefore cannot be a referrers entry — it is an ordinary
image tagged by digest.

Call sites affected: pkg/meta/hooks.go (OnUpdateManifest,
OnDeleteManifest, OnGetManifest) and pkg/meta/parse.go (parseRepo,
both loops). pkg/extensions/sync also calls it, in
service.go and destination.go.

To reproduce

1. Configuration — any storage backend, with the search extension on:

   { "extensions": { "search": { "enable": true } } }

2. Client tool used — anything that can push a manifest under a chosen tag
   (regctl, skopeo, oras, or oc-mirror against a digest-pinned source).

   Push a manifest to repo, note its digest sha256:X, then tag that same
   manifest sha256-X — i.e. the tag names the digest of the manifest it
   resolves to. This is what oc-mirror produces from repo@sha256:X.

3. Seen error — no error is logged. The tag is served normally:

   • GET /v2/repo/tags/list → returns sha256-X
   • GET /v2/repo/manifests/sha256-X → 200, the manifest

   but it is absent from metadb:

   • ImageList(repo) → no result
   • GlobalSearch → does not return the repository
   • the UI does not list the repository at all when it has no other tag

   The same happens on a restart against existing storage, because parseRepo
   applies the same filter while re-indexing.

Expected behavior

An image tagged with its own digest is an ordinary image and should be indexed
in metadb like any other tag, appearing in ImageList, GlobalSearch and the
UI.

Genuine referrers fallback tags should stay excluded — the filter should
discriminate, not be disabled.

Screenshots

Not attached — the visible symptom is simply the absence of the repository from
the UI's repository list. Happy to add before/after captures if useful.

Additional context

Proposed fix — for review before I open a PR.

I have a working change and would like a read on whether the approach is the one
you want before raising it:

main...zaldre:zot:fix/referrers-tag-digest-named

(one commit, +79/−11 across 5 files; branch is based on b689b06 and would be
rebased onto current main before a PR)

It does three things:

1. Anchors IsReferrersTag to the exact schema, ^sha256-[a-fA-F0-9]{64}$.
2. Adds IsDigestNamedTag(tag, digest) — does this tag name its own target?
3. Has pkg/meta skip a reference only when it is referrers-shaped and not
   self-naming, through one shared isReferrersEntry helper used by all four
   call sites.

Both metadb entry points already have the digest in hand (digest in the hooks,
manifest.Digest in parseRepo), so no new plumbing is required.

Tests: new table tests in pkg/common/common_test.go cover the tag shapes above
and the self-naming discriminator. One existing fixture changed —
TestUpdateErrors asserted the early return using the tag sha256-123, which is
three hex characters and is not a referrers tag under the corrected rule; it now
uses a full-length referrers tag, which is what the test appears to have
intended. make check reports 0 issues across all build-tag combinations, and
pkg/common, pkg/meta, pkg/extensions/search,
pkg/extensions/search/convert, pkg/storage/... and pkg/extensions/sync
pass.

Verified manually with two manifests in one repository — one tagged
sha256-<its own digest>, one tagged sha256-<a different digest>:

	tags/list	ImageList	GlobalSearch
before — self-named	yes	—	—
before — referrers	yes	—	—
after — self-named	yes	indexed	repo returned
after — referrers	yes	—	—

Also verified that starting a patched binary against an existing storage
directory re-indexes previously hidden images with no re-push, so an upgrade
picks up content already on disk.

Open questions for maintainers

• pkg/extensions/sync call sites. I left them calling IsReferrersTag
  unchanged. Narrowing that rule does change sync behaviour for the malformed
  shapes — notsha256-<hex>, v1-sha256-<hex> and sha256-zzz were skipped as
  referrers tags and would now be replicated as ordinary tags. The
  sha256-<64 hex> case is classified the same as before, so sync is unaffected
  there. The consequence is that zot-to-zot sync still declines to replicate a
  digest-named tag that metadb would now index. Would you prefer the sync sites
  moved onto the self-naming discriminator in the same PR?

• OnGetManifest has no digest in hand, only the body, so the check there
  needs godigest.FromBytes(body). Happy to arrange it so the hash is only
  computed once the tag is known to be referrers-shaped, if you would rather not
  add work to the manifest GET path.

• Algorithm coverage. The current rule is sha256-only, and I kept it that
  way; the spec's fallback schema is <alg>-<encoded> for any algorithm. Worth
  widening separately, or leave as is?

If the approach looks right I will open the PR with a Fixes #<this issue>.
Happy to take it in a different direction if you would rather solve it elsewhere
— e.g. by inspecting the tagged manifest for referrers structure rather than
comparing digests.

[zaldre]

@andaaron I have discovered that this also breaks the sync function for digest tagged images. Downstream consumers get an error

2026/09/10 21:01:18 WARN: resolving upstream digest for "openshift4/metallb-rhel9-operator:sha256-a46dc6c202db1b7814a44692839dd4d5d5ce6bfc99a31698eb9abf1ccd7aab5a": GET https://mirror.mycompany.com:8443/v2/v16/openshift4/metallb-rhel9-operator/manifests/sha256-a46dc6c202db1b7814a44692839dd4d5d5ce6bfc99a31698eb9abf1ccd7aab5a: MANIFEST_UNKNOWN: manifest unknown; map[description:This error is returned when the manifest, identified by name and tag is unknown to the repository. reference:sha256-a46dc6c202db1b7814a44692839dd4d5d5ce6bfc99a31698eb9abf1ccd7aab5a]

I use Zot as a pull through cache for an Azure Container Registry and hit this bug. This image definitely exists in the upstream ACR. I've got a fix for it, however, since the root cause is very closely linked to this issue i'd like to understand whether you want it tracked here or as a separate issue

[andaaron]

OK will look into it.

[andaaron]

Thanks for the write-up and the branch, metadb approach looks right.
There is no need for a separate issue for sync.

Please rebase onto current main and include the sync call sites in the same PR before we merge.

1. Include sync in this PR. Leaving sync on plain IsReferrersTag means:

• Periodic SyncRepo still skips digest-named tags (pkg/extensions/sync/service.go).
• destination.go can still early-return for a referrers-shaped image manifest, which would drop a real digest-named image if it ever entered that path.
• Metadb would index content that sync never copies (zot-to-zot), and pull-through / on-demand stays broken for the same shape — which matches the follow-up on the issue (ACR pull-through → MANIFEST_UNKNOWN for …:sha256-<hex>).
  Please move the sync sites onto the same self-naming discriminator (at least the SyncRepo skip and the destination referrers early-return).

2. Gate godigest.FromBytes in OnGetManifest
   Computing the digest only after IsReferrersTag(reference) is true is preferable so ordinary GET traffic does not pay for an extra hash.

3. Keep sha256-only here, other algorithms can be a follow-up