Add configurable option to keep tag files in storage backend

[moh2a]

Is your feature request related to a problem? Please describe.
Since PR goharbor/community#265, Harbor no longer lands tag files in the storage backend. I rely on a standby native Distribution registry reading from the same S3 backend for disaster recovery. Without tag files in S3, the failover registry cannot resolve images by tag and becomes unusable.

Describe the solution you'd like
Add a configurable option (e.g. registry_store_tags_in_storage: true|false, default false) that allows users to opt into keeping tag files in the storage backend, accepting the GC performance trade-off.

Describe the main design/architecture of your solution
A configuration flag in harbor.yml (or equivalent env variable). When enabled, Harbor preserves the pre-#22507 behavior: tag files are written to the storage backend during push. When disabled (default), current behavior applies (tags not landed in storage).

Describe the development plan you've considered
Conditional check in the push path where the tag file write was removed by #22507. If the flag is enabled, execute the original write. No changes to GC logic needed, the existing tag deletion in GC already handles cleaning up tag files in storage.

Additional context
Another user reported the same need: @jsingleton785 here : goharbor/community#265 (comment). This also aligns with @reasonerjt's suggestion (r2380819844) to provide an option for users who want tags in storage, and with Option 1 from the original proposal.

[wy65701436]

Thanks @moh2a for bringing this up.

I believe that you understand why we made the change, but let me summary again: decoupling tag metadata from object storage and managing it strictly in PostgreSQL was a deliberate design choice that significantly improved Harbor's reliability, push latency, and GC performance. And more details: goharbor/community#265 (comment)

While adding a toggle like registry_store_tags_in_storage: true seems simple, supporting it introduces substantial complexity across Harbor:

• Database & API: We would need to track storage tag states across creation, re-tagging, and deletion.
• Garbage Collection: GC would need to maintain dual execution paths (fast DB-only vs. slow S3 tag sweeps), increasing maintenance and testing overhead.
• Error Handling: Handling partial failures (e.g., DB succeeds but S3 tag write fails) would require complex rollback and retry logic.

We fully agree that Disaster Recovery is important, but maintaining dual tag storage logic across the Push, DB, API, and GC layers adds a heavy system-wide burden.

We’d love to hear your thoughts and feedback from the community:

• Could Harbor’s native replication or DB + S3 storage replication meet your DR requirements?
• Are there specific constraints in your setup that prevent using standard replication?

[moh2a]

Thanks for the detailed response. I understand the design rationale and the complexity a toggle would introduce.
To answer your questions: our constraint is that we need read-only registries in airgapped zones and DR sites where running a full Harbor instance with PostgreSQL isn't feasible. A lightweight native Distribution reading from the same S3 backend was our fallback, but that depends on tag files in storage.
Rather than asking Harbor to reintroduce tag writes to storage, we would explore a different approach: keep a single global Harbor instance as the source of truth and use regsync (or ocync) to replicate images to downstream Distribution registries deployed in DR and airgapped zones. This could give us:

• No Harbor changes needed: Harbor stays on the current architecture (tags in DB only), no dual paths or GC overhead.
• DR resilience: downstream Distribution registries would be full registries with their own tag files, not passive shared-storage readers.
• Flexibility: replication topology (full/partial sync, frequency) could be controlled independently from Harbor.

This means this feature request would no longer block us. Feel free to close or keep it open for other use cases.
Thank you again for your time.

[Vad1mo]

Everything through 2.14 wrote tag links to storage, and tooling built on that layout (standby registries, storage-level DR, offline mirrors) broke on upgrade.

On the GC concern: I think it goes away if the write side and the delete side are decoupled.

The expensive part is tagStore.Lookup (tagstore.go:144). It lists _manifests/tags/ and then GETs one object per tag in the repo, for every manifest deleted. Writing a tag link on push costs two small PUTs and isn't the problem.

So: restore the push-side write, and don't restore the sweep. Harbor already has the switch. delete_tag gates the v2 DELETE at garbage_collection.go:345. Turn it off and GC only calls registryctl's manifest delete, which passes an empty tags slice and does a single Delete on the revisions dir. That's actually fewer S3 calls than 2.15 does today, since today's default still makes the v2 round trip and finds nothing.

The trade is orphaned tag links. I think that's fine: link contents are digests, so a stale tag 404s rather than serving the wrong image, and GC is DB-driven so orphans don't hold blobs alive. They're 71-byte objects.

One thing to get right: delete_tag defaults to true, so the flag has to force it off, otherwise opting in silently reintroduces the slow path.