Skip to content

Auth token fetch is not retried on transient network errors (unlike blob/manifest fetches) #6981

Description

@cmur2

Contributing guidelines and issue reporting guide

Well-formed report checklist

  • I have found a bug that the documentation does not mention anything about my problem
  • I have found a bug that there are no open or closed issues that are related to my problem
  • I have provided version/information about my environment and done my best to provide a reproducer

Description of bug

Bug description

Intermittently, a transient network error during the registry OAuth token exchange fails the whole build and we found that re-running the same build succeeds.

Observed error:

failed to fetch oauth token: Post "https://auth.docker.io/token":
read tcp ...->104.18.43.178:443: read: connection reset by peer

If I see that correctly, the blob/manifest fetches are retried via util/resolver/retryhandler (ECONNRESET, EOF, timeouts, 5xx, with backoff) but the token fetch in util/resolver/authorizer.go (fetchTokenFetchToken/FetchTokenWithOAuth) is not: a network-level error on the auth request therefore propagates up.

Expected behaviour: transient network errors on the token/auth request are retried.

Reproduction

I unfortunately don't have a deterministic reproducer due to the transitive fault. Reproducing would require a Docker registry that injects a connection reset fault at the right time (for us it is auth.docker.io).

Version information

$ docker buildx version
github.com/docker/buildx v0.34.1 e0b0e77d18d3379bc1e0d55f3b37de288d36fe47
$ docker buildx inspect --bootstrap
Driver:        docker-container
BuildKit image: moby/buildkit:buildx-stable-1
BuildKit version: v0.31.2
# docker engine (from `docker version`): server API 1.55, client API 1.54; runc v1.3.6

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions