Skip to content

Conversation

@euanh
Copy link
Contributor

@euanh euanh commented Sep 23, 2025

Checking in Package.resolved for library packages is discouraged because it can mask failures caused by new versions of upstream dependencies.

https://docs.swift.org/swiftpm/documentation/packagemanagerdocs/resolvingpackageversions#Coordinating-versions-of-dependencies-for-your-package

When containerization is built in CI, or when tests are run inside a checked-out local copy of the repository, Package.resolved pins all its dependencies to fixed versions. New versions of upstream dependencies wil not be tested unless Package.resolved is explicitly updated.

When containerization is built as a dependency of a end-user
project, its Package.resolved file is ignored. Instead, the
dependency constraints from containerization's Package.swift
file are combined with those of the project and any other library
dependencies, so SwiftPM or Xcode can find a set of mutually compatible
packages. This can lead to new versions of containerization's upstream
dependencies being used, even though those versions have never been tested
in CI.

The effects of this could be seen before #298 was merged, because swift-nio had changed upstream, breaking packages which depend on containerization
but not breaking containerization's own CI. swift-nio 2.86.1
removed the NIOFileSystem product (apple/swift-nio#3370)
which is not yet ready to be public. CI builds of containerization were
not affected by this change because Package.resolved pinned swift-nio to
an older version:

{
  "identity" : "swift-nio",
  "kind" : "remoteSourceControl",
  "location" : "https://github.com/apple/swift-nio.git",
  "state" : {
    "revision" : "34d486b01cd891297ac615e40d5999536a1e138d",
    "version" : "2.83.0"
  }
},

In comparison, creating a new package which used containerization showed that the library could no longer be built:

% swift package init --type executable
Creating executable package: test
Creating Package.swift
Creating Sources
Creating Sources/test/test.swift
% cat > Package.swift <<EOF
heredoc> // swift-tools-version: 6.2
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "test",
    platforms: [
        .macOS(.v26),
    ],
    dependencies: [
        .package(url: "https://github.com/apple/containerization", from: "0.7.2"),

    ],
    targets: [
        // Targets are the basic building blocks of a package, defining a module or a test suite.
        // Targets can depend on other targets in this package and products from dependencies.
        .executableTarget(
            name: "test",
            dependencies: [
                .product(name: "Containerization", package: "containerization"),
            ]
        ),
    ]
)
EOF
% swift build
...
/private/tmp/test/.build/checkouts/containerization/Sources/ContainerizationOCI/Client/RegistryClient+Fetch.swift:25:8: error: no such module 'NIOFileSystem'
 23 |
 24 | #if os(macOS)
 25 | import NIOFileSystem
    |        `- error: no such module 'NIOFileSystem'
 26 | #endif
 27 |

Removing Package.resolved would cause the same error to occur in a clean build of containerization and be picked up in CI.

@euanh euanh force-pushed the remove-package-resolved branch from 46433d7 to 69d5612 Compare September 23, 2025 17:06
@dcantah
Copy link
Member

dcantah commented Sep 23, 2025

Github seems to be having having timeouts to download artifacts (which is screwing the protobuf CI step)

@dcantah
Copy link
Member

dcantah commented Sep 29, 2025

I don't know what this would change, but couldn't hurt, could you rebase and see if CI is happier? @euanh

…nges

Checking in `Package.resolved` for library packages is discouraged
because it can mask failures caused by new versions of upstream
dependencies.

When `containerization` is built in CI, or when tests are run inside
a checked-out local copy of the repository, `Package.resolved` pins
all its dependencies to fixed versions.  New versions of upstream
dependencies wil not be tested unless `Package.resolved` is explicitly
updated.

When `containerization` is built as a dependency of a end-user
project, its `Package.resolved` file is ignored.   Instead, the
dependency constraints from `containerization`'s `Package.swift`
file are combined with those of the project and any other library
dependencies, so SwiftPM or Xcode can find a set of mutually compatible
packages.  This can lead to new versions of `containerization`'s upstream
dependencies being used, even though those versions have never been tested
in CI.

The effects of this can currently be seen, because `swift-nio` has
changed upstream, breaking packages which depend on `containerization`
but not breaking `containerization`'s own CI.    `swift-nio` 2.86.1
removes the `NIOFileSystem` product (apple/swift-nio#3370)
which is not yet ready to be public.   CI builds of `containerization` are
not affected by this change because `Package.resolved` pins `swift-nio` to
an older version:

    {
      "identity" : "swift-nio",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/apple/swift-nio.git",
      "state" : {
        "revision" : "34d486b01cd891297ac615e40d5999536a1e138d",
        "version" : "2.83.0"
      }
    },

In comparison, creating a new package which uses `containerization` shows
that the library can no longer be built:

    % swift package init --type executable
    Creating executable package: test
    Creating Package.swift
    Creating Sources
    Creating Sources/test/test.swift
    % cat > Package.swift <<EOF
    heredoc> // swift-tools-version: 6.2
    // The swift-tools-version declares the minimum version of Swift required to build this package.

    import PackageDescription

    let package = Package(
        name: "test",
        platforms: [
            .macOS(.v26),
        ],
        dependencies: [
            .package(url: "https://github.com/apple/containerization", from: "0.7.2"),

        ],
        targets: [
            // Targets are the basic building blocks of a package, defining a module or a test suite.
            // Targets can depend on other targets in this package and products from dependencies.
            .executableTarget(
                name: "test",
                dependencies: [
                    .product(name: "Containerization", package: "containerization"),
                ]
            ),
        ]
    )
    EOF
    % swift build
    ...
    /private/tmp/test/.build/checkouts/containerization/Sources/ContainerizationOCI/Client/RegistryClient+Fetch.swift:25:8: error: no such module 'NIOFileSystem'
     23 |
     24 | #if os(macOS)
     25 | import NIOFileSystem
        |        `- error: no such module 'NIOFileSystem'
     26 | #endif
     27 |

Removing `Package.resolved` causes the same error to occur in a clean build of `containerization`.
@euanh euanh force-pushed the remove-package-resolved branch from 69d5612 to e81f475 Compare September 30, 2025 07:51
@euanh
Copy link
Contributor Author

euanh commented Sep 30, 2025

@dcantah Still timing out on the protobuf dependency download

error: failed downloading 'https://github.com/apple/swift-protobuf/releases/download/protoc-artifactbundle-v31.1/protoc-31.1.artifactbundle.zip' which is required by binary target 'protoc': downloadError("The request timed out.")
make: *** [protoc_gen_grpc_swift] Error 1

https://github.com/apple/containerization/actions/runs/18122772523/job/51570908096?pr=299#step:6:219

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants