From d2bd2ec1b0fe036241d2fecbb210c7239098fcc9 Mon Sep 17 00:00:00 2001 From: Arlo Siemsen Date: Wed, 14 Apr 2021 14:33:26 -0700 Subject: [PATCH 1/9] Add new RFC cargo_alternative_registry_auth. --- text/0000-cargo-alternative-registry-auth.md | 100 +++++++++++++++++++ 1 file changed, 100 insertions(+) create mode 100644 text/0000-cargo-alternative-registry-auth.md diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/0000-cargo-alternative-registry-auth.md new file mode 100644 index 00000000000..0b63cc0e634 --- /dev/null +++ b/text/0000-cargo-alternative-registry-auth.md @@ -0,0 +1,100 @@ +- Feature Name: cargo_alternative_registry_auth +- Start Date: 2021-03-31 +- RFC PR: rust-lang/rfcs#0000 +- Tracking Issue: rust-lang/rust#0000 + +# Summary +Enables Cargo to include the authorization token for all API requests, crate downloads and index updates (when using HTTP) by adding a configuration option to `config.json` in the registry index. + +# Motivation +Organizations need a way to securely publish and distribute internal Rust crates. The current available methods for private crate distribution are awkward: **git repos** do not work well with `cargo update` for resolving semver-compatible dependencies, and do not support the registry API. **Alternative registries** do not support private access and must be operated behind a firewall, or resort to encoding credentials in URLs. + +There are many multi-protocol package managers: Artifactory, AWS CodeArtifact, Azure Artifacts, GitHub Artifacts, Google Artifact Registry, and CloudSmith. However, only CloudSmith and Artifactory support Cargo, and they resort to encoding credentials in the URL or allowing anonymous download of packages. This RFC (especially when combined with the approved http-registry RFC) will make it significantly easier to implement Cargo support on private package managers. + +# Guide-level explanation +Alternative registry operators can set a new key `auth-required = true` in the registry's `config.json` file, which will cause Cargo to include the Authorization token for all API requests, crate downloads, and index updates (if over HTTP). + +```json +{ + "dl": "https://example.com/index/api/v1/crates", + "api": "https://example.com/", + "auth-required": true +} +``` + +If the index is hosted via HTTP using [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) and Cargo receives an `HTTP 401` error when fetching `config.json`, Cargo will automatically re-try the request with the Authorization token included. + + +# Reference-level explanation +A new key, `auth-required`, will be allowed in the [`config.json`](https://doc.rust-lang.org/cargo/reference/registries.html#index-format) file stored in the registry index. When this key is set to `true`, the authorization token will be sent with any HTTP requests made to the registry API, crate downloads, and index (if using http). If a token is not available when Cargo is attempting to make a request, the user would be prompted to run `cargo login --registry NAME` to save a token. + +The authorization token would be sent as an HTTP header, exactly how it is currently sent for operations such as `publish` or `yank`: +``` +Authorization: +``` + +## Interaction with HTTP registries +The approved (but currently unimplemeneted) [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) enables Cargo to fetch the index over HTTP. When fetching `config.json` from an HTTP index, if Cargo receives an `HTTP 401` response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run `cargo login` to add one. + +To avoid the overhead of an extra HTTP request when fetching `config.json`, the user can optionally configure Cargo locally by setting `auth-required = true` in the `[registries]` table. If the local `auth-required` flag is `true` then Cargo will always include the Authorization token fetching `config.json` over HTTP -- skipping the initial unauthorized requiest and `HTTP 401`. The local configuration option does not impact other operations, such as API requests or downloads. It also does not impact git-based registries. + +```toml +[registries] +my-registry = { index = "https://example.com/index", auth-required = true } +``` + +## Security considerations +If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would not be sent to the redirect target. + +The authorization header would only be included for requests using `https` or requests targeting `localhost`. If cargo detected an alternative registry was configured to send the authorization token over an insecure channel, it would exit with an error informing the user. + +## Interaction with `credential-process` +The unstable [credential-process](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#credential-process) feature stores credentials keyed on the registry api url, which is only available in after fetching `config.json` from the index. If access to the index is secured using the authorization token, then Cargo will be unable to fetch the `config.json` file before calling the credential process. + +For example, the following command would need to download `config.json` from the index before storing the credential. +`cargo login --registry my-registry -Z http-registry -Z credential-process` + +To resolve this issue, the credential process feature would use the registry *index url* as the key instead of the *api url*. + +Since the token may be used multiple times in a single cargo session (such as updating the index + downloading crates), Cargo should cache the token if it is provided by a `credential-process` to avoid repeatedly calling the credential process. + +## Command line options +Cargo commands such as `install` or `search` that support an `--index ` command line option to use a registry other than what is available in the configuration file would gain a `--token ` command line option (similar to `publish` today). If a `--token ` command line option is given, the provided authorization token would be sent along with the request. + +# Prior art +[prior-art]: #prior-art + +The proposed **private-registry-auth** RFC [also proposes](https://github.com/jdemilledt/rfcs/blob/master/text/0000-private-registry-auth.md) sending the authorization token with all requests, but is missing detail. + +**NuGet** first attempts to access the index anonymously, then attempts to call credential helpers, then prompts for authentication. + +**NPM** uses a local configuration key [`always-auth`](https://docs.npmjs.com/cli/v7/using-npm/config#always-auth). When set to `true` the authorization token is sent with all requests. + +**Gradle / Maven (Java)** uses a [local configuration option](https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.repositories.MavenArtifactRepository.html) for private package repositories that causes an authorization header to be sent. + +**git** first attempts to fetch without authentication. If the server sends back an HTTP 401, then git will send a username & password (if available), or invoke configured [credential helpers](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). + +# Drawbacks +[drawbacks]: #drawbacks + +* There is not a good way to add the authorization header when downloading the index via `git`, so the index authorization will continue to be handled by `git`, until the http-registry RFC is completed. +* Requires a breaking change to the unstable `credential-process` feature, described above under "Interaction with `credential-process`". + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +This design provides a simple mechanism for cargo to send an authorization header to a registry that works similar to other package managers. Additionally it would work with [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) to serve the index over HTTP, including using a standard web server with basic authentication, since the `token` could be set to `Basic `. + +Alternatives: +* Don't add any configuration options to `config.json` or the `[registries]` table and rely on the auto-detection method for everything by first attempting an unauthenticated request, then on HTTP 401, the request would be re-tried including the token. This carries more risk of the token being sent when the server may not be expecting it, but would avoid a configuration option for the registry operator. It also would require more HTTP requests, since each type of request would need to be first attempted without the token. +* Don't add a configuration option to `config.json` and rely only on the local configuration in the `[registries]` table. This avoids the auto-detection, but requires configuration from the user, which could be set up incorrectly or missed. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +* Do registries need a way to specify which API requests require authorization? Is one switch sufficient? + +# Future possibilities +[future-possibilities]: #future-possibilities + +The `credential-process` system could be extended to support generating tokens rather than only storing them. This would further improve security and allow additional features such as 2FA prompts. \ No newline at end of file From bbe11f89d272ec66bb531d6ff1072f4f3e450f7d Mon Sep 17 00:00:00 2001 From: Arlo Siemsen Date: Wed, 16 Jun 2021 13:44:38 -0700 Subject: [PATCH 2/9] Added section on authentication for git registries; clarified section on https requirements --- text/0000-cargo-alternative-registry-auth.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/0000-cargo-alternative-registry-auth.md index 0b63cc0e634..068b884092f 100644 --- a/text/0000-cargo-alternative-registry-auth.md +++ b/text/0000-cargo-alternative-registry-auth.md @@ -1,6 +1,6 @@ - Feature Name: cargo_alternative_registry_auth - Start Date: 2021-03-31 -- RFC PR: rust-lang/rfcs#0000 +- RFC PR: rust-lang/rfcs#3139 - Tracking Issue: rust-lang/rust#0000 # Summary @@ -40,13 +40,13 @@ To avoid the overhead of an extra HTTP request when fetching `config.json`, the ```toml [registries] -my-registry = { index = "https://example.com/index", auth-required = true } +my-registry = { index = "sparse+https://example.com/index", auth-required = true } ``` -## Security considerations -If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would not be sent to the redirect target. +## Security +If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would *not* be sent to the redirect target. -The authorization header would only be included for requests using `https` or requests targeting `localhost`. If cargo detected an alternative registry was configured to send the authorization token over an insecure channel, it would exit with an error informing the user. +The authorization header would only be included for requests using `https://`. Under no circumstances would cargo pass an authorization header over an unencrypted `http://` connection. If cargo detected an alternative registry was configured to send the authorization token over an insecure channel, it would exit with an error. ## Interaction with `credential-process` The unstable [credential-process](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#credential-process) feature stores credentials keyed on the registry api url, which is only available in after fetching `config.json` from the index. If access to the index is secured using the authorization token, then Cargo will be unable to fetch the `config.json` file before calling the credential process. @@ -58,6 +58,7 @@ To resolve this issue, the credential process feature would use the registry *in Since the token may be used multiple times in a single cargo session (such as updating the index + downloading crates), Cargo should cache the token if it is provided by a `credential-process` to avoid repeatedly calling the credential process. + ## Command line options Cargo commands such as `install` or `search` that support an `--index ` command line option to use a registry other than what is available in the configuration file would gain a `--token ` command line option (similar to `publish` today). If a `--token ` command line option is given, the provided authorization token would be sent along with the request. @@ -97,4 +98,10 @@ Alternatives: # Future possibilities [future-possibilities]: #future-possibilities -The `credential-process` system could be extended to support generating tokens rather than only storing them. This would further improve security and allow additional features such as 2FA prompts. \ No newline at end of file +## Credential Process +The `credential-process` system could be extended to support generating tokens rather than only storing them. This would further improve security and allow additional features such as 2FA prompts. + +## Authentication for Git-based registries +Private registries may want to use the same Authorization header for controlling access to a git-based index over `https`, rather than letting git handle the authentication separately. + +This could be enabled by the same local configuration key `auth-required = true` in the `[registries]` table. Both `libgit2` and the `git` command line have a mechanism for specifying an additional header that could be used to pass the Authorization header. \ No newline at end of file From cc2579d53a8537d54b950217ce16f4bdfdcce723 Mon Sep 17 00:00:00 2001 From: Arlo Siemsen Date: Thu, 24 Jun 2021 13:45:32 -0700 Subject: [PATCH 3/9] Clarify what the local `auth-required` flag does --- text/0000-cargo-alternative-registry-auth.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/0000-cargo-alternative-registry-auth.md index 068b884092f..94fabdc1e97 100644 --- a/text/0000-cargo-alternative-registry-auth.md +++ b/text/0000-cargo-alternative-registry-auth.md @@ -36,7 +36,9 @@ Authorization: ## Interaction with HTTP registries The approved (but currently unimplemeneted) [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) enables Cargo to fetch the index over HTTP. When fetching `config.json` from an HTTP index, if Cargo receives an `HTTP 401` response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run `cargo login` to add one. -To avoid the overhead of an extra HTTP request when fetching `config.json`, the user can optionally configure Cargo locally by setting `auth-required = true` in the `[registries]` table. If the local `auth-required` flag is `true` then Cargo will always include the Authorization token fetching `config.json` over HTTP -- skipping the initial unauthorized requiest and `HTTP 401`. The local configuration option does not impact other operations, such as API requests or downloads. It also does not impact git-based registries. +To avoid the overhead of an extra HTTP request when fetching `config.json`, the user can optionally configure Cargo locally by setting `auth-required` in the `[registries]` table. If the local `auth-required` flag is `true` then Cargo will include the Authorization token when initially fetching `config.json` over HTTP. If it is `false`, Cargo will never include the Authorization token when fetching `config.json`. If it is unset, Cargo performs the auto-detection described above. + +This local configuration option does not impact other registry operations, such as API requests or downloads (which are controlled by the flag in `config.json`). It also does not impact git-based registries. ```toml [registries] From 883a985880452995a1e18906c1ebf5bf1ad66e95 Mon Sep 17 00:00:00 2001 From: Arlo Siemsen Date: Mon, 12 Jul 2021 12:01:53 -0700 Subject: [PATCH 4/9] Move the local configuration option to future possibilities to avoid potential confusion --- text/0000-cargo-alternative-registry-auth.md | 21 ++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/0000-cargo-alternative-registry-auth.md index 94fabdc1e97..4c3883a5215 100644 --- a/text/0000-cargo-alternative-registry-auth.md +++ b/text/0000-cargo-alternative-registry-auth.md @@ -36,15 +36,6 @@ Authorization: ## Interaction with HTTP registries The approved (but currently unimplemeneted) [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) enables Cargo to fetch the index over HTTP. When fetching `config.json` from an HTTP index, if Cargo receives an `HTTP 401` response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run `cargo login` to add one. -To avoid the overhead of an extra HTTP request when fetching `config.json`, the user can optionally configure Cargo locally by setting `auth-required` in the `[registries]` table. If the local `auth-required` flag is `true` then Cargo will include the Authorization token when initially fetching `config.json` over HTTP. If it is `false`, Cargo will never include the Authorization token when fetching `config.json`. If it is unset, Cargo performs the auto-detection described above. - -This local configuration option does not impact other registry operations, such as API requests or downloads (which are controlled by the flag in `config.json`). It also does not impact git-based registries. - -```toml -[registries] -my-registry = { index = "sparse+https://example.com/index", auth-required = true } -``` - ## Security If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would *not* be sent to the redirect target. @@ -103,7 +94,17 @@ Alternatives: ## Credential Process The `credential-process` system could be extended to support generating tokens rather than only storing them. This would further improve security and allow additional features such as 2FA prompts. +## Local configuration option +To avoid the overhead of an extra HTTP request when fetching `config.json`, the user could optionally configure Cargo locally by setting `auth-required` in the `[registries]` table. If the local `auth-required` flag is `true`, then Cargo could include the Authorization token when initially fetching `config.json` over HTTP. If it is `false`, Cargo would not include the Authorization token when fetching `config.json`. If it is unset, Cargo would perform the auto-detection described above. + +This local configuration option would not impact other registry operations, such as API requests or downloads (which are controlled by the flag in `config.json`). It also would not impact git-based registries. + +```toml +[registries] +my-registry = { index = "sparse+https://example.com/index", auth-required = true } +``` + ## Authentication for Git-based registries Private registries may want to use the same Authorization header for controlling access to a git-based index over `https`, rather than letting git handle the authentication separately. -This could be enabled by the same local configuration key `auth-required = true` in the `[registries]` table. Both `libgit2` and the `git` command line have a mechanism for specifying an additional header that could be used to pass the Authorization header. \ No newline at end of file +This could be enabled by the same local configuration key `auth-required = true` in the `[registries]` table. Both `libgit2` and the `git` command line have a mechanism for specifying an additional header that could be used to pass the Authorization header. From 8a993d3d9f581d25fc94d3eb0fc246377edfbd1c Mon Sep 17 00:00:00 2001 From: Arlo Siemsen Date: Mon, 24 Jan 2022 11:58:51 -0800 Subject: [PATCH 5/9] Update RFC based on feedback --- text/0000-cargo-alternative-registry-auth.md | 38 ++++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/0000-cargo-alternative-registry-auth.md index 4c3883a5215..d07fa8345c0 100644 --- a/text/0000-cargo-alternative-registry-auth.md +++ b/text/0000-cargo-alternative-registry-auth.md @@ -34,13 +34,11 @@ Authorization: ``` ## Interaction with HTTP registries -The approved (but currently unimplemeneted) [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) enables Cargo to fetch the index over HTTP. When fetching `config.json` from an HTTP index, if Cargo receives an `HTTP 401` response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run `cargo login` to add one. +The approved (but currently unimplemented) [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) enables Cargo to fetch the index over HTTP. When fetching `config.json` from an HTTP index, if Cargo receives an `HTTP 401` response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run `cargo login` to add one. The `HTTP 401` response from the registry server may also include an `X-Cargo-Token-Url: ` header to specify where the user should go to get a token. In that case, `cargo` can display a more helpful message such as "please paste the Token found on https://example.com/token-url-from-header below" ## Security If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would *not* be sent to the redirect target. -The authorization header would only be included for requests using `https://`. Under no circumstances would cargo pass an authorization header over an unencrypted `http://` connection. If cargo detected an alternative registry was configured to send the authorization token over an insecure channel, it would exit with an error. - ## Interaction with `credential-process` The unstable [credential-process](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#credential-process) feature stores credentials keyed on the registry api url, which is only available in after fetching `config.json` from the index. If access to the index is secured using the authorization token, then Cargo will be unable to fetch the `config.json` file before calling the credential process. @@ -51,6 +49,19 @@ To resolve this issue, the credential process feature would use the registry *in Since the token may be used multiple times in a single cargo session (such as updating the index + downloading crates), Cargo should cache the token if it is provided by a `credential-process` to avoid repeatedly calling the credential process. +## Token Lookup by Index Url + +Cargo doesn't always know a registry's name. Sometimes only the index url is known. Consider the following scenario: we have two private registries A, and B. A allows published crates to depend on crates in B. When cargo builds such a crate, the crate's normalized cargo.toml file won't have the name of the dependent registry, only it's index URL. This becomes a problem when Cargo needs to look up the authentication token for B. + +``` +[dependencies.B] +version = "0.1" +registry-index = "https://index-url-for-registry-containing-b/" +``` + +`Cargo.lock` files also only contain the index url, not the registry name. + +Registry credentials stored in the 'credentials' file are keyed on the registry name, not the index url. Cargo would search for a token by checking all (index, token) pairs for one that matches the index. To unambiguously find a credential by index URL, Cargo would issue an error if two registries were configured with the same index URL. This approach of finding the credentials by index URL does not support the environment variable based configuration overrides (since Cargo wouldn't know the environment variable to look up). ## Command line options Cargo commands such as `install` or `search` that support an `--index ` command line option to use a registry other than what is available in the configuration file would gain a `--token ` command line option (similar to `publish` today). If a `--token ` command line option is given, the provided authorization token would be sent along with the request. @@ -71,7 +82,7 @@ The proposed **private-registry-auth** RFC [also proposes](https://github.com/jd # Drawbacks [drawbacks]: #drawbacks -* There is not a good way to add the authorization header when downloading the index via `git`, so the index authorization will continue to be handled by `git`, until the http-registry RFC is completed. +* There is not a good way to add the authorization header when downloading the index via `git`, so index authorization will continue to be handled by `git`, until the http-registry RFC is completed. * Requires a breaking change to the unstable `credential-process` feature, described above under "Interaction with `credential-process`". # Rationale and alternatives @@ -79,32 +90,29 @@ The proposed **private-registry-auth** RFC [also proposes](https://github.com/jd This design provides a simple mechanism for cargo to send an authorization header to a registry that works similar to other package managers. Additionally it would work with [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) to serve the index over HTTP, including using a standard web server with basic authentication, since the `token` could be set to `Basic `. -Alternatives: +## Alternatives: * Don't add any configuration options to `config.json` or the `[registries]` table and rely on the auto-detection method for everything by first attempting an unauthenticated request, then on HTTP 401, the request would be re-tried including the token. This carries more risk of the token being sent when the server may not be expecting it, but would avoid a configuration option for the registry operator. It also would require more HTTP requests, since each type of request would need to be first attempted without the token. * Don't add a configuration option to `config.json` and rely only on the local configuration in the `[registries]` table. This avoids the auto-detection, but requires configuration from the user, which could be set up incorrectly or missed. # Unresolved questions [unresolved-questions]: #unresolved-questions -* Do registries need a way to specify which API requests require authorization? Is one switch sufficient? +* Do registries need a more fine-grained switch for which API commands require authentication? # Future possibilities [future-possibilities]: #future-possibilities ## Credential Process -The `credential-process` system could be extended to support generating tokens rather than only storing them. This would further improve security and allow additional features such as 2FA prompts. +The `credential-process` feature could be extended to support generating tokens rather than only storing them. This would further improve security and allow additional features such as 2FA prompts. -## Local configuration option -To avoid the overhead of an extra HTTP request when fetching `config.json`, the user could optionally configure Cargo locally by setting `auth-required` in the `[registries]` table. If the local `auth-required` flag is `true`, then Cargo could include the Authorization token when initially fetching `config.json` over HTTP. If it is `false`, Cargo would not include the Authorization token when fetching `config.json`. If it is unset, Cargo would perform the auto-detection described above. +## Authentication for Git-based registries +Private registries may want to use the same Authorization header for authenticating to a git-based index over `https`, rather than letting git handle the authentication. -This local configuration option would not impact other registry operations, such as API requests or downloads (which are controlled by the flag in `config.json`). It also would not impact git-based registries. +This could be enabled by a local configuration key `cargo-handles-auth = true` in the `[registries]` table. Both `libgit2` and the `git` command line have a mechanism for including an additional header that could be used to pass the Authorization header. ```toml [registries] -my-registry = { index = "sparse+https://example.com/index", auth-required = true } +my-registry = { index = "sparse+https://example.com/index", cargo-handles-auth = true } ``` -## Authentication for Git-based registries -Private registries may want to use the same Authorization header for controlling access to a git-based index over `https`, rather than letting git handle the authentication separately. - -This could be enabled by the same local configuration key `auth-required = true` in the `[registries]` table. Both `libgit2` and the `git` command line have a mechanism for specifying an additional header that could be used to pass the Authorization header. +Using the http sparse index will likely be a preferred path for private registries, because it avoids the complexity of the git protocol. From f3aecb96eeb95542d81d6dc6b0a22c1245383604 Mon Sep 17 00:00:00 2001 From: Arlo Siemsen Date: Mon, 24 Jan 2022 12:53:44 -0800 Subject: [PATCH 6/9] Fix minor typo --- text/0000-cargo-alternative-registry-auth.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/0000-cargo-alternative-registry-auth.md index d07fa8345c0..1e4523ce29a 100644 --- a/text/0000-cargo-alternative-registry-auth.md +++ b/text/0000-cargo-alternative-registry-auth.md @@ -40,7 +40,7 @@ The approved (but currently unimplemented) [RFC2789](https://github.com/rust-lan If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would *not* be sent to the redirect target. ## Interaction with `credential-process` -The unstable [credential-process](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#credential-process) feature stores credentials keyed on the registry api url, which is only available in after fetching `config.json` from the index. If access to the index is secured using the authorization token, then Cargo will be unable to fetch the `config.json` file before calling the credential process. +The unstable [credential-process](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#credential-process) feature stores credentials keyed on the registry api url, which is only available after fetching `config.json` from the index. If access to the index is secured using the authorization token, then Cargo will be unable to fetch the `config.json` file before calling the credential process. For example, the following command would need to download `config.json` from the index before storing the credential. `cargo login --registry my-registry -Z http-registry -Z credential-process` From aa18f9239a6684e0c5a64ef9308d0e76a1da87e3 Mon Sep 17 00:00:00 2001 From: Arlo Siemsen Date: Tue, 22 Feb 2022 15:01:56 -0800 Subject: [PATCH 7/9] Add note about token specifics --- text/0000-cargo-alternative-registry-auth.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/0000-cargo-alternative-registry-auth.md index 1e4523ce29a..430c9cc5644 100644 --- a/text/0000-cargo-alternative-registry-auth.md +++ b/text/0000-cargo-alternative-registry-auth.md @@ -26,13 +26,15 @@ If the index is hosted via HTTP using [RFC2789](https://github.com/rust-lang/rfc # Reference-level explanation -A new key, `auth-required`, will be allowed in the [`config.json`](https://doc.rust-lang.org/cargo/reference/registries.html#index-format) file stored in the registry index. When this key is set to `true`, the authorization token will be sent with any HTTP requests made to the registry API, crate downloads, and index (if using http). If a token is not available when Cargo is attempting to make a request, the user would be prompted to run `cargo login --registry NAME` to save a token. +A new optional key, `auth-required`, will be allowed in the [`config.json`](https://doc.rust-lang.org/cargo/reference/registries.html#index-format) file stored in the registry index. When this key is set to `true`, the authorization token will be sent with any HTTP requests made to the registry API, crate downloads, and index (if using http). If a token is not available when Cargo is attempting to make a request, the user would be prompted to run `cargo login --registry NAME` to save a token. The authorization token would be sent as an HTTP header, exactly how it is currently sent for operations such as `publish` or `yank`: ``` Authorization: ``` +This RFC does not specify or change the format of the Authorization Token. For the purposes of this RFC, tokens are opaque; no particular format or protocol is specified, and third-party registry authentication should not assume support for any particular format. This includes shared-secret tokens, even though crates.io and the existing publish support for third-party registries currently supports such bearer tokens. Future RFCs (such as [RFC2789](https://github.com/rust-lang/rfcs/pull/3231)) may update the format and protocol used for tokens. + ## Interaction with HTTP registries The approved (but currently unimplemented) [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) enables Cargo to fetch the index over HTTP. When fetching `config.json` from an HTTP index, if Cargo receives an `HTTP 401` response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run `cargo login` to add one. The `HTTP 401` response from the registry server may also include an `X-Cargo-Token-Url: ` header to specify where the user should go to get a token. In that case, `cargo` can display a more helpful message such as "please paste the Token found on https://example.com/token-url-from-header below" From 5dd2b7c970c5dd30ea68813918b9b3bdbca0be1c Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Fri, 11 Mar 2022 15:53:28 -0800 Subject: [PATCH 8/9] Switch to unix line endings. --- text/0000-cargo-alternative-registry-auth.md | 240 +++++++++---------- 1 file changed, 120 insertions(+), 120 deletions(-) diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/0000-cargo-alternative-registry-auth.md index 430c9cc5644..b29081a46b5 100644 --- a/text/0000-cargo-alternative-registry-auth.md +++ b/text/0000-cargo-alternative-registry-auth.md @@ -1,120 +1,120 @@ -- Feature Name: cargo_alternative_registry_auth -- Start Date: 2021-03-31 -- RFC PR: rust-lang/rfcs#3139 -- Tracking Issue: rust-lang/rust#0000 - -# Summary -Enables Cargo to include the authorization token for all API requests, crate downloads and index updates (when using HTTP) by adding a configuration option to `config.json` in the registry index. - -# Motivation -Organizations need a way to securely publish and distribute internal Rust crates. The current available methods for private crate distribution are awkward: **git repos** do not work well with `cargo update` for resolving semver-compatible dependencies, and do not support the registry API. **Alternative registries** do not support private access and must be operated behind a firewall, or resort to encoding credentials in URLs. - -There are many multi-protocol package managers: Artifactory, AWS CodeArtifact, Azure Artifacts, GitHub Artifacts, Google Artifact Registry, and CloudSmith. However, only CloudSmith and Artifactory support Cargo, and they resort to encoding credentials in the URL or allowing anonymous download of packages. This RFC (especially when combined with the approved http-registry RFC) will make it significantly easier to implement Cargo support on private package managers. - -# Guide-level explanation -Alternative registry operators can set a new key `auth-required = true` in the registry's `config.json` file, which will cause Cargo to include the Authorization token for all API requests, crate downloads, and index updates (if over HTTP). - -```json -{ - "dl": "https://example.com/index/api/v1/crates", - "api": "https://example.com/", - "auth-required": true -} -``` - -If the index is hosted via HTTP using [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) and Cargo receives an `HTTP 401` error when fetching `config.json`, Cargo will automatically re-try the request with the Authorization token included. - - -# Reference-level explanation -A new optional key, `auth-required`, will be allowed in the [`config.json`](https://doc.rust-lang.org/cargo/reference/registries.html#index-format) file stored in the registry index. When this key is set to `true`, the authorization token will be sent with any HTTP requests made to the registry API, crate downloads, and index (if using http). If a token is not available when Cargo is attempting to make a request, the user would be prompted to run `cargo login --registry NAME` to save a token. - -The authorization token would be sent as an HTTP header, exactly how it is currently sent for operations such as `publish` or `yank`: -``` -Authorization: -``` - -This RFC does not specify or change the format of the Authorization Token. For the purposes of this RFC, tokens are opaque; no particular format or protocol is specified, and third-party registry authentication should not assume support for any particular format. This includes shared-secret tokens, even though crates.io and the existing publish support for third-party registries currently supports such bearer tokens. Future RFCs (such as [RFC2789](https://github.com/rust-lang/rfcs/pull/3231)) may update the format and protocol used for tokens. - -## Interaction with HTTP registries -The approved (but currently unimplemented) [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) enables Cargo to fetch the index over HTTP. When fetching `config.json` from an HTTP index, if Cargo receives an `HTTP 401` response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run `cargo login` to add one. The `HTTP 401` response from the registry server may also include an `X-Cargo-Token-Url: ` header to specify where the user should go to get a token. In that case, `cargo` can display a more helpful message such as "please paste the Token found on https://example.com/token-url-from-header below" - -## Security -If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would *not* be sent to the redirect target. - -## Interaction with `credential-process` -The unstable [credential-process](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#credential-process) feature stores credentials keyed on the registry api url, which is only available after fetching `config.json` from the index. If access to the index is secured using the authorization token, then Cargo will be unable to fetch the `config.json` file before calling the credential process. - -For example, the following command would need to download `config.json` from the index before storing the credential. -`cargo login --registry my-registry -Z http-registry -Z credential-process` - -To resolve this issue, the credential process feature would use the registry *index url* as the key instead of the *api url*. - -Since the token may be used multiple times in a single cargo session (such as updating the index + downloading crates), Cargo should cache the token if it is provided by a `credential-process` to avoid repeatedly calling the credential process. - -## Token Lookup by Index Url - -Cargo doesn't always know a registry's name. Sometimes only the index url is known. Consider the following scenario: we have two private registries A, and B. A allows published crates to depend on crates in B. When cargo builds such a crate, the crate's normalized cargo.toml file won't have the name of the dependent registry, only it's index URL. This becomes a problem when Cargo needs to look up the authentication token for B. - -``` -[dependencies.B] -version = "0.1" -registry-index = "https://index-url-for-registry-containing-b/" -``` - -`Cargo.lock` files also only contain the index url, not the registry name. - -Registry credentials stored in the 'credentials' file are keyed on the registry name, not the index url. Cargo would search for a token by checking all (index, token) pairs for one that matches the index. To unambiguously find a credential by index URL, Cargo would issue an error if two registries were configured with the same index URL. This approach of finding the credentials by index URL does not support the environment variable based configuration overrides (since Cargo wouldn't know the environment variable to look up). - -## Command line options -Cargo commands such as `install` or `search` that support an `--index ` command line option to use a registry other than what is available in the configuration file would gain a `--token ` command line option (similar to `publish` today). If a `--token ` command line option is given, the provided authorization token would be sent along with the request. - -# Prior art -[prior-art]: #prior-art - -The proposed **private-registry-auth** RFC [also proposes](https://github.com/jdemilledt/rfcs/blob/master/text/0000-private-registry-auth.md) sending the authorization token with all requests, but is missing detail. - -**NuGet** first attempts to access the index anonymously, then attempts to call credential helpers, then prompts for authentication. - -**NPM** uses a local configuration key [`always-auth`](https://docs.npmjs.com/cli/v7/using-npm/config#always-auth). When set to `true` the authorization token is sent with all requests. - -**Gradle / Maven (Java)** uses a [local configuration option](https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.repositories.MavenArtifactRepository.html) for private package repositories that causes an authorization header to be sent. - -**git** first attempts to fetch without authentication. If the server sends back an HTTP 401, then git will send a username & password (if available), or invoke configured [credential helpers](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). - -# Drawbacks -[drawbacks]: #drawbacks - -* There is not a good way to add the authorization header when downloading the index via `git`, so index authorization will continue to be handled by `git`, until the http-registry RFC is completed. -* Requires a breaking change to the unstable `credential-process` feature, described above under "Interaction with `credential-process`". - -# Rationale and alternatives -[rationale-and-alternatives]: #rationale-and-alternatives - -This design provides a simple mechanism for cargo to send an authorization header to a registry that works similar to other package managers. Additionally it would work with [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) to serve the index over HTTP, including using a standard web server with basic authentication, since the `token` could be set to `Basic `. - -## Alternatives: -* Don't add any configuration options to `config.json` or the `[registries]` table and rely on the auto-detection method for everything by first attempting an unauthenticated request, then on HTTP 401, the request would be re-tried including the token. This carries more risk of the token being sent when the server may not be expecting it, but would avoid a configuration option for the registry operator. It also would require more HTTP requests, since each type of request would need to be first attempted without the token. -* Don't add a configuration option to `config.json` and rely only on the local configuration in the `[registries]` table. This avoids the auto-detection, but requires configuration from the user, which could be set up incorrectly or missed. - -# Unresolved questions -[unresolved-questions]: #unresolved-questions - -* Do registries need a more fine-grained switch for which API commands require authentication? - -# Future possibilities -[future-possibilities]: #future-possibilities - -## Credential Process -The `credential-process` feature could be extended to support generating tokens rather than only storing them. This would further improve security and allow additional features such as 2FA prompts. - -## Authentication for Git-based registries -Private registries may want to use the same Authorization header for authenticating to a git-based index over `https`, rather than letting git handle the authentication. - -This could be enabled by a local configuration key `cargo-handles-auth = true` in the `[registries]` table. Both `libgit2` and the `git` command line have a mechanism for including an additional header that could be used to pass the Authorization header. - -```toml -[registries] -my-registry = { index = "sparse+https://example.com/index", cargo-handles-auth = true } -``` - -Using the http sparse index will likely be a preferred path for private registries, because it avoids the complexity of the git protocol. +- Feature Name: cargo_alternative_registry_auth +- Start Date: 2021-03-31 +- RFC PR: rust-lang/rfcs#3139 +- Tracking Issue: rust-lang/rust#0000 + +# Summary +Enables Cargo to include the authorization token for all API requests, crate downloads and index updates (when using HTTP) by adding a configuration option to `config.json` in the registry index. + +# Motivation +Organizations need a way to securely publish and distribute internal Rust crates. The current available methods for private crate distribution are awkward: **git repos** do not work well with `cargo update` for resolving semver-compatible dependencies, and do not support the registry API. **Alternative registries** do not support private access and must be operated behind a firewall, or resort to encoding credentials in URLs. + +There are many multi-protocol package managers: Artifactory, AWS CodeArtifact, Azure Artifacts, GitHub Artifacts, Google Artifact Registry, and CloudSmith. However, only CloudSmith and Artifactory support Cargo, and they resort to encoding credentials in the URL or allowing anonymous download of packages. This RFC (especially when combined with the approved http-registry RFC) will make it significantly easier to implement Cargo support on private package managers. + +# Guide-level explanation +Alternative registry operators can set a new key `auth-required = true` in the registry's `config.json` file, which will cause Cargo to include the Authorization token for all API requests, crate downloads, and index updates (if over HTTP). + +```json +{ + "dl": "https://example.com/index/api/v1/crates", + "api": "https://example.com/", + "auth-required": true +} +``` + +If the index is hosted via HTTP using [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) and Cargo receives an `HTTP 401` error when fetching `config.json`, Cargo will automatically re-try the request with the Authorization token included. + + +# Reference-level explanation +A new optional key, `auth-required`, will be allowed in the [`config.json`](https://doc.rust-lang.org/cargo/reference/registries.html#index-format) file stored in the registry index. When this key is set to `true`, the authorization token will be sent with any HTTP requests made to the registry API, crate downloads, and index (if using http). If a token is not available when Cargo is attempting to make a request, the user would be prompted to run `cargo login --registry NAME` to save a token. + +The authorization token would be sent as an HTTP header, exactly how it is currently sent for operations such as `publish` or `yank`: +``` +Authorization: +``` + +This RFC does not specify or change the format of the Authorization Token. For the purposes of this RFC, tokens are opaque; no particular format or protocol is specified, and third-party registry authentication should not assume support for any particular format. This includes shared-secret tokens, even though crates.io and the existing publish support for third-party registries currently supports such bearer tokens. Future RFCs (such as [RFC2789](https://github.com/rust-lang/rfcs/pull/3231)) may update the format and protocol used for tokens. + +## Interaction with HTTP registries +The approved (but currently unimplemented) [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) enables Cargo to fetch the index over HTTP. When fetching `config.json` from an HTTP index, if Cargo receives an `HTTP 401` response, the request will be re-attempted with the Authorization header included. If no authorization token is available, Cargo will suggest that the user run `cargo login` to add one. The `HTTP 401` response from the registry server may also include an `X-Cargo-Token-Url: ` header to specify where the user should go to get a token. In that case, `cargo` can display a more helpful message such as "please paste the Token found on https://example.com/token-url-from-header below" + +## Security +If the server responds with an HTTP redirect, the redirect would be followed, but the Authorization header would *not* be sent to the redirect target. + +## Interaction with `credential-process` +The unstable [credential-process](https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#credential-process) feature stores credentials keyed on the registry api url, which is only available after fetching `config.json` from the index. If access to the index is secured using the authorization token, then Cargo will be unable to fetch the `config.json` file before calling the credential process. + +For example, the following command would need to download `config.json` from the index before storing the credential. +`cargo login --registry my-registry -Z http-registry -Z credential-process` + +To resolve this issue, the credential process feature would use the registry *index url* as the key instead of the *api url*. + +Since the token may be used multiple times in a single cargo session (such as updating the index + downloading crates), Cargo should cache the token if it is provided by a `credential-process` to avoid repeatedly calling the credential process. + +## Token Lookup by Index Url + +Cargo doesn't always know a registry's name. Sometimes only the index url is known. Consider the following scenario: we have two private registries A, and B. A allows published crates to depend on crates in B. When cargo builds such a crate, the crate's normalized cargo.toml file won't have the name of the dependent registry, only it's index URL. This becomes a problem when Cargo needs to look up the authentication token for B. + +``` +[dependencies.B] +version = "0.1" +registry-index = "https://index-url-for-registry-containing-b/" +``` + +`Cargo.lock` files also only contain the index url, not the registry name. + +Registry credentials stored in the 'credentials' file are keyed on the registry name, not the index url. Cargo would search for a token by checking all (index, token) pairs for one that matches the index. To unambiguously find a credential by index URL, Cargo would issue an error if two registries were configured with the same index URL. This approach of finding the credentials by index URL does not support the environment variable based configuration overrides (since Cargo wouldn't know the environment variable to look up). + +## Command line options +Cargo commands such as `install` or `search` that support an `--index ` command line option to use a registry other than what is available in the configuration file would gain a `--token ` command line option (similar to `publish` today). If a `--token ` command line option is given, the provided authorization token would be sent along with the request. + +# Prior art +[prior-art]: #prior-art + +The proposed **private-registry-auth** RFC [also proposes](https://github.com/jdemilledt/rfcs/blob/master/text/0000-private-registry-auth.md) sending the authorization token with all requests, but is missing detail. + +**NuGet** first attempts to access the index anonymously, then attempts to call credential helpers, then prompts for authentication. + +**NPM** uses a local configuration key [`always-auth`](https://docs.npmjs.com/cli/v7/using-npm/config#always-auth). When set to `true` the authorization token is sent with all requests. + +**Gradle / Maven (Java)** uses a [local configuration option](https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.repositories.MavenArtifactRepository.html) for private package repositories that causes an authorization header to be sent. + +**git** first attempts to fetch without authentication. If the server sends back an HTTP 401, then git will send a username & password (if available), or invoke configured [credential helpers](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). + +# Drawbacks +[drawbacks]: #drawbacks + +* There is not a good way to add the authorization header when downloading the index via `git`, so index authorization will continue to be handled by `git`, until the http-registry RFC is completed. +* Requires a breaking change to the unstable `credential-process` feature, described above under "Interaction with `credential-process`". + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +This design provides a simple mechanism for cargo to send an authorization header to a registry that works similar to other package managers. Additionally it would work with [RFC2789](https://github.com/rust-lang/rfcs/pull/2789) to serve the index over HTTP, including using a standard web server with basic authentication, since the `token` could be set to `Basic `. + +## Alternatives: +* Don't add any configuration options to `config.json` or the `[registries]` table and rely on the auto-detection method for everything by first attempting an unauthenticated request, then on HTTP 401, the request would be re-tried including the token. This carries more risk of the token being sent when the server may not be expecting it, but would avoid a configuration option for the registry operator. It also would require more HTTP requests, since each type of request would need to be first attempted without the token. +* Don't add a configuration option to `config.json` and rely only on the local configuration in the `[registries]` table. This avoids the auto-detection, but requires configuration from the user, which could be set up incorrectly or missed. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +* Do registries need a more fine-grained switch for which API commands require authentication? + +# Future possibilities +[future-possibilities]: #future-possibilities + +## Credential Process +The `credential-process` feature could be extended to support generating tokens rather than only storing them. This would further improve security and allow additional features such as 2FA prompts. + +## Authentication for Git-based registries +Private registries may want to use the same Authorization header for authenticating to a git-based index over `https`, rather than letting git handle the authentication. + +This could be enabled by a local configuration key `cargo-handles-auth = true` in the `[registries]` table. Both `libgit2` and the `git` command line have a mechanism for including an additional header that could be used to pass the Authorization header. + +```toml +[registries] +my-registry = { index = "sparse+https://example.com/index", cargo-handles-auth = true } +``` + +Using the http sparse index will likely be a preferred path for private registries, because it avoids the complexity of the git protocol. From 4538429eabc84a6231344167f46268ca4af82655 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Fri, 11 Mar 2022 15:54:13 -0800 Subject: [PATCH 9/9] Update 3139 to prepare for merge. --- ...gistry-auth.md => 3139-cargo-alternative-registry-auth.md} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename text/{0000-cargo-alternative-registry-auth.md => 3139-cargo-alternative-registry-auth.md} (98%) diff --git a/text/0000-cargo-alternative-registry-auth.md b/text/3139-cargo-alternative-registry-auth.md similarity index 98% rename from text/0000-cargo-alternative-registry-auth.md rename to text/3139-cargo-alternative-registry-auth.md index b29081a46b5..e0942d71d2d 100644 --- a/text/0000-cargo-alternative-registry-auth.md +++ b/text/3139-cargo-alternative-registry-auth.md @@ -1,7 +1,7 @@ - Feature Name: cargo_alternative_registry_auth - Start Date: 2021-03-31 -- RFC PR: rust-lang/rfcs#3139 -- Tracking Issue: rust-lang/rust#0000 +- RFC PR: [rust-lang/rfcs#3139](https://github.com/rust-lang/rfcs/pull/3139) +- Tracking Issue: [rust-lang/cargo#10474](https://github.com/rust-lang/cargo/issues/10474) # Summary Enables Cargo to include the authorization token for all API requests, crate downloads and index updates (when using HTTP) by adding a configuration option to `config.json` in the registry index.