From 8cbd3c90b4bfa30f5623d634f4e1af9d25e80af0 Mon Sep 17 00:00:00 2001 From: Brendan Magee Date: Fri, 28 Mar 2025 19:28:50 +0000 Subject: [PATCH 1/5] chore(build-and-publish.md): fix typo, "simle" --> "simple" --- docs/tutorials/build-and-publish.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/build-and-publish.md b/docs/tutorials/build-and-publish.md index 0717765..9ac57f0 100644 --- a/docs/tutorials/build-and-publish.md +++ b/docs/tutorials/build-and-publish.md @@ -10,7 +10,7 @@ Flox aims to be the one tool that you need for the entire software development l ## Prepare a project -Let's start by creating a simle Go project. +Let's start by creating a simple Go project. We'll create a directory called `myproject` and create a Flox environment inside of it so we can install our tools. ```text From 38bb9fcd70a17119eb666b66f9b2da47483126bd Mon Sep 17 00:00:00 2001 From: Brendan Magee Date: Fri, 28 Mar 2025 20:19:35 +0000 Subject: [PATCH 2/5] feat(cookbook): add Flox Catalog Store setup cookbook --- docs/cookbook/infrastructure/flox-store.md | 94 ++++++++++++++++++++++ mkdocs.yml | 2 + 2 files changed, 96 insertions(+) create mode 100644 docs/cookbook/infrastructure/flox-store.md diff --git a/docs/cookbook/infrastructure/flox-store.md b/docs/cookbook/infrastructure/flox-store.md new file mode 100644 index 0000000..ee21849 --- /dev/null +++ b/docs/cookbook/infrastructure/flox-store.md @@ -0,0 +1,94 @@ +--- +title: Catalog Store +description: Create a Catalog Store for publishing your own Flox packages +--- + +# Setting up a Catalog Store + +Publishing your own software to your organization's Flox Catalog requires some +initial setup, but the process is relatively straightforward. Flox supports +publishing packages to a Catalog Store, which will exist in an AWS S3 bucket +that is managed by your organization. In order to use this bucket to store +binaries built with Flox, you will need to set ingress and egress URIs on the +catalog using a utility published by Flox. Then, all you need to do to publish +your software is to call `flox publish`, and Flox will take care of the rest. + +## Configure an AWS S3 Bucket + +The first step in setting up your Catalog Store is creation and configuration of +an AWS S3 Bucket. There are numerous ways to accomplish this, including the AWS +Console, the AWS CLI, and Terraform (or another infrastructure-as-code tool), +to name a few. These processes are well documented, but to get started, +it's best to refer directly to AWS documentation. + +- [What is Amazon S3?][amazon-s3]{:target="\_blank"} +- [AWS S3 CLI Reference][aws-cli-reference-s3]{:target="\_blank"} +- [Amazon Simple Storage Service API Reference][aws-s3-api-reference]{:target="\_blank"} + +Once your S3 bucket is set up and configured with the access policies deemed +necessary by your organization's internal policies, you're ready to proceed to +the next step. Someone from Flox can help you if you run into trouble during +the setup process. Simply reach out to your designated point of contact, +and we'll work with you to get you up and running. + +[amazon-s3]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html +[aws-cli-reference-s3]: https://docs.aws.amazon.com/cli/latest/reference/s3/ +[aws-s3-api-reference]: https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html + +## Ensure the `nix daemon` Has Access to the S3 Bucket + +As you probably know by now, the underlying technology powering Flox is Nix. +Accordingly, we need to take a couple steps to ensure that the `nix daemon` +has access to the S3 bucket you've just created. To do so, +you have a couple of options: + +1. Set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and, if applicable, `AWS_SESSION_TOKEN` +as environment variables +1. Use the `aws configure` command +[as described in the CLI reference][aws-cli-configure-command]{:target="\_blank"} +to set those same values + +If you follow the second set of steps, you can confirm that everything is set +up correctly by inspecting the values stored in `$HOME/.aws/credentials`. + +[aws-cli-configure-command]: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/configure/index.html#configure + +## Set Catalog Store Ingress and Egress URIs + +Once you have your S3 bucket configured, the next step is to set an ingress URI +and egress URI for your Catalog Store. Flox provides a utility for you +that does exactly what you need, within a Flox environment. To use this, +you'll need to run the following command: + +```sh +flox activate -r flox/flox-catalog-util +``` + +When you run this command, you'll see the following output: + +```console +✅ You are now using the environment 'flox/flox-catalog-util (remote)'. +To stop using this environment, type 'exit' +``` + +Within the active Flox environment, you can simply run the following command: + +```sh +catalog-util store --catalog "" set --store-config '{ "store-type": "nix-copy", "ingress_uri": "s3://", "egress_uri": "s3://" }' +``` + +You'll note that it's possible to set the ingress and egress URIs to the same +value, if you wish to do so. + +## Set Signing Key When Publishing Your Package + +At this point, you should have an appropriately Catalog Store to which you can +publish your own software via the `flox publish` command. The last thing you +need to worry about is configuring a signing key for publishing packages: + +```sh +flox config --set publish.signing_key "" +``` + +Now you're ready to use Flox publish your own software to the Catalog Store +that you just set up. diff --git a/mkdocs.yml b/mkdocs.yml index 02bcaca..77c08f5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -62,6 +62,8 @@ nav: - ... | flat | cookbook/languages/* - Builds: - ... | flat | cookbook/builds/* + - Infrastructure: + - ... | flat | cookbook/infrastructure/* - Reference: - Flox manual: - reference/command-reference/flox.md From c2a515b5003569c0ef9e33845e79378db488698f Mon Sep 17 00:00:00 2001 From: Brendan Magee Date: Fri, 28 Mar 2025 20:20:13 +0000 Subject: [PATCH 3/5] feat(publishing.md): link to Flox Catalog Store setup cookbook --- docs/concepts/publishing.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/concepts/publishing.md b/docs/concepts/publishing.md index 55b2322..114882f 100644 --- a/docs/concepts/publishing.md +++ b/docs/concepts/publishing.md @@ -82,9 +82,10 @@ See the "Catalog Store" cookbook page to learn more about how to provision the s Artifacts uploaded to a Catalog Store may be signed. This key is provided to Flox via the `flox publish --signing-key` argument so that the key can be used to sign artifacts during the publish process. Similarly, in order to install packages signed with this key, Nix must be configured to trust this key. -See the "Catalog Store" cookbook page to learn more about how to configure Nix to trust the signing key. +See the ["Catalog Store" cookbook page][catalog-store-cookbook] to learn more about how to configure Nix to trust the signing key. [builds-concept]: ./manifest-builds.md [early]: https://flox.dev/early/ [catalog-util]: https://github.com/flox/catalog-util [catalog-concept]: ./packages-and-catalog.md +[catalog-store-cookbook]: ../cookbook/infrastructure/flox-store.md From 20fb23a1e9a27198d54c0ac48d03f062d3b28104 Mon Sep 17 00:00:00 2001 From: Brendan Magee Date: Fri, 28 Mar 2025 23:06:41 +0000 Subject: [PATCH 4/5] feat(flox-store.md): add clarifying details to Catalog Store cookbook Also restructure the menu so that the cookbook section is now called "publish" instead of "infrastructure." --- docs/concepts/publishing.md | 2 +- .../{infrastructure => publish}/flox-store.md | 30 ++++++++++++------- mkdocs.yml | 4 +-- 3 files changed, 23 insertions(+), 13 deletions(-) rename docs/cookbook/{infrastructure => publish}/flox-store.md (76%) diff --git a/docs/concepts/publishing.md b/docs/concepts/publishing.md index 114882f..cb016c0 100644 --- a/docs/concepts/publishing.md +++ b/docs/concepts/publishing.md @@ -88,4 +88,4 @@ See the ["Catalog Store" cookbook page][catalog-store-cookbook] to learn more ab [early]: https://flox.dev/early/ [catalog-util]: https://github.com/flox/catalog-util [catalog-concept]: ./packages-and-catalog.md -[catalog-store-cookbook]: ../cookbook/infrastructure/flox-store.md +[catalog-store-cookbook]: ../cookbook/publish/flox-store.md diff --git a/docs/cookbook/infrastructure/flox-store.md b/docs/cookbook/publish/flox-store.md similarity index 76% rename from docs/cookbook/infrastructure/flox-store.md rename to docs/cookbook/publish/flox-store.md index ee21849..f20d6b3 100644 --- a/docs/cookbook/infrastructure/flox-store.md +++ b/docs/cookbook/publish/flox-store.md @@ -7,11 +7,19 @@ description: Create a Catalog Store for publishing your own Flox packages Publishing your own software to your organization's Flox Catalog requires some initial setup, but the process is relatively straightforward. Flox supports -publishing packages to a Catalog Store, which will exist in an AWS S3 bucket -that is managed by your organization. In order to use this bucket to store -binaries built with Flox, you will need to set ingress and egress URIs on the -catalog using a utility published by Flox. Then, all you need to do to publish -your software is to call `flox publish`, and Flox will take care of the rest. +publishing packages to a Catalog Store, which can exist in an AWS S3 bucket +or in any S3 compatible service, like [MinIO][minio-s3-compatible]{:target="\_blank"} +or [Backblaze B2][backblaze-b2-cloud-storage]{:target="\_blank"}. (For the +sake of simplicity, this guide focuses on S3, but there are other providers +available if you prefer them to AWS.) + +In order to use an S3 bucket to store artifacts built with Flox, you will need +to set ingress and egress URIs on the catalog using a utility published by Flox. +Then, all you need to do to publish your software is to call `flox publish`, +and Flox will take care of the rest. + +[minio-s3-compatible]: https://min.io/product/s3-compatibility +[backblaze-b2-cloud-storage]: https://www.backblaze.com/cloud-storage ## Configure an AWS S3 Bucket @@ -35,18 +43,20 @@ and we'll work with you to get you up and running. [aws-cli-reference-s3]: https://docs.aws.amazon.com/cli/latest/reference/s3/ [aws-s3-api-reference]: https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html -## Ensure the `nix daemon` Has Access to the S3 Bucket +## Ensure the Nix Daemon Has Access to the S3 Bucket As you probably know by now, the underlying technology powering Flox is Nix. -Accordingly, we need to take a couple steps to ensure that the `nix daemon` +Accordingly, we need to take a couple steps to ensure that the Nix daemon has access to the S3 bucket you've just created. To do so, you have a couple of options: -1. Set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and, if applicable, `AWS_SESSION_TOKEN` -as environment variables +1. Set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and, if applicable, +`AWS_SESSION_TOKEN` as environment variables, both for Flox and for +the daemon itself 1. Use the `aws configure` command [as described in the CLI reference][aws-cli-configure-command]{:target="\_blank"} -to set those same values +to set those same values, and ensure that the AWS profile and region match those +configured for the S3 bucket If you follow the second set of steps, you can confirm that everything is set up correctly by inspecting the values stored in `$HOME/.aws/credentials`. diff --git a/mkdocs.yml b/mkdocs.yml index 77c08f5..526373a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -62,8 +62,8 @@ nav: - ... | flat | cookbook/languages/* - Builds: - ... | flat | cookbook/builds/* - - Infrastructure: - - ... | flat | cookbook/infrastructure/* + - Publish: + - ... | flat | cookbook/publish/* - Reference: - Flox manual: - reference/command-reference/flox.md From 12b6bb33f412e7b7366f8d84495543266394e68a Mon Sep 17 00:00:00 2001 From: Brendan Magee Date: Sat, 29 Mar 2025 11:02:32 +0000 Subject: [PATCH 5/5] feat(flox-store): add instructions for signing key generation --- docs/cookbook/publish/flox-store.md | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/docs/cookbook/publish/flox-store.md b/docs/cookbook/publish/flox-store.md index f20d6b3..215649e 100644 --- a/docs/cookbook/publish/flox-store.md +++ b/docs/cookbook/publish/flox-store.md @@ -90,14 +90,25 @@ catalog-util store --catalog "" set --store-config '{ "store-ty You'll note that it's possible to set the ingress and egress URIs to the same value, if you wish to do so. -## Set Signing Key When Publishing Your Package +## Create and Set a Signing Key -At this point, you should have an appropriately Catalog Store to which you can -publish your own software via the `flox publish` command. The last thing you -need to worry about is configuring a signing key for publishing packages: +At this point, you should have an appropriately configured Catalog Store +to which you can publish your own software via the `flox publish` command. +The last thing you need to worry about is configuring a signing key +for publishing packages. + +The first step in this process is generating a key. This example illustrates +how you would do so for a key called "my-key": + +sh``` +nix key generate-secret --key-name my-key > my-key.key +nix key convert-secret-to-public < my-key.key +``` +Once you've generated the key, you can configure Flox to sign the packages +you publish with that key: ```sh -flox config --set publish.signing_key "" +flox config --set publish.signing_key "