Skip to content

Commit 2fcbacf

Browse files
fnunecnunciato
andauthored
[docs] Document Terraform modules in the Pulumi Cloud registry (#19453)
* [registry] Document Terraform module hosting in the Pulumi Cloud registry Add a new concept page under IDP describing how Pulumi Cloud hosts Terraform modules: authentication via `terraform login`, the three HCP-compatible publish paths (go-tfe, hashicorp/tfe provider, tfc-workflows-github Action), the standard module layout we extract from at publish, consumption from OpenTofu / Terraform via the Module Registry Protocol, and consumption from a Pulumi program via `pulumi package add terraform-module`. Cross-link from the existing "Use a Terraform Module in Pulumi" guide so consumers landing on the IaC guide can discover the Pulumi-Cloud-hosted path. Design doc: https://app.notion.com/p/Terraform-module-hosting-in-the-Pulumi-Cloud-registry-372fdbdf1cce801ea7a3f4946c1e4154 Fixes pulumi/pulumi-service#44080 ## Test plan - Render the IDP concepts section locally and confirm the new "Terraform Modules" entry appears under the Concepts menu - Click through every internal link in the new page to confirm targets exist * [registry] Add launch blog post; correct registry host to tf.pulumi.com Add the announcement blog post for hosting Terraform modules in the Pulumi Cloud registry, and correct the registry host across the docs and blog from app.pulumi.com to the canonical tf.pulumi.com (per pulumi/pulumi-service#44889; app.pulumi.com is the console only). Move the migration guidance out of the blog and into the concept page, and document the one naming rule that differs from HCP Terraform: module names reject underscores. * [registry] Fix auth: Pulumi access token, not terraform login The TFE discovery document (cmd/service/api/tfe_discover.go) advertises tfe.v2, state.v2, and modules.v1 but no login.v1, so terraform login against tf.pulumi.com is unsupported. Document the real auth instead: a Pulumi access token is the bearer for publish (go-tfe, the tfe provider, the GitHub Action), pulumi login for pulumi package add, and TF_TOKEN_tf_pulumi_com for plain OpenTofu or Terraform. * [registry] Address review: drop GitHub Action, fix injection and layout - Remove the tfc-workflows-github Action from the publish and delete paths and from the auth list. create-run triggers a run; it does not touch the module registry. The testbed only exercised go-tfe and the tfe provider. - Consume from Pulumi: drop the rejected "CLI injects TF_TOKEN_<host>" detail. After pulumi login the provider resolves the module with your Pulumi credentials. Note that terraform-module is a parameterized provider and name its parameters. - Module layout: parse all root .tf files (any filenames); examples and README are captured at publish, not rendered (console rendering is not built yet). - Say packages, not components. * [registry] Brand: sentence-case the concept page title; link state backend Apply the Pulumi brand writing-style rules: sentence case for the concept page title_tag/title/h1 and no over-capitalized concepts, so "Terraform modules in the Pulumi Cloud registry" rather than title case. Drop the "just" minimizer from the migration line. Link "keep Terraform state in Pulumi Cloud" in the blog to the state backend guide. * [registry] Consume via the hcl package instead of terraform-module The Pulumi-program consume path now uses the forthcoming hcl package: pulumi package add hcl module <source> [version], with the version optional (omit for latest, pass to pin). Drops the terraform-module alias parameter, since the hcl module form derives the package name from the module. * [registry] Document conversion as the default Pulumi consumption path Publishing a module version converts it into a Pulumi package, and neither the concept page, the launch post, nor the Terraform module guide said so. All three taught `pulumi package add hcl module ...` as the way to consume from a Pulumi program, which is now the fallback rather than the default. The concept page gains a section on what publishing produces: the `<name>-<system>` package name, that conversion runs per version, and where to see which versions have converted. `pulumi package add <name>-<system>` becomes the documented path. The `hcl module` form stays, for versions still converting, noting that it runs the same conversion locally rather than using the published package. The guide's "Using a Module from Pulumi Cloud" section follows the same order. The pages describe what the package gives a consumer, not only that one exists: the module becomes a multi-language component, so its variables are typed inputs and its outputs typed outputs, with a generated SDK in the project's language, its resources visible individually in previews and the resource graph, an API reference generated from those variables and outputs, and a record of which stacks depend on it and which are behind the latest version. Usage tracking follows the package, so a consumer reaching the module over the Terraform protocol does not report a dependency and does not appear in the usage columns or the "Used by" tab. Both pages say so. Consuming from OpenTofu or Terraform moves below the Pulumi path on the concept page and in the post. It still documents that existing `.tf` consumers are unaffected and how to reference the module and its submodules. All three state that installing a converted package needs Pulumi CLI 3.248.0 or newer, verified by bisect: 3.247.0 fails the plugin handshake, 3.248.0 works. The Terraform module detail page is being removed from the console, so the pages describe conversion state as living on the package's page instead. Fixes #20555 ## Test plan 1. Automated checks - `./scripts/format.sh` on all three files - `vale --config=.vale.ini` on all three: 0 errors; remaining warnings are on lines this change does not touch * [registry] Retitle the launch post and set its author The title names the payoff rather than the hosting. Author set to the existing team entry. * [registry] Stop selling local conversion as a fallback, link install and console URLs Local conversion runs the same `hcl` provider the registry runs, so a module the registry could not convert fails locally for the same reason. All three pages offered it as the answer for a module using Terraform features Pulumi cannot express, which it is not. It is now described as what it is: the same conversion, run at the moment you run it, useful while a version is still converting. The concept page's section is renamed to match and says outright that it is not a way around a failed conversion. The CLI version requirement links to Download & Install Pulumi on all three pages, so a reader on an older CLI has somewhere to go. The concept page picks up aliases for the two docs URLs the console links to, both of which 404 today: - `/docs/terraform-modules/`, from the package install card - `/docs/iac/using-pulumi/pulumi-cloud/registry/terraform-modules/`, from the Terraform modules list page The concept page's deletion section is removed while we decide what deleting a converted package should do to the module it came from (pulumi/pulumi-service#47170). Documenting module deletion now would describe behavior we expect to change. The post is dated 2026-08-04. * [registry] Fix lint failures and the review's link and style findings `make lint` gates the build and was failing: - The post was missing the required `category` front matter. It announces a shipped feature, so `product` - Its `meta_desc` was 182 characters against a 160 limit. Dropped the usage tracking clause, which the post covers in the body - Removing the concept page's deletion section left a trailing blank line From the pre-merge review: - The multi-language component link 404s. `pulumi-hcl` has no `main` branch; `https://github.com/pulumi/pulumi-hcl/blob/master/docs/mlc.md` resolves. Fixed in the post and the concept page - The access token link went through an alias. The page's canonical route is `/docs/administration/access-identity/access-tokens/`; the old path is line 19 of that page's own aliases list - The closing line of "Converting a module locally" restated what the Authenticate section already establishes - Style: two `It is` openings and one `mostly` - The guide's new heading was Title Case, against AGENTS.md's sentence case rule for H2 and below The module name rule is now stated as the regex the service enforces, `[a-z0-9][a-z0-9-]*` (`pkg/apitype/registry_artifact.go`), rather than a character list that implied a leading hyphen was valid. Two review findings are not acted on. The registry host is `tf.pulumi.com`: `tfe.pulumi.com` is a dead record that CloudFront-403s since the distribution alias, ALB host rule, and service dispatch moved, so the provider test the review cites is stale. And `canonical_url` pointing at the concept page is what BLOGGING.md prescribes for a feature announcement documented in the docs. * [registry] Address review: heading context, define system, trim inside baseball * [registry] Split the blog into #20648; make local conversion a fallback, not a wait-saver * [registry] Say plainly that local conversion publishes no package --------- Co-authored-by: Christian Nunciato <chris@nunciato.org>
1 parent 3cf8dc4 commit 2fcbacf

2 files changed

Lines changed: 165 additions & 0 deletions

File tree

content/docs/iac/guides/building-extending/using-existing-tools/use-terraform-module.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,26 @@ pulumi package add terraform-module ./path/to/module localmod
8484

8585
Any directory containing `.tf` files and optionally `variables.tf` and `outputs.tf` is considered a valid module.
8686

87+
### Using a module from Pulumi Cloud
88+
89+
If your organization publishes Terraform modules to the [Pulumi Cloud registry](/docs/idp/concepts/terraform-modules/), every published version is converted into a Pulumi package for you. Install it by package name, which is the module's name and system joined with a hyphen. The system is the last segment of the module's address, naming what the module provisions, such as `aws` or `azurerm`:
90+
91+
```bash
92+
pulumi package add <name>-<system> [<version>]
93+
```
94+
95+
A module published as `acme-corp/vpc/aws` installs as `vpc-aws`. This is the same as any other Pulumi package: you get a generated SDK in your language, an [API reference](/docs/idp/concepts/private-registry/#api-documentation) on the package's page, and [usage tracking](/docs/idp/concepts/private-registry/#usage-tracking) showing which of your stacks depend on it and which are behind the latest version. Installing a converted package requires Pulumi CLI 3.248.0 or newer; see [Download & Install Pulumi](/docs/install/) to upgrade.
96+
97+
The package's page in Pulumi Cloud shows whether a given version has converted. If a version you need has no package, you can convert the module locally instead, against the module address rather than the package name:
98+
99+
```bash
100+
pulumi package add hcl module tf.pulumi.com/<namespace>/<name>/<system> [<version>]
101+
```
102+
103+
This converts the module on your machine and generates an SDK for your project. Nothing is published to the registry, so this route has no package page, API reference, or usage tracking, and a module the registry could not convert may well fail here for the same reason. The version is optional; omit it to resolve the latest published version. Self-hosted Pulumi Cloud installations use their own host (`<your-pulumi-host>/<namespace>/<name>/<system>`).
104+
105+
See [Terraform Modules in the Pulumi Cloud Registry](/docs/idp/concepts/terraform-modules/) for the publishing side and the broader module workflow.
106+
87107
## Example: Using the AWS RDS Module
88108

89109
Here's an example of how to use the AWS RDS module to provision a MySQL database in your Pulumi program.
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
---
2+
title_tag: Terraform modules in the Pulumi Cloud registry | Pulumi IDP
3+
title: Terraform modules
4+
h1: "Terraform modules in the Pulumi Cloud registry"
5+
meta_desc: Publish and consume Terraform modules in Pulumi Cloud using the HCP-compatible registry surface.
6+
menu:
7+
idp:
8+
parent: idp-concepts
9+
identifier: idp-concepts-terraform-modules
10+
weight: 15
11+
aliases:
12+
- /docs/terraform-modules/
13+
- /docs/iac/using-pulumi/pulumi-cloud/registry/terraform-modules/
14+
---
15+
16+
Pulumi Cloud hosts Terraform modules as a first-class registry resource alongside [packages](/docs/iac/concepts/packages/) and [templates](/docs/idp/concepts/organization-templates/). Teams migrating from HCP Terraform can publish their existing modules to Pulumi Cloud using the same tooling they already use (the [go-tfe](https://github.com/hashicorp/go-tfe) library or the [hashicorp/tfe Terraform provider](https://registry.terraform.io/providers/hashicorp/tfe/latest/docs)) by pointing those tools at `tf.pulumi.com` instead of `app.terraform.io`. Every module version you publish is also converted into a Pulumi package. The module's variables become typed inputs and its outputs become typed outputs, with a generated SDK in TypeScript, Python, Go, C#, Java, or YAML, an API reference on the package's page, and a record of which stacks depend on it. Conversion is additive: existing `.tf` consumers keep resolving the module over the Terraform protocol.
17+
18+
## Before you begin
19+
20+
1. You need a [Pulumi Cloud](https://app.pulumi.com) account on the Enterprise or Business Critical plan. Publishing is gated to those tiers; reading and listing modules is available on any plan, so you always keep access to modules you have already published.
21+
1. You need the [Pulumi CLI](/docs/install/) installed if you plan to consume modules from a Pulumi program.
22+
1. You need OpenTofu or Terraform installed if you plan to consume modules from a `.tf` file with `tofu init` / `terraform init`.
23+
24+
## Authenticate
25+
26+
Every surface authenticates with a [Pulumi access token](/docs/administration/access-identity/access-tokens/). It is the bearer token for everything Pulumi Cloud exposes over the HashiCorp protocol: the publish API, the state backend, and the module registry.
27+
28+
- Publishing: the go-tfe client and the tfe provider take your Pulumi access token wherever they expect a TFE token today. See [Publish a module](#publish-a-module).
29+
- Consuming from a Pulumi program: run `pulumi login`. `pulumi package add` passes the token through, so there is no separate registry login.
30+
- Consuming from plain OpenTofu or Terraform: set the host token. OpenTofu and Terraform derive the variable name from the host by replacing dots with underscores (and dashes with double underscores), so `tf.pulumi.com` becomes `TF_TOKEN_tf_pulumi_com`:
31+
32+
```bash
33+
export TF_TOKEN_tf_pulumi_com=$PULUMI_ACCESS_TOKEN
34+
```
35+
36+
You can also store the token in the Terraform CLI credentials file (`~/.terraform.d/credentials.tfrc.json`). Self-hosted installations use the same scheme with their own host.
37+
38+
## Publish a module
39+
40+
Pulumi Cloud's publish API is wire-compatible with HCP Terraform's private registry. Existing HCP migration tooling works unmodified, pointed at the new host. The two most common paths:
41+
42+
### go-tfe
43+
44+
Publish from Go, or from any CI pipeline that already drives HCP Terraform through this client, by pointing it at the Pulumi Cloud host:
45+
46+
```go
47+
client, _ := tfe.NewClient(&tfe.Config{
48+
Address: "https://tf.pulumi.com",
49+
Token: os.Getenv("PULUMI_ACCESS_TOKEN"),
50+
})
51+
```
52+
53+
The `RegistryModules` surface (`client.RegistryModules.Create`, `CreateVersion`, `UploadTarGzip`, etc.) accepts the same payloads it accepts against `app.terraform.io`.
54+
55+
### `hashicorp/tfe` Terraform provider
56+
57+
Publish from HCL, if you manage your registry modules declaratively with OpenTofu or Terraform:
58+
59+
```hcl
60+
provider "tfe" {
61+
hostname = "tf.pulumi.com"
62+
token = var.pulumi_access_token
63+
}
64+
65+
resource "tfe_registry_module" "vpc" {
66+
organization = "acme"
67+
...
68+
}
69+
```
70+
71+
### Module layout
72+
73+
At publish time Pulumi Cloud reads the standard [Terraform module structure](https://developer.hashicorp.com/terraform/language/modules/develop/structure):
74+
75+
- Root `.tf` files (any filenames) define the module's inputs, outputs, and required providers.
76+
- `modules/<name>/` subdirectories are parsed the same way as the root and can be consumed as submodules.
77+
- `examples/<name>/` subdirectories and the `README.md` are captured at publish.
78+
79+
## Migrating from HCP Terraform
80+
81+
If you publish from CI today, the move is a host change. Point your existing pipelines at `tf.pulumi.com`, supply a Pulumi access token, and your publish steps run unchanged. Reading and listing modules work on any plan, so you keep access to modules you have already published regardless of your plan.
82+
83+
### Module names
84+
85+
Pulumi Cloud uses the same `<namespace>/<name>/<system>` address form as HCP Terraform. The namespace is your Pulumi organization, and the system is the segment HCP Terraform calls the provider: what the module provisions, such as `aws`, `azurerm`, or `kubernetes`. One rule is stricter: the module name must match `[a-z0-9][a-z0-9-]*`, so it starts with a letter or digit and underscores are rejected at publish. A module that HCP hosts under a name like `control_tower_account_factory` has to be renamed to `control-tower-account-factory` before you publish it. Uppercase in the name is lowercased automatically.
86+
87+
## What happens when you publish
88+
89+
Publishing a module version also converts it into a Pulumi package, with no extra step on your part. The package is named after the module: `<name>-<system>`, published under the same namespace and registry source, so a module published as `acme-corp/vpc/aws` produces a package called `vpc-aws`.
90+
91+
Conversion runs per version, so a module can have some versions with packages and some without. The package's page in Pulumi Cloud shows which versions have converted and gives you the command to install one.
92+
93+
## Consume from a Pulumi program
94+
95+
Once a version has converted, install it by package name:
96+
97+
```bash
98+
pulumi package add <name>-<system> [<version>]
99+
```
100+
101+
The module is a [multi-language component](https://github.com/pulumi/pulumi-hcl/blob/master/docs/mlc.md): its `variable` blocks become typed inputs, its `output` blocks become typed outputs, and Pulumi generates an SDK in the language your project uses. The version you pass is persisted in `Pulumi.yaml`, so `pulumi install` regenerates the same pinned version.
102+
103+
The resources the module creates appear individually in previews and in the resource graph rather than as one opaque unit. In Pulumi Cloud the package gets the same treatment as any other: an [API reference](/docs/idp/concepts/private-registry/#api-documentation) generated from the module's variables and outputs, and [usage tracking](/docs/idp/concepts/private-registry/#usage-tracking) recording which stacks depend on it and which of those are behind the latest version.
104+
105+
Usage tracking only counts consumption through the converted package. A stack or workspace that consumes the module over the Terraform protocol does not report a dependency, so it does not appear in the usage columns or on the package's "Used by" tab.
106+
107+
{{% notes type="info" %}}
108+
Installing a converted package requires Pulumi CLI 3.248.0 or newer. See [Download & Install Pulumi](/docs/install/) to install or upgrade.
109+
{{% /notes %}}
110+
111+
### If a version has no package
112+
113+
Installing by package name is the path to reach for. If a version you need has no package, you can convert the module locally instead, against the module address rather than the package name:
114+
115+
```bash
116+
pulumi package add hcl module tf.pulumi.com/<namespace>/<name>/<system> [<version>]
117+
```
118+
119+
`hcl` is a parameterized provider. The `module` keyword selects module mode, followed by the module address and an optional version. Omit the version to resolve the latest published version; pass one to pin it.
120+
121+
This converts the module on your machine, using your local `hcl` provider, and generates an SDK for your project. Nothing is published: there is no package in the registry, and so no package page, no API reference, and no usage tracking, and a teammate who needs the module runs the same command rather than installing what you produced. A module the registry could not convert may well fail here for the same reason.
122+
123+
Both commands resolve using your Pulumi credentials. See [Use a Terraform Module in Pulumi](/docs/iac/guides/building-extending/using-existing-tools/use-terraform-module/) for examples.
124+
125+
## Consume from OpenTofu or Terraform
126+
127+
Reference the module in a `.tf` file:
128+
129+
```hcl
130+
module "vpc" {
131+
source = "tf.pulumi.com/<namespace>/<name>/<system>"
132+
version = "1.2.3"
133+
}
134+
```
135+
136+
`tofu init` and `terraform init` resolve and download the module from Pulumi Cloud using the token you set above.
137+
138+
Submodules are referenced with the standard `//modules/<name>` source syntax:
139+
140+
```hcl
141+
module "private_subnet" {
142+
source = "tf.pulumi.com/<namespace>/<name>/<system>//modules/<submodule>"
143+
version = "1.2.3"
144+
}
145+
```

0 commit comments

Comments
 (0)