Skip to content

Commit 0fe8c09

Browse files
Consolidate 15 templated AWS-credentials what-is pages into one hub (#20755)
* Consolidate 15 templated AWS-credentials what-is pages into one hub Merges 15 near-duplicate (96.4% character-similar) programmatically generated AWS-credentials pages under /what-is/ into a single canonical troubleshooting hub, aws-cli-dynamic-credentials-with-pulumi-esc, covering the same 5 credential errors and 10 AWS CLI commands with a proper FAQ section (none of the 16 originals had one). run-aws-sts-get-caller-identity-with-dynamic-credentials, the section's #1 traffic page (113 clicks/28d, position 4.81), is left untouched at its existing URL and is now cross-linked from the new hub instead of being folded in, per the consolidation card's constraint. The other 15 old URLs redirect to the hub via Hugo aliases (meta-refresh + noindex, the preferred SEO-safe redirect mechanism for renamed Hugo content per BUILD-AND-DEPLOY.md). Also updates data/what_is_sections.yml's troubleshooting slug list to match the new page set. * Shorten meta_desc on new hub page to pass markdown lint (<160 chars) * Address pinned review low-confidence findings on AWS credentials hub - Point internal links at canonical ESC doc paths (/docs/esc/, /docs/esc/get-started/, /docs/esc/guides/configuring-oidc/aws/) instead of the legacy /docs/pulumi-cloud/esc/ and /docs/esc/environments/... alias paths. - Drop the unverifiable frequency claim on the InvalidAccessKeyId cause and state the possible causes without ranking them. - Align the Pulumi Cloud console walkthrough wording (org switcher in left-hand nav, button labels) with the verified copy in the ESC get-started guide and the AWS OIDC configuration guide. --------- Co-authored-by: workprentice <257153108+workprentice@users.noreply.github.com>
1 parent f569be1 commit 0fe8c09

18 files changed

Lines changed: 185 additions & 1844 deletions
Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
---
2+
title: Fix AWS CLI Credential Errors and Run AWS Commands with Pulumi ESC
3+
meta_desc: |
4+
Learn how Pulumi ESC's dynamic AWS credentials fix errors like ExpiredToken
5+
and let you run any AWS CLI command without local credential setup.
6+
type: what-is
7+
page_title: Fix AWS CLI Credential Errors and Run AWS Commands with Pulumi ESC
8+
authors: ["diana-esteves", "torian-crane"]
9+
aliases:
10+
- /what-is/resolve-list-buckets-expired-token/
11+
- /what-is/resolve-list-buckets-invalid-access-key-id/
12+
- /what-is/resolve-list-buckets-invalid-client-token-id/
13+
- /what-is/resolve-list-buckets-signature-does-not-match/
14+
- /what-is/resolve-unable-to-locate-credentials/
15+
- /what-is/run-aws-cloudwatch-get-metric-data-with-dynamic-credentials/
16+
- /what-is/run-aws-dynamodb-list-tables-with-dynamic-credentials/
17+
- /what-is/run-aws-ec2-describe-instances-with-dynamic-credentials/
18+
- /what-is/run-aws-ec2-start-instances-with-dynamic-credentials/
19+
- /what-is/run-aws-ec2-stop-instances-with-dynamic-credentials/
20+
- /what-is/run-aws-iam-list-users-with-dynamic-credentials/
21+
- /what-is/run-aws-lambda-list-functions-with-dynamic-credentials/
22+
- /what-is/run-aws-s3-cp-with-dynamic-credentials/
23+
- /what-is/run-aws-s3-ls-with-dynamic-credentials/
24+
- /what-is/run-aws-s3-sync-with-dynamic-credentials/
25+
---
26+
27+
Most AWS CLI credential errors, such as `ExpiredToken`, `InvalidAccessKeyId`, `InvalidClientTokenId`, `SignatureDoesNotMatch`, and "Unable to locate credentials," trace back to the same root cause: long-lived credentials that were configured locally, went stale, or were never configured correctly in the first place. Amazon Security Token Service (STS) issues temporary, limited-privilege credentials specifically to reduce this risk, but temporary credentials still require someone to generate, distribute, and refresh them by hand unless a tool does it automatically.
28+
29+
[Pulumi ESC (Environments, Secrets, and Configurations)](/docs/esc/) removes that manual step. With [dynamic credentials from AWS using OIDC](/blog/esc-env-run-aws/), Pulumi ESC requests short-lived AWS credentials on demand and injects them into the shell for the duration of a single command, via `pulumi env run`. There is nothing stored on disk to expire, misconfigure, or leak, and every AWS CLI command, from `aws s3 ls` to `aws sts get-caller-identity`, runs against fresh, correctly scoped credentials every time.
30+
31+
## Common AWS CLI credential errors and their cause
32+
33+
### ExpiredToken
34+
35+
"An error occurred (ExpiredToken) when calling the ListBuckets operation" (or any other operation) means the temporary credentials used for the call, typically issued through an IAM role or STS, have passed their expiration time. AWS expires these credentials by design; the error is expected behavior once the clock runs out, not a sign of misconfiguration.
36+
37+
### InvalidAccessKeyId
38+
39+
"An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation" means the access key ID in the request does not exist in AWS's records for the target account — commonly because the key was deleted, belongs to a different account, or was mistyped.
40+
41+
### InvalidClientTokenId
42+
43+
"An error occurred (InvalidClientTokenId) when calling the ListBuckets operation" is AWS's way of saying the security token or access key it received cannot be validated at all: malformed, revoked, or never issued. It is a closely related but distinct failure mode from an expired or merely incorrect key.
44+
45+
### SignatureDoesNotMatch
46+
47+
"An error occurred (SignatureDoesNotMatch) when calling the ListBuckets operation" means AWS could not verify the cryptographic signature computed from the secret access key. This usually points to a secret key that was copied incorrectly, is out of sync with its access key ID, or a system clock that has drifted enough to invalidate the signed request.
48+
49+
### Unable to locate credentials
50+
51+
The AWS CLI and SDKs raise "Unable to locate credentials" when they cannot find any credentials at all in the usual places: environment variables, the shared credentials file, an EC2 instance profile, or an assumed role. It is the default failure when nothing has been configured yet.
52+
53+
Every one of these errors disappears when credentials are generated fresh for each command rather than stored and reused. That is precisely what Pulumi ESC's dynamic credentials are designed to do.
54+
55+
## Using Pulumi ESC for dynamic credentials with AWS
56+
57+
[Pulumi ESC](https://www.pulumi.com/product/esc/) is a service that helps to alleviate the burden of managing cloud configuration and secrets by providing a centralized way to handle these critical aspects of cloud development. The `pulumi env run` command in particular helps resolve concerns around how to:
58+
59+
- Securely share credentials with teammates in a consistent way.
60+
- Minimize the risks associated with locally configured, long-lived and highly privileged credentials.
61+
- Ensure teams can easily and safely run AWS CLI commands without requiring deep security expertise.
62+
63+
### What is the pulumi env run command?
64+
65+
The [Pulumi documentation for the `pulumi env run` command](/docs/iac/cli/commands/pulumi_env_run/) states the following:
66+
67+
> This command opens the environment with the given name and runs the given command. If the opened environment contains a top-level 'environmentVariables' object, each key-value pair in the object is made available to the command as an environment variable.
68+
69+
In practice, this means any AWS CLI command, `aws sts get-caller-identity`, `aws s3 ls`, `aws ec2 describe-instances`, or dozens of others, can run without configuring AWS credentials locally beforehand. Three things follow from that:
70+
71+
- **Seamless command execution.** `pulumi env run` lets you execute AWS commands effortlessly, freeing you from the intricacies of managing AWS credentials on your local machine.
72+
- **Enhanced security.** Removing local credential storage drastically reduces the risk of accidental exposure. Credentials and secrets stay securely managed within the Pulumi environment.
73+
- **Streamlined collaboration.** Because credentials are centralized, every team member runs commands against the same secure environment, which removes the need to coordinate individual credentials and configurations.
74+
75+
## Getting started with pulumi env run
76+
77+
### Step 1: Install and log in to Pulumi ESC
78+
79+
Install the [Pulumi CLI](/docs/iac/download-install/), then run `pulumi login` and follow the prompts to log in.
80+
81+
```bash
82+
$ pulumi login
83+
Manage your Pulumi stacks by logging in.
84+
Run `pulumi login --help` for alternative login options.
85+
Enter your access token from https://app.pulumi.com/account/tokens
86+
or hit <ENTER> to log in using your browser :
87+
Logged in to pulumi.com as …
88+
```
89+
90+
### Step 2: Create the OIDC configuration
91+
92+
Rather than storing AWS credentials as static secrets in an ESC environment, configure dynamic credentials so Pulumi ESC generates them on demand. Follow the [guide for configuring OIDC between Pulumi and AWS](/docs/esc/guides/configuring-oidc/aws/), and make sure the IAM role you create has sufficient permissions for the AWS operations you plan to run.
93+
94+
### Step 3: Create a new Pulumi ESC environment
95+
96+
Once OIDC is configured, create a new environment in [Pulumi Cloud](https://app.pulumi.com/signin). Make sure you have the correct organization selected in the left-hand navigation, select **Environments**, then **+ Create Environment**, and give it a name.
97+
98+
{{< video title="Open environment in Pulumi ESC console" src="https://www.pulumi.com/uploads/esc-create-new-env.mp4" autoplay="true" loop="true" >}}
99+
100+
### Step 4: Add the AWS provider integration
101+
102+
Clear the placeholder content in the environment editor and replace it with the following, substituting `<your-oidc-iam-role-arn>` with the IAM role ARN from the OIDC step:
103+
104+
```yaml
105+
values:
106+
aws:
107+
login:
108+
fn::open::aws-login:
109+
oidc:
110+
duration: 1h
111+
roleArn: <your-oidc-iam-role-arn>
112+
sessionName: pulumi-environments-session
113+
environmentVariables:
114+
AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
115+
AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
116+
AWS_SESSION_TOKEN: ${aws.login.sessionToken}
117+
```
118+
119+
### Step 5: Run any AWS CLI command with dynamic credentials
120+
121+
First, confirm your local environment has no AWS credentials configured:
122+
123+
```bash
124+
$ aws configure list
125+
Name Value Type Location
126+
---- ----- ---- --------
127+
profile <not set> None None
128+
access_key <not set> None None
129+
secret_key <not set> None None
130+
region <not set> None None
131+
```
132+
133+
Then run any AWS CLI command through `pulumi env run`, replacing `<your-pulumi-org-name>`, `<your-project-name>`, and `<your-environment-name>` with your own values:
134+
135+
```bash
136+
pulumi env run <your-pulumi-org-name>/<your-project-name>/<your-environment-name> -- aws s3 ls
137+
```
138+
139+
The same pattern works for any AWS CLI operation. A few of the most common:
140+
141+
- `pulumi env run ... -- aws sts get-caller-identity` — see the [dedicated walkthrough](/what-is/run-aws-sts-get-caller-identity-with-dynamic-credentials/) for this specific command, the most-used entry point for verifying dynamic credentials are wired up correctly.
142+
- `pulumi env run ... -- aws s3 cp <source> <destination>`
143+
- `pulumi env run ... -- aws s3 sync <source> <destination>`
144+
- `pulumi env run ... -- aws ec2 describe-instances`
145+
- `pulumi env run ... -- aws ec2 start-instances --instance-ids <id>`
146+
- `pulumi env run ... -- aws ec2 stop-instances --instance-ids <id>`
147+
- `pulumi env run ... -- aws iam list-users`
148+
- `pulumi env run ... -- aws lambda list-functions`
149+
- `pulumi env run ... -- aws dynamodb list-tables`
150+
- `pulumi env run ... -- aws cloudwatch get-metric-data --cli-input-json <file>`
151+
152+
## Frequently asked questions
153+
154+
### What causes the ExpiredToken error when calling AWS APIs?
155+
156+
Temporary AWS credentials, whether issued through an IAM role, STS, or Pulumi ESC's dynamic credentials, carry an expiration time by design. The `ExpiredToken` error simply means that time has passed. Requesting a fresh set of credentials for each command, which is what `pulumi env run` does automatically, prevents the error from occurring at all.
157+
158+
### How do I fix InvalidAccessKeyId or InvalidClientTokenId errors in the AWS CLI?
159+
160+
Both errors mean AWS could not validate the access key or token it received, either because it does not exist, was revoked, or was never issued correctly. Rather than debugging a specific stale key, replace static local credentials with Pulumi ESC's dynamic credentials so a correctly scoped, valid key is generated for every command.
161+
162+
### Why does AWS return SignatureDoesNotMatch?
163+
164+
This error means AWS could not verify the request's cryptographic signature, usually because a secret access key was copied incorrectly, is mismatched with its access key ID, or the system clock has drifted. Dynamic credentials from Pulumi ESC avoid the problem entirely, since the access key and secret are generated together and used immediately.
165+
166+
### What does "Unable to locate credentials" mean and how do I fix it?
167+
168+
The AWS CLI raises this error when it finds no credentials in any of the usual locations it checks: environment variables, the shared credentials file, an instance profile, or an assumed role. Running the command through `pulumi env run` supplies valid credentials as environment variables for that single invocation, so there is nothing to locate or configure beforehand.
169+
170+
### Can I run any AWS CLI command with Pulumi ESC dynamic credentials?
171+
172+
Yes. Once an environment is configured with the `aws-login` OIDC provider, `pulumi env run <org>/<project>/<environment> -- <any aws command>` works for any AWS CLI operation the underlying IAM role is permitted to perform. The command after `--` is unrestricted by Pulumi ESC itself; permissions are governed entirely by the IAM role's policy.
173+
174+
## Conclusion
175+
176+
Pulumi ESC makes it easier than ever to tame AWS credential management, from everyday commands like `aws s3 ls` to the errors that show up when credentials are stale, wrong, or missing entirely. Pulumi ESC supports dynamic credentials using OIDC across AWS, Azure, and Google Cloud. Check out the following links to learn more:
177+
178+
- Follow the [Getting Started](/docs/esc/get-started/) guide.
179+
- Read the [documentation](/docs/esc/) for all the commands and features available.
180+
- Visit the [open source](https://github.com/pulumi/esc) repo for Pulumi ESC.
181+
182+
Feel free to [join the Pulumi community on Slack](https://slack.pulumi.com/) and let us know what you think!

0 commit comments

Comments
 (0)