Skip to content

Skip pulumi install when there is nothing to resolve - #1298

Open
bpalermo wants to merge 1 commit into
pulumi:masterfrom
bpalermo:fix/1297-skip-install-prebuilt-binary
Open

Skip pulumi install when there is nothing to resolve#1298
bpalermo wants to merge 1 commit into
pulumi:masterfrom
bpalermo:fix/1297-skip-install-prebuilt-binary

Conversation

@bpalermo

@bpalermo bpalermo commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

The workspace controller called the agent's Install RPC unconditionally, with no check for whether the project had anything to resolve. There are two such cases, and in both the wasted work turns into a stalled workspace.

A prebuilt program (#1297)

A project using runtime.options.binary runs an already-compiled program, so there is nothing to resolve — but the workspace still needs a complete language toolchain. A missing writable build cache, an unwritable module cache, or a compiler version that wants to download a toolchain each stops every update rather than degrading anything, and none of it is visible from the Stack, which keeps displaying its previous lastUpdate message. It also forces the image to carry a toolchain the program does not need at runtime, ruling out a CLI-only base.

One clarification on frequency, since it isn't obvious from the code: install is gated by the pod's auto.pulumi.com/initialized annotation, so it runs once per pod — but the source revision is part of the StatefulSet, so every new commit or Flux digest rolls a new pod and installs again.

A project-info workspace (#1299)

Worse, and the reason this PR covers two issues. The source-free destroy path from #1222 bootstraps a workspace from a Pulumi.yaml synthesized with only a name and runtime — deliberately no program — so a Stack can be destroyed from backend state after its source is gone. Install ran there too. Confirmed against Pulumi v3.256.0, one temp dir per runtime containing only that file:

runtime pulumi install
go error: installing dependencies: `go mod tidy` failed (go.mod file not found)
nodejs exit status 254
dotnet unable to find program: dotnet
yaml succeeds

So it fails for everything but YAML, the workspace stalls, and the Stack retains its finalizer and never destroys — the resource-orphaning outcome #1222 and #1233 exist to prevent. The stalled-during-deletion finalizer release at stack_controller.go:997 does not help: it requires status.lastUpdate == nil, while reaching this path requires status.projectInfo, which only markStackSucceeded sets.

How each is decided

The agent handles the prebuilt binary. Server.Install now loads the project settings and returns without doing anything when the runtime declares a binary. The agent already read Pulumi.yaml at startup but kept only Runtime.Name(), discarding the options map that carries it — the information was on hand and thrown away. The same rule is applied on the agent's own startup path, so behaviour does not depend on how the agent is driven.

This had to live in the agent: the operator learns a project's runtime only from ProjectInfo on an update result, i.e. after a successful operation, so at install time it knows nothing about the project. (For the same reason, #1297's first suggested option — having pulumi install itself no-op — belongs in pulumi/pulumi's CLI; LocalWorkspace.Install is runtime-agnostic and just shells out. Worth filing upstream; this doesn't block it.)

The operator handles the project-info workspace, because it can decide that locally — and because it must. writeProjectFile passes NewProjectRuntimeInfo(runtime, nil), so the synthesized project drops runtime.options.binary even for a project that declared it; no project-file-based detection can ever fire on that path.

Behaviour change worth calling out: skipping install also skips provider plugin acquisition, so this leans on the engine's automatic acquisition during an update. That is the default, so normally invisible — but a deployment that sets PULUMI_DISABLE_AUTOMATIC_PLUGIN_ACQUISITION must pre-seed plugins in its image.

An explicit opt-out

New Workspace.spec.skipInstall disables the step outright, for a workspace whose image already has what the program needs (vendored dependencies, or dependencies baked in). Because StackSpec.WorkspaceTemplate is a Workspace apply configuration, it works from a Stack with no further plumbing:

apiVersion: pulumi.com/v1
kind: Stack
spec:
  workspaceTemplate:
    spec:
      skipInstall: true

Making skips visible

InstallResult gains skipped / reason (both messages were previously empty, so this is additive and wire-compatible). Without it the operator would log Running pulumi install and report Ready with no sign that anything was skipped — trading one confusing log for another. Every skip, from either side, is logged with its reason and emits a Normal InstallationSkipped event.

On --skip-install

#1297 is right that this reads as intent and doesn't behave like it. It was never the control it looks like: the operator passes it to every agent so the startup install is suppressed and installing happens through the RPC instead, where the controller can report progress and failures on the Workspace. That is now stated in a comment at the call site, with spec.skipInstall as the user-facing setting.

Testing

  • agent/pkg/server: new testdata/prebuilt fixture — runtime go, no go.mod, plus options.binary. Deliberately the same shape as the existing uninstallable fixture, so the pair is the assertion: installing would fail here, and succeeding proves it was skipped rather than attempted. Plus a table test for the detection helper covering absent options, an empty binary, and a non-string value (the options map is map[string]any).
  • operator/internal/controller/auto: mockAutomationServer previously ignored its request and recorded nothing, so it could not express "was not called" — it now counts Install invocations. TestWorkspaceSkipInstall covers install-by-default, skipInstall: true, skipInstall: false, a project-info workspace (RPC never made, workspace still Ready, event emitted), and an agent-reported skip being surfaced.
  • Each new test was confirmed to fail against the pre-fix code rather than pass vacuously.

go build ./..., the full operator + agent suites, and golangci-lint are clean, with no codegen drift. Generated protobuf was produced with protoc 29.3 to match the checked-in files, so the diff is limited to the new fields and agent_grpc.pb.go is untouched. make test-e2e has not been run, and neither failure has been reproduced end-to-end against a live cluster.

Related issues

Fixes #1297
Fixes #1299

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

PR is now waiting for a maintainer to run the acceptance tests. This PR will only perform build and linting.
Note for the maintainer: To run the acceptance tests, please comment /run-acceptance-tests on the PR

The workspace controller called the agent's Install RPC unconditionally, with no
check for whether the project had anything to resolve. Two cases where it does
not, both of which turn wasted work into a stalled workspace.

A project using runtime.options.binary runs an already-compiled program. Nothing
needs resolving, but the workspace still requires a complete language toolchain,
so a missing writable build cache, an unwritable module cache, or a compiler
version that wants to download a toolchain each stops every update rather than
degrading anything. It also forces the image to carry a toolchain the program
does not need at runtime. Note install runs more often than it appears: it is
gated by the pod's auto.pulumi.com/initialized annotation, so once per pod, but
the source revision is part of the StatefulSet, so every new commit or Flux
digest rolls a new pod and installs again.

A project-info workspace is worse. It is bootstrapped from a Pulumi.yaml
synthesized with only a name and runtime -- no program at all -- so that a Stack
can be destroyed from backend state after its source is gone. Installing there
fails for every runtime except yaml, and the failure stalls the workspace, so
the Stack cannot destroy: it retains its finalizer and strands its cloud
resources, which is what the source-free destroy path exists to avoid. The
stalled-during-deletion finalizer release does not help, because it requires
status.lastUpdate to be nil while reaching this path requires a prior successful
update.

- The agent's Install RPC now loads the project settings and returns without
  doing anything when the runtime declares a binary. The agent already read
  Pulumi.yaml at startup but kept only the runtime name, discarding the options
  map that carries it. The same rule is applied on the agent's own startup path,
  so it does not depend on how the agent is driven.

  This also skips plugin acquisition, which install would otherwise perform. The
  engine acquires missing plugins during an update by default, so this is
  normally invisible, but a deployment that disables automatic acquisition has
  to pre-seed them.

- The operator skips the RPC entirely for a project-info workspace. It can
  decide that locally, and must: the synthesized project file is written with
  nil runtime options, so it drops runtime.options.binary even for a project
  that declared it, and no project-file-based detection can fire there.

- New Workspace.spec.skipInstall disables the step outright, for a workspace
  whose image already has what the program needs. Because
  StackSpec.WorkspaceTemplate is a Workspace apply configuration,
  spec.workspaceTemplate.spec.skipInstall works from a Stack with no further
  plumbing.

- InstallResult gains skipped/reason so an agent-side skip is visible. Without
  it the operator would log "Running pulumi install" and report Ready with no
  sign anything was skipped, trading one confusing log for another. Every skip,
  from either side, is logged with its reason and emits an InstallationSkipped
  event.

Note that --skip-install, which the operator passes to every agent, was never
the control it looks like: it only suppresses the agent's startup install so
that installing happens through the RPC where the controller can report status.
It is now commented as such, and spec.skipInstall is the user-facing setting.

Fixes pulumi#1297
Fixes pulumi#1299

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@bpalermo
bpalermo force-pushed the fix/1297-skip-install-prebuilt-binary branch from 5578aef to e163854 Compare August 8, 2026 12:09
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

PR is now waiting for a maintainer to run the acceptance tests. This PR will only perform build and linting.
Note for the maintainer: To run the acceptance tests, please comment /run-acceptance-tests on the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant