tfbridge: match scalar set elements by value when extracting inputs

[Zamiell]

Claude Opus 5 found/fixed this bug while investigating a non-deterministic Pulumi plan.

LLM:

Match scalar set elements by value when extracting inputs

Fixes the third instance of a family of extractInputs bugs where pulumi refresh silently
rewrites the inputs recorded in state, causing permanent drift that Diff() cannot see.

Problem

extractInputs merges the inputs recorded in state with the state returned by the provider. For
arrays it matches elements by position, which is correct for TypeList but wrong for TypeSet.
#3384 and #3410 addressed this by adding matchSetElements, which matches set elements by
content -- but only when every element is an object:

// Only match object elements -- scalar sets have no keys to score on.
for _, v := range oldArray {
    if !v.IsObject() {
        return nil
    }
}

A set of scalars therefore falls back to positional matching, and the scalar branch of
extractInputs returns newState:

case oldInput.IsString() && newState.IsString():
    ...
    return newState, oldInput.StringValue() == newState.StringValue()

The net effect is that refresh replaces the recorded inputs with the provider's ordering.

A set is unordered, so the order the provider returns generally differs from the order the values
appear in the program. Once refresh has rewritten the inputs, they are permanently different from
what the program produces on every subsequent run.

Why this matters

The drift is invisible to Diff(), so nothing is displayed and pulumi preview --expect-no-changes
and pulumi refresh --expect-no-changes both pass.

It is not harmless, though. The engine compares raw input bags in sameSnapshotMutation.mustWrite
when deciding whether a checkpoint write can be elided. A resource whose inputs differ from its
state cannot be elided, so every affected resource forces a full state serialization and upload on
every subsequent update, for as long as the difference persists.

We hit this on a stack with roughly 1400 resources and a 19.5MB state file, where the redundant
writes turned a one-line change into a 16 minute pulumi up. Fixing #3567 dropped the resources
that drift after a refresh from 787 to 34. This bug accounts for 17 of the remaining 34, all of
them a TypeSet of strings; applying this change as well takes those 17 to 0 and brings the same
update down to roughly 90 seconds. The other 17 involve unrelated properties and are not addressed
here.

Fix

Match scalar set elements by exact value.

Matching is only applied when both sides hold the same multiset of values. When set membership
genuinely changed, matchScalarSetElements returns nil so the caller keeps its existing
positional behavior and the change remains visible as drift. Elements that are not plain scalars
(computed, secret, output, nested collections) also fall back to positional matching rather than
being guessed at.

Testing

TestRegressScalarSetOrderRefreshPreservesStateInputs in pkg/tests is an end-to-end test: it runs
a real up followed by a refresh and asserts on the exported stack state.

Asserting on state is deliberate and necessary. ExpectNoChanges passes on both refresh and
preview while this bug is occurring, because Diff() returns DIFF_NONE. State is the only place
the bug is observable, so a test built on ExpectNoChanges would not catch a regression here.

TestRefreshExtractInputsScalarTypeSetReorder in pkg/tfbridge covers the unit-level behavior:

• set_order_preserved -- a reordered scalar set keeps the recorded order
• typelist_still_positional -- TypeList is unaffected and stays positional
• changed_membership_still_visible -- a genuine membership change is not masked
• duplicate_values_preserved -- repeated values are matched one-for-one

Both the end-to-end test and the two order-sensitive unit subtests were verified to fail without the
change in schema.go and pass with it. The two control subtests pass either way, confirming the
existing behavior they cover is unchanged.

go test ./pkg/tfbridge/ passes. go test ./pkg/tests/ passes except for TestAccProviderConfig,
which fails identically on an unmodified checkout in my environment because it requires
make install_plugins.

Note on scope

pkg/pf has its own input extraction path, which I have not examined. This change is limited to the
SDK-based bridge.