Update Orchestrion to v1.12.0 and preserve partial uploads

.bazelignore

@@ -11,6 +11,7 @@ bazel-out
 bazel-testlogs
 third_party/rgo/v0_60_0/base
 third_party/rgo/v0_61_1/base
+third_party/rgo/v0_62_0/base
 tools/tests/rules_go_variant_regressions
 modules/go/bazel-bin
 modules/go/bazel-out

AGENTS.md

@@ -95,21 +95,14 @@ The sync rule creates `@test_optimization_data//` containing:
   ```bash
   # Tests write payloads to TEST_UNDECLARED_OUTPUTS_DIR automatically
   # Bazel collects them to bazel-testlogs/<target>/test.outputs/
-  ./bazelw test //... || test_status=$?; test_status=${test_status:-0}
-  ./bazelw run //<topt-package>:dd_test_optimization_doctor || doctor_status=$?; doctor_status=${doctor_status:-0}
-  if [ "$doctor_status" -ne 0 ]; then
-    if [ "$test_status" -ne 0 ]; then exit "$test_status"; fi
-    exit "$doctor_status"
-  fi
-  ./bazelw run //<topt-package>:dd_upload_payloads -- --dry-run --validate-enrichment || dry_run_status=$?; dry_run_status=${dry_run_status:-0}
-  if [ "$dry_run_status" -ne 0 ]; then
-    if [ "$test_status" -ne 0 ]; then exit "$test_status"; fi
-    exit "$dry_run_status"
-  fi
-  DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" ./bazelw run //<topt-package>:dd_upload_payloads
-  upload_status=$?
-  if [ "$test_status" -ne 0 ]; then exit "$test_status"; fi
-  exit "$upload_status"
+  test_status=0; doctor_status=0; dry_run_status=0; upload_status=0
+  ./bazelw test //... || test_status=$?
+  ./bazelw run //<topt-package>:dd_test_optimization_doctor || doctor_status=$?
+  ./bazelw run //<topt-package>:dd_upload_payloads -- --dry-run --validate-enrichment || dry_run_status=$?
+  DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" ./bazelw run //<topt-package>:dd_upload_payloads || upload_status=$?
+  for status in "$test_status" "$doctor_status" "$dry_run_status" "$upload_status"; do
+    if [ "$status" -ne 0 ]; then exit "$status"; fi
+  done
   ```
 - Force refetch of test optimization data:
   ```bash
@@ -164,7 +157,8 @@ The sync rule creates `@test_optimization_data//` containing:
 - Bazel automatically collects these to `bazel-testlogs/<package>/<target>/test.outputs/`.
 - In consumer workspaces, run `./bazelw run //<topt-package>:dd_test_optimization_doctor`
   after tests complete, then run `./bazelw run //<topt-package>:dd_upload_payloads -- --dry-run --validate-enrichment`, then upload with `./bazelw run //<topt-package>:dd_upload_payloads`.
-  Do not run the real upload if doctor or dry-run enrichment validation fails.
+  When upload is authorized, process every available fresh valid payload even if
+  tests, doctor, or dry-run fail, and preserve the earliest failure as the job result.
 - For Go, route the repository's central `dd_go_test` wrapper through
   `dd_topt_go_test`; `--config=test-optimization` is the only user-facing
   enable switch.

CHANGELOG.md

@@ -34,6 +34,10 @@ versioning.
   Orchestrion integration and public consumer patch profile.
 
 ### Changed
+- Updated the maintained `rules_go` Orchestrion integration to Orchestrion
+  `v1.12.0` and `dd-trace-go/v2` `v2.9.1`, preserving upstream test-variant
+  resolution while retaining the Bazel-specific resolver compatibility layer.
+  Orchestrion `v1.12.0` requires Go `1.25.0` or newer.
 - The public Go Bzlmod extension now defaults `enabled_by_env` to `True`, so
   omitting `--config=test-optimization` disables metadata sync and Orchestrion
   together while the named config enables both.

README.md

@@ -286,8 +286,8 @@ RTO_ARCHIVE_TYPE="tar.gz"
 RULES_GO_UPSTREAM="v0_60_0"
 RULES_GO_VARIANT="base"
 RULES_GO_STRIP_PREFIX="third_party/rgo/v0_60_0/base"
-DD_TRACE_GO_VERSION="v2.9.0"
-ORCHESTRION_VERSION="v1.9.0"
+DD_TRACE_GO_VERSION="v2.9.1"
+ORCHESTRION_VERSION="v1.12.0"
 ```
 
 The archive URL, SHA256, and prefix are tied to the repository commit.
@@ -377,7 +377,7 @@ tools/test_optimization/run_test_optimization_ci.sh \
   --support-bundle .topt/reports/dd-test-optimization-support.zip \
   //...
 
-# Add --upload only when the real upload should run after doctor and dry-run pass.
+# Add --upload to send every available fresh valid payload after validation attempts.
 DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" \
   tools/test_optimization/run_test_optimization_ci.sh \
     --report-dir .topt/reports \
@@ -395,7 +395,7 @@ DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" \
   -SupportBundle .topt\reports\dd-test-optimization-support.zip `
   //...
 
-# Add -Upload only when the real upload should run after doctor and dry-run pass.
+# Add -Upload to send every available fresh valid payload after validation attempts.
 $env:DD_API_KEY = "<your-api-key>"
 $env:DD_SITE = "datadoghq.com"
 .\tools\test_optimization\run_test_optimization_ci.ps1 `
@@ -431,14 +431,16 @@ bazel_dep(name = "rules_go", version = "0.60.0")
 
 Then run the Datadog bootstrap helper once from the workspace that owns your
 Go module. `--dd-trace-go-version` is optional; if you omit it, the default is
-`v2.9.0`.
+`v2.9.1`. Orchestrion `v1.12.0` requires Go `1.25.0` or newer, so the
+`--runtime-version` value and registered Bazel Go toolchain must satisfy that
+minimum.
 
 ```bash
 bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
   --guided \
   --service go-service \
   --runtime-version 1.25.0 \
-  --dd-trace-go-version v2.9.0 \
+  --dd-trace-go-version v2.9.1 \
   --write-bazelrc
 ```
 
@@ -449,7 +451,7 @@ bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
   --guided \
   --service go-service \
   --runtime-version 1.25.0 \
-  --dd-trace-go-version v2.9.0 \
+  --dd-trace-go-version v2.9.1 \
   --go-module-dir path/to/go-module \
   --write-bazelrc
 ```
@@ -1424,7 +1426,7 @@ tools/test_optimization/run_test_optimization_ci.sh \
   --support-bundle .topt/reports/dd-test-optimization-support.zip \
   //...
 
-# Add --upload only when the real upload should run after doctor and dry-run pass.
+# Add --upload to send every available fresh valid payload after validation attempts.
 DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" \
   tools/test_optimization/run_test_optimization_ci.sh \
     --report-dir .topt/reports \
@@ -1440,7 +1442,7 @@ DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" \
   -SupportBundle .topt\reports\dd-test-optimization-support.zip `
   //...
 
-# Add -Upload only when the real upload should run after doctor and dry-run pass.
+# Add -Upload to send every available fresh valid payload after validation attempts.
 $env:DD_API_KEY = "<your-api-key>"
 $env:DD_SITE = "datadoghq.com"
 .\tools\test_optimization\run_test_optimization_ci.ps1 `
@@ -1450,10 +1452,10 @@ $env:DD_SITE = "datadoghq.com"
   //...
 ```
 
-**IMPORTANT**: Always preserve the test exit code. Upload failed-test payloads
-when doctor and dry-run enrichment pass, but do not run the real upload after a
-doctor or dry-run failure. Using plain `;` causes CI to report success even when
-tests fail.
+**IMPORTANT**: Always preserve the test exit code. When upload is enabled, the
+wrapper runs doctor and dry-run for diagnostics, then uploads every available
+fresh valid payload even if tests or either validation step failed. The earliest
+test, doctor, dry-run, or upload failure remains the job result.
 
 ### Important runtime requirements
 
@@ -1768,7 +1770,7 @@ warmed host Go module cache. Test payloads still use the Bazel file-output
 contract: the tracer writes JSON files under `TEST_UNDECLARED_OUTPUTS_DIR`, and
 the uploader enriches those JSON files with repository and Bazel metadata. Pass
 `--dd-trace-go-version <query>` if you want a non-default tracer version;
-otherwise the default is `v2.9.0`.
+otherwise the default is `v2.9.1`.
 
 ```bash
 bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
@@ -1808,7 +1810,7 @@ use_repo(test_optimization_go_sdk, "test_optimization_go_sdk")
 
 orchestrion = use_extension("@rules_go//go:extensions.bzl", "orchestrion")
 orchestrion.from_source(
-    version = "v1.9.0",
+    version = "v1.12.0",
     dd_trace_go_pin_files = [
         "@//:go.mod",
         "@//:go.sum",
@@ -1836,11 +1838,11 @@ use_repo(test_optimization_go_sdk, "test_optimization_go_sdk")
 
 orchestrion = use_extension("@rules_go//go:extensions.bzl", "orchestrion")
 orchestrion.from_source(
-    version = "v1.9.0",
+    version = "v1.12.0",
     dd_trace_go_versions = {
-        "github.com/DataDog/dd-trace-go/v2": "v2.9.0",
-        "github.com/DataDog/dd-trace-go/contrib/net/http/v2": "v2.9.0",
-        "github.com/DataDog/dd-trace-go/contrib/log/slog/v2": "v2.9.0",
+        "github.com/DataDog/dd-trace-go/v2": "v2.9.1",
+        "github.com/DataDog/dd-trace-go/contrib/net/http/v2": "v2.9.1",
+        "github.com/DataDog/dd-trace-go/contrib/log/slog/v2": "v2.9.1",
     },
     go_sdk_root = "@test_optimization_go_sdk//:ROOT",
     go_sdk_version = "<go-version>",
@@ -1853,7 +1855,7 @@ with explicit Bazel flags in the script itself. There is no special repo-root
 `--config=hermetic` shortcut for this flow.
 
 If all three selection settings are omitted, the legacy default is still
-`v2.9.0`. Manual setups must keep the local Go module pins on the same effective
+`v2.9.1`. Manual setups must keep the local Go module pins on the same effective
 versions, or the build will stop with a mismatch error. Do not combine
 `dd_trace_go_pin_files`, `dd_trace_go_version`, or `dd_trace_go_versions` in
 the same `orchestrion.from_source(...)` call.

docs/Configuration_Reference.md

@@ -16,7 +16,7 @@ Guided Go bootstrap accepts:
 
 | Flag | Default | Description |
 |------|---------|-------------|
-| `--dd-trace-go-version` | `v2.9.0` | Go tracer query for bootstrap. Accepts a tag, pseudo-version, branch, or commit SHA and persists the exact resolved versions Bazel should use |
+| `--dd-trace-go-version` | `v2.9.1` | Go tracer query for bootstrap. Accepts a tag, pseudo-version, branch, or commit SHA and persists the exact resolved versions Bazel should use |
 | `--go-binary` | `go` | Go binary used for bootstrap module graph synchronization. Set this to a pinned SDK path named `go` or `go.exe` when the repository must match Bazel's Go SDK; do not include shell syntax or arguments |
 | `--go-mod-sync` | `targeted` | Local Go module synchronization strategy: `targeted` updates and verifies only Orchestrion tool packages, `tidy` also runs `go mod tidy`, and `off` skips Go module commands |
 
@@ -99,7 +99,7 @@ Manual Orchestrion wiring in `MODULE.bazel` accepts:
 | Setting | Default | Description |
 |---------|---------|-------------|
 | `orchestrion.from_source(..., dd_trace_go_pin_files = ["@//:go.mod", "@//:go.sum"])` | none | Preferred consumer mode. Derives exact direct and transitive supported tracer versions from one checked-in `go.mod` and `go.sum` using the Bazel-managed Go SDK and `-mod=readonly` |
-| `orchestrion.from_source(..., dd_trace_go_version = "...")` | legacy `v2.9.0` when no selection mode is set | Explicit shared-version escape hatch that Bazel validates against the target Go module |
+| `orchestrion.from_source(..., dd_trace_go_version = "...")` | legacy `v2.9.1` when no selection mode is set | Explicit shared-version escape hatch that Bazel validates against the target Go module |
 | `orchestrion.from_source(..., dd_trace_go_versions = {...})` | none | Exact canonical per-module tracer versions that Bazel validates against the target Go module for `github.com/DataDog/dd-trace-go/v2`, `github.com/DataDog/dd-trace-go/contrib/net/http/v2`, and `github.com/DataDog/dd-trace-go/contrib/log/slog/v2` |
 | `orchestrion.from_source(..., go_sdk_root = "@repo//:ROOT")` | none | Bazel-managed Go SDK root used to build Orchestrion instead of discovering a host `go` binary |
 | `orchestrion.from_source(..., go_sdk_version = "...")` | none | Exact version of `go_sdk_root`; enables bootstrap-cache lookup before SDK materialization and is verified after materialization on a miss |
@@ -114,6 +114,7 @@ Notes:
 - Guided bootstrap declares `go_sdk_root` and `go_sdk_version` from
   `--runtime-version`. Manual wiring must set both together and keep the version
   equal to the registered Go toolchain and Test Optimization `runtime_version`.
+- Orchestrion `v1.12.0` requires Go `1.25.0` or newer.
 - Bootstrap repins the local Go module to the same effective versions.
 - Bootstrap uses targeted module sync by default and does not run
   `go mod tidy` unless `--go-mod-sync=tidy` is selected.
@@ -304,8 +305,8 @@ Rule: `dd_payload_uploader(...)`
 | `filter_prefix` | bool | `False` | Only upload files matching `span_events_*.json` or `coverage_*.json` |
 | `gzip_payloads` | bool | `False` | Gzip test payloads before upload |
 | `data` | label_list | `[]` | Data files to include (for example, `context.json` for enrichment) |
-| `expected_targets` | string_list | `[]` | Optional exact local labels expected in the matching BEP. Fresh and cached results jointly satisfy coverage; only fresh outputs are inspected or uploaded |
-| `expected_targets_file` | label | unset | Optional schema-v1 exact-target file. Static and file inputs must match when both are non-empty |
+| `expected_targets` | string_list | `[]` | Optional exact local labels expected in the matching BEP. Fresh and cached results jointly satisfy coverage; missing results are reported while other fresh outputs continue to upload |
+| `expected_targets_file` | label | unset | Optional schema-v1 exact-target file. Static and file inputs must match when both are non-empty; missing results do not block other fresh uploads |
 
 ## Doctor rule attributes
 
@@ -376,8 +377,8 @@ workspace root package.
 | `sync_repo_name` | string | `"test_optimization_data"` | Repository exposing `:test_optimization_context` |
 | `doctor_name` | string | `"dd_test_optimization_doctor"` | Generated doctor target name |
 | `uploader_name` | string | `"dd_upload_payloads"` | Generated uploader target name |
-| `expected_targets` | string_list | `[]` | Strict labels passed to both doctor and uploader. List only instrumented runtime test targets that emit payloads |
-| `expected_targets_file` | label or `None` | `None` | Generated exact-target JSON file forwarded to both doctor and uploader for manifest-driven invocations |
+| `expected_targets` | string_list | `[]` | Exact labels passed to both tools. Doctor validates them strictly; uploader reports missing results while continuing with other fresh payloads |
+| `expected_targets_file` | label or `None` | `None` | Generated exact-target JSON file forwarded to both tools with the same doctor/uploader semantics for manifest-driven invocations |
 | `context_data` | label_list or `None` | `["@<sync_repo>//:test_optimization_context"]` | Explicit context data labels when the default sync repo label is not enough |
 | `doctor_kwargs` | dict or `None` | `{}` | Extra attrs for `dd_test_optimization_doctor`; cannot override `name`, `data`, `expected_targets`, or `expected_targets_file` |
 | `uploader_kwargs` | dict or `None` | `{}` | Extra attrs for `dd_payload_uploader`; cannot override `name`, `data`, `expected_targets`, or `expected_targets_file` |

docs/Initial_documentation.md

@@ -70,8 +70,8 @@ The steps are:
    as `//tools/test_optimization`.
    Usage: run `bazel test`, then the doctor target, then the uploader with
    `--dry-run --validate-enrichment`, then the real uploader target. Preserve
-   the test exit code, but do not run the real upload if doctor or dry-run
-   enrichment validation fails.
+   the earliest failure, but still let the real uploader process every available
+   fresh valid payload when upload is enabled.
 
 4. **Language macros (optional)**:
    Thin wrappers (for Go/Python/Java/NodeJS/.NET/Ruby) set up the right runfiles/env so test code can read the synced files and write payloads to `TEST_UNDECLARED_OUTPUTS_DIR`.

docs/Installation_Reference.md

@@ -126,8 +126,8 @@ RTO_ARCHIVE_TYPE="tar.gz"
 RULES_GO_UPSTREAM="v0_60_0"
 RULES_GO_VARIANT="base"
 RULES_GO_STRIP_PREFIX="third_party/rgo/v0_60_0/base"
-DD_TRACE_GO_VERSION="v2.9.0"
-ORCHESTRION_VERSION="v1.9.0"
+DD_TRACE_GO_VERSION="v2.9.1"
+ORCHESTRION_VERSION="v1.12.0"
 ```
 
 The archive URL, SHA256, and prefix are tied to the repository commit.
@@ -237,7 +237,7 @@ bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
   --guided \
   --service go-service \
   --runtime-version 1.25.0 \
-  --dd-trace-go-version v2.9.0 \
+  --dd-trace-go-version v2.9.1 \
   --write-bazelrc
 ```
 
@@ -248,13 +248,13 @@ bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
   --guided \
   --service go-service \
   --runtime-version 1.25.0 \
-  --dd-trace-go-version v2.9.0 \
+  --dd-trace-go-version v2.9.1 \
   --go-module-dir path/to/go-module \
   --write-bazelrc
 ```
 
 `--dd-trace-go-version` is optional. If omitted, the workspace uses the default
-`v2.9.0`. It accepts a tag, pseudo-version,
+`v2.9.1`. It accepts a tag, pseudo-version,
 branch, or commit SHA. Bootstrap resolves that input to exact tracer versions,
 keeps the local Go module pins on those same versions, and prevents Bazel and
 the Go module from silently drifting apart.
@@ -441,7 +441,7 @@ tools/test_optimization/run_test_optimization_ci.sh \
   --support-bundle .topt/reports/dd-test-optimization-support.zip \
   //...
 
-# Add --upload only when the real upload should run after doctor and dry-run pass.
+# Add --upload to send every available fresh valid payload after validation attempts.
 DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" \
   tools/test_optimization/run_test_optimization_ci.sh \
     --report-dir .topt/reports \
@@ -458,7 +458,7 @@ DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" \
   -SupportBundle .topt\reports\dd-test-optimization-support.zip `
   //...
 
-# Add -Upload only when the real upload should run after doctor and dry-run pass.
+# Add -Upload to send every available fresh valid payload after validation attempts.
 $env:DD_API_KEY = "<your-api-key>"
 $env:DD_SITE = "datadoghq.com"
 .\tools\test_optimization\run_test_optimization_ci.ps1 `
@@ -521,7 +521,9 @@ python3 tools/test_optimization/render_report_summary.py \
   --output .topt/reports/upload-diagnostics.md
 ```
 
-Do not run the real uploader if the doctor or dry-run enrichment step fails.
+When upload is enabled, run the real uploader after the doctor and dry-run
+attempts even if either validation failed. It processes every available fresh
+valid payload while the wrapper preserves the earlier failure as the job result.
 
 For manual Go extension wiring, set `module_path` to the Go module path from
 `go.mod`: