uploader: add opt-in parallel Python runtime - #233
Conversation
Introduce a cross-platform Python uploader with bounded file workers, shared CODEOWNERS preparation, preventive 4.5 MiB splitting, retries, dry-run, debug diagnostics, and final statistics. Keep the legacy runtimes as the default during rollout and document the remaining cross-platform and consumer validation gates.
![datadog-datadog-prod-us1-2[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
The opt-in Python uploader checks credentials before it can return a no-op result. It also treats false CI markers as true and writes an empty staging path in schema-v1 reports.
🤖 Datadog Autotest · Commit 44cca3b · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7f90062726
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fbf426cd40
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0f591b5807
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5a5bba148d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 33cbe29432
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 73fb10c0ff
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3dc8478f72
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 85ddbe264c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
🔄 Datadog auto-retried 11 jobs - 4 passed on retry 🔗 Commit SHA: 464306b | Docs | View more details | Give us feedback! |
Preserve cross-runtime lock and CODEOWNERS lookup parity, validate endpoint ports, and reject non-standard JSON before upload. Expand split, retry, replay, backpressure, launcher, and dry-run coverage and align the migration tracker and troubleshooting docs.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9c9cb5afd2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Reduce orchestration and transport duplication, clarify module boundaries and lifecycle reporting, and centralize uploader test runfile setup. Keep the existing uploader behavior while making the implementation and migration tracker easier to review and maintain. Validated with the 449-test Python tooling suite, all 199 //tools tests, shellcheck, PowerShell parsing, buildifier, compileall, and git diff checks.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d5e2a974a1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Use domain-specific names for telemetry, result, resource, and legacy-report state. Remove a redundant report wrapper and unused telemetry property while preserving configuration precedence and output contracts. Validated with the 449-test Python tooling suite, all 199 //tools tests, template lint, compileall, and git diff checks.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: de4b02e6c5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Distinguish HTTP outcomes from file results, give resource and telemetry state domain-specific names, and replace string-based report aggregation with explicit fields. Remove stale rule commentary while preserving runtime and report contracts. Validated with the 449-test Python suite, all 199 //tools tests, compileall, template lint, buildifier, and git diff checks.
Share payload-type counting across discovery filters and replace terse enrichment, splitting, and runfiles names with domain-specific ones. Preserve unsupported-type handling while removing duplicated aggregation code. Validated with the 449-test Python suite, all 199 //tools tests, compileall, template lint, and git diff checks.
Pass PreparedHttpRequest through the transport boundary intact and distinguish prepared requests, body streams, HTTP requests, response excerpts, and transport failures. Reuse canonical payload counters for empty discovery. Validated with the 449-test Python suite, all 199 //tools tests, compileall, template lint, and git diff checks.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 89b9b1e6f2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2ff6e34be9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9ba8869e62
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 464306b4a3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| sidecar = task.test_outputs_dir / "bazel_target_metadata.json" | ||
| if not sidecar.is_file(): | ||
| return None, None | ||
| payload, failure = _read_json_object(sidecar, "Bazel metadata") |
There was a problem hiding this comment.
When a Unix test emits bazel_target_metadata.json as a symlink to a JSON file outside test.outputs, is_file() and the subsequent read both follow it. The uploader then merges every key/value from that host-side file into outbound event metadata, allowing sandboxed test code to exfiltrate JSON that it could not read during the test; require a non-symlink regular file whose resolved path remains beneath task.test_outputs_dir.
AGENTS.md reference: AGENTS.md:L163-L164
Useful? React with 👍 / 👎.
| not freshness_preparation.scan_roots | ||
| and config.fail_on_error | ||
| and not all_expected_outputs_cached | ||
| ): |
There was a problem hiding this comment.
With fail_on_error=True, optional BEP freshness, and an existing but empty bazel-testlogs directory, an expected remote-only output passes this guard because a scan root exists; it is not added to eligible_outputs, and the later tests_executed() marker check is false, so the invocation exits 0 despite the documented requirement that missing fresh payloads fail. The fresh evidence beyond the resolved cached-coverage thread is this existing-root path, which still reproduces with one cached expected target and one remote-only expected target; fail the plan based on the uncovered remote-only expected output rather than only on whether the root exists.
Useful? React with 👍 / 👎.
1 participant
Summary
use_python_uploaderrollout switchRollout
This is an implementation milestone, not the final migration switch. The legacy uploader remains the default. Linux, macOS, Windows, consumer-fixture, documentation, default-switch, and legacy-removal gates remain tracked in
docs/python_parallel_uploader_migration_plan.md.Validation
python3 -m unittest discover -s tools/tests/python -p test*_tools.py: 426 tests, 425 passed, 1 platform-specific skip./bazelw test //tools/tests/python:python_tools_test --noexperimental_split_xml_generation --nocache_test_results: passedbazel run: passed with final statisticsgit diff --check: passed