Reproduce flaky: shutdown 'executes only once'#5584
Draft
Conversation
Hypothesis: the test fails when the HTTP round-trip to the agent takes longer than DEFAULT_SHUTDOWN_TIMEOUT (1 second). The worker thread is mid-HTTP-call when shutdown! calls join(1), the join times out, and traces_flushed is still 0 when the assertion runs. This commit forces the race condition by stubbing the transport's send_traces to add a 2-second sleep, exceeding the 1-second shutdown timeout. The test should fail deterministically in CI. Co-Authored-By: Claude <noreply@anthropic.com>
This was referenced Apr 11, 2026
Contributor
|
✨ Fix all issues with BitsAI or with Cursor
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Reproducer for flaky test
spec/datadog/tracing/integration_spec.rb:592— the shutdown "executes only once" integration test. Forces the suspected race condition to validate the hypothesis in CI. Expected result: the test fails deterministically.Do not merge — this PR exists for CI validation only.
Motivation:
Flaky test reported in PR #5581. The test intermittently fails with
traces_flushed: 0under CI load.Hypothesis: The test fails when the HTTP round-trip to the agent takes longer than
DEFAULT_SHUTDOWN_TIMEOUT(1 second). The worker thread is mid-HTTP-call whenshutdown!callsjoin(1), the join times out, andtraces_flushedis still 0 when the assertion runs.Reproducer technique: Stubs
tracer.writer.transport.send_traceswithand_wrap_originalto add a 2-second sleep before the real HTTP call. This exceeds the 1-second shutdown timeout, forcing the join to time out deterministically.Change log entry
None.
How to test the change?
CI should show the "executes only once" / "flushed trace" test failing deterministically with
traces_flushed: 0. If it doesn't, the hypothesis is wrong.