Skip to content

Parallelize test execution to reduce runtime #2511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Copilot wants to merge 20 commits into
base: main
Choose a base branch
from
+19 −14
Draft

Parallelize test execution to reduce runtime #2511

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
f489cec
Initial plan
Copilot Nov 22, 2025
0ee5f9f
Implement parallel test execution and fix port conflicts
Copilot Nov 22, 2025
f1540f3
Fix port reference in test assertion and stderr handling
Copilot Nov 22, 2025
a7ead23
Remove unnecessary blank line for cleaner formatting
Copilot Nov 22, 2025
e1af25a
Simplify parallel test execution without arrays per feedback
Copilot Nov 22, 2025
12b7d4a
Fix wait command to work with background jobs
Copilot Nov 22, 2025
bf3ad48
Use process substitution to handle filenames with spaces
Copilot Nov 22, 2025
d9c28a5
Fix syntax in run_tests.sh for better readability
plengauer Nov 23, 2025
08e4aca
Update run_tests.sh
plengauer Nov 23, 2025
7a8eea9
Refactor test execution loop in run_tests.sh
plengauer Nov 23, 2025
0cca292
Rename test_auto_performance_script.sh to test_performance_script.sh
plengauer Nov 23, 2025
8f285c9
Rename test_auto_performance_full.sh to test_performance_full.sh
plengauer Nov 23, 2025
bee264b
Add performance directory to test runner
plengauer Nov 23, 2025
51a88fe
Update run_tests.sh
plengauer Nov 23, 2025
34c7538
Update run_tests.sh
plengauer Nov 25, 2025
a004ce7
Fix port conflict: change test_auto_injection_netcat.sh to use port 1…
Copilot Nov 27, 2025
7b7c7a4
Merge branch 'main' into copilot/speed-up-test-parallel-processing
plengauer Nov 28, 2025
df1b118
Fix port conflict: change test_auto_injection_node.sh to use port 8081
Copilot Nov 28, 2025
4bf343b
Fix race condition in test_auto_netcat.sh TEST 0 by adding sleep befo…
Copilot Nov 29, 2025
76500d8
Add perl output buffering to prevent interleaved test output
Copilot Nov 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions tests/auto/test_auto_injection_netcat.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ set -e
. otel.sh

echo TEST 1 >&2
OTEL_SHELL_AUTO_INJECTED=FALSE \otel4netcat ncat -l -p 12345 -c cat &
OTEL_SHELL_AUTO_INJECTED=FALSE \otel4netcat ncat -l -p 12355 -c cat &
pid="$!"
sleep 3
assert_equals "hello world" "$(echo -n hello world | ncat -i 10 --no-shutdown 127.0.0.1 12345)"
assert_equals "hello world" "$(echo -n hello world | ncat -i 10 --no-shutdown 127.0.0.1 12355)"
wait "$pid"

span="$(resolve_span '.name | endswith("cat")')"
Expand All @@ -22,18 +22,18 @@ assert_equals "SpanKind.CONSUMER" $(\echo "$span" | jq -r '.kind')
# assert_equals "ncat -l -p 12345 -c cat" $(\echo "$span" | jq -r '.name')
# assert_equals "SpanKind.INTERNAL" $(\echo "$span" | jq -r '.kind')

span="$(resolve_span '.name == "ncat -i 10 --no-shutdown 127.0.0.1 12345"')"
span="$(resolve_span '.name == "ncat -i 10 --no-shutdown 127.0.0.1 12355"')"
assert_equals "SpanKind.INTERNAL" $(\echo "$span" | jq -r '.kind')
span_id=$(\echo "$span" | jq -r '.context.span_id')
span="$(resolve_span '.parent_id == "'$span_id'"')"
assert_equals "send/receive" $(\echo "$span" | jq -r '.name')
assert_equals "SpanKind.PRODUCER" $(\echo "$span" | jq -r '.kind')

echo TEST 2 >&2
OTEL_SHELL_AUTO_INJECTED=FALSE \otel4netcat_http ncat -l -p 12345 -c 'printf "HTTP/1.1 200 OK\r\n\r\nhello world"' &
OTEL_SHELL_AUTO_INJECTED=FALSE \otel4netcat_http ncat -l -p 12355 -c 'printf "HTTP/1.1 200 OK\r\n\r\nhello world"' &
pid="$!"
sleep 3
assert_equals "hello world" "$(curl http://127.0.0.1:12345/)"
assert_equals "hello world" "$(curl http://127.0.0.1:12355/)"
wait "$pid"

span="$(resolve_span '.name == "GET") | select(.kind == "SpanKind.SERVER"')"
Expand All @@ -43,4 +43,4 @@ assert_equals "GET" "$(\printf '%s' "$span" | jq -r '.name')"
assert_equals "SpanKind.CLIENT" "$(\printf '%s' "$span" | jq -r '.kind')"
span_id=$(\printf '%s' "$span" | jq -r '.parent_id')
span="$(resolve_span '.context.span_id == "'"$span_id"'"')"
assert_equals "curl http://127.0.0.1:12345/" "$(\printf '%s' "$span" | jq -r '.name')"
assert_equals "curl http://127.0.0.1:12355/" "$(\printf '%s' "$span" | jq -r '.name')"
0 tests/auto/test_auto_performance_full.sh → tests/performance/test_performance_full.sh
View file
Open in desktop
File renamed without changes.
0 tests/auto/test_auto_performance_script.sh → tests/performance/test_performance_script.sh
View file
Open in desktop
File renamed without changes.
14 changes: 9 additions & 5 deletions tests/run_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ else
export TEST_SHELL="$SHELL"
fi

for dir in unit sdk auto integration; do
{ find $dir -iname 'test_*.sh'; find $dir -iname 'test_*.'"$SHELL"; } | sort -u | while read -r file; do
rm /tmp/opentelemetry_shell_*_instrumentation_cache_*.aliases 2> /dev/null || true
failed_flag="$(mktemp -u)"

for dir in unit sdk auto integration performance; do
while IFS= read -r file; do
export OTEL_EXPORT_LOCATION="$(mktemp -u)".sdk.out
export OTEL_SHELL_SDK_STDOUT_REDIRECT="$(mktemp -u -p "$(mktemp -d)")".pipe
export OTEL_TRACES_EXPORTER=console
Expand All @@ -36,7 +37,10 @@ for dir in unit sdk auto integration; do
touch "$stdout" "$stderr"
chmod 0666 "$stdout" "$stderr"
export OTEL_SHELL_SDK_STDERR_REDIRECT="$stderr"
timeout $((60 * 60 * 3)) $TEST_SHELL $options "$file" 1> "$stdout" && echo "$file SUCCEEDED" || (echo "$file FAILED" && echo "stdout:" && cat "$stdout" && echo "stderr:" && cat "$stderr" && echo "otlp:" && cat "$OTEL_EXPORT_LOCATION" && exit 1)
done
( timeout $((60 * 60 * 3)) $TEST_SHELL $options "$file" 1> "$stdout" 2> "$stderr" && echo "$file SUCCEEDED" || (echo "$file FAILED" && echo "stdout:" && cat "$stdout" && echo "stderr:" && cat "$stderr" && echo "otlp:" && cat "$OTEL_EXPORT_LOCATION" && touch "$failed_flag" && exit 1) ) &
[ "$dir" != performance ] || wait
done < <({ find $dir -iname 'test_*.sh'; find $dir -iname 'test_*.'"$SHELL"; } | sort -u)
wait
if [ -r "$failed_flag" ]; then exit 1; fi
done
echo "ALL TESTS SUCCESSFUL"
Loading