Skip to content

Move system-tests to GHA #8692

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 15 commits into from
Apr 22, 2025
Merged

Move system-tests to GHA #8692

merged 15 commits into from
Apr 22, 2025

Conversation

cbeauchesne
Copy link
Contributor

@cbeauchesne cbeauchesne commented Apr 11, 2025 •
edited
Loading

What Does This Do

Waiting for a proper runner in gitlab, let's move system-tests in GHA

Needs DataDog/system-tests#4538

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Apr 11, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master cbeauchesne/system-tests-gha
git_commit_date 1745246084 1745333847
git_commit_sha 7dc621f 49ed7ed
release_version 1.49.0-SNAPSHOT~7dc621f3bb 1.49.0-SNAPSHOT~49ed7ed892
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745337124 1745337124
ci_job_id 906935952 906935952
ci_pipeline_id 62922738 62922738
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ekfrqbyz-project-304-concurrent-0-mhxrk5lb 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ekfrqbyz-project-304-concurrent-0-mhxrk5lb 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 65 metrics, 6 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~49ed7ed892, baseline=1.49.0-SNAPSHOT~7dc621f3bb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.068 s) : 0, 1068201
Total [baseline] (8.715 s) : 0, 8715222
Agent [candidate] (1.066 s) : 0, 1065546
Total [candidate] (8.703 s) : 0, 8702596
section iast
Agent [baseline] (1.195 s) : 0, 1195173
Total [baseline] (9.266 s) : 0, 9265628
Agent [candidate] (1.187 s) : 0, 1186997
Total [candidate] (9.247 s) : 0, 9247199
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.183 s) : 0, 1182691
Total [baseline] (9.232 s) : 0, 9232145
Agent [candidate] (1.184 s) : 0, 1184428
Total [candidate] (9.266 s) : 0, 9266231
section iast_TELEMETRY_OFF
Agent [baseline] (1.182 s) : 0, 1182360
Total [baseline] (9.275 s) : 0, 9274983
Agent [candidate] (1.184 s) : 0, 1184177
Total [candidate] (9.23 s) : 0, 9230277
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.068 s -
Agent iast 1.195 s 126.972 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.183 s 114.49 ms (10.7%)
Agent iast_TELEMETRY_OFF 1.182 s 114.159 ms (10.7%)
Total tracing 8.715 s -
Total iast 9.266 s 550.406 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.232 s 516.923 ms (5.9%)
Total iast_TELEMETRY_OFF 9.275 s 559.761 ms (6.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent iast 1.187 s 121.451 ms (11.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.184 s 118.883 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.184 s 118.631 ms (11.1%)
Total tracing 8.703 s -
Total iast 9.247 s 544.603 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.266 s 563.635 ms (6.5%)
Total iast_TELEMETRY_OFF 9.23 s 527.681 ms (6.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~49ed7ed892, baseline=1.49.0-SNAPSHOT~7dc621f3bb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (728.858 ms) : 0, 728858
BytebuddyAgent [candidate] (726.568 ms) : 0, 726568
GlobalTracer [baseline] (241.35 ms) : 0, 241350
GlobalTracer [candidate] (241.499 ms) : 0, 241499
AppSec [baseline] (55.71 ms) : 0, 55710
AppSec [candidate] (55.795 ms) : 0, 55795
Debugger [baseline] (5.15 ms) : 0, 5150
Debugger [candidate] (5.159 ms) : 0, 5159
Remote Config [baseline] (4.352 ms) : 0, 4352
Remote Config [candidate] (2.131 ms) : 0, 2131
Telemetry [baseline] (9.331 ms) : 0, 9331
Telemetry [candidate] (10.881 ms) : 0, 10881
section iast
BytebuddyAgent [baseline] (847.644 ms) : 0, 847644
BytebuddyAgent [candidate] (842.09 ms) : 0, 842090
GlobalTracer [baseline] (231.592 ms) : 0, 231592
GlobalTracer [candidate] (229.756 ms) : 0, 229756
IAST [baseline] (23.961 ms) : 0, 23961
IAST [candidate] (23.532 ms) : 0, 23532
AppSec [baseline] (54.981 ms) : 0, 54981
AppSec [candidate] (54.844 ms) : 0, 54844
Debugger [baseline] (4.212 ms) : 0, 4212
Debugger [candidate] (4.21 ms) : 0, 4210
Remote Config [baseline] (617.378 µs) : 0, 617
Remote Config [candidate] (607.45 µs) : 0, 607
Telemetry [baseline] (8.638 ms) : 0, 8638
Telemetry [candidate] (8.608 ms) : 0, 8608
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (838.034 ms) : 0, 838034
BytebuddyAgent [candidate] (839.509 ms) : 0, 839509
GlobalTracer [baseline] (229.044 ms) : 0, 229044
GlobalTracer [candidate] (229.235 ms) : 0, 229235
IAST [baseline] (23.644 ms) : 0, 23644
IAST [candidate] (22.732 ms) : 0, 22732
AppSec [baseline] (55.114 ms) : 0, 55114
AppSec [candidate] (56.063 ms) : 0, 56063
Debugger [baseline] (4.247 ms) : 0, 4247
Debugger [candidate] (4.231 ms) : 0, 4231
Remote Config [baseline] (605.03 µs) : 0, 605
Remote Config [candidate] (612.418 µs) : 0, 612
Telemetry [baseline] (8.658 ms) : 0, 8658
Telemetry [candidate] (8.667 ms) : 0, 8667
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (836.871 ms) : 0, 836871
BytebuddyAgent [candidate] (840.358 ms) : 0, 840358
GlobalTracer [baseline] (230.28 ms) : 0, 230280
GlobalTracer [candidate] (228.839 ms) : 0, 228839
IAST [baseline] (22.358 ms) : 0, 22358
IAST [candidate] (22.421 ms) : 0, 22421
AppSec [baseline] (56.071 ms) : 0, 56071
AppSec [candidate] (55.65 ms) : 0, 55650
Debugger [baseline] (4.153 ms) : 0, 4153
Debugger [candidate] (4.182 ms) : 0, 4182
Remote Config [baseline] (623.649 µs) : 0, 624
Remote Config [candidate] (633.698 µs) : 0, 634
Telemetry [baseline] (8.672 ms) : 0, 8672
Telemetry [candidate] (8.644 ms) : 0, 8644
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~49ed7ed892, baseline=1.49.0-SNAPSHOT~7dc621f3bb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057422
Total [baseline] (10.455 s) : 0, 10454723
Agent [candidate] (1.065 s) : 0, 1065454
Total [candidate] (10.45 s) : 0, 10450139
section appsec
Agent [baseline] (1.197 s) : 0, 1196722
Total [baseline] (10.796 s) : 0, 10795971
Agent [candidate] (1.201 s) : 0, 1200507
Total [candidate] (10.789 s) : 0, 10788778
section iast
Agent [baseline] (1.185 s) : 0, 1184985
Total [baseline] (11.052 s) : 0, 11052309
Agent [candidate] (1.188 s) : 0, 1187793
Total [candidate] (11.085 s) : 0, 11084777
section profiling
Agent [baseline] (1.311 s) : 0, 1311091
Total [baseline] (11.022 s) : 0, 11021697
Agent [candidate] (1.309 s) : 0, 1308961
Total [candidate] (10.988 s) : 0, 10988186
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.197 s 139.3 ms (13.2%)
Agent iast 1.185 s 127.562 ms (12.1%)
Agent profiling 1.311 s 253.669 ms (24.0%)
Total tracing 10.455 s -
Total appsec 10.796 s 341.248 ms (3.3%)
Total iast 11.052 s 597.586 ms (5.7%)
Total profiling 11.022 s 566.974 ms (5.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent appsec 1.201 s 135.052 ms (12.7%)
Agent iast 1.188 s 122.338 ms (11.5%)
Agent profiling 1.309 s 243.507 ms (22.9%)
Total tracing 10.45 s -
Total appsec 10.789 s 338.639 ms (3.2%)
Total iast 11.085 s 634.638 ms (6.1%)
Total profiling 10.988 s 538.047 ms (5.1%) 
gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~49ed7ed892, baseline=1.49.0-SNAPSHOT~7dc621f3bb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (721.573 ms) : 0, 721573
BytebuddyAgent [candidate] (727.599 ms) : 0, 727599
GlobalTracer [baseline] (239.433 ms) : 0, 239433
GlobalTracer [candidate] (240.883 ms) : 0, 240883
AppSec [baseline] (55.822 ms) : 0, 55822
AppSec [candidate] (54.767 ms) : 0, 54767
Debugger [baseline] (4.363 ms) : 0, 4363
Debugger [candidate] (4.46 ms) : 0, 4460
Remote Config [baseline] (697.119 µs) : 0, 697
Remote Config [candidate] (711.302 µs) : 0, 711
Telemetry [baseline] (12.123 ms) : 0, 12123
Telemetry [candidate] (13.439 ms) : 0, 13439
section appsec
BytebuddyAgent [baseline] (738.335 ms) : 0, 738335
BytebuddyAgent [candidate] (740.326 ms) : 0, 740326
GlobalTracer [baseline] (235.479 ms) : 0, 235479
GlobalTracer [candidate] (236.429 ms) : 0, 236429
AppSec [baseline] (175.643 ms) : 0, 175643
AppSec [candidate] (176.344 ms) : 0, 176344
Debugger [baseline] (4.199 ms) : 0, 4199
Debugger [candidate] (4.228 ms) : 0, 4228
Remote Config [baseline] (634.218 µs) : 0, 634
Remote Config [candidate] (641.455 µs) : 0, 641
Telemetry [baseline] (8.173 ms) : 0, 8173
Telemetry [candidate] (8.254 ms) : 0, 8254
IAST [baseline] (21.898 ms) : 0, 21898
IAST [candidate] (21.848 ms) : 0, 21848
section iast
BytebuddyAgent [baseline] (838.839 ms) : 0, 838839
BytebuddyAgent [candidate] (841.361 ms) : 0, 841361
GlobalTracer [baseline] (229.939 ms) : 0, 229939
GlobalTracer [candidate] (230.572 ms) : 0, 230572
AppSec [baseline] (56.226 ms) : 0, 56226
AppSec [candidate] (55.985 ms) : 0, 55985
Debugger [baseline] (4.24 ms) : 0, 4240
Debugger [candidate] (4.226 ms) : 0, 4226
Remote Config [baseline] (614.775 µs) : 0, 615
Remote Config [candidate] (599.955 µs) : 0, 600
Telemetry [baseline] (8.715 ms) : 0, 8715
Telemetry [candidate] (8.758 ms) : 0, 8758
IAST [baseline] (22.936 ms) : 0, 22936
IAST [candidate] (22.848 ms) : 0, 22848
section profiling
BytebuddyAgent [baseline] (716.432 ms) : 0, 716432
BytebuddyAgent [candidate] (713.516 ms) : 0, 713516
GlobalTracer [baseline] (374.733 ms) : 0, 374733
GlobalTracer [candidate] (376.131 ms) : 0, 376131
AppSec [baseline] (54.102 ms) : 0, 54102
AppSec [candidate] (53.125 ms) : 0, 53125
Debugger [baseline] (4.368 ms) : 0, 4368
Debugger [candidate] (4.396 ms) : 0, 4396
Remote Config [baseline] (669.654 µs) : 0, 670
Remote Config [candidate] (662.231 µs) : 0, 662
Telemetry [baseline] (8.88 ms) : 0, 8880
Telemetry [candidate] (9.004 ms) : 0, 9004
ProfilingAgent [baseline] (101.399 ms) : 0, 101399
ProfilingAgent [candidate] (101.957 ms) : 0, 101957
Profiling [baseline] (101.424 ms) : 0, 101424
Profiling [candidate] (101.984 ms) : 0, 101984
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-22T15:20:12 2025-04-22T15:27:53
git_branch master cbeauchesne/system-tests-gha
git_commit_date 1745246084 1745333847
git_commit_sha 7dc621f 49ed7ed
release_version 1.49.0-SNAPSHOT~7dc621f3bb 1.49.0-SNAPSHOT~49ed7ed892
start_time 2025-04-22T15:19:58 2025-04-22T15:27:39
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745336069 1745336069
ci_job_id 906935953 906935953
ci_pipeline_id 62922738 62922738
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ekfrqbyz-project-304-concurrent-1-jqxgmrwj 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ekfrqbyz-project-304-concurrent-1-jqxgmrwj 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~49ed7ed892, baseline=1.49.0-SNAPSHOT~7dc621f3bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (375.544 µs) : 355, 396
.   : milestone, 376,
iast (509.184 µs) : 487, 531
.   : milestone, 509,
iast_FULL (726.365 µs) : 704, 748
.   : milestone, 726,
iast_GLOBAL (559.353 µs) : 536, 583
.   : milestone, 559,
iast_HARDCODED_SECRET_DISABLED (510.67 µs) : 489, 532
.   : milestone, 511,
iast_INACTIVE (457.281 µs) : 436, 479
.   : milestone, 457,
iast_TELEMETRY_OFF (500.087 µs) : 478, 522
.   : milestone, 500,
tracing (453.896 µs) : 433, 475
.   : milestone, 454,
section candidate
no_agent (377.151 µs) : 355, 399
.   : milestone, 377,
iast (516.041 µs) : 494, 538
.   : milestone, 516,
iast_FULL (727.556 µs) : 706, 749
.   : milestone, 728,
iast_GLOBAL (554.743 µs) : 533, 576
.   : milestone, 555,
iast_HARDCODED_SECRET_DISABLED (513.846 µs) : 492, 536
.   : milestone, 514,
iast_INACTIVE (456.913 µs) : 436, 478
.   : milestone, 457,
iast_TELEMETRY_OFF (498.388 µs) : 476, 520
.   : milestone, 498,
tracing (454.665 µs) : 434, 476
.   : milestone, 455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.544 µs [354.827 µs, 396.261 µs] -
iast 509.184 µs [487.314 µs, 531.054 µs] 133.64 µs (35.6%)
iast_FULL 726.365 µs [704.289 µs, 748.44 µs] 350.821 µs (93.4%)
iast_GLOBAL 559.353 µs [536.152 µs, 582.554 µs] 183.809 µs (48.9%)
iast_HARDCODED_SECRET_DISABLED 510.67 µs [489.118 µs, 532.222 µs] 135.126 µs (36.0%)
iast_INACTIVE 457.281 µs [435.957 µs, 478.604 µs] 81.737 µs (21.8%)
iast_TELEMETRY_OFF 500.087 µs [478.346 µs, 521.827 µs] 124.543 µs (33.2%)
tracing 453.896 µs [433.156 µs, 474.635 µs] 78.352 µs (20.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.151 µs [354.873 µs, 399.428 µs] -
iast 516.041 µs [493.842 µs, 538.239 µs] 138.89 µs (36.8%)
iast_FULL 727.556 µs [705.713 µs, 749.399 µs] 350.405 µs (92.9%)
iast_GLOBAL 554.743 µs [533.262 µs, 576.223 µs] 177.592 µs (47.1%)
iast_HARDCODED_SECRET_DISABLED 513.846 µs [491.753 µs, 535.939 µs] 136.695 µs (36.2%)
iast_INACTIVE 456.913 µs [435.922 µs, 477.904 µs] 79.762 µs (21.1%)
iast_TELEMETRY_OFF 498.388 µs [476.291 µs, 520.484 µs] 121.237 µs (32.1%)
tracing 454.665 µs [433.517 µs, 475.813 µs] 77.514 µs (20.6%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~49ed7ed892, baseline=1.49.0-SNAPSHOT~7dc621f3bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.356 ms) : 1337, 1376
.   : milestone, 1356,
appsec (1.742 ms) : 1719, 1766
.   : milestone, 1742,
appsec_no_iast (1.739 ms) : 1717, 1762
.   : milestone, 1739,
code_origins (1.66 ms) : 1632, 1687
.   : milestone, 1660,
iast (1.504 ms) : 1479, 1528
.   : milestone, 1504,
profiling (1.492 ms) : 1468, 1515
.   : milestone, 1492,
tracing (1.499 ms) : 1475, 1523
.   : milestone, 1499,
section candidate
no_agent (1.348 ms) : 1329, 1367
.   : milestone, 1348,
appsec (1.719 ms) : 1696, 1741
.   : milestone, 1719,
appsec_no_iast (1.73 ms) : 1705, 1755
.   : milestone, 1730,
code_origins (1.698 ms) : 1671, 1724
.   : milestone, 1698,
iast (1.514 ms) : 1491, 1538
.   : milestone, 1514,
profiling (1.508 ms) : 1485, 1531
.   : milestone, 1508,
tracing (1.505 ms) : 1481, 1529
.   : milestone, 1505,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.356 ms [1.337 ms, 1.376 ms] -
appsec 1.742 ms [1.719 ms, 1.766 ms] 385.724 µs (28.4%)
appsec_no_iast 1.739 ms [1.717 ms, 1.762 ms] 383.112 µs (28.2%)
code_origins 1.66 ms [1.632 ms, 1.687 ms] 303.434 µs (22.4%)
iast 1.504 ms [1.479 ms, 1.528 ms] 147.334 µs (10.9%)
profiling 1.492 ms [1.468 ms, 1.515 ms] 135.368 µs (10.0%)
tracing 1.499 ms [1.475 ms, 1.523 ms] 142.589 µs (10.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.329 ms, 1.367 ms] -
appsec 1.719 ms [1.696 ms, 1.741 ms] 370.796 µs (27.5%)
appsec_no_iast 1.73 ms [1.705 ms, 1.755 ms] 381.978 µs (28.3%)
code_origins 1.698 ms [1.671 ms, 1.724 ms] 349.746 µs (26.0%)
iast 1.514 ms [1.491 ms, 1.538 ms] 166.641 µs (12.4%)
profiling 1.508 ms [1.485 ms, 1.531 ms] 159.929 µs (11.9%)
tracing 1.505 ms [1.481 ms, 1.529 ms] 157.09 µs (11.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master cbeauchesne/system-tests-gha
git_commit_date 1745246084 1745333847
git_commit_sha 7dc621f 49ed7ed
release_version 1.49.0-SNAPSHOT~7dc621f3bb 1.49.0-SNAPSHOT~49ed7ed892
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1745336594 1745336594
ci_job_id 906935955 906935955
ci_pipeline_id 62922738 62922738
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-tksdm6ze-project-304-concurrent-0-1qauchta 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-tksdm6ze-project-304-concurrent-0-1qauchta 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~49ed7ed892, baseline=1.49.0-SNAPSHOT~7dc621f3bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.351 ms) : 2307, 2395
.   : milestone, 2351,
iast (2.14 ms) : 2084, 2196
.   : milestone, 2140,
iast_GLOBAL (2.175 ms) : 2118, 2231
.   : milestone, 2175,
profiling (1.982 ms) : 1938, 2026
.   : milestone, 1982,
tracing (1.963 ms) : 1921, 2006
.   : milestone, 1963,
section candidate
no_agent (1.483 ms) : 1472, 1495
.   : milestone, 1483,
appsec (2.36 ms) : 2317, 2404
.   : milestone, 2360,
iast (2.139 ms) : 2083, 2195
.   : milestone, 2139,
iast_GLOBAL (2.177 ms) : 2122, 2233
.   : milestone, 2177,
profiling (1.975 ms) : 1931, 2019
.   : milestone, 1975,
tracing (1.967 ms) : 1924, 2009
.   : milestone, 1967,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 2.351 ms [2.307 ms, 2.395 ms] 875.399 µs (59.3%)
iast 2.14 ms [2.084 ms, 2.196 ms] 664.399 µs (45.0%)
iast_GLOBAL 2.175 ms [2.118 ms, 2.231 ms] 698.771 µs (47.3%)
profiling 1.982 ms [1.938 ms, 2.026 ms] 506.162 µs (34.3%)
tracing 1.963 ms [1.921 ms, 2.006 ms] 487.693 µs (33.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.483 ms [1.472 ms, 1.495 ms] -
appsec 2.36 ms [2.317 ms, 2.404 ms] 877.176 µs (59.1%)
iast 2.139 ms [2.083 ms, 2.195 ms] 655.73 µs (44.2%)
iast_GLOBAL 2.177 ms [2.122 ms, 2.233 ms] 694.328 µs (46.8%)
profiling 1.975 ms [1.931 ms, 2.019 ms] 491.901 µs (33.2%)
tracing 1.967 ms [1.924 ms, 2.009 ms] 483.716 µs (32.6%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~49ed7ed892, baseline=1.49.0-SNAPSHOT~7dc621f3bb
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.886 s) : 14886000, 14886000
.   : milestone, 14886000,
appsec (15.029 s) : 15029000, 15029000
.   : milestone, 15029000,
iast (18.73 s) : 18730000, 18730000
.   : milestone, 18730000,
iast_GLOBAL (17.993 s) : 17993000, 17993000
.   : milestone, 17993000,
profiling (15.071 s) : 15071000, 15071000
.   : milestone, 15071000,
tracing (15.024 s) : 15024000, 15024000
.   : milestone, 15024000,
section candidate
no_agent (14.984 s) : 14984000, 14984000
.   : milestone, 14984000,
appsec (14.939 s) : 14939000, 14939000
.   : milestone, 14939000,
iast (19.174 s) : 19174000, 19174000
.   : milestone, 19174000,
iast_GLOBAL (17.99 s) : 17990000, 17990000
.   : milestone, 17990000,
profiling (15.094 s) : 15094000, 15094000
.   : milestone, 15094000,
tracing (14.743 s) : 14743000, 14743000
.   : milestone, 14743000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.886 s [14.886 s, 14.886 s] -
appsec 15.029 s [15.029 s, 15.029 s] 143.0 ms (1.0%)
iast 18.73 s [18.73 s, 18.73 s] 3.844 s (25.8%)
iast_GLOBAL 17.993 s [17.993 s, 17.993 s] 3.107 s (20.9%)
profiling 15.071 s [15.071 s, 15.071 s] 185.0 ms (1.2%)
tracing 15.024 s [15.024 s, 15.024 s] 138.0 ms (0.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.984 s [14.984 s, 14.984 s] -
appsec 14.939 s [14.939 s, 14.939 s] -45.0 ms (-0.3%)
iast 19.174 s [19.174 s, 19.174 s] 4.19 s (28.0%)
iast_GLOBAL 17.99 s [17.99 s, 17.99 s] 3.006 s (20.1%)
profiling 15.094 s [15.094 s, 15.094 s] 110.0 ms (0.7%)
tracing 14.743 s [14.743 s, 14.743 s] -241.0 ms (-1.6%)

@cbeauchesne
Copy link
Contributor Author

Needs DataDog/system-tests#4546

@cbeauchesne cbeauchesne mentioned this pull request Apr 22, 2025
Merged
7 tasks
@cbeauchesne cbeauchesne marked this pull request as ready for review April 22, 2025 13:10
@cbeauchesne cbeauchesne requested a review from a team as a code owner April 22, 2025 13:10
@cbeauchesne cbeauchesne requested a review from dougqh April 22, 2025 13:10
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 22, 2025 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Remove the tag from the pull request title

If you need help, please check our contributing guidelines.

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Apr 22, 2025
smola
smola requested changes Apr 22, 2025
View reviewed changes
.github/workflows/system-tests.yaml
main:
needs:
- build
uses: DataDog/system-tests/.github/workflows/system-tests.yml@main
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No pinning of system-tests commit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, .github/workflows/system-tests.yml does not support it.

If it's a blocker, I can work on this. Though, the success rate on dd-trace-java main branch is constantly above 90% for the past 5 months.

Let me know if you want to keep the pin.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤞 Let's try.

@cbeauchesne cbeauchesne requested a review from smola April 22, 2025 15:02
@cbeauchesne cbeauchesne changed the title [ci] Move system-tests to GHA Move system-tests to GHA Apr 22, 2025
@cbeauchesne cbeauchesne merged commit c4bd105 into master Apr 22, 2025
575 of 576 checks passed
@cbeauchesne cbeauchesne deleted the cbeauchesne/system-tests-gha branch April 22, 2025 16:26
@github-actions github-actions bot added this to the 1.49.0 milestone Apr 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.49.0
Development

Successfully merging this pull request may close these issues.
2 participants
@cbeauchesne @smola