ci: enable performance quality gates

[igoragoli]

What does this PR do?

Enables pre-release performance quality gates on dd-trace-rb.

• Microbenchmarks: microbenchmarks-check-big-regressions job (20% threshold via fail_on_regression)
• Macrobenchmarks: macrobenchmarks-check-slo-breaches + macrobenchmarks-notify-slo-breaches jobs with SLO thresholds via fail_on_breach
  • 36 scenarios, 66 thresholds
  • normal_operation: p50/p99 latency
  • high_load: throughput
  • utilization monitors: CPU% and RSS
  • baseline scenarios excluded (not actionable)

Motivation:

Catch performance regressions before release. Aligns dd-trace-rb with dd-trace-go and dd-trace-py.

Change log entry

None.

Additional Notes:

SLO generation:

• Generated with benchmark_analyzer generate slos --strategy tight --significant-impact-threshold 0.10 (T=10%)
• Source: single pipeline run of all 8 macrobenchmark configurations
• One RSS threshold manually bumped (high_load--profiling-and-tracing-and-appsec--puma-utilization: 2.73 GB → 3.25 GB) due to cross-run variance

Quality gates setup:

• All gate jobs use allow_failure: true until thresholds are validated
• Slack notifications go to apm-dcs-performance-alerts (TODO: switch to #guild-dd-ruby)
• tracing-and-appsec macrobenchmark produced no k6 results, so it has no SLO thresholds yet
• Depends on ci: add dd-octo-sts policy for GitLab SLO change tracking #5570

How to test the change?

CI pipeline validates gate jobs run correctly after benchmarks complete.

[github-actions]

Thank you for updating Change log entry section 👏

^{Visited at: 2026-04-09 14:50:40 UTC}

[igoragoli]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• ci: enable performance quality gates #5571 👈 (View in Graphite)
• ci: add dd-octo-sts policy for GitLab SLO change tracking #5570
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[pr-commenter]

Benchmarks

Benchmark execution time: 2026-04-10 11:41:04

Comparing candidate commit d1e7605 in PR branch augusto/enable-perf-quality-gates with baseline commit 1595023 in branch augusto/add-perf-quality-gate-dd-octo-sts-policy.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'