Add DI condition/template evaluation wall-time timeout (RFC C4) - #6223
Add DI condition/template evaluation wall-time timeout (RFC C4)#6223p-datadog wants to merge 3 commits into
Conversation
|
👋 Hey @DataDog/ruby-guild, please fill "Change log entry" section in the pull request description. If changes need to be present in CHANGELOG.md you can state it this way **Change log entry**
Yes. A brief summary to be placed into the CHANGELOG.md(possible answers Yes/Yep/Yeah) Or you can opt out like that **Change log entry**
None.(possible answers No/Nope/None) Visited at: 2026-08-20 22:24:44 UTC |
dd-octo-sts
Bot
added
the
debugger
Typing analysisNote: Ignored files are excluded from the next sections. Untyped methodsThis PR introduces 1 partially typed method, and clears 1 partially typed method. It increases the percentage of typed methods from 70.06% to 70.09% (+0.03%). Partially typed methods (+1-1)❌ Introduced:If you believe a method or an attribute is rightfully untyped or partially typed, you can add |
|
…erators Add a wall-time deadline check to Evaluator#filter, #all, #any (every EVALUATION_DEADLINE_CHECK_INTERVAL items) that raises a new DI::Error::EvaluationTimeout (subclass of ExpressionEvaluationError) when the per-invocation deadline is exceeded. The deadline lives on the per-invocation Context (Context#deadline_ns) to avoid racing the shared Evaluator instance across application threads. nil deadline preserves existing unbounded behavior.
…e eval Add the max_time_to_evaluate_ms setting (env DD_DYNAMIC_INSTRUMENTATION_EVALUATION_TIMEOUT_MS, default 50) and resolve a per-invocation deadline (Context#deadline_ns) before condition evaluation in both instrumenter hit paths and before template segment evaluation in ProbeNotificationBuilder. An EvaluationTimeout is surfaced through the existing condition-eval-failed callback as an evaluation-error snapshot with no captures, with a telemetry counter and debug log.
Evaluator cooperative-timeout tests (filter/all/any with deadline in the past, no deadline, far-future deadline, and deadline crossed midway via a stubbed clock). Instrumenter condition-timeout test asserting an over-budget condition produces an evaluation-error snapshot with no captures via the existing callback, plus an evaluation_timeouts telemetry counter. Builder settings double defaults max_time_to_evaluate_ms to nil.
3119c88 to
6983411
Compare
BenchmarksBenchmark execution time: 2026-08-24 14:34:38 Comparing candidate commit 6983411 in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 48 metrics, 0 unstable metrics.
|
2 participants
Implements backlog item
eval-timeout(DEBUG-6078). RFC area: C4 (condition/template evaluation wall-time timeout). Design:design/eval-timeout.mdin p-datadog/claude-projects.What
Bounds condition (
when) and template-segment evaluation wall-time. When the budget is exhausted mid-evaluation, the evaluation is aborted and surfaced as an evaluation-error snapshot with no captured user data — the same shape a runtime condition error already produces — so a pathological expression (ReDoS regex, huge collection filter) cannot stall the instrumented method or leak a fully-captured snapshot.How
max_time_to_evaluate_ms(envDD_DYNAMIC_INSTRUMENTATION_EVALUATION_TIMEOUT_MS, default 50 ms — RFC upper bound, conservative).Evaluator#filter/#all/#anyevery 64 items, raising a newDI::Error::EvaluationTimeout(subclass ofExpressionEvaluationErrorso the existing condition-eval-failed callback produces the error snapshot with no captures).Context#deadline_ns(avoids racing the sharedEvaluatorinstance across threads). Resolved before condition eval in both instrumenter hit paths and before template eval inProbeNotificationBuilder.EvaluationTimeoutincrements adynamic_instrumentation.evaluation_timeoutstelemetry counter and logs at debug.Non-goals / follow-ups
matchesregex bound (MATCHES_TIMEOUT_SECONDS = 0.5, 500 ms) is left unchanged: regex backtracking runs in the C regexp engine and cannot be cooperatively interrupted, so lowering it 10–50× is a customer-visible regression on legitimate regex conditions. It exceeds the RFC 10–50 ms evaluation range; aligning it is deferred.Tests
Local (Ruby 3.3.12): evaluator, instrumenter, probe_notification_builder, capture_expression_evaluator, instrumenter_circuit_breaker, serializer, probe specs — 373 examples, 0 failures. standard/steep deferred to PR CI.
System test
Test_Debugger_Evaluation_Timeout_Line_Probe(DataDog/system-tests#7287) ismissing_featureuntil the Ruby weblog gains the/debugger/expression/stringsand/debugger/snapshot/limitsfixtures and line mappings and the manifest is flipped (separate system-tests work).