Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Profiler] Use chrono types instead of integer type #6288

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

gleocadie
Copy link
Collaborator

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

@github-actions github-actions bot added the area:profiler Issues related to the continous-profiler label Nov 14, 2024
@gleocadie gleocadie force-pushed the gleocadie/use-strong-type-for-time-instead-of-int branch from a16e088 to 2a7c290 Compare November 14, 2024 16:26
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Nov 14, 2024

Datadog Report

Branch report: gleocadie/use-strong-type-for-time-instead-of-int
Commit report: f6bd75e
Test service: dd-trace-dotnet

❌ 1 Failed (0 Known Flaky), 460525 Passed, 3238 Skipped, 21h 16m 33.72s Total Time

❌ Failed Tests (1)

  • SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.KafkaTests - Details

    Expand for error
     Expected relevantSpans to contain at least 122 item(s) because we want to ensure that we don't timeout while waiting for spans from the mock tracer agent, but found 63: {{TraceId: 13500928364603174659, SpanId: 16355150308545544112, Name: kafka.consume, Resource: Consume Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 15107301983078651061, SpanId: 2861791058393693474, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 12825117334969749086, SpanId: 13803226637313805338, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 7828582273746697791, SpanId: 17857544722889626581, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 1161100027495668797, SpanId: 4871382068230767124, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 3060974486289068103, SpanId: 13591864246685057035, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 1784702671320819054, SpanId: 8627961084873134140, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 18085573801849959878, SpanId: 2835872407648712626, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 14154396355201575123, SpanId: 7717743163340719068, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 7776730335224106783, SpanId: 4077797736471482390, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 13518448688873955174, SpanId: 2538735762282987122, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 6776204479129492554, SpanId: 7252180571386850051, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 8256296830740883058, SpanId: 5338162516703791565, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 12024836454360201155, SpanId: 8075961654451197345, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 3296801701377742171, SpanId: 13368513597562274087, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 10667999335199483208, SpanId: 10549459348127780423, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 14099129562398804153, SpanId: 6229547115260693500, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 6279659497626455415, SpanId: 14017089371484444913, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 12806190050954945163, SpanId: 9905891923033713577, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 7135381087571153492, SpanId: 2575560503658734755, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 10388996474525768823, SpanId: 8997556409789563131, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 14659264844901806091, SpanId: 1518917808598938582, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 14881008291983237552, SpanId: 814192150408049722, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 4032405285483620738, SpanId: 14095479859185271839, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 13660138854137153313, SpanId: 11901110571063167400, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 6939269780944182501, SpanId: 10293526687179072980, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 17052987476623719425, SpanId: 1842129629927608616, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 11557084496529644390, SpanId: 13440099970189318237, Name: kafka.produce, Resource: Produce Topic sample-topic-Release-net8-0-1-9-3, Service: Samples.Kafka-kafka}, {TraceId: 164689
    

@gleocadie gleocadie force-pushed the gleocadie/use-strong-type-for-time-instead-of-int branch 2 times, most recently from 7b57237 to f50ff96 Compare November 14, 2024 16:53
@andrewlock
Copy link
Member

andrewlock commented Nov 14, 2024

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6288) - mean (71ms)  : 63, 80
     .   : milestone, 71,
    master - mean (74ms)  : 65, 83
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (6288) - mean (1,108ms)  : 1089, 1128
     .   : milestone, 1108,
    master - mean (1,110ms)  : 1085, 1135
     .   : milestone, 1110,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6288) - mean (108ms)  : 106, 110
     .   : milestone, 108,
    master - mean (108ms)  : 106, 110
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6288) - mean (761ms)  : 746, 776
     .   : milestone, 761,
    master - mean (767ms)  : 752, 783
     .   : milestone, 767,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6288) - mean (92ms)  : 90, 93
     .   : milestone, 92,
    master - mean (92ms)  : 89, 95
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6288) - mean (719ms)  : 704, 734
     .   : milestone, 719,
    master - mean (727ms)  : 714, 739
     .   : milestone, 727,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6288) - mean (191ms)  : 185, 197
     .   : milestone, 191,
    master - mean (190ms)  : 187, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6288) - mean (1,211ms)  : 1189, 1232
     .   : milestone, 1211,
    master - mean (1,212ms)  : 1184, 1240
     .   : milestone, 1212,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6288) - mean (278ms)  : 273, 282
     .   : milestone, 278,
    master - mean (276ms)  : 272, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6288) - mean (942ms)  : 923, 962
     .   : milestone, 942,
    master - mean (942ms)  : 926, 959
     .   : milestone, 942,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6288) - mean (266ms)  : 262, 271
     .   : milestone, 266,
    master - mean (266ms)  : 262, 270
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (6288) - mean (926ms)  : 910, 942
     .   : milestone, 926,
    master - mean (927ms)  : 910, 945
     .   : milestone, 927,

Loading

@gleocadie gleocadie force-pushed the gleocadie/use-strong-type-for-time-instead-of-int branch from f50ff96 to 55acb33 Compare November 14, 2024 18:47
@gleocadie gleocadie force-pushed the gleocadie/use-strong-type-for-time-instead-of-int branch 2 times, most recently from 7faabb5 to fbf9825 Compare November 14, 2024 20:36
@gleocadie gleocadie changed the title [Profiler] Use timing types instead of integer type [Profiler] Use chrono types instead of integer type Nov 14, 2024
@gleocadie gleocadie force-pushed the gleocadie/use-strong-type-for-time-instead-of-int branch from fbf9825 to f6bd75e Compare November 14, 2024 20:54
@andrewlock
Copy link
Member

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6288) (11.166M)   : 0, 11166306
    master (11.106M)   : 0, 11106407
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6288) (7.213M)   : 0, 7213062
    master (7.204M)   : 0, 7204056
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.595M)   : 0, 7595165

    section Manual
    master (11.008M)   : 0, 11007592

    section Manual + Automatic
    This PR (6288) (6.624M)   : 0, 6623725
    master (6.725M)   : 0, 6725238

    section DD_TRACE_ENABLED=0
    master (10.307M)   : 0, 10307106

Loading
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6288) (9.610M)   : 0, 9609538
    master (9.667M)   : 0, 9666990
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6288) (6.507M)   : 0, 6507391
    master (6.549M)   : 0, 6549462

    section Trace stats
    master (6.703M)   : 0, 6702885

    section Manual
    master (9.608M)   : 0, 9608456

    section Manual + Automatic
    This PR (6288) (5.962M)   : 0, 5962217
    master (5.958M)   : 0, 5958327

    section DD_TRACE_ENABLED=0
    master (8.816M)   : 0, 8816192

Loading
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6288) (9.653M)   : 0, 9652550
    master (9.761M)   : 0, 9761080
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6288) (6.274M)   : 0, 6274285
    master (6.222M)   : 0, 6221888
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (7.068M)   : 0, 7067863

    section Manual
    master (9.826M)   : 0, 9826453

    section Manual + Automatic
    This PR (6288) (5.996M)   : 0, 5996334
    master (5.724M)   : 0, 5723553

    section DD_TRACE_ENABLED=0
    master (9.178M)   : 0, 9177760

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:profiler Issues related to the continous-profiler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants