Add TestTracer to new TestTracingKit #180

slashmo · 2025-08-30T13:25:27Z

Motivation

Library authors and end users integrating Swift Distributed Tracing need some form of test tracer implementation that they can use in unit tests. Up until now, Swift Distributed Tracing didn't provide a built-in solution to this problem, leading to TestTracer being redefined in many libraries and presumably some end-user applications. Most of these implementations share the common need of inspecting spans and context propagation to verify their code behaves as expected so they end up being extremely similar or straight up duplicates.

Therefore, I'd like to create a first-party TestTracer in a new TracingTestKit library. The name TracingTestKit aligns with MetricsTestKit from swift-metrics.

Modifications

Adds TracingTestKit library
Adds TestTracer to TracingTestKit to allow inspection of spans in unit tests

Remaining tasks

Add documentation to TracingTestKit

Result

Library users can now extensively test the parts of their code which make use of Swift Distributed Tracing without needing to build their own TestTracer type. Instead, they can now depend on TracingTestKit.

Future Direction

Using `TestTracer` in `TracingTests`

To keep this pull request focused on providing a fully unit-tested TestTracer implementation I opted not to use the new TestTracer in the existing TracingTests. IMHO, this can come in a follow-up PR.

Task-local `TestTracer`

@adam-fowler built a great task-local tracer for Hummingbird which allows parallel test execution while still using the global InstrumentationSystem: https://github.com/hummingbird-project/hummingbird/blob/3ae359b1bb1e72378ed43b59fdcd4d44cac5d7a4/Tests/HummingbirdTests/TestTracer.swift#L184
Now that we'd have a built-in TestTracer it'd be great to also add this to TracingTestKit.

FranzBusch

Great idea! Overall can we add public API docs and some docs into our DocC catalog for this please?

FranzBusch · 2025-09-01T16:31:47Z

Sources/TracingTestKit/TestSpan.swift

+public struct TestSpan: Span {
+    public let context: ServiceContext
+    public let spanContext: TestSpanContext
+    public let kind: SpanKind
+    public let startInstant: any TracerInstant


Can we add some doc comments to this? Also can all of these properties be vars instead?

FranzBusch · 2025-09-01T16:31:58Z

Sources/TracingTestKit/TestSpan.swift

+        self.onEnd = onEnd
+    }
+
+    public var isRecording: Bool {


Same here. Can we add doc comments to all of those please?

These have docs on Span already so not sure if that's necessary here

Then we could add // swift-format-ignore: AllPublicDeclarationsHaveDocumentation and enable AllPublicDeclarationsHaveDocumentation in the swift-format config file. I'd also like to see all public APIs having comments or opt out this way if they inherit them.

That said - probably worth doing in a separate PR.

Filed #182

FranzBusch · 2025-09-01T16:32:30Z

Sources/TracingTestKit/TestSpan.swift

+    }
+}
+
+public struct FinishedTestSpan: Sendable {


Can we move this into a new file, add doc comments and make all the properties vars instead?

Not sure this needs new file but +1 on the var, there's no reason not to :)

FranzBusch · 2025-09-01T16:32:56Z

Tests/TracingTestKitTests/TestTracerTests.swift

+@_spi(Locking) import Instrumentation
+import Testing
+import Tracing
+@testable import TracingTestKit


Do we need a testable import here?

FranzBusch · 2025-09-01T16:33:47Z

Tests/TracingTestKitTests/TestTracerTests.swift

Is it possible to add a test that uses the global withSpan APIs and the test tracer?

slashmo · 2025-09-01T16:56:10Z

Overall can we add public API docs and some docs into our DocC catalog for this please?

Thanks for the review. Yes, I'll add docs next but wanted to get some general feedback on the API first.

ktoso · 2025-09-05T03:34:47Z

Sources/TracingTestKit/TestSpan.swift

+        self.onEnd = onEnd
+    }
+
+    public var isRecording: Bool {


These have docs on Span already so not sure if that's necessary here

Sources/TracingTestKit/TestSpan.swift

ktoso · 2025-09-05T03:36:18Z

Sources/TracingTestKit/TestSpan.swift

+    }
+}
+
+public struct FinishedTestSpan: Sendable {


Not sure this needs new file but +1 on the var, there's no reason not to :)

ktoso · 2025-09-05T03:50:07Z

Sources/TracingTestKit/TestTracer.swift

+@_spi(Locking) import Instrumentation
+import Tracing
+
+public struct TestTracer: Tracer {


The implementation is fine, but we should call it some InMemoryStorageTracer, and rename the module as well perhaps. This isn't strictly required for tests but just accumulates in memory; you could imagine some silly use-cases that would want to use this and I don't necessarily want to taint them with the Test... word, WDYT?

Sources/TracingTestKit/TestTracer.swift

Sources/TracingTestKit/TestSpan.swift

ktoso · 2025-09-05T04:21:40Z

Sources/TracingTestKit/TestSpan.swift

+            errors: errors,
+            status: status
+        )
+        _isRecording.withValue { $0 = false }


This is racy with the assertIsRecording; you have to check + swap to false at the same time as asserting, otherwise you can get two tasks end()-ing they'll both pass the assert and then both try to set -- but we would have double-ended.

ktoso · 2025-09-05T04:34:03Z

Package.swift

@@ -6,6 +6,7 @@ let package = Package(
    products: [
        .library(name: "Instrumentation", targets: ["Instrumentation"]),
        .library(name: "Tracing", targets: ["Tracing"]),
+        .library(name: "TracingTestKit", targets: ["TracingTestKit"]),


I think we should call this InMemoryTracing maybe, and not have it be strictly called "just for testing" there may be various use cases to use this

ktoso · 2025-09-10T02:47:30Z

We discussed this one a bit more -- just pending a rename and the cleanups and happy to merge afterwards!

ktoso · 2025-09-11T04:28:33Z

I took a stab at pushing this over the finish line: #183

Gonna chat with @slashmo if he'll pick those over here or how to proceed

Co-authored-by: Moritz Lang <[email protected]>

slashmo requested review from ktoso, simonjbeaumont, czechboy0 and fabianfett August 30, 2025 13:25

slashmo added the 🆕 semver/minor Adds new public API. label Aug 30, 2025

Add TestTracer to new TestTracingKit

7df3df8

slashmo force-pushed the feature/tracing-test-kit branch from 17f4c8d to 7df3df8 Compare August 30, 2025 13:29

slashmo added 2 commits August 30, 2025 15:32

Fix compilation in Swift 5.10

f6b7db6

Expose SpanKind in TestSpan and FinishedTestSpan

b80efc1

FranzBusch requested changes Sep 1, 2025

View reviewed changes

slashmo mentioned this pull request Sep 4, 2025

Add Distributed Tracing support valkey-io/valkey-swift#177

Open

ktoso reviewed Sep 5, 2025

View reviewed changes

This was referenced Sep 10, 2025

Introduce built-in swift-distributed-tracing support swift-server/async-http-client#857

Open

Pushing InMemoryTracer (#180) over the finish line #183

Merged

ktoso closed this Sep 11, 2025

ktoso added a commit that referenced this pull request Sep 11, 2025

Pushing InMemoryTracer (#180) over the finish line (#183)

6600888

Co-authored-by: Moritz Lang <[email protected]>

slashmo deleted the feature/tracing-test-kit branch September 11, 2025 12:52

Add TestTracer to new TestTracingKit #180

Add TestTracer to new TestTracingKit #180

Conversation

slashmo commented Aug 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Remaining tasks

Result

Future Direction

Using TestTracer in TracingTests

Task-local TestTracer

Uh oh!

FranzBusch left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

czechboy0 Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slashmo commented Sep 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ktoso commented Sep 10, 2025

Uh oh!

ktoso commented Sep 11, 2025

Uh oh!

Uh oh!

slashmo commented Aug 30, 2025 •

edited

Loading

Using `TestTracer` in `TracingTests`

Task-local `TestTracer`

czechboy0 Sep 5, 2025 •

edited

Loading