Skip to content

Fix race conditions on hub and span #1050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 11 commits into
base: master
Choose a base branch
from
Draft

Fix race conditions on hub and span #1050

wants to merge 11 commits into from

Conversation

giortzisg
Copy link
Contributor

Closes both #1048 and #1049.

Description

Adds tests for Hubs, Scopes and Spans to identify race conditions and fixes everything uncovered.

Changes include:

  • Add getAtomicClientAndScope. Previously when accessing the client and scope it was possible for one of them to change between the two separate calls. Should be accessed under a single lock.
  • Add scope locks when parsing trace and baggage

@giortzisg giortzisg requested a review from cleptric July 2, 2025 10:50
@giortzisg giortzisg self-assigned this Jul 2, 2025
@giortzisg giortzisg added the Bug Issue type label Jul 2, 2025
Copy link

codecov bot commented Jul 2, 2025

Codecov Report

Attention: Patch coverage is 92.50000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 87.09%. Comparing base (221b2ac) to head (dc65a13).
Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
hub.go 90.32% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1050      +/-   ##
==========================================
+ Coverage   86.71%   87.09%   +0.38%     
==========================================
  Files          55       55              
  Lines        5804     5859      +55     
==========================================
+ Hits         5033     5103      +70     
+ Misses        629      613      -16     
- Partials      142      143       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

Comment on lines +434 to 440
scope.mu.RLock()
defer scope.mu.RUnlock()

if scope.span != nil {
return scope.span.ToBaggage()
span := scope.span
if span != nil {
return span.ToBaggage()
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
scope.mu.RLock()
defer scope.mu.RUnlock()
if scope.span != nil {
return scope.span.ToBaggage()
span := scope.span
if span != nil {
return span.ToBaggage()
}
if span := scope.GetSpan(); span != nil {
return span.ToBaggage()
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scope.GetSpan() already does the locking. Additionally we can do it like this which afaik is more idiomatic :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually just remembered why I did the change like this. After the if block we are still accessing scope.propagationContext so we need to lock the scope again.

Comment on lines +414 to 420
scope.mu.RLock()
defer scope.mu.RUnlock()

if scope.span != nil {
return scope.span.ToSentryTrace()
span := scope.span
if span != nil {
return span.ToSentryTrace()
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
scope.mu.RLock()
defer scope.mu.RUnlock()
if scope.span != nil {
return scope.span.ToSentryTrace()
span := scope.span
if span != nil {
return span.ToSentryTrace()
}
if span := scope.GetSpan(); span != nil {
return span.ToSentryTrace()
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above comment.

@lcian
Copy link
Member

lcian commented Jul 7, 2025

There seem to be some more places where both the hub and scope are used without using the new function which could possibly still lead to race conditions, as an example:

  • gin/sentrygin.go in the handle function
  • fiber/sentryfiber.go in the handle function
  • dynamic_sampling_context.go
    etc.

I would suggest reviewing all usages of hub.Client() and hub.Scope() and checking whether they need to be corrected.

cursor[bot]

This comment was marked as outdated.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Race Condition in `ToBaggage()` Method

The ToBaggage() method introduces a race condition. It acquires a read lock on the current span (s) but then attempts to modify the dynamicSamplingContext field of the transaction span (t = s.GetTransaction()). This write operation is performed without holding a write lock on t. If t is s, the write occurs under s's read lock, violating the read-write mutex contract. If t is a different span, t's lock is not held at all, leading to concurrent writes without proper synchronization.

tracing.go#L326-L342

sentry-go/tracing.go

Lines 326 to 342 in 4ac25ad

func (s *Span) ToBaggage() string {
s.mu.RLock()
defer s.mu.RUnlock()
t := s.GetTransaction()
if t == nil {
return ""
}
// In case there is currently no frozen DynamicSamplingContext attached to the transaction,
// create one from the properties of the transaction.
if !s.dynamicSamplingContext.IsFrozen() {
// This will return a frozen DynamicSamplingContext.
if dsc := DynamicSamplingContextFromTransaction(t); dsc.HasEntries() {
t.dynamicSamplingContext = dsc
}
}

Fix in CursorFix in Web


Was this report helpful? Give feedback by reacting with 👍 or 👎

@cleptric cleptric marked this pull request as draft July 21, 2025 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Issue type
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants