-
Notifications
You must be signed in to change notification settings - Fork 241
Fix race conditions on hub and span #1050
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1050 +/- ##
==========================================
+ Coverage 86.71% 87.09% +0.38%
==========================================
Files 55 55
Lines 5804 5859 +55
==========================================
+ Hits 5033 5103 +70
+ Misses 629 613 -16
- Partials 142 143 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
scope.mu.RLock() | ||
defer scope.mu.RUnlock() | ||
|
||
if scope.span != nil { | ||
return scope.span.ToBaggage() | ||
span := scope.span | ||
if span != nil { | ||
return span.ToBaggage() | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
scope.mu.RLock() | |
defer scope.mu.RUnlock() | |
if scope.span != nil { | |
return scope.span.ToBaggage() | |
span := scope.span | |
if span != nil { | |
return span.ToBaggage() | |
} | |
if span := scope.GetSpan(); span != nil { | |
return span.ToBaggage() | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
scope.GetSpan()
already does the locking. Additionally we can do it like this which afaik is more idiomatic :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually just remembered why I did the change like this. After the if block we are still accessing scope.propagationContext
so we need to lock the scope again.
scope.mu.RLock() | ||
defer scope.mu.RUnlock() | ||
|
||
if scope.span != nil { | ||
return scope.span.ToSentryTrace() | ||
span := scope.span | ||
if span != nil { | ||
return span.ToSentryTrace() | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
scope.mu.RLock() | |
defer scope.mu.RUnlock() | |
if scope.span != nil { | |
return scope.span.ToSentryTrace() | |
span := scope.span | |
if span != nil { | |
return span.ToSentryTrace() | |
} | |
if span := scope.GetSpan(); span != nil { | |
return span.ToSentryTrace() | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as above comment.
There seem to be some more places where both the hub and scope are used without using the new function which could possibly still lead to race conditions, as an example:
I would suggest reviewing all usages of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Race Condition in `ToBaggage()` Method
The ToBaggage()
method introduces a race condition. It acquires a read lock on the current span (s
) but then attempts to modify the dynamicSamplingContext
field of the transaction span (t = s.GetTransaction()
). This write operation is performed without holding a write lock on t
. If t
is s
, the write occurs under s
's read lock, violating the read-write mutex contract. If t
is a different span, t
's lock is not held at all, leading to concurrent writes without proper synchronization.
tracing.go#L326-L342
Lines 326 to 342 in 4ac25ad
func (s *Span) ToBaggage() string { | |
s.mu.RLock() | |
defer s.mu.RUnlock() | |
t := s.GetTransaction() | |
if t == nil { | |
return "" | |
} | |
// In case there is currently no frozen DynamicSamplingContext attached to the transaction, | |
// create one from the properties of the transaction. | |
if !s.dynamicSamplingContext.IsFrozen() { | |
// This will return a frozen DynamicSamplingContext. | |
if dsc := DynamicSamplingContextFromTransaction(t); dsc.HasEntries() { | |
t.dynamicSamplingContext = dsc | |
} | |
} | |
Was this report helpful? Give feedback by reacting with 👍 or 👎
Closes both #1048 and #1049.
Description
Adds tests for Hubs, Scopes and Spans to identify race conditions and fixes everything uncovered.
Changes include:
getAtomicClientAndScope
. Previously when accessing the client and scope it was possible for one of them to change between the two separate calls. Should be accessed under a single lock.