Skip to content

Commit 351dcdc

Browse files
authored
Add the native histogram metric types (#279)
NativeValueHistogram and NativeDurationHistogram record a distribution whose buckets are derived from the observations rather than declared up front, and report it as one serialized payload per interval rather than a counter per bucket. Both mirror the equivalent methods on Histogram, so the only thing that changes for a caller moving over is the constructor. Two types rather than one. Histogram gets unit safety for free -- it is handed ValueBuckets or DurationBuckets at construction and drops calls that do not match (stats.go). A native histogram declares no buckets, so that signal is gone, and mixing the APIs is worse here than there: the sample is not dropped, it is accumulated. RecordDuration(250ms) contributes 0.25 and RecordValue(250) contributes 250, and because a scope memoizes by name, the two calls need not even be in the same file. Nothing downstream can separate them again. Splitting the type moves that to compile time -- a NativeValueHistogram has no RecordDuration method to call. RecordDuration records SECONDS, matching DurationBuckets.AsValues(). Reporters needing another unit on the wire convert on the way out. A drain is a delta: it marshals what has accumulated since the last one and clears. It clears after a failed marshal too. Marshalling is deterministic, so re-presenting the same samples cannot succeed, and keeping them would wedge the histogram permanently when it is the accumulated state the encoder rejected -- Clear is the only way back. One lost interval beats silence from then on. Without a ScopeOptions.NativeHistogramFactory the scope falls back to a backend that counts observations and refuses to marshal them, so the metric is inert rather than broken. tally depends on no histogram implementation. NativeHistogramData is the entire seam.
1 parent 3babbc2 commit 351dcdc

2 files changed

Lines changed: 563 additions & 0 deletions

File tree

native_histogram.go

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
// Copyright (c) 2026 Uber Technologies, Inc.
2+
//
3+
// Permission is hereby granted, free of charge, to any person obtaining a copy
4+
// of this software and associated documentation files (the "Software"), to deal
5+
// in the Software without restriction, including without limitation the rights
6+
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7+
// copies of the Software, and to permit persons to whom the Software is
8+
// furnished to do so, subject to the following conditions:
9+
//
10+
// The above copyright notice and this permission notice shall be included in
11+
// all copies or substantial portions of the Software.
12+
//
13+
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14+
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15+
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16+
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17+
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18+
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
19+
// THE SOFTWARE.
20+
21+
package tally
22+
23+
import (
24+
"errors"
25+
"sync"
26+
"time"
27+
)
28+
29+
// DefaultNativeHistogramMaxBuckets is the bucket budget used when a scope
30+
// does not configure one. Matches the OpenTelemetry exponential histogram
31+
// default max size.
32+
const DefaultNativeHistogramMaxBuckets = 160
33+
34+
// ErrNativeHistogramBackendNotConfigured is returned by MarshalBinary when a
35+
// scope has no ScopeOptions.NativeHistogramFactory. Values are still counted,
36+
// but there is no encoder to serialize them with, so nothing is reported.
37+
var ErrNativeHistogramBackendNotConfigured = errors.New(
38+
"native histogram backend not configured: " +
39+
"set ScopeOptions.NativeHistogramFactory",
40+
)
41+
42+
// NativeHistogramData is a mergeable accumulator backing a NativeHistogram,
43+
// supplied by the application through ScopeOptions.NativeHistogramFactory.
44+
//
45+
// Implementations need not be safe for concurrent use; the owning scope
46+
// serializes every call. MarshalBinary is expected to produce a self-describing
47+
// encoding of everything accumulated since the last Clear, in whatever format
48+
// the configured reporter understands.
49+
type NativeHistogramData interface {
50+
// Update records a single observation.
51+
Update(value float64)
52+
53+
// Count returns the number of observations accumulated since the last
54+
// Clear.
55+
Count() uint64
56+
57+
// Clear discards everything accumulated so far.
58+
Clear()
59+
60+
// MarshalBinary serializes everything accumulated since the last Clear.
61+
MarshalBinary() ([]byte, error)
62+
}
63+
64+
// NativeValueHistogram is the interface for emitting native histogram metrics
65+
// over unitless values: a distribution whose buckets are derived from the
66+
// observed values rather than declared up front, reported as one serialized
67+
// payload per interval instead of as a counter per bucket.
68+
//
69+
// Values and durations are separate types so that one distribution cannot mix
70+
// the two. A native histogram has no declared buckets to signal which it
71+
// holds, so unlike Histogram there is nothing to catch the mistake later: the
72+
// samples are simply accumulated together and the percentiles come out wrong.
73+
type NativeValueHistogram interface {
74+
// RecordValue records a specific value directly.
75+
RecordValue(value float64)
76+
}
77+
78+
// NativeDurationHistogram is the NativeValueHistogram equivalent for
79+
// durations, which it records as seconds.
80+
type NativeDurationHistogram interface {
81+
// RecordDuration records a specific duration directly, as seconds.
82+
RecordDuration(value time.Duration)
83+
84+
// Start gives you a specific point in time to then record a duration.
85+
Start() Stopwatch
86+
}
87+
88+
// Asserted here rather than left to the scope: on its own this file has no
89+
// call site that would catch a variant drifting out of its interface.
90+
var (
91+
_ NativeValueHistogram = nativeValueHistogram{}
92+
_ NativeDurationHistogram = nativeDurationHistogram{}
93+
_ StopwatchRecorder = nativeDurationHistogram{}
94+
)
95+
96+
type nativeHistogram struct {
97+
// mtx guards data, whose implementations are supplied by the application
98+
// and are not required to be safe for concurrent use.
99+
mtx sync.Mutex
100+
data NativeHistogramData
101+
102+
cachedNativeHistogram CachedNativeHistogram
103+
}
104+
105+
func newNativeHistogram(
106+
data NativeHistogramData,
107+
cachedNativeHistogram CachedNativeHistogram,
108+
) *nativeHistogram {
109+
return &nativeHistogram{
110+
data: data,
111+
cachedNativeHistogram: cachedNativeHistogram,
112+
}
113+
}
114+
115+
// record is the only way into the accumulator. It is deliberately unexported
116+
// and unit-agnostic: both variants embed *nativeHistogram, so an exported
117+
// Record method here would be promoted into each of them and hand a caller
118+
// back the API the split exists to withhold.
119+
func (h *nativeHistogram) record(value float64) {
120+
h.mtx.Lock()
121+
defer h.mtx.Unlock()
122+
123+
h.data.Update(value)
124+
}
125+
126+
// drain marshals everything accumulated since the last drain and resets the
127+
// accumulator, so each payload is a delta covering one reporting interval
128+
// rather than the distribution's whole history.
129+
//
130+
// A marshal error clears the accumulator as well. Marshalling is
131+
// deterministic, so re-presenting the same samples cannot succeed, and holding
132+
// them would wedge the histogram for good: when it is the accumulated state
133+
// that the encoder rejected, Clear is the only way back. Losing one interval
134+
// beats never reporting again.
135+
func (h *nativeHistogram) drain() ([]byte, uint64, bool) {
136+
h.mtx.Lock()
137+
defer h.mtx.Unlock()
138+
139+
samples := h.data.Count()
140+
if samples == 0 {
141+
return nil, 0, false
142+
}
143+
144+
payload, err := h.data.MarshalBinary()
145+
h.data.Clear()
146+
if err != nil {
147+
return nil, 0, false
148+
}
149+
150+
return payload, samples, true
151+
}
152+
153+
func (h *nativeHistogram) report(name string, tags map[string]string, r StatsReporter) {
154+
payload, samples, ok := h.drain()
155+
if !ok {
156+
return
157+
}
158+
159+
r.ReportNativeHistogram(name, tags, payload, samples)
160+
}
161+
162+
func (h *nativeHistogram) cachedReport() {
163+
payload, samples, ok := h.drain()
164+
if !ok {
165+
return
166+
}
167+
168+
h.cachedNativeHistogram.ReportNativeHistogram(payload, samples)
169+
}
170+
171+
func (h *nativeHistogram) snapshot() uint64 {
172+
h.mtx.Lock()
173+
defer h.mtx.Unlock()
174+
175+
return h.data.Count()
176+
}
177+
178+
// nativeValueHistogram is the unitless view of an accumulator.
179+
type nativeValueHistogram struct {
180+
*nativeHistogram
181+
}
182+
183+
func (h nativeValueHistogram) RecordValue(value float64) {
184+
h.record(value)
185+
}
186+
187+
// nativeDurationHistogram is the duration view of an accumulator.
188+
type nativeDurationHistogram struct {
189+
*nativeHistogram
190+
}
191+
192+
// RecordDuration records value in SECONDS, matching the units that
193+
// DurationBuckets.AsValues() reports durations in. Reporters that need another
194+
// unit on the wire convert on the way out.
195+
func (h nativeDurationHistogram) RecordDuration(value time.Duration) {
196+
h.record(value.Seconds())
197+
}
198+
199+
func (h nativeDurationHistogram) Start() Stopwatch {
200+
return NewStopwatch(globalNow(), h)
201+
}
202+
203+
func (h nativeDurationHistogram) RecordStopwatch(stopwatchStart time.Time) {
204+
d := globalNow().Sub(stopwatchStart)
205+
h.RecordDuration(d)
206+
}
207+
208+
// defaultNativeHistogramData counts observations but cannot serialize them.
209+
// It is what a scope falls back to when no NativeHistogramFactory is set, so
210+
// that the native histogram methods are safe to call but emit nothing.
211+
type defaultNativeHistogramData struct {
212+
count uint64
213+
}
214+
215+
func defaultNativeHistogramFactory(maxBuckets int) NativeHistogramData {
216+
return &defaultNativeHistogramData{}
217+
}
218+
219+
func (d *defaultNativeHistogramData) Update(value float64) {
220+
d.count++
221+
}
222+
223+
func (d *defaultNativeHistogramData) Count() uint64 {
224+
return d.count
225+
}
226+
227+
func (d *defaultNativeHistogramData) Clear() {
228+
d.count = 0
229+
}
230+
231+
func (d *defaultNativeHistogramData) MarshalBinary() ([]byte, error) {
232+
return nil, ErrNativeHistogramBackendNotConfigured
233+
}

0 commit comments

Comments
 (0)