Commit 205f15f
Experimental timeline view for stitched Python+native stacks (#1040)
* Experimental timeline view for stitched Python+native stacks
Adds a new GUI section beneath the existing flame chart that renders the
captured stitched stacks chronologically. x-axis is wallclock time,
y-axis is stack depth (icicle, outermost on top). Two summary tracks
above the main panel show GC and I/O activity, classified by frame
patterns inspired by Chrome / Firefox profiler conventions.
Backend:
- ScaleneStatistics gains combined_stacks_timeline: a run-length-encoded
list of (timestamp, stack_key, count). Consecutive identical stacks
coalesce, so a tight loop firing the same stack thousands of times in
a row is one entry. Soft cap at 100K runs.
- combined_stacks keys move from tagged tuples to frozen dataclasses
(PyFrameKey, NativeFrameKey) so the stack-key shape is type-checked
end to end.
- Native-frame helpers (_is_scalene_handler_frame,
_is_cpython_runtime_frame, _trim_native_stack) now operate on a
frozen _ResolvedNativeFrame dataclass. The on-wire
[module, symbol, ip, offset] list shape only materializes at the JSON
output boundary, preserving compatibility with consumers.
- Pydantic CombinedStackTimelineEvent and CombinedStackFrame describe
the wire shape.
GUI:
- renderCombinedStacksTimeline() draws the timeline. Each run becomes a
vertical strip of stacked colored frames; clicking a [py] frame jumps
to its source line.
- GC and I/O classifier patterns use (?:\b|_)...(?:_|\b) tokenization so
symbols like select_kqueue_control_impl, os_read, _io_FileIO_read,
accept4 match correctly. The previous \b...\b patterns missed them
because '_' is a regex word character.
Async-aware reconstruction:
- New ScaleneTracing.should_trace_for_stitched_stack: looser variant
that lets asyncio/* and selectors.py through. Used only by the
timeline path; line-level CPU/memory accounting still uses the
strict should_trace.
- process_cpu_sample now starts the f_back walk from the original
innermost frame (not the already-walked-back filtered frame), so the
full asyncio call chain becomes visible on the timeline.
- When a sample lands inside the event loop with coroutines suspended,
the timeline emits one synthetic stitched run per suspended task at
its await point - mirroring the existing per-line "await %"
attribution. The synthetic display name "[await] <func> (<task>)" is
recognized by the I/O classifier.
Tests:
- tests/test_native_stacks.py: end-to-end smoke tests covering the
GC/IO classifier and async-blocking attribution. Pure-Python tests
for the trim helpers updated to construct _ResolvedNativeFrame;
combined_stack tests updated to construct PyFrameKey / NativeFrameKey.
- tests/test_async_profiling.py: SuspendedTaskInfo unpacking test
updated for the new func_name field.
- test/timeline_view_demo.py: standalone demo workload that exercises
CPU, sync I/O, forced GC, and asyncio.sleep so all three timeline
classifications are visible at once.
Full pytest suite passes (397 + 13 skipped); mypy and ruff clean on
the scalene package.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* GUI: render disclosure triangles as plain text, matching CSrankings
The toggle triangles (▶/▼) inherit Bootstrap 5's font-family stack
which includes "Apple Color Emoji". On macOS/iOS, Safari can pick up
U+25BA (►) from the emoji font and render it as a colored "play
button" emoji instead of a plain text triangle, especially in the
collapsed (right-pointing) state. The expanded triangle (U+25BC, ▼)
doesn't have this problem because no emoji presentation exists for it.
The result was a visually mismatched pair when collapsed.
Match what CSrankings does: route every toggle through a single
.disclosure-triangle CSS class that pins font-family to Helvetica /
Arial / sans-serif and sets font-variant-emoji: text. Same effective
styling (cursor: pointer; color: blue) as the previous inline styles,
but emoji presentation is now suppressed.
- gui-elements.ts: add the missing trailing semicolons to RightTriangle
/ DownTriangle entity strings (matches CSrankings exactly; HTML5
parsers tolerate omission but it's not strictly compliant).
- index.html.template: add the .disclosure-triangle CSS rule to the
global <style> block.
- scalene-gui.ts: replace inline cursor/color styles on the three
toggle spans (button-combined-stacks, button-combined-timeline,
per-file button-${id}) with class="disclosure-triangle".
Verified via Selenium that the rendered toggle now reports
fontFamily="Helvetica, Arial, sans-serif" and fontVariantEmoji="text",
matching CSrankings exactly (15.84px × 18px). All 397 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Drop should_trace_for_stitched_stack: synthetic await frames cover it
The looser stitched-stack-only filter was added to surface asyncio /
selectors frames in the timeline call chain. The subsequent async-aware
reconstruction (add_async_await_run + suspended-task snapshot) replaced
those frames with one synthetic "[await] <func> (<task>)" entry per
suspended task, mirroring the existing "await %" attribution.
In default mode the looser filter no longer contributes any frames to
the timeline (verified on test/timeline_view_demo.py: 11 synthetic
frames, 0 raw asyncio path frames). It only had effect under
--no-async, where the user has explicitly opted out of async-aware
attribution; serving that minority with truncated <module>+native_leaf
stacks is consistent with the flag's intent.
Removes:
- ScaleneTracing.should_trace_for_stitched_stack and the
_is_async_io_stdlib_module helper.
- Scalene._should_trace_for_stitched_stack static delegate.
- The should_trace_for_stitched_stack kwarg on
ScaleneCPUProfiler.process_cpu_sample.
- The orig_frame plumbing in process_cpu_sample (revert to
main_thread_frame for consistency with add_stack — strict
should_trace produces the same Python chain from either start frame).
Net -71 lines across 3 files. Full test suite still 397 + 13 skipped;
mypy and ruff clean. Demo produces the same 11 [await] synthetic
frames as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Multi-frame stitched stacks for synthetic [await] runs
Each suspended task's synthetic timeline run was previously a single
[await] frame at the innermost suspension point. The user-visible call
chain that led up to that await — including any intermediate user-async
functions — was missing.
Wire the existing ScaleneAsync.walk_await_chain through to
add_async_await_run so the synthetic stack mirrors the actual nested
coroutine chain at suspend time.
Backend
- SuspendedTaskInfo gains an optional ``chain`` field — a tuple of
(filename, lineno, func_name) tuples, outermost-first (matches the
combined_stacks convention).
- _poll_suspended_tasks (Strategy A) populates chain via
walk_await_chain(coro).
- _on_yield (Strategy B / sys.monitoring) populates chain via
walk_await_chain(task.get_coro()) at the moment of yield.
- add_async_await_run renders chain entries as PyFrameKey frames in the
stitched stack, with the [await] marker (and the (Task-N) suffix)
applied only to the innermost frame the timeline keeps. Filtering is
done first so the marker stays on the deepest user-code frame even
when the chain ends in stdlib asyncio frames that should_trace
rejects. Falls back to the original single-frame stack when chain is
empty.
Tests / demo
- tests/test_async_profiling.py — SuspendedTaskInfo unpacking test now
unpacks the new ``chain`` field (defaults to ``()``).
- test/timeline_view_demo.py — async phase now exercises a
three-deep user-async chain (_slow_io -> _fetch_one ->
_wait_for_data) so the multi-frame synthetic stack is visible in the
rendered timeline.
End-to-end on the demo: each of the 10 concurrent tasks now produces a
3-frame stack ``_slow_io -> _fetch_one -> [await] _wait_for_data
(Task-N)`` with 70 hits each, instead of the prior single-frame
collapsed run.
Bug fix while in here: is-leaf detection in add_async_await_run was
checked against the unfiltered chain. When the deepest frame got
filtered out (e.g. an asyncio.sleep frame at the bottom of the chain),
the [await] marker and the (Task-N) suffix were lost on the next-deepest
frame, and concurrent tasks awaiting at the same line collapsed into one
undifferentiated row. Filter first, then index.
Full test suite still 397 + 13 skipped; mypy and ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 1fd91c4 commit 205f15f
13 files changed
Lines changed: 1533 additions & 157 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
8 | | - | |
| 7 | + | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
27 | 41 | | |
28 | 42 | | |
29 | 43 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2457 | 2457 | | |
2458 | 2458 | | |
2459 | 2459 | | |
| 2460 | + | |
2460 | 2461 | | |
2461 | 2462 | | |
| 2463 | + | |
2462 | 2464 | | |
2463 | 2465 | | |
2464 | 2466 | | |
| |||
69969 | 69971 | | |
69970 | 69972 | | |
69971 | 69973 | | |
69972 | | - | |
69973 | | - | |
| 69974 | + | |
| 69975 | + | |
69974 | 69976 | | |
69975 | 69977 | | |
69976 | 69978 | | |
| |||
71795 | 71797 | | |
71796 | 71798 | | |
71797 | 71799 | | |
71798 | | - | |
| 71800 | + | |
71799 | 71801 | | |
71800 | 71802 | | |
71801 | 71803 | | |
| |||
71817 | 71819 | | |
71818 | 71820 | | |
71819 | 71821 | | |
| 71822 | + | |
| 71823 | + | |
| 71824 | + | |
| 71825 | + | |
| 71826 | + | |
| 71827 | + | |
| 71828 | + | |
| 71829 | + | |
| 71830 | + | |
| 71831 | + | |
| 71832 | + | |
| 71833 | + | |
| 71834 | + | |
| 71835 | + | |
| 71836 | + | |
| 71837 | + | |
| 71838 | + | |
| 71839 | + | |
| 71840 | + | |
| 71841 | + | |
| 71842 | + | |
| 71843 | + | |
| 71844 | + | |
| 71845 | + | |
| 71846 | + | |
| 71847 | + | |
| 71848 | + | |
| 71849 | + | |
| 71850 | + | |
| 71851 | + | |
| 71852 | + | |
| 71853 | + | |
| 71854 | + | |
| 71855 | + | |
| 71856 | + | |
| 71857 | + | |
| 71858 | + | |
| 71859 | + | |
| 71860 | + | |
| 71861 | + | |
| 71862 | + | |
| 71863 | + | |
| 71864 | + | |
| 71865 | + | |
| 71866 | + | |
| 71867 | + | |
| 71868 | + | |
| 71869 | + | |
| 71870 | + | |
| 71871 | + | |
| 71872 | + | |
| 71873 | + | |
| 71874 | + | |
| 71875 | + | |
| 71876 | + | |
| 71877 | + | |
| 71878 | + | |
| 71879 | + | |
| 71880 | + | |
| 71881 | + | |
| 71882 | + | |
| 71883 | + | |
| 71884 | + | |
| 71885 | + | |
| 71886 | + | |
| 71887 | + | |
| 71888 | + | |
| 71889 | + | |
| 71890 | + | |
| 71891 | + | |
| 71892 | + | |
| 71893 | + | |
| 71894 | + | |
| 71895 | + | |
| 71896 | + | |
| 71897 | + | |
| 71898 | + | |
| 71899 | + | |
| 71900 | + | |
| 71901 | + | |
| 71902 | + | |
| 71903 | + | |
| 71904 | + | |
| 71905 | + | |
| 71906 | + | |
| 71907 | + | |
| 71908 | + | |
| 71909 | + | |
| 71910 | + | |
| 71911 | + | |
| 71912 | + | |
| 71913 | + | |
| 71914 | + | |
| 71915 | + | |
| 71916 | + | |
| 71917 | + | |
| 71918 | + | |
| 71919 | + | |
| 71920 | + | |
| 71921 | + | |
| 71922 | + | |
| 71923 | + | |
| 71924 | + | |
| 71925 | + | |
| 71926 | + | |
| 71927 | + | |
| 71928 | + | |
| 71929 | + | |
| 71930 | + | |
| 71931 | + | |
| 71932 | + | |
| 71933 | + | |
| 71934 | + | |
| 71935 | + | |
| 71936 | + | |
| 71937 | + | |
| 71938 | + | |
| 71939 | + | |
| 71940 | + | |
| 71941 | + | |
| 71942 | + | |
| 71943 | + | |
| 71944 | + | |
| 71945 | + | |
| 71946 | + | |
| 71947 | + | |
| 71948 | + | |
| 71949 | + | |
| 71950 | + | |
| 71951 | + | |
| 71952 | + | |
| 71953 | + | |
| 71954 | + | |
| 71955 | + | |
| 71956 | + | |
| 71957 | + | |
| 71958 | + | |
| 71959 | + | |
| 71960 | + | |
| 71961 | + | |
| 71962 | + | |
| 71963 | + | |
| 71964 | + | |
| 71965 | + | |
| 71966 | + | |
| 71967 | + | |
| 71968 | + | |
| 71969 | + | |
| 71970 | + | |
| 71971 | + | |
| 71972 | + | |
| 71973 | + | |
| 71974 | + | |
| 71975 | + | |
| 71976 | + | |
| 71977 | + | |
| 71978 | + | |
| 71979 | + | |
| 71980 | + | |
| 71981 | + | |
| 71982 | + | |
| 71983 | + | |
| 71984 | + | |
| 71985 | + | |
| 71986 | + | |
| 71987 | + | |
| 71988 | + | |
| 71989 | + | |
| 71990 | + | |
| 71991 | + | |
| 71992 | + | |
| 71993 | + | |
| 71994 | + | |
| 71995 | + | |
| 71996 | + | |
| 71997 | + | |
| 71998 | + | |
| 71999 | + | |
| 72000 | + | |
| 72001 | + | |
| 72002 | + | |
| 72003 | + | |
| 72004 | + | |
| 72005 | + | |
| 72006 | + | |
| 72007 | + | |
| 72008 | + | |
| 72009 | + | |
| 72010 | + | |
| 72011 | + | |
| 72012 | + | |
| 72013 | + | |
| 72014 | + | |
| 72015 | + | |
| 72016 | + | |
| 72017 | + | |
| 72018 | + | |
| 72019 | + | |
| 72020 | + | |
| 72021 | + | |
| 72022 | + | |
| 72023 | + | |
| 72024 | + | |
| 72025 | + | |
| 72026 | + | |
| 72027 | + | |
| 72028 | + | |
| 72029 | + | |
| 72030 | + | |
| 72031 | + | |
| 72032 | + | |
| 72033 | + | |
| 72034 | + | |
| 72035 | + | |
| 72036 | + | |
| 72037 | + | |
| 72038 | + | |
71820 | 72039 | | |
71821 | 72040 | | |
71822 | 72041 | | |
| |||
72087 | 72306 | | |
72088 | 72307 | | |
72089 | 72308 | | |
72090 | | - | |
| 72309 | + | |
72091 | 72310 | | |
72092 | 72311 | | |
72093 | 72312 | | |
| |||
72205 | 72424 | | |
72206 | 72425 | | |
72207 | 72426 | | |
| 72427 | + | |
72208 | 72428 | | |
72209 | 72429 | | |
72210 | 72430 | | |
| |||
72530 | 72750 | | |
72531 | 72751 | | |
72532 | 72752 | | |
| 72753 | + | |
72533 | 72754 | | |
72534 | 72755 | | |
72535 | 72756 | | |
| |||
0 commit comments