Skip to content

performance-profiling.md

Latest commit

 

History

History
144 lines (113 loc) · 6.91 KB

performance-profiling.md

File metadata and controls

144 lines (113 loc) · 6.91 KB
Raw

Structured performance profiling

Meganeura has two complementary profiling outputs:

  • Perfetto traces show CPU spans, submissions, waits, and GPU passes on a timeline.
  • Structured session profiles retain repeated hardware timestamp samples and attach each dispatch to its selected pipeline, workgroup geometry, execution phase, and coarse kernel family.

The structured profile is the default tool for explaining a benchmark gap. It is reproducible across Vulkan and Metal, can be collected by Inferena after the ordinary benchmark without changing its reported latency, and produces JSON that can be compared across revisions. RenderDoc and vendor-specific capture tools remain useful for a single unexplained shader, but are not part of the benchmark protocol.

Quick check

Run the small self-contained example:

MEGANEURA_GPU_TIMING=1 \
  cargo run --release --example profile_session -- gap-profile.json

The example first measures the ordinary grouped-pass execution, then collects five instrumented samples. The JSON records both times and their ratio.

For paper workloads, use Inferena's profile option so the profile and normal measurement share the exact graph, inputs, precision policy, and revision:

INFERENA_MEGANEURA_PATH=../meganeura \
  ./run.sh --frameworks meganeura --model Whisper-tiny \
  --profile --profile-samples 5 --results-dir results/gap-study

Inferena writes one sidecar per execution mode under results/gap-study/profiles/.

Timing contract

Set MEGANEURA_GPU_TIMING=1 before constructing the first GPU context. Blade then allocates hardware timestamp queries. During structured capture, Meganeura deliberately records one compute pass per timed plan dispatch — see "Plans larger than the timestamp budget" below for what happens to the rest. Once the completion fence signals, Blade resolves every pass start and the final completion timestamp directly onto the process monotonic clock; no follow-up execution is needed.

A dispatch interval starts at its calibrated pass-start timestamp and ends at the next pass start, or at the submission completion timestamp for the final pass. It therefore includes work and any inter-pass queue gap before the next timestamp. This is appropriate for ranking end-to-end dispatch costs, but it is not an instruction-level kernel metric.

One-pass-per-dispatch execution is more intrusive than normal execution, which groups dispatches and uses inline barriers. Never substitute the profiled wall time for benchmark latency. The artifact includes:

  • the normal benchmark median supplied by the caller;
  • every profiled wall-time and timestamped-GPU sample;
  • the instrumentation wall-time ratio;
  • per-dispatch median, quartiles, and raw samples;
  • phase and family aggregates;
  • pipeline variants and driver-reported executable statistics, when available;
  • device, driver, plan, barrier, and memory metadata.

The family shares use the sum of each dispatch's median so they add to 100%. The separate median of each family's per-run total is retained as well.

Capture API

Call meganeura::profiler::capture_session_profile after the normal benchmark and pass the normal median through CaptureOptions::unprofiled_median_ms. The input-preparation closure runs once before each replay.

The collector describes dispatches in the compiled execution plan. Disable optimizer, gradient-accumulation, and gradient-clipping passes before capture; those passes are appended by the runtime and do not have plan metadata. The collector rejects a timestamp-count mismatch instead of silently assigning an auxiliary pass to the wrong shader.

Plans larger than the timestamp budget

Blade writes at most limits::PASS_COUNT timestamps per submission and silently skips the rest, so no single replay can measure a larger plan — and a decode or training step of a real model runs well past that.

The collector therefore splits such a plan into windows of dispatch indices and replays the session once per window. The window's dispatches each get their own timestamped pass; everything outside it still executes, but batched into one grouped pass on either side, which keeps the plan's barriers while costing only two of the timestamp slots however many dispatches are out there. The preparation closure runs before every replay, so each window observes the same execution, and the per-dispatch results are stitched back together. Every dispatch is still measured samples times; only the number of replays grows.

measurement.window_count reports how many replays one sample took and is 1 for a plan that fits, in which case capture behaves exactly as it did before. CaptureOptions::max_timed_passes_per_replay lowers the budget below Blade's limit, cutting the plan into more and smaller windows so that each replay perturbs the schedule less.

When window_count exceeds 1, each profiled_wall_samples_ms entry is the sum of all replay wall times needed for that sample. Consequently, instrumentation_wall_ratio reports total capture cost rather than the perturbation of one execution. gpu_total_samples_ms likewise sums a sample's windows, so both aggregates still describe one complete profile sample.

Escalating beyond the built-in profile

Start with the largest family and dispatch deltas across GPUs or revisions. Use driver pipeline statistics to check register pressure and spilling, then inspect the generated WGSL/SPIR-V for the small set of suspect pipelines.

Use RenderDoc only when resource bindings, barriers, generated shader state, or a driver-specific anomaly needs visual inspection. For instruction throughput, occupancy, cache behavior, or tensor-core utilization, use the vendor tool for the affected platform (for example Nsight, Radeon GPU Profiler, or Xcode GPU capture). Those captures are supporting forensic evidence, not portable benchmark artifacts.

State-checked training localization

examples/profile_training.rs provides a fixed, synthetic F+L+B roster with normal timing blocks before and after capture, full profiled-result comparisons before the next profiled execution, exact dispatch contracts, provenance and telemetry. It enables timestamp queries through GpuOptions, with no environment variable required. The retained protocol and results separate localization from optimizer-backed timing and candidate acceptance.

Those records show why both controls matter: all 45 profiled full states match bitwise, but short normal timing blocks can still drift badly. Before/after normal samples are not interleaved candidate comparisons. Timing-enabled contexts also do not reproduce the profiler-disabled production configuration.

The current phase label is the scheduled prefix/suffix around the last loss dispatch. Independent gradient seeds can be hoisted into the prefix. Use shader direction, requires_full_precision and node origins for semantic attribution; do not assume everything labeled forward belongs to the undifferentiated graph.