Skip to content

Instrumentation

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

Instrumentation

README.md

NeRArchSim Neural Rendering Instrumentation

Instrumentation Overview

This folder provides tools to collect execution traces from neural rendering models (e.g., Nerfstudio) and to transform/visualize those traces for NeRArchSim analysis.

Folder structure

  • nerfstudio_vendor/: files used to do instrumentation in nerfstudio
    • instrumentation/tracing.py and trace_config.json
    • scripts/eval.py (evaluation with tracing)
    • scripts/train.py (training with tracing support)
  • operator_mapping.py: maps traced functions to normalized operator taxonomy
  • dag_to_operators_integration.py: converts traced DAGs to realistic operators
  • plot_transformed_operators.py: visualizes transformed operator graphs
  • plot_dot_subgraph.py: extract and render a specific subgraph cluster from a grouped DOT file

For a quick end-to-end example (training -> trace -> analyze), see the root README.md. This page contains a more detailed setup, training, and evaluation guide.

Table of Contents

Quick Setup

See the root README.md for Quick setup (Nerfstudio + tracing). This page provides the full detailed setup, training, and evaluation guide.

Manual Setup

Setup CUDA

  • Install CUDA-11.8
wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
chmod +x ./cuda_11.8.0_520.61.05_linux.run
./cuda_11.8.0_520.61.05_linux.run --silent --tmpdir=. --toolkit --toolkitpath=/change/this/path/cuda-11.8/
export PATH=/change/this/path/cuda-11.8/bin${PATH:+:${PATH}}

Setup Nerfstudio

Install dependencies

# Use local environment (recommended) or global environment
conda create --prefix $PWD/nerarchsim python=3.8 -y
conda activate $PWD/nerarchsim

pip install --upgrade pip setuptools
pip install torch==2.1.2+cu118 torchvision==0.16.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118
conda install -c nvidia/label/cuda-11.8.0 cuda-toolkit -y
pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
conda install -c conda-forge graphviz python-graphviz -y

Install Nerfstudio

git clone https://github.com/nerfstudio-project/nerfstudio.git
cd nerfstudio && git checkout 50e0e3c70c775e89333256213363badbf074f29d && pip install -e . && cd ..

Copy instrumentation files

# Optionally copy the vendor reference files into your nerfstudio checkout
mkdir nerfstudio/nerfstudio/instrumentation/
cp -f Instrumentation/nerfstudio_vendor/instrumentation/tracing.py nerfstudio/nerfstudio/instrumentation/ || true
cp -f Instrumentation/nerfstudio_vendor/instrumentation/trace_config.json nerfstudio/nerfstudio/instrumentation/ || true
cp -f Instrumentation/nerfstudio_vendor/scripts/eval.py nerfstudio/nerfstudio/scripts/ || true
cp -f Instrumentation/nerfstudio_vendor/scripts/train.py nerfstudio/nerfstudio/scripts/train_instrumented.py || true
cp -f Instrumentation/nerfstudio_vendor/utils/eval_utils.py nerfstudio/nerfstudio/utils/ || true
cp -f Instrumentation/nerfstudio_vendor/utils/train_utils.py nerfstudio/nerfstudio/utils/ || true
cd nerfstudio && pip install -e . && cd ..

Download dataset

Two standard datasets are used: the NeRF-synthetic (Blender) scenes and the Mip-NeRF 360 real scenes. Both are publicly available.

mkdir -p /tmp/nerf && cd /tmp/nerf

# NeRF-synthetic (Blender) scenes: chair, drums, ficus, hotdog, lego, materials, mic, ship.
# Official release: https://www.matthewtancik.com/nerf (Google Drive "nerf_synthetic.zip").
# Download nerf_synthetic.zip into /tmp/nerf, then:
unzip nerf_synthetic.zip -d nerf_synthetic

# Mip-NeRF 360 real scenes: bicycle, bonsai, counter, garden, kitchen, room, stump.
wget http://storage.googleapis.com/gresearch/refraw360/360_v2.zip
unzip 360_v2.zip -d mip360

cd -  # Return to the NeRArchSim directory

Reconstruct a scene

Pick the dataparser by scene type: synthetic (Blender) scenes use blender-data; real Mip-NeRF 360 (COLMAP) scenes use colmap.

# --- Synthetic (Blender) scenes: use the blender-data dataparser ---
# Train three pipelines on the same scene (adjust data path as needed)
CUDA_VISIBLE_DEVICES=0 nohup ns-train vanilla-nerf --output-dir output_result --max-num-iterations=10000 --data /tmp/nerf/nerf_synthetic/mic blender-data > vanilla-nerf-mic.log &
CUDA_VISIBLE_DEVICES=1 nohup ns-train instant-ngp  --output-dir output_result --max-num-iterations=10000 --data /tmp/nerf/nerf_synthetic/mic blender-data > instant-ngp-mic.log &
CUDA_VISIBLE_DEVICES=2 nohup ns-train splatfacto  --output-dir output_result --max-num-iterations=100000 --data /tmp/nerf/nerf_synthetic/mic blender-data > splatfacto-mic.log &
# --- Real Mip-NeRF 360 (COLMAP) scenes: use the colmap dataparser, NOT blender-data ---
# The scenes ship a COLMAP reconstruction (images/, images_2|4|8/, sparse/0/) instead of
# transforms_*.json. Point colmap at sparse/0 and select a prebuilt downscale folder.
# --downscale-rounding-mode round matches how the prebuilt images_N folders were generated
# (the default, floor, can mismatch camera vs image dimensions by one pixel).
CUDA_VISIBLE_DEVICES=0 nohup ns-train vanilla-nerf --output-dir output_result --experiment-name bicycle \
  --max-num-iterations=30000 --data /tmp/nerf/mip360/bicycle \
  colmap --colmap-path sparse/0 --images-path images --downscale-factor 4 --downscale-rounding-mode round \
  > vanilla-nerf-bicycle.log &

Note: vanilla-nerf has no scene contraction, so quality on unbounded 360 scenes is limited regardless of step count; instant-ngp / splatfacto are better suited to those scenes. See scripts/run_scene_validation.sh for an end-to-end multi-scene driver (train -> instrumented eval/trace -> transform -> validate).

Training with Tracing

# Train with tracing enabled at specific iterations
python nerfstudio/nerfstudio/scripts/train_instrumented.py vanilla-nerf \
  --output-dir output_result \
  --max-num-iterations=1000 \
  --data /tmp/nerf/nerf_synthetic/mic blender-data \
  --enable-trace \
  --trace-iterations 100 500 1000 \
  --trace-config-path nerfstudio/nerfstudio/instrumentation/trace_config.json

# This will save traces at iterations 100, 500, and 1000 to:
# output_result/vanilla-nerf-*/traces/execution_dag_iter_*.pkl

Render a scene / Collect Traces

# Locate trained configs for each pipeline
CONFIG_VANILLA=$(find output_result -path "*vanilla-nerf*/*/config.yml" | head -n1)
CONFIG_INGP=$(find output_result -path "*instant-ngp*/*/config.yml" | head -n1)
CONFIG_SPLAT=$(find output_result -path "*splatfacto*/*/config.yml" | head -n1)

# Evaluate one image with tracing enabled (writes execution_dag.pkl)
# vanilla-nerf
OUT=traces/vanilla_$(date +%Y%m%d_%H%M%S)
mkdir -p "$OUT"
DISABLE_TRACE_PLOT=1 \
ns-eval \
  --load-config "$CONFIG_VANILLA" \
  --render-output-path "$OUT" \
  --enable-trace \
  --trace-config-path nerfstudio/nerfstudio/instrumentation/trace_config.json \
  --eval-image-indices 0

# instant-ngp
OUT=traces/instantngp_$(date +%Y%m%d_%H%M%S)
mkdir -p "$OUT"
DISABLE_TRACE_PLOT=1 \
ns-eval \
  --load-config "$CONFIG_INGP" \
  --render-output-path "$OUT" \
  --enable-trace \
  --trace-config-path nerfstudio/nerfstudio/instrumentation/trace_config.json \
  --eval-image-indices 0

# splatfacto (3D Gaussian Splatting)
OUT=traces/splatfacto_$(date +%Y%m%d_%H%M%S)
mkdir -p "$OUT"
DISABLE_TRACE_PLOT=1 \
ns-eval \
  --load-config "$CONFIG_SPLAT" \
  --render-output-path "$OUT" \
  --enable-trace \
  --trace-config-path nerfstudio/nerfstudio/instrumentation/trace_config.json \
  --eval-image-indices 0

# Traces: $OUT/execution_dag.pkl in each output directory

Key flags and env vars

  • --load-config PATH: config.yml produced by ns-train
  • --render-output-path PATH: where images and trace files are saved
  • --enable-trace: turn on instrumentation to record the execution DAG
  • --trace-config-path PATH: JSON with functions to trace (defaults to Nerfstudio’s instrumentation directory)
  • --eval-image-indices ...: restrict which eval images to run; pass indices as separate integers (e.g., 0 1 2).
  • DISABLE_TRACE_PLOT=1 (env): disables plotting pngs during trace to speed up runs and avoid GUI deps; the trace execution_dag.pkl is still written.

Transform and Visualize Operators

# Transform DAG to realistic operators and generate visualizations
python Instrumentation/plot_transformed_operators.py "$OUT/execution_dag.pkl"

Outputs include coarse/fine graphs and a text summary of operator characteristics.

Optional: plot a DOT subgraph cluster

If you have a grouped DOT (e.g., execution_dag_grouped.dot), you can extract and render a specific cluster:

python Instrumentation/plot_transformed_operators.py "$OUT/execution_dag.pkl" \
  --plot-dot-subgraph \
  --dot execution_dag_grouped.dot \
  --cluster-index coarse:0 \
  --out-prefix execution_dag_component0

(Or call the helper directly: python Instrumentation/plot_dot_subgraph.py --dot ... --cluster-index ... --out-prefix ...)

Custom Pipeline Instrumentation

  • In Instrumentation/nerfstudio_vendor/instrumentation/trace_config.json, specify functions to trace.
  • Operator taxonomy categories (for mapping and visualization):
    • SAMPLING: Ray sampling and point generation
    • ENCODING: Positional, hash, or frequency encoding
    • FIELD_COMPUTATION: MLP, network inference
    • BLENDING: Volume rendering, alpha compositing

Test instrumentation quickly:

ns-eval --load-config your_config.yml --enable-trace --eval-image-indices 0

Verify the output contains your operators in execution_dag.pkl.

Testing and Validation

# Minimal smoke test for NeRArchSim core
python -c "import rendersim_cpp as rs; print('ok')"

For quick start and an end-to-end example command, see the root README.md (ns-eval example and analyze a trace).