Skip to content

CLAUDE.md

Latest commit

 

History

History
107 lines (68 loc) · 8.43 KB

CLAUDE.md

File metadata and controls

107 lines (68 loc) · 8.43 KB
Raw

Project: ROSS

Rensselaer's Optimistic Simulation System — a parallel discrete-event simulator (PDES) written in C, using MPI for distribution. Models are collections of Logical Processes (LPs) that exchange timestamped events. The Time Warp mechanism allows optimistic execution with rollback; ROSS implements this via reverse computation (each event handler has a paired reverse handler that undoes its effect) rather than state saving. Downstream consumers include CODES (which links ROSS via pkg-config) and RISA (in-situ analysis submodule).

Build

Typical manual configure:

cmake -S . -B build -DROSS_BUILD_MODELS=ON -DCMAKE_BUILD_TYPE=Debug
cmake --build build -j
cmake --install build

Models are not built by default — pass -DROSS_BUILD_MODELS=ON to build the bundled phold. External models should be standalone CMake projects that consume ROSS via find_package(ROSS); see models/README.md. The older symlink-into-models/ workflow still works during a deprecation period but is no longer the documented path. Test discovery is gated by ROSS_BUILD_TESTING (defaults ON at top level, inherits from BUILD_TESTING if the parent set it; defaults OFF under add_subdirectory()/FetchContent — parent must opt in).

MPI is required and auto-discovered via find_package(MPI) — do not set CC=mpicc or -DCMAKE_C_COMPILER=mpicc. For non-standard installs, hint with -DMPI_HOME=... or module load <mpi> before configuring. The top-level CMakeLists.txt does per-arch detection via CMAKE_SYSTEM_PROCESSOR and falls back to a gtod clock if unrecognized. Set -DROSS_CLOCK_OVERRIDE=YES to force the gtod clock.

Default is a static library. Pass -DROSS_BUILD_SHARED_LIBS=ON to build shared. The option defaults from BUILD_SHARED_LIBS if a parent project set it (e.g., a superbuild), otherwise OFF.

Running a model

Models accept --synch=N to pick the scheduler:

  • 1 sequential, 2 conservative, 3 optimistic, 4 optimistic debug (single-rank), 5 optimistic realtime, 6 rollback-check (single-rank, runs forward then reverse-executes all events to catch missing reverse handlers).
./phold --synch=1
mpirun -np 2 ./phold --synch=3 --extramem=100000
./phold --args-file=sample.txt     # flags one-per-line

--synch=6 is the go-to when adding/modifying an event handler: it surfaces reverse-computation bugs that optimistic mode might hide.

Tests

ctest --test-dir build          # full suite
ctest --test-dir build -R phold_SCHED_Optimistic   # single test by regex
ctest --test-dir build -V -R <name>                # with output

Tests live in models/phold/CMakeLists.txt and are generated by the ROSS_TEST_SCHEDULERS and ROSS_TEST_INSTRUMENTATION functions in models/CMakeLists.txt. Each new model variant (e.g. phold_comm_test, phold_gvt_hook_test) gets the full scheduler matrix applied via these functions. When adding a model or test variant, call these helpers rather than hand-writing ADD_TEST.

ROSS_BUILD_TESTING is the gate for ctest discovery: it controls whether include(CTest) runs at all, and the helpers above early-return when it's OFF so models build without registering their tests. Phold binaries still build under -DROSS_BUILD_TESTING=OFF -DROSS_BUILD_MODELS=ON; only test registration is suppressed.

Architecture

Simulation hierarchy (top to bottom)

  • PE (tw_pe, one per MPI rank) — owns the priority queue, the event buffer pool, network driver, and GVT state. Code: tw-pe.c, tw-sched.c.
  • KP (tw_kp) — Kernel Process, a rollback granularity unit. Multiple KPs per PE; events are committed/rolled back at KP scope. Code: tw-kp.c.
  • LP (tw_lp) — the model-facing object. Each LP has a tw_lptype vtable (init/pre-run/event/rc-event/commit/final/map). Code: tw-lp.c.

Type definitions: core/ross-types.h. Public API entry point: core/ross.h.

Pluggable components

Compiled-in choices are hard-coded in CMakeLists.txt:

Optional subsystems:

  • AVL tree vs hash for remote-event dedup in optimistic mode (AVL_TREE=ON default; files avl_tree.c, hash-quadratic.c).
  • RIO — checkpoint/restart (core/rio/), opt-in via USE_RIO=ON.
  • Instrumentation (core/instrumentation/) — ships ROSS-internal analysis LPs; enabled by runtime flags --engine-stats, --event-trace, --model-stats, --kp-data, --lp-data.
  • check-revent (core/check-revent/) — backs --synch=6 sequential rollback validation.

Damaris/RISA in-situ vis is currently disabled — the USE_DAMARIS CMake option has been removed pending a follow-up that strips the inert C source paths and the core/risa/ submodule.

Memory discipline

In ROSS_INTERNAL scope (core/*.c files), malloc/calloc/realloc/free/strdup are macro-poisoned in core/ross-base.h — use tw_calloc and the event buffer pool (tw_event_new / tw_event_send) instead. Events come from a preallocated pool sized by --nevents and --extramem. Don't allocate from the system heap inside event handlers.

Writing a model (forward + reverse)

Every event handler must have a paired reverse handler. The reverse handler has to exactly undo state mutations the forward one performed — including any RNG draws (call the same tw_rand_* reverse variant) and any tw_event_send calls (rollback walks them automatically, but the count must match). The tw_bf bitfield argument to the forward handler is the standard place to record which branches ran so the reverse handler can conditionally undo.

Code style

The de facto C style in core/ is:

  • Naming: snake_case with a tw_ prefix for the public ROSS surface — functions (tw_pe_init), structs (tw_pe), typedefs. Function-pointer typedefs end in _f (e.g. init_f, event_f, revent_f, map_f). Local variables are plain snake_case.
  • typedef pattern: struct tw_foo { ... }; typedef struct tw_foo tw_foo; — struct tag and typedef name match, both tw_-prefixed.
  • Header guards: #ifndef INC_<filename>_h / #define INC_<filename>_h / #endif. Not #pragma once.
  • static on file-local functions: applied consistently. Internal helpers in a .c file should be static.
  • Includes: system headers first (stdio/stdlib/string/assert/mpi), then project headers. Grouped, not strictly alphabetized.
  • Line length: soft ~100 chars. No hard cap.
  • Comments: both // and /* */ appear; /** ... */ Doxygen-style is used for some function-level docs but isn't required.

Consumer-facing API surface

Installed headers live under <prefix>/include/ross/. External consumers can #include <ross.h> (the umbrella) or any top-level ross-*.h. The supported entry-point set is intentionally narrow:

  • Supported: <ross.h> and any top-level <ross-*.h> (e.g. <ross-extern.h>, <ross-types.h>).
  • Unsupported but installed: *-internal.h siblings (ross-gvt-internal.h, ross-random-internal.h) ship because supported public headers transitively #include them. Do not include directly. Subdirectory headers (<instrumentation/...>, <check-revent/...>, <queue/...>, <rio/...>) and other internals (<buddy.h>, <lz4.h>, <hash-quadratic.h>) likewise ship but are not part of the supported surface and may be hidden in the future.

CMake consumers do find_package(ROSS REQUIRED) + target_link_libraries(... ROSS::ROSS) and inherit the include path automatically. pkg-config consumers do pkg_check_modules(ROSS REQUIRED IMPORTED_TARGET ross) and get the same path via ${ROSS_INCLUDE_DIRS}.

Consumer compatibility (CODES)

CODES discovers ROSS via pkg_check_modules(ROSS REQUIRED IMPORTED_TARGET ross) using ross.pc. It relies on the ROSS_INCLUDE_DIRS variable and links PkgConfig::ROSS. Any install-layout change (header paths, pkg-config contents) must preserve that surface or CODES breaks.