Arcaflow Plugin: rusty-comms IPC Benchmark

An Arcaflow plugin that wraps the rusty-comms IPC benchmark suite. It executes the ipc-benchmark binary, parses the structured JSON output, and returns strongly-typed latency and throughput results to the Arcaflow engine.

Supported IPC Mechanisms

Mechanism	CLI Name	Description
Unix Domain Socket	uds	Local inter-process socket
TCP Socket	tcp	TCP loopback socket
Shared Memory	shm	POSIX shared memory (with optional direct mode)
POSIX Message Queue	pmq	Kernel-managed message queue

Use all to benchmark every mechanism in a single run.

Building the Container Image

The plugin ships as a multi-stage container image that compiles ipc-benchmark from source (Rust) and bundles it with the Python plugin code.

podman build -t arcaflow-plugin-rusty-comms .

Running with Arcaflow

Prerequisites

• Arcaflow engine binary (or built from source)
• A container runtime: Podman (default) or Docker

Quick Start

1. Build the container image (see above).

2. Create an input file (or use inputs/example.yaml):

tests:
  - mechanisms:
      - uds
    message_size: 1024
    msg_count: 10000

3. Run the workflow:

arcaflow --input inputs/example.yaml \
         --config config.yaml \
         --workflow workflow.yaml

The engine deploys the plugin container, runs the benchmark, and prints the structured results to stdout.

Pre-built Test Suites

Two test input files are included for comprehensive benchmarking:

• quick-rusty-comms-arcaflow-testing.yaml — A small set of tests across UDS, TCP, PMQ, and SHM for quick validation.
• comprehensive-rusty-comms-arcaflow-testing.yaml — A full benchmark matrix covering multiple mechanisms, message sizes, blocking/async modes, concurrency levels, one-way vs round-trip, and send-delay latency profiling.

Example:

arcaflow --input comprehensive-rusty-comms-arcaflow-testing.yaml \
         --config config.yaml

Autogenerated Input/Output Documentation by Arcaflow-Docsgen Below

Run IPC Benchmark (run-benchmark)

Executes one or more rusty-comms IPC benchmark test runs and returns structured latency and throughput results.

Input

Type:	scope
Root object:	InputParams
Properties	tests (list[reference[TestRunConfig]]) Name: Test Runs Description: List of benchmark test configurations. Each entry specifies mechanisms and parameters for one run. Required: Yes Type: list[reference[TestRunConfig]] Minimum items: 1 List items Type: reference[TestRunConfig] Referenced object: TestRunConfig
Objects InputParams (object) Type: object Properties tests (list[reference[TestRunConfig]]) Name: Test Runs Description: List of benchmark test configurations. Each entry specifies mechanisms and parameters for one run. Required: Yes Type: list[reference[TestRunConfig]] Minimum items: 1 List items Type: reference[TestRunConfig] Referenced object: TestRunConfig TestRunConfig (object) Type: object Properties blocking (bool) Name: Blocking Mode Description: Use blocking I/O. Defaults to true when not specified. Set to false for async Tokio mode. Required: No Type: bool buffer_size (int) Name: Buffer Size Description: Internal buffer size in bytes for message queues and shared memory. Required: No Type: int Minimum: 1 client_affinity (int) Name: Client CPU Affinity Description: Pin the client (sender) process to this CPU core. Required: No Type: int Minimum: 0 concurrency (int) Name: Concurrency Description: Number of concurrent workers/threads. Required: No Type: int Minimum: 1 continue_on_error (bool) Name: Continue on Error Description: Continue running remaining mechanisms if one fails. Required: No Type: bool duration (string) Name: Duration Description: Fixed test duration, e.g. '30s', '5m', '1h'. Takes precedence over message count. Required: No Type: string extra_args (list[string]) Name: Extra Arguments Description: Additional CLI arguments passed directly to the ipc-benchmark binary for advanced usage. Required: No Type: list[string] List items Type: string host (string) Name: TCP Host Description: Host address for TCP socket tests. Default: 127.0.0.1. Required: No Type: string include_first_message (bool) Name: Include First Message Description: Include the first (canary) message in results instead of discarding it. Required: No Type: bool mechanisms (list[enum[string]]) Name: IPC Mechanisms Description: IPC mechanisms to benchmark. Use 'all' to test every available mechanism. Valid values: uds, shm, tcp, pmq, all. Required: Yes Type: list[enum[string]] Minimum items: 1 List items Type: enum[string] Values all: all pmq: pmq shm: shm tcp: tcp uds: uds message_size (int) Name: Message Size Description: Message payload size in bytes. Required: No Type: int Minimum: 1 msg_count (int) Name: Message Count Description: Number of messages to send. Ignored when duration is set. Required: No Type: int Minimum: 1 one_way (bool) Name: One-Way Test Description: Run one-way latency tests. If neither one_way nor round_trip is set, both run by default. Required: No Type: bool percentiles (list[float]) Name: Percentiles Description: Percentile values to compute, e.g. [50.0, 95.0, 99.0, 99.9]. Required: No Type: list[float] List items Type: float pmq_priority (int) Name: PMQ Priority Description: Message priority for POSIX Message Queues. Required: No Type: int Minimum: 0 port (int) Name: TCP Port Description: Port for TCP socket tests. Default: 8080. Required: No Type: int Minimum: 1 quiet (bool) Name: Quiet Mode Description: Silence informational console output. Required: No Type: bool round_trip (bool) Name: Round-Trip Test Description: Run round-trip latency tests. If neither one_way nor round_trip is set, both run by default. Required: No Type: bool send_delay (string) Name: Send Delay Description: Delay between sends, e.g. '10ms', '50us'. Useful for latency-focused testing under controlled load. Required: No Type: string server_affinity (int) Name: Server CPU Affinity Description: Pin the server (receiver) process to this CPU core. Required: No Type: int Minimum: 0 shm_direct (bool) Name: SHM Direct Memory Description: Use high-performance direct memory shared memory. Auto-enables blocking mode. Unix only, 8KB max payload. Required: No Type: bool warmup_iterations (int) Name: Warmup Iterations Description: Number of warmup messages before measurement starts. Required: No Type: int Minimum: 0

Outputs

error

Type:	scope
Root object:	ErrorOutput
Properties	error (string) Name: Error Description: Error message describing the failure. Required: Yes Type: string
Objects ErrorOutput (object) Type: object Properties error (string) Name: Error Description: Error message describing the failure. Required: Yes Type: string

success

Type:	scope
Root object:	SuccessOutput
Properties	metadata (reference[BenchmarkMetadata]) Name: Metadata Description: Run metadata and system information. Required: Yes Type: reference[BenchmarkMetadata] Referenced object: BenchmarkMetadata results (list[reference[BenchmarkResult]]) Name: Results Description: Per-mechanism benchmark results. Required: Yes Type: list[reference[BenchmarkResult]] List items Type: reference[BenchmarkResult] Referenced object: BenchmarkResult summary (reference[OverallSummary]) Name: Overall Summary Description: Aggregated summary across mechanisms. Required: Yes Type: reference[OverallSummary] Referenced object: OverallSummary
Objects BenchmarkMetadata (object) Type: object Properties system_info (reference[SystemInfo]) Name: System Info Description: System information. Required: Yes Type: reference[SystemInfo] Referenced object: SystemInfo timestamp (string) Name: Timestamp Description: ISO-8601 timestamp when run started. Required: Yes Type: string total_tests (int) Name: Total Tests Description: Number of mechanism tests executed. Required: Yes Type: int version (string) Name: Version Description: rusty-comms version. Required: Yes Type: string BenchmarkResult (object) Type: object Properties input_blocking (bool) Name: Input: Blocking Description: Original blocking flag from the test input. True = blocking, False = async, None = default. Required: No Type: bool input_concurrency (int) Name: Input: Concurrency Description: Original concurrency value from the test input. Required: No Type: int input_one_way (bool) Name: Input: One-Way Description: Original one_way flag from the test input. Required: No Type: bool input_round_trip (bool) Name: Input: Round-Trip Description: Original round_trip flag from the test input. Required: No Type: bool input_send_delay (string) Name: Input: Send Delay Description: Original send_delay value from the test input. Required: No Type: string input_shm_direct (bool) Name: Input: SHM Direct Description: Original shm_direct flag from the test input. Required: No Type: bool mechanism (string) Name: Mechanism Description: IPC mechanism name, e.g. UnixDomainSocket, SharedMemory, TcpSocket, PosixMessageQueue. Required: Yes Type: string one_way_results (reference[PerformanceMetrics]) Name: One-Way Results Description: One-way latency/throughput results. Required: No Type: reference[PerformanceMetrics] Referenced object: PerformanceMetrics round_trip_results (reference[PerformanceMetrics]) Name: Round-Trip Results Description: Round-trip latency/throughput results. Required: No Type: reference[PerformanceMetrics] Referenced object: PerformanceMetrics status (any) Name: Status Description: 'Success' or {'Failure': 'reason'}. Required: Yes Type: any summary (reference[BenchmarkSummary]) Name: Summary Description: Summary statistics for this mechanism. Required: Yes Type: reference[BenchmarkSummary] Referenced object: BenchmarkSummary system_info (reference[SystemInfo]) Name: System Info Description: System info captured during this test. Required: Yes Type: reference[SystemInfo] Referenced object: SystemInfo test_config (reference[TestConfiguration]) Name: Test Configuration Description: Configuration used for this test. Required: Yes Type: reference[TestConfiguration] Referenced object: TestConfiguration test_duration (map[string,int]) Name: Test Duration Description: Wall-clock duration as {secs, nanos}. Required: Yes Type: map[string,int] Key type Type: string Value type Type: int timestamp (string) Name: Timestamp Description: ISO-8601 timestamp of this test. Required: Yes Type: string BenchmarkSummary (object) Type: object Properties average_latency_ns (int) Name: Average Latency (ns) Description: Average latency in nanoseconds. Required: No Type: int average_throughput_mbps (float) Name: Average Throughput (MB/s) Description: Average throughput in megabytes per second. Required: Yes Type: float error_count (int) Name: Error Count Description: Number of errors during the benchmark. Required: Yes Type: int max_latency_ns (int) Name: Max Latency (ns) Description: Maximum latency in nanoseconds. Required: No Type: int min_latency_ns (int) Name: Min Latency (ns) Description: Minimum latency in nanoseconds. Required: No Type: int p95_latency_ns (int) Name: P95 Latency (ns) Description: 95th percentile latency in nanoseconds. Required: No Type: int p99_latency_ns (int) Name: P99 Latency (ns) Description: 99th percentile latency in nanoseconds. Required: No Type: int peak_throughput_mbps (float) Name: Peak Throughput (MB/s) Description: Peak throughput in megabytes per second. Required: Yes Type: float total_bytes_transferred (int) Name: Total Bytes Transferred Description: Total bytes transferred. Required: Yes Type: int total_messages_sent (int) Name: Total Messages Sent Description: Total messages sent across all test types. Required: Yes Type: int LatencyMetrics (object) Type: object Properties latency_type (string) Name: Latency Type Description: 'OneWay' or 'RoundTrip'. Required: Yes Type: string max_ns (int) Name: Max (ns) Description: Maximum observed latency in nanoseconds. Required: Yes Type: int mean_ns (int) Name: Mean (ns) Description: Mean latency in nanoseconds. Required: Yes Type: int median_ns (int) Name: Median (ns) Description: Median (P50) latency in nanoseconds. Required: Yes Type: int min_ns (int) Name: Min (ns) Description: Minimum observed latency in nanoseconds. Required: Yes Type: int percentiles (list[reference[PercentileValue]]) Name: Percentiles Description: Requested percentile values. Required: Yes Type: list[reference[PercentileValue]] List items Type: reference[PercentileValue] Referenced object: PercentileValue std_dev_ns (int) Name: Std Dev (ns) Description: Standard deviation of latency in nanoseconds. Required: Yes Type: int total_samples (int) Name: Total Samples Description: Number of latency samples collected. Required: Yes Type: int MechanismSummary (object) Type: object Properties average_throughput_mbps (float) Name: Average Throughput (MB/s) Description: Average throughput for this mechanism. Required: Yes Type: float mechanism (string) Name: Mechanism Description: IPC mechanism enum variant name. Required: Yes Type: string p95_latency_ns (int) Name: P95 Latency (ns) Description: P95 latency in nanoseconds. Required: No Type: int p99_latency_ns (int) Name: P99 Latency (ns) Description: P99 latency in nanoseconds. Required: No Type: int total_messages (int) Name: Total Messages Description: Total messages for this mechanism. Required: Yes Type: int OverallSummary (object) Type: object Properties fastest_mechanism (string) Name: Fastest Mechanism Description: Mechanism with highest throughput. Required: No Type: string lowest_latency_mechanism (string) Name: Lowest Latency Mechanism Description: Mechanism with lowest latency. Required: No Type: string mechanisms (map[string,reference[MechanismSummary]]) Name: Mechanisms Description: Per-mechanism summary keyed by display name. Required: Yes Type: map[string,reference[MechanismSummary]] Key type Type: string Value type Type: reference[MechanismSummary] Referenced object: MechanismSummary total_bytes (int) Name: Total Bytes Description: Sum of bytes across all mechanisms. Required: Yes Type: int total_errors (int) Name: Total Errors Description: Sum of errors across all mechanisms. Required: Yes Type: int total_messages (int) Name: Total Messages Description: Sum of messages across all mechanisms. Required: Yes Type: int PercentileValue (object) Type: object Properties percentile (float) Name: Percentile Description: Percentile level, e.g. 95.0 for P95. Required: Yes Type: float value_ns (int) Name: Value (ns) Description: Latency at this percentile in nanoseconds. Required: Yes Type: int PerformanceMetrics (object) Type: object Properties latency (reference[LatencyMetrics]) Name: Latency Description: Latency measurements (if collected). Required: No Type: reference[LatencyMetrics] Referenced object: LatencyMetrics throughput (reference[ThroughputMetrics]) Name: Throughput Description: Throughput measurements. Required: Yes Type: reference[ThroughputMetrics] Referenced object: ThroughputMetrics timestamp (string) Name: Timestamp Description: ISO-8601 timestamp of measurement. Required: Yes Type: string SuccessOutput (object) Type: object Properties metadata (reference[BenchmarkMetadata]) Name: Metadata Description: Run metadata and system information. Required: Yes Type: reference[BenchmarkMetadata] Referenced object: BenchmarkMetadata results (list[reference[BenchmarkResult]]) Name: Results Description: Per-mechanism benchmark results. Required: Yes Type: list[reference[BenchmarkResult]] List items Type: reference[BenchmarkResult] Referenced object: BenchmarkResult summary (reference[OverallSummary]) Name: Overall Summary Description: Aggregated summary across mechanisms. Required: Yes Type: reference[OverallSummary] Referenced object: OverallSummary SystemInfo (object) Type: object Properties architecture (string) Name: Architecture Description: CPU architecture, e.g. x86_64. Required: Yes Type: string benchmark_version (string) Name: Benchmark Version Description: rusty-comms version string. Required: Yes Type: string cpu_cores (int) Name: CPU Cores Description: Number of CPU cores. Required: Yes Type: int memory_gb (float) Name: Memory (GB) Description: System memory in gigabytes. Required: Yes Type: float os (string) Name: OS Description: Operating system name. Required: Yes Type: string rust_version (string) Name: Rust Version Description: Rust compiler version used to build. Required: Yes Type: string TestConfiguration (object) Type: object Properties buffer_size (int) Name: Buffer Size Description: Internal buffer size in bytes. Required: Yes Type: int concurrency (int) Name: Concurrency Description: Number of concurrent workers. Required: Yes Type: int duration (map[string,int]) Name: Duration Description: Configured duration as {secs, nanos} (None if count). Required: No Type: map[string,int] Key type Type: string Value type Type: int message_size (int) Name: Message Size Description: Payload size in bytes. Required: Yes Type: int msg_count (int) Name: Message Count Description: Configured message count (None if duration). Required: No Type: int one_way_enabled (bool) Name: One-Way Enabled Description: Whether one-way tests were enabled. Required: Yes Type: bool percentiles (list[float]) Name: Percentiles Description: Percentile levels computed. Required: Yes Type: list[float] List items Type: float round_trip_enabled (bool) Name: Round-Trip Enabled Description: Whether round-trip tests were enabled. Required: Yes Type: bool warmup_iterations (int) Name: Warmup Iterations Description: Warmup messages before measurement. Required: Yes Type: int ThroughputMetrics (object) Type: object Properties bytes_per_second (int) Name: Bytes/sec Description: Data transmission rate in bytes per second. Required: Yes Type: int duration_ns (int) Name: Duration (ns) Description: Measurement duration in nanoseconds. Required: Yes Type: int messages_per_second (int) Name: Messages/sec Description: Message transmission rate. Required: Yes Type: int total_bytes (int) Name: Total Bytes Description: Total bytes transmitted. Required: Yes Type: int total_messages (int) Name: Total Messages Description: Total messages transmitted. Required: Yes Type: int

Utilities

The utils/ directory contains helper scripts for multi-run benchmarking and result analysis:

• utils/run_benchmarks.sh — Builds the container and runs the comprehensive suite N times (default 5), producing an averaged CSV.
• utils/python/run_comprehensive.py — Python runner that executes multiple Arcaflow iterations and averages results.
• utils/python/parse_arcaflow_output.py — Parses Arcaflow engine output YAML into a CSV summary for analysis.

Running the Full Benchmark Suite

# Run 5 iterations and produce averaged CSV
./utils/run_benchmarks.sh

# Run 3 iterations
./utils/run_benchmarks.sh 3

# Parse existing outputs only (skip running)
./utils/run_benchmarks.sh --skip

Results are written to utils/out/comprehensive_averaged.csv.

Development

Running Tests

From the repository root:

PYTHONPATH=arcaflow_plugin_rusty_comms \
  python -m pytest tests/ -v

Or using unittest directly:

cd arcaflow_plugin_rusty_comms
python -m unittest discover -s ../tests -v

Dependencies

Managed with Poetry:

poetry install

Runtime dependency: arcaflow-plugin-sdk >= 0.14.0

License

Apache License, Version 2.0. See LICENSE for details.