Harmonia: Enhancing Data Placement and Migration in Hybrid Storage Systems via Multi-Agent Reinforcement Learning

Harmonia is a multi-agent online reinforcement learning (RL) framework for holistic data management in hybrid storage systems (HSS). Unlike prior work that optimizes either data placement or data migration in isolation, Harmonia jointly optimizes both policies in a coordinated manner using two lightweight, autonomous RL agents that adapt online to the current workload and HSS configuration.

Harmonia is described in our paper:

Rakesh Nadig, Vamanan Arulchelvan, Rahul Bera, Taha Shahroodi, Gagandeep Singh, Andreas Kosmas Kakolyris, İsmail Emir Yüksel, Mohammad Sadrosadati, Jisung Park, and Onur Mutlu. "Harmonia: Enhancing Data Placement and Migration in Hybrid Storage Systems via Multi-Agent Reinforcement Learning." In Proceedings of the 2026 International Conference on Supercomputing (ICS '26), Belfast, United Kingdom.

The full paper is available at https://arxiv.org/pdf/2503.20507

Repository Layout

harmonia/src/
  execute.py                      # Entry point: builds the environment + placement agent, trains, evaluates
  hybridstorage.py                # HSS model (devices, mapping table, metadata, I/O)
  hybridstorageenvironment.py     # TF-Agents environment for the placement agent
  migration.py                    # Migration logic (candidate selection, migration execution)
  migrationThread.py              # Sets up / trains / evaluates the migration agent
  migrationenvironment.py         # TF-Agents environment for the migration agent
  utils.py                        # Networks and evaluation helpers
driver/
  hss_lib.c                       # HSS block-device driver
  hss_lib.so                      # Compiled shared library
run_all.py                        # Batch-runs the workload suites and collects results

Prerequisites

• Python 3.11 (recommended)
• A Linux host with the target storage devices accessible as block devices (the driver opens device paths such as /dev/nvme0n1).

Python dependencies are pinned in requirements.txt:

tensorflow==2.15
tf-agents==0.19
tf-keras==2.15
pandas==2.2.2
tqdm==4.66.5
openpyxl==3.1.5
matplotlib==3.9.2
xlsxwriter

Install build tools if needed (Debian/Ubuntu):

sudo apt update
sudo apt install build-essential python3.11-dev

Installation

git clone <your-repo-url>
cd Harmonia
python3.11 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python setup.py develop

Building the HSS Driver

Harmonia uses a user-space HSS driver to interface with the real block devices. Build the shared library the harness loads:

cd driver
gcc -fPIC -shared -o hss_lib.so hss_lib.c

The driver opens the configured device paths (e.g., /dev/nvme0n1 for the fast device). Edit driver/hss_lib.c to match the device paths on your system, then rebuild. Running against real devices typically requires elevated privileges (sudo).

Workloads

Harmonia is evaluated on 25 data-intensive block-I/O traces drawn from SYSTOR17, RocksDB, YCSB, MLPerf Storage, LLM inference/training, and Baleen. These traces are openly available from their respective public sources. Once obtained, convert each trace to CSV and organize it by suite (e.g., YCSB_RocksDB/, YCSB/, SYSTOR/, Mixed/, MLPerf/, LLM/), then pass the trace path to execute.py.

Running

harmonia/src/execute.py takes the following positional arguments:

python3 harmonia/src/execute.py \
    <type_env> <workload_path> <so_path> <trace_type> <trace_length> \
    <migration_chunk_size> <experience_buffer_size> <migration_queue_size> \
    <resultfilename> <is_perf_optimized>

Argument	Description
type_env	HSS type. dual (two-device) is supported by execute.py.
workload_path	Path to the block-I/O trace (CSV).
so_path	Path to the compiled driver, e.g. driver/hss_lib.so.
trace_type	Numeric identifier for the workload suite.
trace_length	Number of I/O requests to process.
migration_chunk_size	Pages migrated per migration operation (e.g., 50).
experience_buffer_size	RL replay-buffer capacity (e.g., 1000).
migration_queue_size	Max pending migrations queued (e.g., 10).
resultfilename	Output .xlsx file for results.
is_perf_optimized	1 for a performance-optimized HSS, 0 for cost-optimized.

Optional hyperparameter flags: --rl_algo (DQN, DDQN, C51, REINFORCE, PPO; default C51), --batch, --lr, --eps, --gam, --buf_cap, --init_collect, --num_itr, --eval_itr.

Example:

python3 harmonia/src/execute.py dual YCSB_RocksDB/ssd-02.csv driver/hss_lib.so \
    1 1000000 50 1000 10 harmonia_results.xlsx 1

Saving and Loading Policies

Training saves both agents. The placement agent (in execute.py) saves its model and policy:

tf.saved_model.save(tf_agent, "harmonia_agent")
policy_saver.PolicySaver(tf_agent.policy).save("harmonia_policy")

The migration agent (in migrationThread.py) saves its own model and policy to migration_agent / migration_policy. During evaluation, the saved policies are reloaded for inference:

harmonia_saved_policy = tf.saved_model.load("harmonia_policy")
migration_saved_policy = tf.saved_model.load("migration_policy")

Ensure each policy directory has been fully written before attempting to load it.

Citation

If you use Harmonia in your research, please cite:

@inproceedings{nadig2026harmonia,
  title={Harmonia: Enhancing Data Placement and Migration in Hybrid Storage Systems via Multi-Agent Reinforcement Learning},
  author={Nadig, Rakesh and Arulchelvan, Vamanan and Bera, Rahul and Shahroodi, Taha and Singh, Gagandeep and Kakolyris, Andreas Kosmas and Y{\"u}ksel, {\.I}smail Emir and Sadrosadati, Mohammad and Park, Jisung and Mutlu, Onur},
  booktitle={Proceedings of the 40th ACM International Conference on Supercomputing},
  year={2026}
}

Contact

Project maintainer: Rakesh Nadig (nadigr at ethz dot ch)