Disk Benchmarks

A tool for running automated disk performance benchmarks across different cloud providers and instance types. Currently supports DigitalOcean with plans to support more providers.

Features

• Automated disk performance testing using FIO
• Support for multiple instance types and regions
• Parallel testing of multiple instances
• Automatic instance provisioning and cleanup
• Detailed results in both YAML and human-readable formats
• Disk size sanity checks before running benchmarks
• Comprehensive error handling and state tracking

Prerequisites

• Python 3.x
• yq command-line tool
• doctl (for DigitalOcean)
• SSH key configured in your cloud provider
• Required Python packages: pyyaml, tabulate

Installation

1. Clone the repository:

git clone https://github.com/ethpandaops/disk-benchmarker.git
cd disk-benchmarker

2. Install Python dependencies:

pip install pyyaml tabulate

3. Set up your environment variables in .env:

DO_SSH_KEY_NAME="your-ssh-key-name"

Usage

1. Create a job file (see jobs/simple-do.yaml for an example):

name: simple-benchmark
provider: digitalocean
benchmark:
  size: 50G
  bs: 4k
  iodepth: 64
  rwmixread: 75
instances:
  - type: s-1vcpu-1gb
    region: nyc1
    count: 1

2. Run the benchmark:

./run.sh jobs/your-job.yaml

The script will:

• Verify disk sizes are sufficient
• Create instances
• Run FIO benchmarks
• Collect and format results
• Clean up resources

Output

Results are stored in the runs/ directory with a timestamp-based folder structure:

runs/
└── YYYYMMDD-HHMMSS/
    ├── job.yaml          # Copy of the job file
    ├── state.yaml        # Execution state and instance details
    ├── results.yaml      # Raw benchmark results
    └── instance-xxx/     # Per-instance results
        └── fio.json      # Raw FIO output

Results Format

The script generates both YAML results and a human-readable table showing:

• Instance Type
• Region
• Read IOPS
• Write IOPS
• Total IOPS
• Read Bandwidth
• Write Bandwidth
• Disk Utilization
• Runtime

Error Handling

The script includes comprehensive error handling for:

• Instance creation failures
• SSH connectivity issues
• Benchmark execution problems
• Insufficient disk space
• Missing dependencies

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.