Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions doc/src/usage/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,35 @@ This page covers a few handy `fuzzamoto-cli` workflows. The CLI is built from th
`fuzzamoto-cli` crate in this repository and provides utilities for working with
IR corpora, scenarios, and coverage reports.

## Initialize a campaign with a seed mempool

The `init` subcommand accepts an optional `--seedfile` flag. When provided, the
raw bytes of the seed file are used to create a `mempool.dat` file, which is then
imported via the `importmempool` RPC **before** the snapshot is taken. This means
every fuzz input will start from a node that already has those transactions in its
mempool, giving the fuzzer richer starting state.

### Using `--seedfile` with `fuzzamoto-cli init`

Pass the path to the seed file with `--seedfile`:

```bash
fuzzamoto-cli init \
--sharedir /tmp/fuzzamoto_share \
--crash-handler /path/to/crash_handler.so \
--bitcoind /path/to/bitcoind \
--scenario /path/to/scenario-generic \
--nyx-dir /path/to/nyx \
--seedfile /tmp/my_seed.dat
```

The generated `fuzz_no_pt.sh` script will automatically download the seed file
into the Nyx VM and pass it to the scenario binary via `--seedfile`. During
scenario initialization the seed file's bytes are written as `mempool.dat` and
imported into the node, then the snapshot is taken with that mempool state so
every test case executes against a node that starts with pre-populated
transactions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it's helpful to explain here how the mempool.dat is created? I guess you need to mine the same number of blocks on a regtest node, create transactions, and then dump the file? Thinking about it more, the blocks need to be identical to what fuzzamoto creates?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it'd be better to allow an IR testcase as seedfile that is then executed before the snapshot. That seems a little more generic, and would enable other states as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We were trying to figure out what RPCs to call to build the context and if in-mempool txns should be included in the context. I think we'd need dumptxoutset for the build_txos call and getrawmempool + getmempoolentry if mempool txns are included? Then we'd also need the block headers, though that seems straightforward.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it'd be better to allow an IR testcase as seedfile that is then executed before the snapshot. That seems a little more generic, and would enable other states as well.

This would be good, but the current approach is much simpler for now. But yes, the idea is having something like that as well, we just need to figure out what is necessary as @Crypt-iQ mentioned.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dergoegge would you prefer one general option here instead of two? If so, I can look at what RPCs we'd need. I think we would be able to reuse most of this PR.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd prefer the general option, unless that is for some reason not possible or way more complex.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, one general option would be better, moved to draft for now.


## Generate `ir.context`

The CLI’s `--context` flag expects a context file dumped by the IR scenario. You can produce one outside Nyx as follows:
Expand Down
29 changes: 25 additions & 4 deletions fuzzamoto-cli/src/commands/init.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,26 @@ use crate::error::{CliError, Result};
use crate::utils::{file_ops, nyx, process};
use std::path::{Path, PathBuf};

pub struct InitOptions<'a> {
pub secondary_bitcoind: Option<&'a PathBuf>,
pub rpc_path: Option<&'a PathBuf>,
pub seedfile: Option<&'a PathBuf>,
}

pub struct InitCommand;

impl InitCommand {
pub fn execute(
sharedir: &Path,
crash_handler: &Path,
bitcoind: &Path,
secondary_bitcoind: Option<&PathBuf>,
scenario: &Path,
nyx_dir: &Path,
rpc_path: Option<&PathBuf>,
opts: &InitOptions<'_>,
) -> Result<()> {
let secondary_bitcoind = opts.secondary_bitcoind;
let rpc_path = opts.rpc_path;
let seedfile = opts.seedfile;
file_ops::ensure_sharedir_not_exists(sharedir)?;
file_ops::create_dir_all(sharedir)?;

Expand All @@ -30,6 +38,11 @@ impl InitCommand {
file_ops::copy_file_to_dir(rpc, sharedir)?;
}

if let Some(seed) = seedfile {
file_ops::ensure_file_exists(seed)?;
file_ops::copy_file_to_dir(seed, sharedir)?;
}

let mut all_deps = Vec::new();
let mut binary_names = Vec::new();

Expand Down Expand Up @@ -117,14 +130,22 @@ impl InitCommand {
.and_then(|p| p.file_name())
.and_then(|name| name.to_str());

let seedfile_name = seedfile
.as_ref()
.and_then(|p| p.file_name())
.and_then(|name| name.to_str());

nyx::create_nyx_script(
sharedir,
&all_deps,
&binary_names,
&crash_handler_name,
scenario_name,
secondary_name,
rpc_name,
&nyx::NyxScriptOptions {
secondary_bitcoind: secondary_name,
rpc_path: rpc_name,
seedfile: seedfile_name,
},
)?;

Ok(())
Expand Down
15 changes: 13 additions & 2 deletions fuzzamoto-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ mod error;
mod utils;

use clap::{Parser, Subcommand};
use commands::init::InitOptions;
use commands::{CoverageCommand, InitCommand, IrCommand, ir};
use error::Result;
use std::path::PathBuf;
Expand Down Expand Up @@ -51,6 +52,12 @@ enum Commands {
help = "Path to the file with the RPC commands that should be copied into the share directory"
)]
rpc_path: Option<PathBuf>,

#[arg(
long,
help = "Path to a mempool.dat seed file to be imported into the node's mempool before the snapshot is taken"
)]
seedfile: Option<PathBuf>,
},

/// Create a html coverage report for a given corpus
Expand Down Expand Up @@ -133,14 +140,18 @@ fn main() -> Result<()> {
scenario,
nyx_dir,
rpc_path,
seedfile,
} => InitCommand::execute(
sharedir,
crash_handler,
bitcoind,
secondary_bitcoind.as_ref(),
scenario,
nyx_dir,
rpc_path.as_ref(),
&InitOptions {
secondary_bitcoind: secondary_bitcoind.as_ref(),
rpc_path: rpc_path.as_ref(),
seedfile: seedfile.as_ref(),
},
),
Commands::Coverage {
output,
Expand Down
26 changes: 22 additions & 4 deletions fuzzamoto-cli/src/utils/nyx.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,23 @@ pub fn generate_nyx_config(nyx_path: &Path, sharedir: &Path) -> Result<()> {
Ok(())
}

pub struct NyxScriptOptions<'a> {
pub secondary_bitcoind: Option<&'a str>,
pub rpc_path: Option<&'a str>,
pub seedfile: Option<&'a str>,
}

pub fn create_nyx_script(
sharedir: &Path,
all_deps: &[String],
binary_names: &[String],
crash_handler_name: &str,
scenario_name: &str,
secondary_bitcoind: Option<&str>,
rpc_path: Option<&str>,
opts: &NyxScriptOptions<'_>,
) -> Result<()> {
let secondary_bitcoind = opts.secondary_bitcoind;
let rpc_path = opts.rpc_path;
let seedfile = opts.seedfile;
let mut script = vec![
"chmod +x hget".to_string(),
"cp hget /tmp".to_string(),
Expand All @@ -71,6 +79,10 @@ pub fn create_nyx_script(
script.push(format!("./hget {rpc_path} {rpc_path}"));
}

if let Some(seedfile) = seedfile {
script.push(format!("./hget {seedfile} {seedfile}"));
}

// Make executables
for exe in &["habort", "hcat", "ld-linux-x86-64.so.2", crash_handler_name] {
script.push(format!("chmod +x {exe}"));
Expand Down Expand Up @@ -109,12 +121,18 @@ pub fn create_nyx_script(
script.push(format!("echo \"{proxy_script}\" >> ./bitcoind_proxy"));
script.push("chmod +x ./bitcoind_proxy".to_string());

// Build the seedfile argument for the scenario command
let seedfile_arg = seedfile
.map(|s| format!(" --seedfile {s}"))
.unwrap_or_default();

// Run the scenario
script.push(format!(
"RUST_LOG=debug LD_LIBRARY_PATH=/tmp LD_BIND_NOW=1 ./{} ./bitcoind_proxy {} ./{} > log.txt 2>&1",
"RUST_LOG=debug LD_LIBRARY_PATH=/tmp LD_BIND_NOW=1 ./{} ./bitcoind_proxy {} ./{}{} > log.txt 2>&1",
scenario_name,
rpc_path.unwrap_or(""),
secondary_bitcoind.unwrap_or("")
secondary_bitcoind.unwrap_or(""),
seedfile_arg
));

// Debug info
Expand Down
24 changes: 22 additions & 2 deletions fuzzamoto/src/scenarios/generic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ pub struct GenericScenario<TX: Transport, T: Target<TX>> {
const INTERVAL: u64 = 1;

impl<TX: Transport, T: Target<TX>> GenericScenario<TX, T> {
fn from_target(mut target: T) -> Result<Self, String> {
fn from_target_with_seedfile(mut target: T, seedfile: Option<&str>) -> Result<Self, String> {
let genesis_block = bitcoin::blockdata::constants::genesis_block(bitcoin::Network::Regtest);

let mut time = u64::from(genesis_block.header.time);
Expand Down Expand Up @@ -209,6 +209,22 @@ impl<TX: Transport, T: Target<TX>> GenericScenario<TX, T> {
connection.send_and_recv(&("inv".to_string(), encode::serialize(&inv)), false)?;
}

// Import mempool from seed file if provided. The seed file's raw bytes are written as
// mempool.dat and imported into the node before the snapshot is taken, so fuzz inputs
// can interact with pre-existing transactions.
if let Some(path) = seedfile {
match std::fs::read(path) {
Ok(bytes) => {
if let Err(e) = target.import_mempool(&bytes) {
log::warn!("Failed to import mempool from seed file '{path}': {e}");
} else {
log::info!("Imported mempool from seed file: {path}");
}
}
Err(e) => log::warn!("Failed to read seed file '{path}': {e}"),
}
}

Ok(Self {
target,
time,
Expand All @@ -222,7 +238,11 @@ impl<TX: Transport, T: Target<TX>> GenericScenario<TX, T> {
impl<TX: Transport, T: Target<TX>> Scenario<'_, TestCase> for GenericScenario<TX, T> {
fn new(args: &[String]) -> Result<Self, String> {
let target = T::from_path(&args[1])?;
Self::from_target(target)
let seedfile = args
.windows(2)
.find(|w| w[0] == "--seedfile")
.map(|w| w[1].clone());
Self::from_target_with_seedfile(target, seedfile.as_deref())
}

fn run(&mut self, testcase: TestCase) -> ScenarioResult {
Expand Down
17 changes: 17 additions & 0 deletions fuzzamoto/src/targets/bitcoin_core.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,23 @@ impl TargetNode for BitcoinCoreTarget {
})
}

fn import_mempool(&self, bytes: &[u8]) -> Result<(), String> {
let mempool_path = self.node.workdir().join("mempool.dat");
std::fs::write(&mempool_path, bytes)
.map_err(|e| format!("Failed to write mempool.dat: {e}"))?;
self.node
.client
.call::<serde_json::Value>(
"importmempool",
&[mempool_path
.to_str()
.ok_or("mempool.dat path is not valid UTF-8")?
.into()],
)
.map(|_| ())
.map_err(|e| format!("Failed to import mempool: {e:?}"))
}

fn set_mocktime(&mut self, time: u64) -> Result<(), String> {
let client = &self.node.client;

Expand Down
13 changes: 13 additions & 0 deletions fuzzamoto/src/targets/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,19 @@ pub trait TargetNode: Sized {

/// Check if the target is still alive.
fn is_alive(&self) -> Result<(), String>;

/// Create a `mempool.dat` file from the given raw bytes and import it into the node's mempool.
///
/// The implementation writes `bytes` to a `mempool.dat` file in the node's working directory
/// and then calls `importmempool` on that file. The default implementation is a no-op for
/// targets that do not support this operation.
///
/// # Arguments
///
/// * `bytes` - Raw bytes to write as the `mempool.dat` file content.
fn import_mempool(&self, _bytes: &[u8]) -> Result<(), String> {
Ok(())
}
}

/// `Target` is the interface that the test harness will use to interact with the target Bitcoin
Expand Down