Skip to content

Add fsync to atomic writes in sedimentree_fs_storage for crash durability #131

@expede

Description

@expede

sedimentree_fs_storage::save_loose_commit and save_fragment use the standard temp-file + rename pattern to ensure atomicity with respect to concurrent readers: readers either see the old value or the new value, never a half-written file. That's correct.

However, neither function calls fsync (or equivalent) on either the temp file or the parent directory before returning. On a clean return, the newly-written bytes may still be sitting in the OS page cache, not on physical storage. A kernel panic, power loss, or forced VM reset within the next few seconds can lose the write entirely — or, worse, lose the rename metadata while keeping the temp file, leaving the target absent even though its bytes reached disk.

For the sync server deployment and any other long-running process writing commits and fragments, the current guarantee is "atomic for readers in this process" — not "durable across a crash." That's weaker than what most users would assume from a storage backend built on rename(2).

The symptom under crash: commits that were reported as saved (the future resolved Ok(())) may be silently absent after restart. Peers would then re-sync the lost commits on reconnect, so there's no permanent data loss in a well-connected network, but:

  • Standalone / offline instances would lose local work.
  • A restart window where peers query "do you have commit X?" gets "no" for commits that were supposedly persisted.
  • awaitSettled-style durability contracts the caller might rely on are misleading.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions