perf(fibre): back the blob matrix with huge pages and stop cloning input - #1004
Draft
citizen-stig wants to merge 1 commit into
Draft
perf(fibre): back the blob matrix with huge pages and stop cloning input#1004citizen-stig wants to merge 1 commit into
citizen-stig wants to merge 1 commit into
Conversation
…e input Blob::new allocated the (K+N) x row_size matrix as a plain Vec and kept a second copy of the caller's data. For a 128 MiB blob that is 512 MiB of fresh 4 KiB pages (131k page faults, every one of which the parity scatter, the leaf hashing and the RLC pass then pay TLB misses on) plus a 128 MiB memcpy and another 32k faults for the clone. The matrix is now allocated through the global allocator and its 2 MiB aligned interior is marked MADV_HUGEPAGE before first touch, so with the usual transparent_hugepage=madvise setting the kernel backs it with 2 MiB pages. data() and data_size() are served from the first K rows of the encoded matrix instead of a clone; reconstructed blobs keep their own buffer as before. On its own (single-threaded parity encoder on main) the gain is modest; it matters once parity is encoded by many threads scattering into the matrix, where 4 KiB first-touch faults dominated. criterion, target-cpu=native, Ryzen 9 3950X, THP=madvise: fibre blob_new/128MB 985 ms -> 841 ms fibre blob_new/8MB 64.1 ms -> 58.2 ms Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VPtihoV1GJ9VcNkd7oMRUf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Blob::new allocated the (K+N) x row_size matrix as a plain Vec and kept a second copy of the caller's data. For a 128 MiB blob that is 512 MiB of fresh 4 KiB pages (131k page faults, every one of which the parity scatter, the leaf hashing and the RLC pass then pay TLB misses on) plus a 128 MiB memcpy and another 32k faults for the clone.
The matrix is now allocated through the global allocator and its 2 MiB aligned interior is marked MADV_HUGEPAGE before first touch, so with the usual transparent_hugepage=madvise setting the kernel backs it with 2 MiB pages. data() and data_size() are served from the first K rows of the encoded matrix instead of a clone; reconstructed blobs keep their own buffer as before.
On its own (single-threaded parity encoder on main) the gain is modest; it matters once parity is encoded by many threads scattering into the matrix, where 4 KiB first-touch faults dominated.
Overview