Merge bitcoin/bitcoin#34616: Cluster mempool: SFL cost model (take 2)

744d47f clusterlin: adopt trained cost model (feature) (Pieter Wuille)
4eefdfc clusterlin: rescale costs (preparation) (Pieter Wuille)
ecc9a84 clusterlin: use 'cost' terminology instead of 'iters' (refactor) (Pieter Wuille)
9e7129d clusterlin: introduce CostModel class (preparation) (Pieter Wuille)

Pull request description:

  Part of #30289, replaces earlier #34138.

  This introduces a more accurate cost model for SFL, to control how much CPU time is spent inside the algorithm for clusters that cannot be linearized perfectly within a reasonable amount of time.

  The goal is having a metric for the amount of work performed, so that txmempool can impose limits on that work: a lower bound that is always performed (unless optimality is reached before that point, of course), and an upper bound to limit the latency and total CPU time spent on this. There are conflicting design goals here:
  * On the one hand, it seems ideal if this metric is closely correlated to actual CPU time, because otherwise the limits become inaccurate.
  * On the other hand, it seems a nightmare to have the metric be platform/system dependent, as it makes network-wide reasoning nearly impossible. It's expected that slower systems take longer to do the same thing; this holds for everything, and we don't need to compensate for this.

  There are multiple solutions to this:
  * One extreme is just measuring the time. This is very accurate, but extremely platform dependent, and also non-deterministic due to random scheduling/cache effects.
  * The other extreme is using a very abstract metric like counting how many times certain loops/function inside the algorithm run. That is what is implemented in master right now, just counting the sum of the numbers of transactions updated across all `UpdateChunks()` calls. It however necessarily fails to account for significant portions of runtime spent elsewhere, resulting in a rather wide range of "ns per cost" values.
  * This PR takes a middle ground, counting many function calls / branches / loops, with weights that were determined through benchmarking on an average on a number of systems.

  Specifically, the cost model was obtained by:
  * For a variety of machines:
    * Running a fixed collection of ~385000 clusters found through random generation and fuzzing, optimizing for difficulty of linearization.
      * Linearize each 1000-5000 times, with different random seeds. Sometimes without input linearization, sometimes with a bad one.
        * Gather cycle counts for each of the operations included in this cost model, broken down by their parameters.
    * Correct the data by subtracting the runtime of obtaining the cycle count.
    * Drop the 5% top and bottom samples from each cycle count dataset, and compute the average of the remaining samples.
    * For each operation, fit a least-squares linear function approximation through the samples.
  * Rescale all machine expressions to make their total time match, as we only care about relative cost of each operation.
  * Take the per-operation average of operation expressions across all machines, to construct expressions for an average machine.
  * Approximate the result with integer coefficients.

  The benchmarks were performed by `l0rinc <pap.lorinc@gmail.com>` and myself, on AMD Ryzen 5950X, AMD Ryzen 7995WX, AMD Ryzen 9980X, Apple M4 Max, Intel Core i5-12500H, Intel Core Ultra 7 155H, Intel N150 (Umbrel), Intel Core i7-7700, Intel Core i9-9900K, Intel Haswell (VPS, virtualized), Intel Xeon E5-2637, ARM Cortex-A76 (Raspberry Pi 5), ARM Cortex-A72 (Raspberry Pi 4).

  Based on final benchmarking, the "acceptable" iteration count (which is the minimum spent on every cluster) is to 75000 units, which corresponds to roughly 50 μs on Ryzen 5950X and similar modern desktop hardware.

ACKs for top commit:
  instagibbs:
    ACK 744d47f
  murchandamus:
    reACK 744d47f

Tree-SHA512: 5cb37a6bdd930389937c435f910410c3581e53ce609b9b594a8dc89601e6fca6e6e26216e961acfe9540581f889c14bf289b6a08438a2d7adafd696fc81ff517

Author: fanquake

src/bench/cluster_linearize.cpp

@@ -55,7 +55,7 @@ void BenchLinearizeOptimallyTotal(benchmark::Bench& bench, const std::string& na
         // Benchmark the total time to optimal.
         uint64_t rng_seed = 0;
         bench.name(bench_name).run([&] {
-            auto [_lin, optimal, _cost] = Linearize(depgraph, /*max_iterations=*/10000000, rng_seed++, IndexTxOrder{});
+            auto [_lin, optimal, _cost] = Linearize(depgraph, /*max_cost=*/10000000, rng_seed++, IndexTxOrder{});
             assert(optimal);
         });
     }
@@ -72,15 +72,15 @@ void BenchLinearizeOptimallyPerCost(benchmark::Bench& bench, const std::string&
         // Determine the cost of 100 rng_seeds.
         uint64_t total_cost = 0;
         for (uint64_t iter = 0; iter < 100; ++iter) {
-            auto [_lin, optimal, cost] = Linearize(depgraph, /*max_iterations=*/10000000, /*rng_seed=*/iter, IndexTxOrder{});
+            auto [_lin, optimal, cost] = Linearize(depgraph, /*max_cost=*/10000000, /*rng_seed=*/iter, IndexTxOrder{});
             total_cost += cost;
         }
 
         // Benchmark the time per cost.
         bench.name(bench_name).unit("cost").batch(total_cost).run([&] {
             uint64_t recompute_cost = 0;
             for (uint64_t iter = 0; iter < 100; ++iter) {
-                auto [_lin, optimal, cost] = Linearize(depgraph, /*max_iterations=*/10000000, /*rng_seed=*/iter, IndexTxOrder{});
+                auto [_lin, optimal, cost] = Linearize(depgraph, /*max_cost=*/10000000, /*rng_seed=*/iter, IndexTxOrder{});
                 assert(optimal);
                 recompute_cost += cost;
             }

src/bench/txgraph.cpp

@@ -51,9 +51,9 @@ void BenchTxGraphTrim(benchmark::Bench& bench)
     static constexpr int NUM_DEPS_PER_BOTTOM_TX = 100;
     /** Set a very large cluster size limit so that only the count limit is triggered. */
     static constexpr int32_t MAX_CLUSTER_SIZE = 100'000 * 100;
-    /** Set a very high number for acceptable iterations, so that we certainly benchmark optimal
+    /** Set a very high number for acceptable cost, so that we certainly benchmark optimal
      *  linearization. */
-    static constexpr uint64_t NUM_ACCEPTABLE_ITERS = 100'000'000;
+    static constexpr uint64_t HIGH_ACCEPTABLE_COST = 100'000'000;
 
     /** Refs to all top transactions. */
     std::vector<TxGraph::Ref> top_refs;
@@ -65,7 +65,7 @@ void BenchTxGraphTrim(benchmark::Bench& bench)
     std::vector<size_t> top_components;
 
     InsecureRandomContext rng(11);
-    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, NUM_ACCEPTABLE_ITERS, PointerComparator);
+    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, HIGH_ACCEPTABLE_COST, PointerComparator);
 
     // Construct the top chains.
     for (int chain = 0; chain < NUM_TOP_CHAINS; ++chain) {

src/cluster_linearize.h

@@ -88,9 +88,15 @@ void TestOptimalLinearization(std::span<const uint8_t> enc, std::initializer_lis
                 is_topological = false;
                 break;
             }
-            std::tie(lin, opt, cost) = Linearize(depgraph, 1000000000000, rng.rand64(), IndexTxOrder{}, lin, is_topological);
+            std::tie(lin, opt, cost) = Linearize(
+                /*depgraph=*/depgraph,
+                /*max_cost=*/1000000000000,
+                /*rng_seed=*/rng.rand64(),
+                /*fallback_order=*/IndexTxOrder{},
+                /*old_linearization=*/lin,
+                /*is_topological=*/is_topological);
             BOOST_CHECK(opt);
-            BOOST_CHECK(cost <= MaxOptimalLinearizationIters(depgraph.TxCount()));
+            BOOST_CHECK(cost <= MaxOptimalLinearizationCost(depgraph.TxCount()));
             SanityCheck(depgraph, lin);
             BOOST_CHECK(std::ranges::equal(lin, optimal_linearization));
         }

src/test/cluster_linearize_tests.cpp

@@ -984,7 +984,7 @@ FUZZ_TARGET(clusterlin_sfl)
 
     // Verify that optimality is reached within an expected amount of work. This protects against
     // hypothetical bugs that hugely increase the amount of work needed to reach optimality.
-    assert(sfl.GetCost() <= MaxOptimalLinearizationIters(depgraph.TxCount()));
+    assert(sfl.GetCost() <= MaxOptimalLinearizationCost(depgraph.TxCount()));
 
     // The result must be as good as SimpleLinearize.
     auto [simple_linearization, simple_optimal] = SimpleLinearize(depgraph, MAX_SIMPLE_ITERATIONS / 10);
@@ -1011,16 +1011,17 @@ FUZZ_TARGET(clusterlin_linearize)
 {
     // Verify the behavior of Linearize().
 
-    // Retrieve an RNG seed, an iteration count, a depgraph, and whether to make it connected from
-    // the fuzz input.
+    // Retrieve an RNG seed, a maximum amount of work, a depgraph, and whether to make it connected
+    // from the fuzz input.
     SpanReader reader(buffer);
     DepGraph<TestBitSet> depgraph;
     uint64_t rng_seed{0};
-    uint64_t iter_count{0};
+    uint64_t max_cost{0};
     uint8_t flags{7};
     try {
-        reader >> VARINT(iter_count) >> Using<DepGraphFormatter>(depgraph) >> rng_seed >> flags;
+        reader >> VARINT(max_cost) >> Using<DepGraphFormatter>(depgraph) >> rng_seed >> flags;
     } catch (const std::ios_base::failure&) {}
+    if (depgraph.TxCount() <= 1) return;
     bool make_connected = flags & 1;
     // The following 3 booleans have 4 combinations:
     // - (flags & 6) == 0: do not provide input linearization.
@@ -1043,8 +1044,14 @@ FUZZ_TARGET(clusterlin_linearize)
     }
 
     // Invoke Linearize().
-    iter_count &= 0x7ffff;
-    auto [linearization, optimal, cost] = Linearize(depgraph, iter_count, rng_seed, IndexTxOrder{}, old_linearization, /*is_topological=*/claim_topological_input);
+    max_cost &= 0x3fffff;
+    auto [linearization, optimal, cost] = Linearize(
+        /*depgraph=*/depgraph,
+        /*max_cost=*/max_cost,
+        /*rng_seed=*/rng_seed,
+        /*fallback_order=*/IndexTxOrder{},
+        /*old_linearization=*/old_linearization,
+        /*is_topological=*/claim_topological_input);
     SanityCheck(depgraph, linearization);
     auto chunking = ChunkLinearization(depgraph, linearization);
 
@@ -1056,8 +1063,8 @@ FUZZ_TARGET(clusterlin_linearize)
         assert(cmp >= 0);
     }
 
-    // If the iteration count is sufficiently high, an optimal linearization must be found.
-    if (iter_count > MaxOptimalLinearizationIters(depgraph.TxCount())) {
+    // If the maximum amount of work is sufficiently high, an optimal linearization must be found.
+    if (max_cost > MaxOptimalLinearizationCost(depgraph.TxCount())) {
         assert(optimal);
     }
 
@@ -1145,7 +1152,7 @@ FUZZ_TARGET(clusterlin_linearize)
 
         // Redo from scratch with a different rng_seed. The resulting linearization should be
         // deterministic, if both are optimal.
-        auto [linearization2, optimal2, cost2] = Linearize(depgraph, MaxOptimalLinearizationIters(depgraph.TxCount()) + 1, rng_seed ^ 0x1337, IndexTxOrder{});
+        auto [linearization2, optimal2, cost2] = Linearize(depgraph, MaxOptimalLinearizationCost(depgraph.TxCount()) + 1, rng_seed ^ 0x1337, IndexTxOrder{});
         assert(optimal2);
         assert(linearization2 == linearization);
     }
@@ -1236,7 +1243,7 @@ FUZZ_TARGET(clusterlin_postlinearize_tree)
 
     // Try to find an even better linearization directly. This must not change the diagram for the
     // same reason.
-    auto [opt_linearization, _optimal, _cost] = Linearize(depgraph_tree, 100000, rng_seed, IndexTxOrder{}, post_linearization);
+    auto [opt_linearization, _optimal, _cost] = Linearize(depgraph_tree, 1000000, rng_seed, IndexTxOrder{}, post_linearization);
     auto opt_chunking = ChunkLinearization(depgraph_tree, opt_linearization);
     auto cmp_opt = CompareChunks(opt_chunking, post_chunking);
     assert(cmp_opt == 0);

src/test/fuzz/cluster_linearize.cpp

@@ -325,8 +325,8 @@ FUZZ_TARGET(txgraph)
     auto max_cluster_count = provider.ConsumeIntegralInRange<DepGraphIndex>(1, MAX_CLUSTER_COUNT_LIMIT);
     /** The maximum total size of transactions in a (non-oversized) cluster. */
     auto max_cluster_size = provider.ConsumeIntegralInRange<uint64_t>(1, 0x3fffff * MAX_CLUSTER_COUNT_LIMIT);
-    /** The number of iterations to consider a cluster acceptably linearized. */
-    auto acceptable_iters = provider.ConsumeIntegralInRange<uint64_t>(0, 10000);
+    /** The amount of work to consider a cluster acceptably linearized. */
+    auto acceptable_cost = provider.ConsumeIntegralInRange<uint64_t>(0, 10000);
 
     /** The set of uint64_t "txid"s that have been assigned before. */
     std::set<uint64_t> assigned_txids;
@@ -342,7 +342,7 @@ FUZZ_TARGET(txgraph)
     auto real = MakeTxGraph(
         /*max_cluster_count=*/max_cluster_count,
         /*max_cluster_size=*/max_cluster_size,
-        /*acceptable_iters=*/acceptable_iters,
+        /*acceptable_cost=*/acceptable_cost,
         /*fallback_order=*/fallback_order);
 
     std::vector<SimTxGraph> sims;
@@ -758,9 +758,9 @@ FUZZ_TARGET(txgraph)
                 break;
             } else if (command-- == 0) {
                 // DoWork.
-                uint64_t iters = provider.ConsumeIntegralInRange<uint64_t>(0, alt ? 10000 : 255);
-                bool ret = real->DoWork(iters);
-                uint64_t iters_for_optimal{0};
+                uint64_t max_cost = provider.ConsumeIntegralInRange<uint64_t>(0, alt ? 10000 : 255);
+                bool ret = real->DoWork(max_cost);
+                uint64_t cost_for_optimal{0};
                 for (unsigned level = 0; level < sims.size(); ++level) {
                     // DoWork() will not optimize oversized levels, or the main level if a builder
                     // is present. Note that this impacts the DoWork() return value, as true means
@@ -773,24 +773,24 @@ FUZZ_TARGET(txgraph)
                     if (ret) {
                         sims[level].real_is_optimal = true;
                     }
-                    // Compute how many iterations would be needed to make everything optimal.
+                    // Compute how much work would be needed to make everything optimal.
                     for (auto component : sims[level].GetComponents()) {
-                        auto iters_opt_this_cluster = MaxOptimalLinearizationIters(component.Count());
-                        if (iters_opt_this_cluster > acceptable_iters) {
-                            // If the number of iterations required to linearize this cluster
-                            // optimally exceeds acceptable_iters, DoWork() may process it in two
+                        auto cost_opt_this_cluster = MaxOptimalLinearizationCost(component.Count());
+                        if (cost_opt_this_cluster > acceptable_cost) {
+                            // If the amount of work required to linearize this cluster
+                            // optimally exceeds acceptable_cost, DoWork() may process it in two
                             // stages: once to acceptable, and once to optimal.
-                            iters_for_optimal += iters_opt_this_cluster + acceptable_iters;
+                            cost_for_optimal += cost_opt_this_cluster + acceptable_cost;
                         } else {
-                            iters_for_optimal += iters_opt_this_cluster;
+                            cost_for_optimal += cost_opt_this_cluster;
                         }
                     }
                 }
                 if (!ret) {
-                    // DoWork can only have more work left if the requested number of iterations
+                    // DoWork can only have more work left if the requested amount of work
                     // was insufficient to linearize everything optimally within the levels it is
                     // allowed to touch.
-                    assert(iters <= iters_for_optimal);
+                    assert(max_cost <= cost_for_optimal);
                 }
                 break;
             } else if (sims.size() == 2 && !sims[0].IsOversized() && !sims[1].IsOversized() && command-- == 0) {
@@ -1165,7 +1165,7 @@ FUZZ_TARGET(txgraph)
             auto real_redo = MakeTxGraph(
                 /*max_cluster_count=*/max_cluster_count,
                 /*max_cluster_size=*/max_cluster_size,
-                /*acceptable_iters=*/acceptable_iters,
+                /*acceptable_cost=*/acceptable_cost,
                 /*fallback_order=*/fallback_order);
             /** Vector (indexed by SimTxGraph::Pos) of TxObjects in real_redo). */
             std::vector<std::optional<SimTxObject>> txobjects_redo;

src/test/fuzz/txgraph.cpp

@@ -15,9 +15,9 @@ BOOST_AUTO_TEST_SUITE(txgraph_tests)
 
 namespace {
 
-/** The number used as acceptable_iters argument in these tests. High enough that everything
+/** The number used as acceptable_cost argument in these tests. High enough that everything
  *  should be optimal, always. */
-constexpr uint64_t NUM_ACCEPTABLE_ITERS = 100'000'000;
+constexpr uint64_t HIGH_ACCEPTABLE_COST = 100'000'000;
 
 std::strong_ordering PointerComparator(const TxGraph::Ref& a, const TxGraph::Ref& b) noexcept
 {
@@ -48,7 +48,7 @@ BOOST_AUTO_TEST_CASE(txgraph_trim_zigzag)
     static constexpr int32_t MAX_CLUSTER_SIZE = 100'000 * 100;
 
     // Create a new graph for the test.
-    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, NUM_ACCEPTABLE_ITERS, PointerComparator);
+    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, HIGH_ACCEPTABLE_COST, PointerComparator);
 
     // Add all transactions and store their Refs.
     std::vector<TxGraph::Ref> refs;
@@ -111,7 +111,7 @@ BOOST_AUTO_TEST_CASE(txgraph_trim_flower)
     /** Set a very large cluster size limit so that only the count limit is triggered. */
     static constexpr int32_t MAX_CLUSTER_SIZE = 100'000 * 100;
 
-    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, NUM_ACCEPTABLE_ITERS, PointerComparator);
+    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, HIGH_ACCEPTABLE_COST, PointerComparator);
 
     // Add all transactions and store their Refs.
     std::vector<TxGraph::Ref> refs;
@@ -197,7 +197,7 @@ BOOST_AUTO_TEST_CASE(txgraph_trim_huge)
     std::vector<size_t> top_components;
 
     FastRandomContext rng;
-    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, NUM_ACCEPTABLE_ITERS, PointerComparator);
+    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, HIGH_ACCEPTABLE_COST, PointerComparator);
 
     // Construct the top chains.
     for (int chain = 0; chain < NUM_TOP_CHAINS; ++chain) {
@@ -270,7 +270,7 @@ BOOST_AUTO_TEST_CASE(txgraph_trim_big_singletons)
     static constexpr int NUM_TOTAL_TX = 100;
 
     // Create a new graph for the test.
-    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, NUM_ACCEPTABLE_ITERS, PointerComparator);
+    auto graph = MakeTxGraph(MAX_CLUSTER_COUNT, MAX_CLUSTER_SIZE, HIGH_ACCEPTABLE_COST, PointerComparator);
 
     // Add all transactions and store their Refs.
     std::vector<TxGraph::Ref> refs;
@@ -304,7 +304,7 @@ BOOST_AUTO_TEST_CASE(txgraph_trim_big_singletons)
 BOOST_AUTO_TEST_CASE(txgraph_chunk_chain)
 {
     // Create a new graph for the test.
-    auto graph = MakeTxGraph(50, 1000, NUM_ACCEPTABLE_ITERS, PointerComparator);
+    auto graph = MakeTxGraph(50, 1000, HIGH_ACCEPTABLE_COST, PointerComparator);
 
     auto block_builder_checker = [&graph](std::vector<std::vector<TxGraph::Ref*>> expected_chunks) {
         std::vector<std::vector<TxGraph::Ref*>> chunks;
@@ -383,7 +383,7 @@ BOOST_AUTO_TEST_CASE(txgraph_staging)
     /* Create a new graph for the test.
      * The parameters are max_cluster_count, max_cluster_size, acceptable_iters
      */
-    auto graph = MakeTxGraph(10, 1000, NUM_ACCEPTABLE_ITERS, PointerComparator);
+    auto graph = MakeTxGraph(10, 1000, HIGH_ACCEPTABLE_COST, PointerComparator);
 
     std::vector<TxGraph::Ref> refs;
     refs.reserve(2);

src/test/txgraph_tests.cpp

@@ -394,26 +394,26 @@ void SanityCheck(const DepGraph<SetType>& depgraph, std::span<const DepGraphInde
     }
 }
 
-inline uint64_t MaxOptimalLinearizationIters(DepGraphIndex cluster_count)
+inline uint64_t MaxOptimalLinearizationCost(DepGraphIndex cluster_count)
 {
     // These are the largest numbers seen returned as cost by Linearize(), in a large randomized
     // trial. There exist almost certainly far worse cases, but they are unlikely to be
     // encountered in randomized tests. The purpose of these numbers is guaranteeing that for
     // *some* reasonable cost bound, optimal linearizations are always found.
-    static constexpr uint64_t ITERS[65] = {
+    static constexpr uint64_t COSTS[65] = {
         0,
-        0, 4, 10, 34, 76, 156, 229, 380,
-        441, 517, 678, 933, 1037, 1366, 1464, 1711,
-        2111, 2542, 3068, 3116, 4029, 3467, 5324, 5402,
-        6481, 7161, 7441, 8183, 8843, 9353, 11104, 11455,
-        11791, 12570, 13480, 14259, 14525, 12426, 14477, 20201,
-        18737, 16581, 23622, 28486, 30652, 33021, 32942, 32745,
-        34046, 26227, 34662, 38019, 40814, 31113, 41448, 33968,
-        35024, 59207, 42872, 41277, 42365, 51833, 63410, 67035
+        0, 545, 928, 1633, 2647, 4065, 5598, 8258,
+        9505, 11471, 14137, 19553, 20460, 26191, 28397, 32599,
+        41631, 47419, 56329, 57767, 72196, 63652, 95366, 96537,
+        115653, 125407, 131734, 145090, 156349, 164665, 194224, 203953,
+        207710, 225878, 239971, 252284, 256534, 222142, 251332, 357098,
+        325788, 295867, 410053, 497483, 533892, 576572, 577845, 572400,
+        592536, 455082, 609249, 659130, 714091, 544507, 718788, 562378,
+        601926, 1025081, 732725, 708896, 738224, 900445, 1092519, 1139946
     };
-    assert(cluster_count < std::size(ITERS));
+    assert(cluster_count < std::size(COSTS));
     // Multiply the table number by two, to account for the fact that they are not absolutes.
-    return ITERS[cluster_count] * 2;
+    return COSTS[cluster_count] * 2;
 }
 
 } // namespace