Skip to content

Cap light client execution proof request processing time - #12500

Draft
dmitry-markin wants to merge 30 commits into
masterfrom
dm-light-req-exec-cap
Draft

Cap light client execution proof request processing time#12500
dmitry-markin wants to merge 30 commits into
masterfrom
dm-light-req-exec-cap

Conversation

@dmitry-markin

@dmitry-markin dmitry-markin commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Introduce a capped WASM executor using epoch-based wasmtime interruption mechanism and use it for serving light client execution proof requests.

The execution is bounded by 3s per runtime call. This gives plenty of headroom for legitimate requests. We can consider reducing the default in the future.

Follow-ups

  1. Introduce prometheus metrics to collect actual request processing time
    Add metrics for light request processing time #12541
  2. Make light request processing concurrent
  3. Update light request protocol to include explicit "timeout" error response (TBC)

@dmitry-markin dmitry-markin changed the title Cap light request WASM execution Cap light client request execution time Jun 29, 2026
@dmitry-markin dmitry-markin changed the title Cap light client request execution time Cap light client execution proof request processing time Jun 29, 2026
@dmitry-markin

Copy link
Copy Markdown
Contributor Author

/cmd prdoc --audience node_dev --bump major

@dmitry-markin dmitry-markin added the T0-node This PR/Issue is related to the topic “node”. label Jun 29, 2026
@dmitry-markin
dmitry-markin marked this pull request as ready for review June 29, 2026 13:00
@dmitry-markin
dmitry-markin requested a review from koute as a code owner June 29, 2026 13:00
@dmitry-markin

Copy link
Copy Markdown
Contributor Author

A sloppy analysis of whether 10ms is enough for different classes of web3-app calls

Class Runtime call dApp purpose Driven over light path? Est. time (capped wasm + proof) vs 10 ms cap
Foundational Core_version spec/tx version for signing; upgrade & feature detect Yes — full proof pipeline (reads no state, but client can't know that) < 0.2 ms ✅ far under
Foundational Metadata_metadata / _at_version type/pallet layout → encode tx, decode state/events Yes — full proof pipeline (client can't know it reads no state) ~5–25 ms on large runtimes (CPU-bound encode, no proof overhead) can exceed on large chains
Tx building AccountNonceApi_account_nonce next nonce to build a signable extrinsic Yes (reads System::Account) ~0.3–1.5 ms ✅ under
Tx building TaggedTransactionQueue_validate_transaction pre-submit validity / signed-extension check Rarely (submission is gossip) ~1–5 ms (sig verify dominates) ✅ usually under
Fee / preview TransactionPaymentApi_query_info / query_fee_details "Fee: 0.012 DOT" for a signed extrinsic Yes (reads NextFeeMultiplier) ~0.5–3 ms ✅ under
Fee / preview TransactionPaymentCallApi_query_call_info / _fee_details fee preview for a bare call (pre-signature) Yes ~0.5–3 ms ✅ under
XCM XcmPaymentApi_query_weight_to_asset_fee / query_acceptable_payment_assets cross-chain fee in a chosen asset RPC tooling, not light ~0.5–2 ms ✅ under
XCM DryRunApi_dry_run_call / dry_run_xcm simulate XCM routing + effects (cross-chain only) No — RPC tooling; not a general web3 need ~3–50 ms+ n/a for light path
Staking/pools NominationPoolsApi_* / StakingApi_* (point lookups) rewards, points↔balance dashboards Possibly (smoldot dashboards) ~1–8 ms ✅ usually under
Staking/pools iterate-heavy staking calls (quota, electable set…) derived figures over large collections Possibly ~10–100 ms+ ❌ can exceed
Contracts ContractsApi_get_storage read one contract storage cell Rare (contract dApps use RPC) ~1–4 ms ✅ under
Contracts ContractsApi_callview/getter balanceOf, ownerOf, price quote Rare ~1–10 ms (dominated by code load+instantiate, not logic) ⚠️ borderline
Contracts ContractsApi_callgas-estimation dry-run pre-tx revert check + gas for a real interaction (swap/lend/cross-contract) Rare ~10–30 ms+ ❌ can exceed (legit)
Contracts ContractsApi_callmalicious (gas_limitWeight::MAX, looping contract) DoS Attacker unbounded (gas meter not clamped to block weight; ~u64 ps ceiling) ❌ trapped at cap — the target

@dmitry-markin
dmitry-markin marked this pull request as draft June 30, 2026 10:06
@dmitry-markin

dmitry-markin commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

Benchmarks of Metadata_metadata_at_version(<format>), N=100 each:

chain format min median p90 max
polkadot v16 7.670452ms 7.779901ms 7.998652ms 11.769488ms
asset-hub-polkadot v16 10.336305ms 11.459427ms 13.418529ms 16.613895ms
kusama v16 8.2666ms 8.47162ms 8.88233ms 27.414483ms
asset-hub-kusama v16 12.042182ms 12.416222ms 13.085494ms 19.637135ms
moonbeam TODO TODO TODO TODO TODO

Comment thread substrate/client/executor/wasmtime/src/runtime.rs Outdated
@dmitry-markin
dmitry-markin marked this pull request as ready for review July 3, 2026 10:15
@dmitry-markin

Copy link
Copy Markdown
Contributor Author

Due to issues with syncing Moonbeam (state not fitting into RAM for warp-sync, sync from genesis broken, snapshot links dead), Moonbeam was not benchmarked. The current timeout of 3s should give plenty of headroom, and we can revisit tightening the default timeout later.

Comment thread substrate/client/executor/src/wasm_runtime.rs Outdated
Comment thread substrate/client/executor/src/executor.rs Outdated
Comment thread substrate/client/executor/src/executor.rs Outdated
@dmitry-markin
dmitry-markin requested a review from skunert July 6, 2026 11:25
Comment thread substrate/client/service/src/builder.rs Outdated
Comment thread substrate/client/executor/wasmtime/src/runtime.rs Outdated
Comment thread substrate/client/executor/src/executor.rs Outdated
Comment thread substrate/client/executor/wasmtime/src/runtime.rs Outdated
Comment thread substrate/client/executor/wasmtime/src/runtime.rs Outdated
dmitry-markin and others added 3 commits July 14, 2026 13:57
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <mail@skunert.dev>

@bkchr bkchr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said already and now with looking closer into the pr, we should directly initialize our own executor inside the light client request handler. You get the params and the name and do not need any client that forwards the execution request.

Comment thread substrate/client/api/src/proof_provider.rs Outdated
Comment thread substrate/client/service/src/config.rs Outdated
Comment thread substrate/client/service/src/builder.rs Outdated
Comment thread substrate/client/network/light/src/light_client_requests/handler.rs Outdated
Comment thread substrate/client/service/src/client/client.rs Outdated
@dmitry-markin
dmitry-markin marked this pull request as draft July 15, 2026 04:38
@dmitry-markin

Copy link
Copy Markdown
Contributor Author

... we should directly initialize our own executor inside the light client request handler. You get the params and the name and do not need any client that forwards the execution request.

@bkchr not sure if this is feasible. We need complete LocalCallExecutor (including extensions and CodeProvider with code overrides/substitutes), and to build it we need Backend & ClientConfig. The least issue is that sc-network-light knows nothing about backend, but the real problem is that ClientConfig is part of sc-service, which in turn depends on sc-network-light.

May be there is a way around this circular dependency issue, but I don't see it now.

@dmitry-markin

dmitry-markin commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

For a future reference, here is the actual slowdown with epoch interrupts enabled. Measured on Asset Hub Kusama, each data point was collected in a 10-second loop.

Call Capped med Vanilla med Slowdown
Metadata_metadata (v14) 10.359 ms 9.673 ms ×1.071
Metadata_metadata_at_version(v14) 10.586 ms 9.817 ms ×1.078
Metadata_metadata_at_version(v15) 10.935 ms 10.152 ms ×1.077
Metadata_metadata_at_version(v16) 11.121 ms 10.308 ms ×1.079
Core_execute_block(#19056584) 4.670 ms 4.411 ms ×1.059

The measured slowdown seems to be around 8%, which matches the docs and indeed is unacceptable overhead for a main executor. Lower slowdown for quicker calls might indicate a constant overhead in every call, so real slowdown of WASM execution can be indeed around 10%.

@dmitry-markin

Copy link
Copy Markdown
Contributor Author

This branch became too much of a scratch pad, so I extracted core changes as meaningful commits into a new PR #12725.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T0-node This PR/Issue is related to the topic “node”.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants