KIP 0041: SPV Expiration Window #73
Conversation
There was a problem hiding this comment.
I think, this KIP needs a bit more thought and, ideally, a quantitative analysis of the issue and the proposed solution.
At the moment verification of SPV proofs requires to retain two types of data indefinitely:
- Block hashes for the header oracle against which proof roots are verified and
- IDs that uniquely identify each proven event that are used as nullifiers to prevent double spends.
Both types require about 20 bytes per storage entry if optimized in a straight forward way.
Block Hash Storage
The storage of block hashes scales linearly with the block height of the chain (i.e. at constant rate), independent of usage.
Block hashes could be retained sparsely at the cost of somewhat larger verification cost both in storage as well as in compute. However, this kind of storage could be pruned normally. This trade-off allows to achieve sub-linear or even constant storage costs in exchange for more expensive SPV proofs, because proofs must span the distance from the event block to the next available block header in the oracle. Access to an archiving nodes would be required to create those proofs in first place.
ZK technology would be needed to make constant storage feasible while supporting proofs of arbitrarily old events. A solution that would allow proofs over moderately large block spans (say in the order of several thousand block heights) could be implemented at relatively low cost with existing off-the-shelf technology.
Overall, just limiting the availability of older headers in the header oracle, either by thinning the at older block heights or implementing a strict cut-off depth, would be very easy to implement without the need to make any changes to SPV proof verification. The effect would be that proving older events would become infeasible due to increased proof size requirements.
The addition of more compact proof formats (e.g. via ZK) could reenable proofs for older events at any time without any further adjustments.
Event ID Storage
The storage of Event IDs scales linear in the number of proven events.
Event ID storage cannot be thinned without comprising soundness. However, it would be possible to cryptographically commit to the set of nullified events on-chain and to store older event IDs only off-chain (e.g. only in archiving nodes). It would allow users to redeem funds even for older events at the added costs of having to provide a proof that the respective event hasn't yet been included in the nullifier set. Instead of completely pruning old event IDs from the db, only the actual content would be pruned and a compact cryptographic reference would be retained.
The implementation of this is not trivial. The most straightforward solution that I am aware of would use a sparse Merkle trie for storing the Event IDs and include a commitment to the root of that trie in the Pact state. Because pruned parts of the trie cannot be updated, one would have to keep adding layers to the top of the trie, essentially partitioning the trie in epochs. Proofs for recent events would remain cheap while proofs for older epochs would get increasingly more expensive.
The implementation requires to include the respective data structures (e.g. a sparse extensible Merkle trie) into the Pact state, which is a non-trivial change. An implementation could probably benefit from the ongoing work on adding a state root for the Pact state. Additionally, frontend and API level infrastructure would be needed, but could also be added later on based on actual demand for proving older events.
ZK technology could be used to reduce prove sizes and verification costs of old events to sizes that are feasible for on-chain use at the expense of more expensive proof creation costs. Proofs cost would increase gradually with the age of the proven event. This feature could be implemented independently at a later time based on demand.
Summary
I propose, that we analyze more carefully what the exact costs in terms of storage and scale are for the current situation and the proposed solutions.
We should then consider these and other possible options based on the time and effort that their implementation would require and weigh these against the expected benefits.
|
|
||
| Chainweb's cross-chain transfers are facilitated through SPV proofs. To briefly summarise, these proofs allow verification that the funds were burned on the source chain, thereby allowing the funds to be minted on the target chain. | ||
|
|
||
| Currently, SPV proofs have no expiration. This requires chainweb-node to retain certain block history indefinitely to validate cross-chain transfers, leading to one source of unbounded storage growth. As the network ages, storage requirements for maintaining complete block header history grow linearly with time, creating scalability concerns for node operators. Allowing the pruning of old state is especially important given the long-term goals of shallow Chainweb nodes, which will run with minimal chain history. Currently, the block state that needs to be retained indefinitely is on the order of tens of gigabytes. |
There was a problem hiding this comment.
Currently, the block state that needs to be retained indefinitely is on the order of tens of gigabytes.
How much of this is related to information needed to verify SPV proofs? My guess is that it is only a fraction of the total state space.
|
|
||
| # Design | ||
|
|
||
| We propose defining a constant `SPV_PROOF_EXPIRATION_BLOCKS`, the "SPV proof expiration window", which defines the maximum age of a valid SPV proof, in a number of block heights. An SPV proof is created at a certain blockheight, and will be valid for the expiration window after its creation. |
There was a problem hiding this comment.
An SPV proof is created at a certain blockheight,
Can you specify what exactly this block height is? There are at least block heights involved in an SPV proof:
- The height of the block where the event was emitted and
- The height of the block at which the event is proven, i.e. the root of the proof.
In theory, these two block heights can be arbitrary far apart, although, in practice, this is limited by the proof size and gas costs for verifying the proof. Typically proofs are of minimal size such that they only span the graph diameter in block height.
ZK compressed proofs make proofs across much larger block ranges feasible at constant cost size and verification costs.
| @@ -0,0 +1,36 @@ | |||
| --- | |||
| KIP: "0041" | |||
| Title: SPV Proof Expiration for Cross-Chain Transfers | |||
There was a problem hiding this comment.
I think, this title could use some clarification. It is not clear to me whether it is the SPV proof that is supposed to expire or rather the event that is proven. If it is the former, would it be possible to regenerate a new (valid) SPV proof for the same event, possibly with a more recent proof root?
|
I also propose that, for the short term we consider optimizing the existing (unlimited) storage to just 20 (or even 32) bytes per storage entry. |
|
Did we ever scan the mainnet history for unfinished cross-chain events (and finish unfinished event where possible)? We may do so and prune those parts of the history for which there provably exist no unfinished cross-chain events. |
|
The storage requirements for the block hashes in the header oracle for the mainnet history are currently less than 2GB (assuming 20 bytes per hash). The requirements for storing all cross chain event IDs are expected to be much smaller than that number, because only a small fraction of all blocks on mainnet contains x-yields. So, just pruning the header oracle would probably make up for most of the potential savings in storage space. |
This is a proposal to add an expiration window (in blockheight) to SPV Proofs as a way to help further trim the block state, as well as disallowing infinitely dangling cross-chain transfers. Implementation is at https://github.com/kadena-io/chainweb-node/pull/2050/files.