Keyset ID V2

[davidcaseria]

Summary

This PR introduces Keyset ID v2 (version byte 01) and updates the spec + vectors accordingly.

The Keyset ID v2 derivation is designed to be unambiguous across keysets by hashing both the key amounts and keyset metadata.

Changes

Keyset ID v2 (NUT-02)

• Full keyset ID: 33 bytes (hex string length 66)
  • 01 (version byte) + SHA256(preimage) (32 bytes)
• Short keyset ID (Token i field): first 8 bytes of the full keyset ID (hex string length 16)
  • Wallets MUST resolve short IDs to full IDs; ambiguous short IDs MUST fail parsing.

Derivation preimage (v2)

Preimage is constructed as UTF-8 bytes in this exact order:

• For each key (sorted by amount ascending): append "{amount}:{pubkey_hex}"
• Separate each amount:pubkey_hex pair with a comma (,)
• Append "|unit:{unit}"
• If input_fee_ppk is present and non-zero, append "|input_fee_ppk:{input_fee_ppk}"
  • If input_fee_ppk is omitted / null / 0, it MUST be omitted from the preimage.
• If final_expiry is present and non-zero, append "|final_expiry:{final_expiry}"

Then id = "01" + sha256(preimage).hexdigest().

Deterministic secrets (NUT-13)

• Version-based derivation: secret derivation method determined by keyset ID version.
  • 00: legacy BIP32 derivation
  • 01: HMAC-SHA256 derivation
• HMAC-SHA256 message:
  • message = b"Cashu_KDF_HMAC_SHA256" || keyset_id_bytes || counter_k_bytes || derivation_type_byte

Spec/test-vector updates in this PR

• Updated tests/02-tests.md V2 keyset IDs to match the new preimage rules (including input_fee_ppk behavior).
• Updated 02.md examples to use fully specified vector keysets (no ellipses), so IDs are verifiable.
• Updated tests/13-tests.md V2 secrets/r vectors to match the updated V2 keyset ID.
• Added a small helper script tools/regenerate_vectors.py to regenerate/sanity-check the affected vectors.

Implementation Status

• CDK Keysets V2 cdk#702 Keyset ID V2 Implementation (NUT-02) cdk#1505
• cashu-ts feat: update Keyset ID V2 derivation to latest spec cashu-ts#458
• Nutshell feat: keysets v2 nutshell#798
• gonuts
• nutmix change keyset algo lescuer97/nutmix#215

[callebtc]

Thank you!

Please:

• add a verbose description in the PR what this is about
• we should keep at least 7 bytes, as others have mentioned
• this PR seems to replace the old keyset ID instead of adding a new version. Everything would break. We should instead add a new additional version.

[a1denvalu3]

Should we maybe add a paragraph to this that describes the way the Mint handles proofs with Short IDv2?
The Mint calculates each keyset's short IDv2 in addition to IDv2 and uses the short IDv2s for lookups in cases where it receives proofs with short IDv2.

Alternatively a substitution could be performed by the client, but this means that it has to request v1/keys so it's an additional more request before a swap.

[prusnak]

Quite honest, I am not sure such small change justifies creating a new keyset version and the headache attached to it.

Maybe we should keep this improvement in mind and apply it together with some future proposal that changes something significant?

[davidcaseria]

@prusnak this was suggested to enable wallets to store proofs without referencing mint URLs. @thesimplekid @ok300 do you think this is still necessary?

[thesimplekid]

this was suggested to enable wallets to store proofs without referencing mint URLs.

The longer keyset ids to reduce the chance of a collision and the keyset expiry are things we've talked about awhile and I would like to see, I think it makes sense to move forward with this. I cant think of anything else we wanted to include that we should hold this for, but if there is I am open to it.

[prusnak]

the keyset expiry are things we've talked about awhile and I would like to see, I think it makes sense to move forward with this

I agree. This is exactly the kind of change I was mentioning when I said to "justify" creating a new keyset version.

[ok300]

A few NITs, otherwise it's an ACK from me:

The specified behavior is a bit ambiguous in 2 places.

The keyset id is in each Proof so it can be used by wallets to identify which mint and keyset it was generated from.

This doesn't say if it's the long or short ID. It should probably say "short ID" or rename the variable to s_id for clarity.

To save space, a Token, as defined in [NUT-00][00], SHOULD contain an abbreviated version of the keyset id in the Proof, (the s_id).

This makes it sound preferable, but optional to use the shorter s_id in the proof, whereas the PR text indicates it's mandatory, since the Token size is kept unchanged.

[a1denvalu3]

@davidcaseria I think it would be better if the long keyset ID was 32-bytes:

• Keyset version byte
• the first 31-bytes out of the 32-bytes SHA-256 digest

[a1denvalu3]

CDK draft: cashubtc/cdk#702

[prusnak]

I think it would be better if the long keyset ID was 32-bytes:

• Keyset version byte
• the first 31-bytes out of the 32-bytes SHA-256 digest

Why?

[Egge21M]

Re: Derivation and invalid results (k > N or k == 0)

Usually you would k % N to get the value into range, but because of limitations of some implementations this is not possible. BIP-32 defines

In case parse256(IL) ≥ n or ki = 0, the resulting key is invalid, and one should proceed with the next value for i. (Note: this has probability lower than 1 in 2127.)

I think this should be our default for all derivations too. It requires implementations to bubble up that error to the application layer to properly increment counters (e.g. counter 2 is requested, but produces an invalid result, so counter 3 will be consumed), but that is a sound way to handle this edge case

[robwoodgate]

Re: Derivation and invalid results (k > N or k == 0)

In an ideal world, we would reduce mod N, which then reduces the failure to r = 0 only, a 1 in 2^256 event equivalent to guessing a private key.

But if we cannot do this reliably in all implementations, it may be "prudent" to throw for values > N, specifically in the case of SHA256 digests, as it removes the miniscule risk of some implementations not being able to derive blinding factors for certain hashes... which Murphy's law says will be the most expensive mistake possible.

Maybe that's why BIP32 specified as discard and retry, not reduce %N.

Really wish mod n wasn't an issue, the single subtract is so elegant.

[robwoodgate]

Cross referencing the NUT-00 conventions PR, which currently warns against doing MOD N. #322

Will need to update this if we proceed with the the proposed blinding scheme here.

[callebtc]

I think all points should be addressed now.

[robwoodgate]

@callebtc - the changes are looking great. I think we are there!
Just one housekeeping comment - should verify_02.py be placed under tools instead of in the root folder?

[lescuer97]

should we add a couple test vectors for the preimage?

[robwoodgate]

ACK @ 6e785b2

Epic work, I think this is a much stronger spec now.

[prusnak]

Looking at this code it occurred to me, that this works fine only if there are no duplicate entries in key values of keys. And this seems to be the case since keys is expected to be a dict. (otherwise the keys would need to be a list of tuples instead of a dict).

However, does the keyset spec disallow multiple public keys with the same denomination? I don't think we should allow this, but in that case, we should add this limitation to the keyset spec. Or is such limitation already in the keyset spec?

[thesimplekid]

I think it's worth noting that including the fee in the ID now means that mints cannot change the fee of past keysets. This to some extent reduces the mint's ability to dynamically change fees in response to a DoS, for example, as the first operation will still have the lower fee, only the second operation will incur the higher fee of the new keyset. I am not necessarily against this change; I just think we should be aware of it and make sure we are making this tradeoff intentionally.