Enhance wallet backup and recovery with a mnemonic hsm_secret and standard taproot wallet derivations #8400
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustyrussell
requested changes
Jul 8, 2025
sangbida
force-pushed
the
bip39-hsm
branch
2 times, most recently
from
5de3e77 to
1ae8dc0
Compare
sangbida
force-pushed
the
bip39-hsm
branch
3 times, most recently
from
e8c0864 to
3e25fd9
Compare
Comment on lines
+194
to
+200
| *err = HSM_SECRET_OK; | ||
| return hsms; | ||
| } |
There was a problem hiding this comment.
This gets called extract_hsm_secret and expects the err field to be populated.
| hsmd_mutual_version = maxversion < our_maxversion ? maxversion : our_maxversion; | ||
| return req_reply(conn, c, hsmd_init(hsm_secret, hsmd_mutual_version, | ||
|
|
||
| /* This was tallocated off NULL, and memleak complains if we don't free it */ |
There was a problem hiding this comment.
Yeah, really, it's because not freeing it is wrong! Don't complain about your tools!
sangbida
force-pushed
the
bip39-hsm
branch
18 times, most recently
from
cee3a15 to
c8177fa
Compare
Update the exposesecret plugin to work with the new unified HSM secret format that supports BIP39 mnemonics. Changelog-Added - exposesecret now has a mnemonic field
BIP86 derivation requires the full 64-byte seed that comes from the BIP39 mnemonic. The first 32 bytes goes towards to master seed material and the nest 32 bytes go towards the chaincode, so we need the entire 64 bytes for deterministic derivations. I've kept the old secret struct in for now for backwards compatibility and also added some accessors which will eventually die in this branch's git multiverse but that's a spoiler, they're on the ride for the next few commits at least to help us migrate to this length aware API throughout the rest of the code without making a lot of breaking changes.
hsmd: plumb length-aware secret into hsmd_init; keep 32B mirror BIP86 (from BIP39) wants the full 64-byte BIP32 seed. This commit plumbs a variable-length (32/64B) secret into hsmd and uses the accessors from the previous commit. We keep the old 32B hsm_secret mirror and, for now, only use the first 32 bytes so legacy paths keep working. Spoiler: HKDFs will keep using the 32B seed; only wallet address derivation will switch to the full 64B in a follow-up.
Here's some *foreshadowing* for what's to come. Here's what we're aiming for with our derivation flow:
Derivation split (hardened vs unhardened)
========================================
┌───────────────┐
│ HSM │ (secrets live here)
│ │
│ BIP39 → seed (64B)
│ ↓
│ m/86'/0'/0' ← derive hardened base (private)
│ ↓ (neuter)
│ BIP86 base xpub ← public-only + chain code
│ ↓
│ [send once over wire]
└───────────────┘
│
▼
┌───────────────────────┐
│ lightningd / wallet │
│ │
│ local (unhardened) derivations:
│ /0/i → external
│ /1/i → change
│ │
│ P2TR(BIP86) from pubkey_i
│ (optionally: CHECK with HSM)
└───────────────────────┘
We want to do part of the derivation inside hsmd and then send this base "pubkey" over the wire so our wallet can do the remaining derivation based on the address type and index. This lays the foundation for the base key wire message.
BIP86 wants the full 64-byte BIP32 seed (from BIP39). This wires up BIP86 support so the HSM derives the hardened base m/86'/0'/0' inside the box, and exposes helpers: • derive_bip86_base_key() // m/86'/0'/0' • bip86_key(index) // m/86'/0'/0'/0/index Spoiler: derive_bip86_base_key() and bip86_key() now live in libhsmd.c as they will later be used to check the derived wallet address against hsmd's derivation, this is just to sanity check that we haven't had an accidental bit flip while we have generated this address.
RIP to this commit there's a good chance a lot of this code doesn't even make this into the final PR. Pour one out for the fallen lines of code. This commit is doing the rest of the derivation. There was a significant overlap between the bip32_pubkey derivation and the bip86_pubkey derivation so that has been refactored in one place.
Add the UTXO_P2TR_BIP86 in preparation to add BIP86 wallet functions such as newaddr, listaddr etc. We also add a new index in the database for BIP86 as this is using a completely different derivation path and hsm_secret.
We should now be able to get BIP86 Taproot addresses through lightning-cli! For now we're just adding taproot addresses.
In the case where we receive a taproot utxo we want to be able to tell if it was derived using a BIP32 seed or a BIP86 seed. Considering we will only be supporting BI86 type wallet addresses for mnemonics we can check if the out secret is 64 bytes long and if it is we can use our BIP86 for the withdrawal.
This commit is updating hsmtool and exposesecrets to use the new pattern for storing the secret, which is the secret_data and secret_len, to support both 64 byte and 32 byte seeds.
This commit fixes an issue where BIP86 addresses were not being discovered during wallet recovery/rescan operations. The root cause was that init_txfilter() only populated the transaction filter with BIP32-derived keys, preventing lightningd from recognizing BIP86 UTXOs during blockchain scans. Now both BIP32 and BIP86 derived scripts are included in the filter when BIP86 derivation is enabled. This ensures that wallets restored from BIP39 mnemonics can properly discover and display previously funded BIP86 addresses without requiring manual address generation first.
We're removing --use-bip86-derivation. Since a mnemonic will now be the standard hsm_secret BIP86 base wallet addresses will also be the standard.
This simplifies the UTXO type system by removing the separate BIP86 enum value. P2TR addresses will now use unified derivation logic based on the wallet's HSM secret type rather than having separate enum values."
Add TLV field to hsmd_init_reply_v4 to communicate the HSM secret type (mnemonic vs legacy) from HSM to lightningd. This allows lightningd to automatically determine whether to use BIP86 or BIP32 derivation without needing separate address types.
…IP32 approach Simplify wallet address generation by using a unified approach where the derivation method (BIP86 vs BIP32) is determined by the wallet's HSM secret type rather than having separate address types.
Changelog-Removed: Remove hsm_encryption files as they have now been replaced by hsm_secret
This schema change updates newaddr to remove bip86 which was previously added, since don't want to make unnecessary schema changes this is being removed. The generated files for the exposesecret schema change are also being added
Introduces a generic utility function to replace the repeated pattern of sodium_mlock() + tal_add_destructor()
sangbida
force-pushed
the
bip39-hsm
branch
2 times, most recently
from
c89a9ef to
1a79fab
Compare
sangbida
changed the title
sangbida
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR significantly changes how the hsm_secret works in Core Lightning. It's now being changed from 32 bytes to a BIP39 mnemonic with an option for passphrase.
Changes
Closes #8381
Checklist
Before submitting the PR, ensure the following tasks are completed. If an item is not applicable to your PR, please mark it as checked: