Multi party, networked offline heads (#1851)

This is a change I encountered when rebasing `raft-network` for #1720 and was useful back in the spike, but would have also been valuable in the `hydra-doom` use case. Anyways, this is adding a `--offline-head-seed` argument to offline mode and fixes the "simulation" opened head to be deterministic across multiple instances, resulting that the nodes can and do talk to each other and consequently sign snapshots. --- * [x] CHANGELOG updated * [x] Documentation updated * [x] Haddocks updated * [x] No new TODOs introduced
cardano-scaling · Feb 17, 2025 · 7c3c3cb · 7c3c3cb
2 parents b6d963e + ac61246
commit 7c3c3cb
Show file tree

Hide file tree

Showing 15 changed files with 456 additions and 448 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -8,12 +8,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 As a minor extension, we also keep a semantic version for the `UNRELEASED`
 changes.
 
-## [0.20.1] - UNRELEASED
+## [0.21.0] - UNRELEASED
 
-- Fix a bug where decoding `Party` information from chain would crash the node
-  or chain observer. A problematic transaction will now be ignored and not
-  deemed a valid head protocol transaction. An example was if the datum would
-  contain CBOR instead of just hex encoded bytes.
+- Fix a bug where decoding `Party` information from chain would crash the node or chain observer.
+  - A problematic transaction will now be ignored and not deemed a valid head protocol transaction.
+  - An example was if the datum would contain CBOR instead of just hex encoded bytes.
+
+- **BREAKING** Enable multi-party, networked "offline" heads by providing an `--offline-head-seed` option to `hydra-node`.
+  - Drop `hydra-nodde offline` as a sub-command. Use `--offline-head-seed` and `--initial-utxo` options to switch to offline mode.
 
 - Stream historical data from disk in the hydra-node API server.
 

diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md
@@ -173,7 +173,9 @@ If the `hydra-node` already tracks a head in its `state` and `--start-chain-from
 
 Hydra supports an offline mode that allows for disabling the layer 1 interface – the underlying Cardano blockchain from which Hydra heads acquire funds and to which funds are eventually withdrawn. Disabling layer 1 interactions allows use cases that would otherwise require running and configuring an entire layer 1 private devnet. For example, the offline mode can be used to quickly validate a series of transactions against a UTXO, without having to spin up an entire layer 1 Cardano node.
 
-To initialize the layer 2 ledger's UTXO state, offline mode takes an obligatory `--initial-utxo` parameter, which points to a JSON-encoded UTxO file. See the [API reference](https://hydra.family/head-protocol/api-reference/#schema-UTxO) for the schema.
+As an offline head will not connect to any chain, we need to provide an `--offline-head-seed` manually, which is a hexadecimal byte string. Offline heads can still use the L2 network and to make multiple `hydra-node` "see" the same offline head, the offline head seed needs to match along with provided [hydra keys](#hydra-keys).
+
+To initialize UTxO state available on the L2 ledger, offline mode takes an obligatory `--initial-utxo` parameter, which points to a JSON-encoded UTxO file. See the [API reference](https://hydra.family/head-protocol/api-reference/#schema-UTxO) for the schema.
 
 Using this example UTxO:
 ```json utxo.json
@@ -187,12 +189,13 @@ Using this example UTxO:
 }
 ```
 
-An offline mode hydra-node can be started with:
+A (single participant) offline Hydra head can be started with:
 ```shell
-hydra-node offline \
+hydra-node \
+  --offline-head-seed 0001 \
+  --initial-utxo utxo.json \
   --hydra-signing-key hydra.sk \
-  --ledger-protocol-parameters protocol-parameters.json \
-  --initial-utxo utxo.json
+  --ledger-protocol-parameters protocol-parameters.json
 ```
 
 As the node is not connected to a real network, genesis parameters that normally influence things like time-based transaction validation cannot be fetched and are set to defaults. To configure block times, set `--ledger-genesis` to a Shelley genesis file similar to the [shelley-genesis.json](https://book.world.dev.cardano.org/environments/mainnet/shelley-genesis.json).

diff --git a/hydra-cluster/hydra-cluster.cabal b/hydra-cluster/hydra-cluster.cabal
@@ -110,7 +110,6 @@ library
     , process
     , QuickCheck
     , req
-    , temporary
     , text
     , time
     , typed-process

diff --git a/hydra-cluster/src/HydraNode.hs b/hydra-cluster/src/HydraNode.hs
@@ -33,8 +33,8 @@ import Network.HTTP.Req (GET (..), HttpException, JsonResponse, NoReqBody (..),
 import Network.HTTP.Req qualified as Req
 import Network.HTTP.Simple (httpLbs, setRequestBodyJSON)
 import Network.WebSockets (Connection, ConnectionException, HandshakeException, receiveData, runClient, sendClose, sendTextData)
+import System.Directory (createDirectoryIfMissing)
 import System.FilePath ((<.>), (</>))
-import System.IO.Temp (withSystemTempDirectory)
 import System.Info (os)
 import System.Process (
   CreateProcess (..),
@@ -338,73 +338,75 @@ withHydraNode' ::
 withHydraNode' tracer chainConfig workDir hydraNodeId hydraSKey hydraVKeys allNodeIds mGivenStdOut action = do
   -- NOTE: AirPlay on MacOS uses 5000 and we must avoid it.
   when (os == "darwin") $ port `shouldNotBe` (5_000 :: Network.PortNumber)
-  withSystemTempDirectory "hydra-node" $ \dir -> do
-    let cardanoLedgerProtocolParametersFile = dir </> "protocol-parameters.json"
-    case chainConfig of
-      Offline _ ->
-        readConfigFile "protocol-parameters.json"
-          >>= writeFileBS cardanoLedgerProtocolParametersFile
-      Direct DirectChainConfig{nodeSocket, networkId} -> do
-        -- NOTE: This implicitly tests of cardano-cli with hydra-node
-        protocolParameters <- cliQueryProtocolParameters nodeSocket networkId
-        Aeson.encodeFile cardanoLedgerProtocolParametersFile $
-          protocolParameters
-            & atKey "txFeeFixed" ?~ toJSON (Number 0)
-            & atKey "txFeePerByte" ?~ toJSON (Number 0)
-            & key "executionUnitPrices" . atKey "priceMemory" ?~ toJSON (Number 0)
-            & key "executionUnitPrices" . atKey "priceSteps" ?~ toJSON (Number 0)
-            & atKey "utxoCostPerByte" ?~ toJSON (Number 0)
-            & atKey "treasuryCut" ?~ toJSON (Number 0)
-            & atKey "minFeeRefScriptCostPerByte" ?~ toJSON (Number 0)
-
-    let hydraSigningKey = dir </> (show hydraNodeId <> ".sk")
-    void $ writeFileTextEnvelope (File hydraSigningKey) Nothing hydraSKey
-    hydraVerificationKeys <- forM (zip [1 ..] hydraVKeys) $ \(i :: Int, vKey) -> do
-      let filepath = dir </> (show i <> ".vk")
-      filepath <$ writeFileTextEnvelope (File filepath) Nothing vKey
-    let p =
-          ( hydraNodeProcess $
-              -- NOTE: Using 0.0.0.0 over 127.0.0.1 will make the hydra-node
-              -- crash if it can't bind the interface and make tests fail more
-              -- obvious when e.g. a hydra-node instance is already running.
-              RunOptions
-                { verbosity = Verbose "HydraNode"
-                , nodeId = NodeId $ show hydraNodeId
-                , host = "0.0.0.0"
-                , port = fromIntegral $ 5_000 + hydraNodeId
-                , peers
-                , apiHost = "0.0.0.0"
-                , apiPort = fromIntegral $ 4_000 + hydraNodeId
-                , tlsCertPath = Nothing
-                , tlsKeyPath = Nothing
-                , monitoringPort = Just $ fromIntegral $ 6_000 + hydraNodeId
-                , hydraSigningKey
-                , hydraVerificationKeys
-                , persistenceDir = workDir </> "state-" <> show hydraNodeId
-                , chainConfig
-                , ledgerConfig =
-                    CardanoLedgerConfig
-                      { cardanoLedgerProtocolParametersFile
-                      }
-                }
-          )
-            { std_out = maybe CreatePipe UseHandle mGivenStdOut
-            , std_err = CreatePipe
-            }
-
-    traceWith tracer $ HydraNodeCommandSpec $ show $ cmdspec p
-
-    withCreateProcess p $ \_stdin mCreatedStdOut mCreatedStdErr processHandle ->
-      case (mCreatedStdOut <|> mGivenStdOut, mCreatedStdErr) of
-        (Just out, Just err) -> action out err processHandle
-        (Nothing, _) -> error "Should not happen™"
-        (_, Nothing) -> error "Should not happen™"
+  let stateDir = workDir </> "state-" <> show hydraNodeId
+  createDirectoryIfMissing True stateDir
+  let cardanoLedgerProtocolParametersFile = stateDir </> "protocol-parameters.json"
+  case chainConfig of
+    Offline _ ->
+      readConfigFile "protocol-parameters.json"
+        >>= writeFileBS cardanoLedgerProtocolParametersFile
+    Direct DirectChainConfig{nodeSocket, networkId} -> do
+      -- NOTE: This implicitly tests of cardano-cli with hydra-node
+      protocolParameters <- cliQueryProtocolParameters nodeSocket networkId
+      Aeson.encodeFile cardanoLedgerProtocolParametersFile $
+        protocolParameters
+          & atKey "txFeeFixed" ?~ toJSON (Number 0)
+          & atKey "txFeePerByte" ?~ toJSON (Number 0)
+          & key "executionUnitPrices" . atKey "priceMemory" ?~ toJSON (Number 0)
+          & key "executionUnitPrices" . atKey "priceSteps" ?~ toJSON (Number 0)
+          & atKey "utxoCostPerByte" ?~ toJSON (Number 0)
+          & atKey "treasuryCut" ?~ toJSON (Number 0)
+          & atKey "minFeeRefScriptCostPerByte" ?~ toJSON (Number 0)
+
+  let hydraSigningKey = stateDir </> "me.sk"
+  void $ writeFileTextEnvelope (File hydraSigningKey) Nothing hydraSKey
+  hydraVerificationKeys <- forM (zip [1 ..] hydraVKeys) $ \(i :: Int, vKey) -> do
+    let filepath = stateDir </> ("other-" <> show i <> ".vk")
+    filepath <$ writeFileTextEnvelope (File filepath) Nothing vKey
+  let p =
+        ( hydraNodeProcess $
+            -- NOTE: Using 0.0.0.0 over 127.0.0.1 will make the hydra-node
+            -- crash if it can't bind the interface and make tests fail more
+            -- obvious when e.g. a hydra-node instance is already running.
+            RunOptions
+              { verbosity = Verbose "HydraNode"
+              , nodeId = NodeId $ show hydraNodeId
+              , host = "0.0.0.0"
+              , port = fromIntegral $ 5_000 + hydraNodeId
+              , peers
+              , apiHost = "0.0.0.0"
+              , apiPort = fromIntegral $ 4_000 + hydraNodeId
+              , tlsCertPath = Nothing
+              , tlsKeyPath = Nothing
+              , monitoringPort = Just $ fromIntegral $ 6_000 + hydraNodeId
+              , hydraSigningKey
+              , hydraVerificationKeys
+              , persistenceDir = stateDir
+              , chainConfig
+              , ledgerConfig =
+                  CardanoLedgerConfig
+                    { cardanoLedgerProtocolParametersFile
+                    }
+              }
+        )
+          { std_out = maybe CreatePipe UseHandle mGivenStdOut
+          , std_err = CreatePipe
+          }
+
+  traceWith tracer $ HydraNodeCommandSpec $ show $ cmdspec p
+
+  withCreateProcess p $ \_stdin mCreatedStdOut mCreatedStdErr processHandle ->
+    case (mCreatedStdOut <|> mGivenStdOut, mCreatedStdErr) of
+      (Just out, Just err) -> action out err processHandle
+      (Nothing, _) -> error "Should not happen™"
+      (_, Nothing) -> error "Should not happen™"
  where
   port = fromIntegral $ 5_000 + hydraNodeId
 
+  -- NOTE: See comment above about 0.0.0.0 vs 127.0.0.1
   peers =
     [ Host
-        { Network.hostname = "127.0.0.1"
+        { Network.hostname = "0.0.0.0"
         , Network.port = fromIntegral $ 5_000 + i
         }
     | i <- allNodeIds

diff --git a/hydra-cluster/test/Test/CardanoNodeSpec.hs b/hydra-cluster/test/Test/CardanoNodeSpec.hs
@@ -18,7 +18,7 @@ import System.Directory (doesFileExist)
 import Test.Hydra.Cluster.Utils (forEachKnownNetwork)
 
 supportedNetworks :: [KnownNetwork]
-supportedNetworks = [Mainnet, Preproduction, Preview, Sanchonet]
+supportedNetworks = [Mainnet, Preproduction, Preview]
 
 supportedCardanoNodeVersion :: String
 supportedCardanoNodeVersion = "10.1.4"

diff --git a/hydra-cluster/test/Test/EndToEndSpec.hs b/hydra-cluster/test/Test/EndToEndSpec.hs
@@ -117,46 +117,79 @@ withClusterTempDir = withTempDir "hydra-cluster"
 
 spec :: Spec
 spec = around (showLogsOnFailure "EndToEndSpec") $ do
-  it "End-to-end offline mode" $ \tracer -> do
-    withClusterTempDir $ \tmpDir -> do
-      (aliceCardanoVk, aliceCardanoSk) <- keysFor Alice
-      (bobCardanoVk, _) <- keysFor Bob
-      initialUTxO <- generate $ do
-        a <- genUTxOFor aliceCardanoVk
-        b <- genUTxOFor bobCardanoVk
-        pure $ a <> b
-      Aeson.encodeFile (tmpDir </> "utxo.json") initialUTxO
-      let offlineConfig =
-            Offline
-              OfflineChainConfig
-                { initialUTxOFile = tmpDir </> "utxo.json"
-                , ledgerGenesisFile = Nothing
-                }
-      -- Start a hydra-node in offline mode and submit a transaction from alice to bob
-      aliceToBob <- withHydraNode (contramap FromHydraNode tracer) offlineConfig tmpDir 1 aliceSk [] [1] $ \node -> do
-        let Just (aliceSeedTxIn, aliceSeedTxOut) = UTxO.find (isVkTxOut aliceCardanoVk) initialUTxO
-        let Right aliceToBob =
+  describe "End-to-end offline mode" $ do
+    it "can process transactions in single participant offline head persistently" $ \tracer -> do
+      withClusterTempDir $ \tmpDir -> do
+        (aliceCardanoVk, aliceCardanoSk) <- keysFor Alice
+        (bobCardanoVk, _) <- keysFor Bob
+        initialUTxO <- generate $ do
+          a <- genUTxOFor aliceCardanoVk
+          b <- genUTxOFor bobCardanoVk
+          pure $ a <> b
+        Aeson.encodeFile (tmpDir </> "utxo.json") initialUTxO
+        let offlineConfig =
+              Offline
+                OfflineChainConfig
+                  { offlineHeadSeed = "test"
+                  , initialUTxOFile = tmpDir </> "utxo.json"
+                  , ledgerGenesisFile = Nothing
+                  }
+        -- Start a hydra-node in offline mode and submit a transaction from alice to bob
+        aliceToBob <- withHydraNode (contramap FromHydraNode tracer) offlineConfig tmpDir 1 aliceSk [] [1] $ \node -> do
+          let Just (aliceSeedTxIn, aliceSeedTxOut) = UTxO.find (isVkTxOut aliceCardanoVk) initialUTxO
+          let Right aliceToBob =
+                mkSimpleTx
+                  (aliceSeedTxIn, aliceSeedTxOut)
+                  (mkVkAddress testNetworkId bobCardanoVk, txOutValue aliceSeedTxOut)
+                  aliceCardanoSk
+          send node $ input "NewTx" ["transaction" .= aliceToBob]
+          waitMatch 10 node $ \v -> do
+            guard $ v ^? key "tag" == Just "SnapshotConfirmed"
+          pure aliceToBob
+
+        -- Restart a hydra-node in offline mode expect we can reverse the transaction (it retains state)
+        withHydraNode (contramap FromHydraNode tracer) offlineConfig tmpDir 1 aliceSk [] [1] $ \node -> do
+          let
+            bobTxOut = toCtxUTxOTxOut $ List.head (txOuts' aliceToBob)
+            Right bobToAlice =
               mkSimpleTx
-                (aliceSeedTxIn, aliceSeedTxOut)
-                (mkVkAddress testNetworkId bobCardanoVk, txOutValue aliceSeedTxOut)
+                (mkTxIn aliceToBob 0, bobTxOut)
+                (mkVkAddress testNetworkId bobCardanoVk, txOutValue bobTxOut)
                 aliceCardanoSk
-        send node $ input "NewTx" ["transaction" .= aliceToBob]
-        waitMatch 10 node $ \v -> do
-          guard $ v ^? key "tag" == Just "SnapshotConfirmed"
-        pure aliceToBob
-
-      -- Restart a hydra-node in offline mode expect we can reverse the transaction (it retains state)
-      withHydraNode (contramap FromHydraNode tracer) offlineConfig tmpDir 1 aliceSk [] [1] $ \node -> do
-        let
-          bobTxOut = toCtxUTxOTxOut $ List.head (txOuts' aliceToBob)
-          Right bobToAlice =
-            mkSimpleTx
-              (mkTxIn aliceToBob 0, bobTxOut)
-              (mkVkAddress testNetworkId bobCardanoVk, txOutValue bobTxOut)
-              aliceCardanoSk
-        send node $ input "NewTx" ["transaction" .= bobToAlice]
-        waitMatch 10 node $ \v -> do
-          guard $ v ^? key "tag" == Just "SnapshotConfirmed"
+          send node $ input "NewTx" ["transaction" .= bobToAlice]
+          waitMatch 10 node $ \v -> do
+            guard $ v ^? key "tag" == Just "SnapshotConfirmed"
+
+    it "supports multi-party networked heads" $ \tracer -> do
+      withClusterTempDir $ \tmpDir -> do
+        (aliceCardanoVk, aliceCardanoSk) <- keysFor Alice
+        (bobCardanoVk, _) <- keysFor Bob
+        initialUTxO <- generate $ do
+          a <- genUTxOFor aliceCardanoVk
+          b <- genUTxOFor bobCardanoVk
+          pure $ a <> b
+        Aeson.encodeFile (tmpDir </> "utxo.json") initialUTxO
+        let offlineConfig =
+              Offline
+                OfflineChainConfig
+                  { offlineHeadSeed = "test"
+                  , initialUTxOFile = tmpDir </> "utxo.json"
+                  , ledgerGenesisFile = Nothing
+                  }
+        let tr = contramap FromHydraNode tracer
+        -- Start two hydra-nodes in offline mode and submit a transaction from alice to bob
+        withHydraNode tr offlineConfig tmpDir 1 aliceSk [bobVk] [1, 2] $ \aliceNode -> do
+          withHydraNode tr offlineConfig tmpDir 2 bobSk [aliceVk] [1, 2] $ \bobNode -> do
+            waitForNodesConnected tr 20 $ aliceNode :| [bobNode]
+            let Just (aliceSeedTxIn, aliceSeedTxOut) = UTxO.find (isVkTxOut aliceCardanoVk) initialUTxO
+            let Right aliceToBob =
+                  mkSimpleTx
+                    (aliceSeedTxIn, aliceSeedTxOut)
+                    (mkVkAddress testNetworkId bobCardanoVk, txOutValue aliceSeedTxOut)
+                    aliceCardanoSk
+            send aliceNode $ input "NewTx" ["transaction" .= aliceToBob]
+            waitMatch 10 bobNode $ \v -> do
+              guard $ v ^? key "tag" == Just "SnapshotConfirmed"
 
   describe "End-to-end on Cardano devnet" $ do
     describe "single party hydra head" $ do