Skip to content

Add LSQ leashing 0.30.0.1 #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tweag-ev-ak wants to merge 14 commits into
base: leashing-stub-branch-for-pr
Choose a base branch
from
Draft

Add LSQ leashing 0.30.0.1 #18

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
c65cc4b
Add leashing
tweag-ev-ak Jan 12, 2026
f33caa4
Add leashing
tweag-ev-ak Jan 12, 2026
5ac599d
Add leashing 0.27.0.0 leash (#13)
axman6 Feb 19, 2026
9020145
wip: fixing tests
tweag-ev-ak Feb 17, 2026
ef8c646
Add leashing
tweag-ev-ak Feb 19, 2026
4179875
Add leashing
tweag-ev-ak Feb 19, 2026
4210cb3
Fix review comments
tweag-ev-ak Mar 2, 2026
f113d1e
Stop throwing AcquireFailurePointStateIsBusy error
tweag-ev-ak Mar 9, 2026
be4260c
Add crucialLsqClients config
tweag-ev-ak Mar 9, 2026
a3161f4
Cleanup LoE variables
tweag-ev-ak Mar 10, 2026
278cd8f
Fix tests
tweag-ev-ak Mar 11, 2026
1be3526
Update the ouroboros-network pin.
tweag-ev-ak Mar 13, 2026
fa25012
Update the ouroboros-network pin 2
tweag-ev-ak Mar 16, 2026
9f9c384
Provide an extra STM action for the forker's acquirement
tweag-ev-ak Mar 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions cabal.project
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,11 @@ if impl (ghc >= 9.12)
allow-newer:
-- https://github.com/kapralVV/Unique/issues/11
, Unique:hashable

source-repository-package
type: git
location: https://github.com/tweag/ouroboros-network
tag: 461bbd3f6a27bc282d8a2b7c54e9ef929f30efb0
--sha256: sha256-lEw68KGx16CkBktEeS4xyth6zh9doLewQqxHAiUrxcQ=
subdir:
ouroboros-network-protocols
2 changes: 1 addition & 1 deletion flake.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
supportedSystems = [
"x86_64-linux"
"x86_64-darwin"
#"aarch64-linux"
"aarch64-linux"
Comment thread
tweag-ev-ak marked this conversation as resolved.
"aarch64-darwin"
];
in
Expand Down
11 changes: 7 additions & 4 deletions ...s-diffusion/src/ouroboros-consensus-diffusion/Ouroboros/Consensus/Network/NodeToClient.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ mkHandlers ::
NodeKernelArgs m addrNTN addrNTC blk ->
NodeKernel m addrNTN addrNTC blk ->
Handlers m addrNTC blk
mkHandlers NodeKernelArgs{cfg, tracers} NodeKernel{getChainDB, getMempool} =
mkHandlers NodeKernelArgs{cfg, tracers} NodeKernel{getLeashingStateVar, getChainDB, getMempool} =
Handlers
{ hChainSyncServer =
chainSyncBlocksServer
Expand All @@ -144,9 +144,12 @@ mkHandlers NodeKernelArgs{cfg, tracers} NodeKernel{getChainDB, getMempool} =
localTxSubmissionServer
(Node.localTxSubmissionServerTracer tracers)
getMempool
, hStateQueryServer =
localStateQueryServer (ExtLedgerCfg cfg)
. ChainDB.getReadOnlyForkerAtPoint getChainDB
, hStateQueryServer = \rr ->
localStateQueryServer
(ExtLedgerCfg cfg)
getLeashingStateVar
(ChainDB.getCurrentChainWithTime getChainDB)
(ChainDB.getReadOnlyForkerAtPoint getChainDB $ rr)
, hTxMonitorServer =
localTxMonitorServer
getMempool
Expand Down
18 changes: 13 additions & 5 deletions ouroboros-consensus-diffusion/src/ouroboros-consensus-diffusion/Ouroboros/Consensus/Node.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ import qualified Codec.CBOR.Encoding as CBOR
import Codec.Serialise (DeserialiseFailure)
import qualified Control.Concurrent.Class.MonadSTM.Strict as StrictSTM
import Control.DeepSeq (NFData)
import Control.Monad (forM_, when)
import Control.Monad (forM_, when, join)
import Control.Monad.Class.MonadTime.SI (MonadTime)
import Control.Monad.Class.MonadTimer.SI (MonadTimer)
import Control.ResourceRegistry
Expand Down Expand Up @@ -504,8 +504,12 @@ runWith RunNodeArgs{..} encAddrNtN decAddrNtN LowLevelRunNodeArgs{..} =
systemStart
(blockchainTimeTracer rnTraceConsensus)

(genesisArgs, setLoEinChainDbArgs) <-
mkGenesisNodeKernelArgs llrnGenesisConfig
genesisArgs <- mkGenesisNodeKernelArgs llrnGenesisConfig
varGetLoEFragment <- newTVarIO $ pure ChainDB.LoEDisabled

@nfrisby nfrisby Feb 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO I didn't scrutinize this very much. Seems plausible as just a copy-paste from what used to be in mkGenesisNodeKernelArgs.

@axman6 axman6 Mar 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we always want this to start as LoEDisabled? What if the CrucialLsqClients is non-empty?

let setLoEinChainDbArgs argsCfg = argsCfg
{ ChainDB.cdbsArgs =
(ChainDB.cdbsArgs argsCfg) { ChainDB.cdbsLoE = join $ readTVarIO varGetLoEFragment }
}

let maybeValidateAll
| lastShutDownWasClean =
Expand Down Expand Up @@ -585,6 +589,7 @@ runWith RunNodeArgs{..} encAddrNtN decAddrNtN LowLevelRunNodeArgs{..} =
genesisArgs
DiffusionPipeliningOn
rnMempoolTimeoutConfig
varGetLoEFragment
nodeKernel <- initNodeKernel nodeKernelArgs
rnNodeKernelHook registry nodeKernel
churnModeVar <- StrictSTM.newTVarIO ChurnModeNormal
Expand Down Expand Up @@ -858,9 +863,10 @@ mkNodeKernelArgs ::
GSM.MarkerFileView m ->
STM m UseBootstrapPeers ->
StrictSTM.StrictTVar m (PublicPeerSelectionState addrNTN) ->
GenesisNodeKernelArgs m blk ->
GenesisNodeKernelArgs ->
DiffusionPipeliningSupport ->
Maybe Mempool.MempoolTimeoutConfig ->
StrictTVar m (ChainDB.GetLoEFragment m blk) ->
m (NodeKernelArgs m addrNTN (ConnectionId addrNTC) blk)
mkNodeKernelArgs
registry
Expand All @@ -880,7 +886,8 @@ mkNodeKernelArgs
publicPeerSelectionStateVar
genesisArgs
getDiffusionPipeliningSupport
mempoolTimeoutConfig =
mempoolTimeoutConfig
varGetLoEFragment =
do
let (kaRng, psRng) = split rng
return
Expand Down Expand Up @@ -911,6 +918,7 @@ mkNodeKernelArgs
, publicPeerSelectionStateVar
, genesisArgs
, getDiffusionPipeliningSupport
, varGetLoEFragment
}

-- | We allow the user running the node to customise the 'NodeKernelArgs'
Expand Down
156 changes: 69 additions & 87 deletions ...consensus-diffusion/src/ouroboros-consensus-diffusion/Ouroboros/Consensus/Node/Genesis.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,21 @@ module Ouroboros.Consensus.Node.Genesis
( -- * 'GenesisConfig'
GenesisConfig (..)
, GenesisConfigFlags (..)
, LoEAndGDDConfig (..)
, GDDConfig (..)
, defaultGenesisConfigFlags
, disableGenesisConfig
, enableGenesisConfigDefault
, mkGenesisConfig

-- * NodeKernel helpers
, GenesisNodeKernelArgs (..)
, LoEAndGDDNodeKernelArgs (..)
, GDDNodeKernelArgs (..)
, mkGenesisNodeKernelArgs
, setGetLoEFragment
) where

import Control.Monad (join)
import Data.Maybe (fromMaybe)
import qualified Data.Map.Strict as Map
import Data.Traversable (for)
import Data.Typeable (Typeable)
import GHC.Generics (Generic)
Expand All @@ -41,22 +41,19 @@ import Ouroboros.Consensus.MiniProtocol.ChainSync.Client.HistoricityCheck
( HistoricityCutoff (..)
)
import qualified Ouroboros.Consensus.Node.GsmState as GSM
import Ouroboros.Consensus.Storage.ChainDB (ChainDbArgs)
import qualified Ouroboros.Consensus.Storage.ChainDB as ChainDB
import qualified Ouroboros.Consensus.Storage.ChainDB.Impl.Args as ChainDB
import Ouroboros.Consensus.Util.Args
import Ouroboros.Consensus.Util.IOLike
import Ouroboros.Network.AnchoredFragment (AnchoredFragment)
import qualified Ouroboros.Network.AnchoredFragment as AF
import Ouroboros.Network.BlockFetch
( GenesisBlockFetchConfiguration (..)
)

-- | Whether to en-/disable the Limit on Eagerness and the Genesis Density
-- | Whether to en-/disable the Genesis Density
-- Disconnector.
data LoEAndGDDConfig a
= LoEAndGDDEnabled !a
| LoEAndGDDDisabled
data GDDConfig a =
GDDEnabled !a
| GDDDisabled
deriving stock (Eq, Generic, Show, Functor, Foldable, Traversable)

-- | Aggregating the various configs for Genesis-related subcomponents.
Expand All @@ -67,15 +64,15 @@ data GenesisConfig = GenesisConfig
{ gcBlockFetchConfig :: !GenesisBlockFetchConfiguration
, gcChainSyncLoPBucketConfig :: !ChainSyncLoPBucketConfig
, gcCSJConfig :: !CSJConfig
, gcLoEAndGDDConfig :: !(LoEAndGDDConfig LoEAndGDDParams)
, gcGDDConfig :: !(GDDConfig GDDParams)
, gcHistoricityCutoff :: !(Maybe HistoricityCutoff)
}
deriving stock (Eq, Generic, Show)

-- | Genesis configuration flags and low-level args, as parsed from config file or CLI
data GenesisConfigFlags = GenesisConfigFlags
{ gcfEnableCSJ :: Bool
, gcfEnableLoEAndGDD :: Bool
, gcfEnableGDD :: Bool
, gcfEnableLoP :: Bool
, gcfBlockFetchGracePeriod :: Maybe DiffTime
, gcfBucketCapacity :: Maybe Integer
Expand All @@ -89,7 +86,7 @@ defaultGenesisConfigFlags :: GenesisConfigFlags
defaultGenesisConfigFlags =
GenesisConfigFlags
{ gcfEnableCSJ = True
, gcfEnableLoEAndGDD = True
, gcfEnableGDD = True
, gcfEnableLoP = True
, gcfBlockFetchGracePeriod = Nothing
, gcfBucketCapacity = Nothing
Expand All @@ -115,7 +112,7 @@ mkGenesisConfig Nothing =
}
, gcChainSyncLoPBucketConfig = ChainSyncLoPBucketDisabled
, gcCSJConfig = CSJDisabled
, gcLoEAndGDDConfig = LoEAndGDDDisabled
, gcGDDConfig = GDDDisabled
, gcHistoricityCutoff = Nothing
}
mkGenesisConfig (Just cfg) =
Expand All @@ -141,10 +138,10 @@ mkGenesisConfig (Just cfg) =
{ csjcJumpSize
}
else CSJDisabled
, gcLoEAndGDDConfig =
if gcfEnableLoEAndGDD
then LoEAndGDDEnabled LoEAndGDDParams{lgpGDDRateLimit}
else LoEAndGDDDisabled
, gcGDDConfig =
if gcfEnableGDD
then GDDEnabled GDDParams{lgpGDDRateLimit}
else GDDDisabled
, -- Duration in seconds of one Cardano mainnet Shelley stability window
-- (3k/f slots times one second per slot) plus one extra hour as a
-- safety margin.
Expand All @@ -154,7 +151,7 @@ mkGenesisConfig (Just cfg) =
GenesisConfigFlags
{ gcfEnableLoP
, gcfEnableCSJ
, gcfEnableLoEAndGDD
, gcfEnableGDD
, gcfBlockFetchGracePeriod
, gcfBucketCapacity
, gcfBucketRate
Expand Down Expand Up @@ -189,7 +186,7 @@ mkGenesisConfig (Just cfg) =
csjcJumpSize = fromMaybe defaultCSJJumpSize gcfCSJJumpSize
lgpGDDRateLimit = fromMaybe defaultGDDRateLimit gcfGDDRateLimit

newtype LoEAndGDDParams = LoEAndGDDParams
newtype GDDParams = GDDParams
{ lgpGDDRateLimit :: DiffTime
-- ^ How often to evaluate GDD. 0 means as soon as possible.
-- Otherwise, no faster than once every T seconds, where T is the
Expand All @@ -198,81 +195,66 @@ newtype LoEAndGDDParams = LoEAndGDDParams
deriving stock (Eq, Generic, Show)

-- | Genesis-related arguments needed by the NodeKernel initialization logic.
data GenesisNodeKernelArgs m blk = GenesisNodeKernelArgs
{ gnkaLoEAndGDDArgs :: !(LoEAndGDDConfig (LoEAndGDDNodeKernelArgs m blk))
data GenesisNodeKernelArgs = GenesisNodeKernelArgs
{ gnkaGDDArgs :: !(GDDConfig GDDNodeKernelArgs)
}

data LoEAndGDDNodeKernelArgs m blk = LoEAndGDDNodeKernelArgs
{ lgnkaLoEFragmentTVar :: !(StrictTVar m (ChainDB.GetLoEFragment m blk))
-- ^ A TVar containing an action that returns the 'ChainDB.GetLoEFragment'
-- action. We use this extra indirection to update this action after we
-- opened the ChainDB (which happens before we initialize the NodeKernel).
-- After that, this TVar will not be modified again.
, lgnkaGDDRateLimit :: DiffTime
data GDDNodeKernelArgs = GDDNodeKernelArgs
{ lgnkaGDDRateLimit :: DiffTime
}

-- | Create the initial 'GenesisNodeKernelArgs" (with a temporary
-- 'ChainDB.GetLoEFragment' that will be replaced via 'setGetLoEFragment') and a
-- function to update the 'ChainDbArgs' accordingly.
-- | Create the initial 'GenesisNodeKernelArgs" .
mkGenesisNodeKernelArgs ::
forall m blk.
(IOLike m, GetHeader blk, Typeable blk) =>
forall m.
(IOLike m) =>
GenesisConfig ->
m
( GenesisNodeKernelArgs m blk
, Complete ChainDbArgs m blk -> Complete ChainDbArgs m blk
)
m GenesisNodeKernelArgs
mkGenesisNodeKernelArgs gcfg = do
gnkaLoEAndGDDArgs <- for (gcLoEAndGDDConfig gcfg) $ \p -> do
loeFragmentTVar <-
newTVarIO $
pure $
-- Use the most conservative LoE fragment until 'setGetLoEFragment'
-- is called.
ChainDB.LoEEnabled $
AF.Empty AF.AnchorGenesis
gnkaGDDArgs <- for (gcGDDConfig gcfg) $ \p -> do
Comment thread
tweag-ev-ak marked this conversation as resolved.
Outdated
pure
LoEAndGDDNodeKernelArgs
{ lgnkaLoEFragmentTVar = loeFragmentTVar
, lgnkaGDDRateLimit = lgpGDDRateLimit p
GDDNodeKernelArgs
{ lgnkaGDDRateLimit = lgpGDDRateLimit p
}
let updateChainDbArgs = case gnkaLoEAndGDDArgs of
LoEAndGDDDisabled -> id
LoEAndGDDEnabled lgnkArgs -> \cfg ->
cfg
{ ChainDB.cdbsArgs =
(ChainDB.cdbsArgs cfg){ChainDB.cdbsLoE = getLoEFragment}
}
where
getLoEFragment = join $ readTVarIO $ lgnkaLoEFragmentTVar lgnkArgs
pure (GenesisNodeKernelArgs{gnkaLoEAndGDDArgs}, updateChainDbArgs)
pure GenesisNodeKernelArgs{gnkaGDDArgs}

-- | Set 'gnkaGetLoEFragment' to the actual logic for determining the current
-- LoE fragment.
-- | Set the actual logic for determining the current LoE fragment.
setGetLoEFragment ::
forall m blk.
(IOLike m, GetHeader blk, Typeable blk) =>
STM m GSM.GsmState ->
-- | The LoE fragment.
STM m (AnchoredFragment (HeaderWithTime blk)) ->
StrictTVar m (ChainDB.GetLoEFragment m blk) ->
m ()
setGetLoEFragment readGsmState readLoEFragment varGetLoEFragment =
atomically $ writeTVar varGetLoEFragment getLoEFragment
where
getLoEFragment :: ChainDB.GetLoEFragment m blk
getLoEFragment =
atomically $
readGsmState >>= \case
-- When the Honest Availability Assumption cannot currently be
-- guaranteed, we should not select any blocks that would cause our
-- immutable tip to advance, so we return the most conservative LoE
-- fragment.
GSM.PreSyncing ->
pure $ ChainDB.LoEEnabled $ AF.Empty AF.AnchorGenesis
-- When we are syncing, return the current LoE fragment.
GSM.Syncing ->
ChainDB.LoEEnabled <$> readLoEFragment
-- When we are caught up, the LoE is disabled.
GSM.CaughtUp ->
pure ChainDB.LoEDisabled
forall m blk. (IOLike m, GetHeader blk, Typeable blk)
=> STM m (ChainDB.LeashingState blk)
-> STM m GSM.GsmState
-> STM m (Maybe (AnchoredFragment (HeaderWithTime blk)))
-- ^ The Genesis LoE fragment.
-> STM m (AnchoredFragment (HeaderWithTime blk))
-- ^ The LoE fragment.
-> StrictTVar m (ChainDB.GetLoEFragment m blk)
-> m ()
setGetLoEFragment readLeashingState readGsmState readGenesisLoEFragment readLoEFragment varGetLoEFragment =

@nfrisby nfrisby Feb 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between readLoEFragment and readGenesisLoEFragment?

My suspicion is that readLoEFragment should have a more distinct name now. TODO return here when I have an opinion about what that name should be

@tweag-ev-ak tweag-ev-ak Mar 2, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see #18 (comment)

atomically $ writeTVar varGetLoEFragment getLoEFragment
where
getLoEFragment :: ChainDB.GetLoEFragment m blk
getLoEFragment = atomically $ do
leashingState <- readLeashingState
if not $ Map.null leashingState then
Comment thread
tweag-ev-ak marked this conversation as resolved.
Outdated
ChainDB.LoEEnabled <$> readLoEFragment
-- readLoEFragment >>= \case
Comment thread
tweag-ev-ak marked this conversation as resolved.
Outdated
-- Just loeFrag -> pure $ ChainDB.LoEEnabled loeFrag
-- Nothing -> pure ChainDB.LoEDisabled
else
readGenesisLoEFragment >>= \case
Just glf -> do
-- leashing is disabled, run old behavior

@nfrisby nfrisby Feb 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my other big comment about "names should mention LSQ".

I think "leashing is disabled" should say "LSQ leashing is disabled", for example.

@nfrisby nfrisby Feb 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate concern: I don't think we need mutual exclusion "either the LSQ LoE or the GDD LoE". Because we can intersect them to correctly combine them, can't we?

@tweag-ev-ak tweag-ev-ak Mar 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate concern: I don't think we need mutual exclusion "either the LSQ LoE or the GDD LoE". Because we can intersect them to correctly combine them, can't we?

Yes, that's what we do right now in the leashing watcher:

    wNotify :: (LsqLeashingWatcherState blk) -> m ()
    wNotify LsqLeashingWatcherState{..} = do
        let
          lsqLeashingCandidates = Map.toList lsqLeashingState
          prefix = maybe curChain id genesisLoEFrag
          loeFrag = fst $ sharedCandidatePrefix prefix lsqLeashingCandidates

if there is a genesis LoE fragment, we use it as a base for sharedCandidatePrefix, otherwise it's current chain. Is it a correct approach?

Only later, in setGetLoEFragment, we decide which the LoE fragment to return, basing on both fragment variables, the lsqLeashingState and the gsm state.

readLoEFragment in setGetLoEFragment returns the resulting LoE fragment calculated by lsqLeashingWatcher.

readGenesisLoEFragment returns the genesis loe fragment calculated by gddWatcher.

Sorry that I confused you because of all these variables mess. I wanted to keep the original implementation as much as possible and extend it rather than changing the existent one. I thought it would be easier to understand the additions. Turned out it only messed up the understanding.

We can do that final calculation in the lsq leashing watcher instead of setGetLoEFragment. I just couldn't decide what's the correct approach. Either we do all the logic in the watchers and remove the getLoEFragment from setGetLoEFragment (it will become just plain readLoEFragment) or combine the whole calculation in setGetLoEFragment (what I ended up with to keep the old behaviour and make our additions more clear).

readGsmState >>= \case
-- When the Honest Availability Assumption cannot currently be
-- guaranteed, we should not select any blocks that would cause our
-- immutable tip to advance, so we return the most conservative LoE
-- fragment.
GSM.PreSyncing ->
pure $ ChainDB.LoEEnabled $ AF.Empty AF.AnchorGenesis
-- When we are syncing, return the current LoE fragment.
GSM.Syncing ->
pure $ ChainDB.LoEEnabled glf
-- When we are caught up, the LoE is disabled.
GSM.CaughtUp ->
pure ChainDB.LoEDisabled
Nothing ->
pure ChainDB.LoEDisabled
Loading
Loading