Implement block-submitting Imp functions

[neilmayhew]

Description

Add functions to:

• submit a list of fixed-up transactions in a block
• capture submitted transactions in a "simulation" and then group them into a block and submit it

Checklist

• Commits in meaningful sequence and with useful messages.
• Tests added or updated when needed.
• CHANGELOG.md files updated for packages with externally visible changes.
  NOTE: New section is never added with the code changes. (See RELEASING.md).
• Versions updated in .cabal and CHANGELOG.md files when necessary, according to the
  versioning process.
• Version bounds in .cabal files updated when necessary.
  NOTE: If bounds change in a cabal file, that package itself must have a version increase. (See RELEASING.md).
• Code formatted (use scripts/fourmolize.sh).
• Cabal files formatted (use scripts/cabal-format.sh).
• CDDL files are up to date (use scripts/gen-cddl.sh)
• hie.yaml updated (use scripts/gen-hie.sh).
• Self-reviewed the diff.

[lehins]

I really do like the direction where this is going, however this PR does need some changes.
I'll be happy to talk more about all the suggestions that I made in this PR

[lehins]

I think in order to have it consistent with trySubmitTx, this function should accept a list of transactions that have not been fixed up and this function itself will call the simulateThenRestore in order to do all the fixing up and then submission as a block.

If we don't do that then usability of these functions will be limited.

That being said, naturally we'll need a help like this that will be capable of submitting transactions that have already been fixed up and ready to go into a block, I just think it needs to have a special name that is distinct from trySubmitBlock. Maybe trySubmitFixedUpBlock 🤷

[neilmayhew]

Yes, this makes sense.

There are two Conway tests that I've updated to use these functions (in eras/conway/impl/testlib/Test/Cardano/Ledger/Conway/Imp/BbodySpec.hs). The first doesn't require running the simulation itself, but the second definitely does, because the transactions all depend on each other. Also, the tests I'm porting from AlonzoBBODY work that way too. I think we'll find that this style will be more common, but time will tell.

Also, I found that it was just more convenient to write test code as if the transactions are being submitted, so that they use the same patterns as the rest of our test code, and then harvest them afterwards (by wrapping the transaction-generating code inside withTxsInBlock) rather than having to save the transactions explicitly. The fact that they're submitted in a block is then abstracted away from the code that creates the transactions.

I'll make all the other improvements (such as splitting events and transactions, and using SMaybe) first and then see how trySubmitBlock looks. Also, I'll push my latest changes to my AlonzoBBODY PR so that we can see what that test code looks like.

[neilmayhew]

I now have tryTxsInBlock that provides all the functionality for the other functions, and I've trimmed down the list of other functions to just:

• submitBlock_
• submitBlock
• submitFailingBlock
• submitFailingBlockM
• withTxsInBlock_
• withTxsInBlock
• withTxsInFailingBlock
• withTxsInFailingBlockM

The submit functions are just simple wrappers for the with functions that run traverse_ submitTx_ on the input list first.

The two main with functions, withTxsInBlock and withTxsInFailingBlockM just force and check the result of tryTxsInBlock and the other two with functions are simple wrappers for these two.

[neilmayhew]

Also, tryTxsInBlock uses impTransactionsFrom instead of simulateThenRestore because we actually want the final state after all the transactions have been through submitTx. simulateThenRestore is now used just in tests that want to run rules functions explicitly (currently, only totalRefScriptSizeInBlock).

[neilmayhew]

OK, this is ready for another review now

[neilmayhew]

With the new commit to switch to using applyBlockEither, I think this is complete. As we discussed, I won't change trySubmitTx for now, but will make a follow-up PR after #5461 is merged.

[neilmayhew]

I had to change the design slightly, because there wasn't a way to submit a block containing failing transactions, because failing transactions aren't recorded. I therefore changed tryTxsInBlock to take an explicit list of (fixed-up) transactions and added a new wrapper function withTxsInBlockEither that has the previous functionality.

Unfortunately, tryTxsInBlock now has to take an additional finalState parameter because there's no longer a way to know what the final ImpTestState should be. This is ugly and I'd like to remove it, but it will require refactoring trySubmitTx to split out the functionality that updates the state, and this PR has hung around long enough already (and has required a couple of tricky rebases).

[neilmayhew]

I suppose another possibility would be to have trySubmitTx record all transactions, not just successful ones.