Revamping paysXXX with more flexible payments

[mmontin]

This PR introduces two payment smart constructors to allow paying a script with no specified value. This idea is to rely on the option txOptEnsureMinAda which should not force users to specify a value when activated.

Following @florentc comments and some more thoughts, this PR actually provides a new convenient universal way of building payments by having a Payable structure that can be populated by various elements. The old helpers are still there, but are now built from it.

Typically, following this change all payments will look like:

myValidator `receives` PayableAnd (InlineDatum dat) (Value val)

or:

alice `receives` (Value val <&&> VisibleHashedDatum dat)

As a side consequence, this in fact solves the initial intent of this PR, as nothing forces users to specify a value if they wish it to be computed automatically from the min ada restriction. Such a statement will be totally valid:

alice `receives` (HiddenHashedDatum dat)

Some remarks/comments/questions:

1. It is currently possible to pay nothing at all (using mempty) which should not have many practical use cases.

2. This PR also renders obvious the prevalent nature of the recipient in the payment. While our payment have the recipient as one of many field, it is actually the only one that is mandatory. There is always a recipient to a payment. This changes makes is more obvious.

3. I randomly chose (&>) as an operator to compose payable elements, but I'm open to better suggestions.

4. I don't like that the users have to tediously use the long constructors for TxSkelOutDatum. Remove the TxSkelOut part would help, but maybe there's an even better way to circumvent this, but I have not yet found any.

5. The Payable structure is composable and relies of the alternative instance for Maybe for all its fields excepts values. Values are instead composed using their semigroup instance. This means that if you pay, let's say, a reference script twice, only the second instance will be kept, while if you pay values twice, they will be added, which should be very convenient.

6. To avoid have to parenthesis everything, I set proper priorities for both introduced operators (receives and &>) so that they can be used conveniently together. This means that the latter has priority over the former.

7. To emphasize the usefulness of receives and lighten the library, I removed all smart constructors related to payments. This has the only drawback of disconnecting typed validators to the type of their datum when paying to them. This is debatable whether that's a concern or not.

[florentc]

Is it worth having other smart constructors to remember to use (and maintain) when the user may just use the classic ones and provide mempty/ada 0/def or Nothing/TxSkelNoDatum themselves?

[mmontin]

@florentc @yannham any of you available for another round of reviews?

[florentc]

I am not a very big fan of the new concrete API but the inner changes and the Payable seem neat.

In my previous message, I was saying I'd rather have explicit mention of the value and datum all the time, even when not there (with explicit mempty or Nothing) than many different smart constructors with various names for the different combinations of "there is a value"/"no value"/"datum"/"no datum" etc. The reason is I think that there is something neat and readable in a predictable structure that is always there: address + value + datum.

Now that things can be passed in any order, and be optional, that makes the definition of an output a bit too liberal and, in the end, in my opinion only (this is very subjective) less consistent and readable in the big picture.

Besides, once again this is only personal preference and I know from experience we are opposed on that 😄, I think infix operators/functions hinder readability and writability. I'd always rather have a classic prefix function (pays/outputs/whatever) followed by a list of parameters, knowing the action before the subject. In code formatting, it often plays better, and it is easier to write. I know that you put effort into setting up the right precedence for the operator here but still, there is often some unexpected edge case that comes to bite us eventually. That's just a cosmetic remark.

I think we gained a lot in terms of ease of use and readability when we dropped the very old way to define transactions skeletons through combination of constraints involving custom operators and instead chose a very simple product type with named fields and default values for each of them. That change was more verbose but more readable. I have the impression that the current change goes in the opposite direction. In fact, maybe we need the same thing we have for skeleton adapted to outputs? Could we have an outputTemplate where we override fields such as outputValue, outputDatum, outputRefScript, etc?

This is a very big text just to express a personal opinion, don't take it for more than what it is 👍

[mmontin]

Thanks for the feedback @florentc I understand your concern. I can at least reintroduce the old constructor paysScript and paysPK as it does not hurt and it will have the upside to keep a connection between a typed validator and its datum type (in the case of paysScript).

[yannham]

I think the general approach is better than what was previously there. Having a class makes the usage more streamlined and uniform at the callsite, instead of having to use a dozen of specialized helpers.

However, maybe because I'm not a die hard Haskeller, I can only agree with @florentc on the operator usage. I generally find that Haskell is already heavy on operators, which means that you need to keep much more in your head (combine or mappend is more self-explanatory than <>, if you come from another language).

Adding idiosyncratic operators is worse: now you need to know operators specific to one library. I'm not sure it's worth it, as the gain is small here IMHO: infix functions in Haskell makes it already ok to use functions as operators, and adding parentheses to the right of receives isn't a big deal. In fact, I would even say that the current precedence of &> quite confusing, because for all built-in operators in any language out there, function application has usually always higher precedence than infix operators: f x && y almost always means (f x) && y, and f x + g y always mean (f x) + (g y) and not f (x + g y). So my brain has a natural tendency of parsing foo `receives` bar &> baz as (foo `receives` bar) &> baz, which I need to consciously fight against.

Though, it is a matter of taste, and I know not everyone agree 🙂

[yannham]

It is currently possible to pay nothing at all (using mempty) which should not have many practical use cases.

About this, a possibility would be to not make Payable a monoid, but just a semigroup, right? Then if you need mempty as a convenience for building payments it's always possible to add a dedicated helper emptyPayment or something. On the other hand it's tempting to make it a monoid, if you have a natural default value...

[mmontin]

Thank you very much for your feedback @florentc and @yannham.

While reading your messages, I realized that I have not motivated this contribution enough, a wrong that I intend to right in this message.

To me this change is absolutely necessary for a lot of reasons:

• Having countless helpers to build payments made little sense and was very obscure, not to mention there were (a lot of) missing ones.
• Implementing the missing ones also made little sense because there were too many of them. There simply are too many combination of possible kinds of payments.
• However, if you happened to want to define a payment that belonged in the "missing" category you had to go all the way into understanding our obscure generic output definition which is unreasonable.
• Our paysPK constructor was pushing users into thinking that we cannot (or should not) have datums in PK outputs, which is extremely misleading as this is very useful in many cases, such as to prevent double satisfaction. The API should push users into using the tool and concepts properly, not the other way around.
• Similarly, our paysScript constructor forced users into always placing a datum in a script output, and that this datum should have the right type, while if fact both are not mandatory.
• The presence of paysPK and paysScript with different signatures and names gave the impression that it is fundamentally different to pay to a script or a PK, while the only difference is the nature of the credential
• We were forced to specify a value in any payment, which does not make any sense whenever the actual ada value is only used to sustain the UTXO cost, which happens quite often, in which case cooked computes and places the right amount for us.
• The functions such as withDatum made little sense as well, because they were basically a hack to be able to change the type of the datum, or to artificially add a datum to a PK output.
• The API was overloaded overall, with both a bunch of (somewhat arbitrarily chosen) smart constructor and a bunch of with functions.

Overall, I found this API very inconvenient to use. Not terrible by any means, as we used it a lot, but inconvenient in the long run, and what is inconvenient can very likely be improved.

However, regarding the content of my contribution itself. I completely agree that it can be improved. In particular, my operator is a bit random and the priorities are convenient but impractical and lead to code chunks unusual to parse. This is however just the tip of the iceberg. The idea here is to have a composable notion of payment that leads to a simpler (and smaller) API, containing basically only two things: having some party receiving some payable structure, and being able to compose payable elements.

The issue is that you cannot really have an instance of Monoid on things that can implement an interface, which is why I had to add a custom operator. As the monoid operator is widely used in the community, this would have been very easily accepted by users.

If we want to adopt the "txSkel" way of doing things with a default payment that users can fill out, this is possible but it has drawbacks:

• we force the fields of the record to have strict types, such as Value, and users to manually call their translation functions (if they want to pay only ada for instance). Maybe we could use GADTs to enforce some constraints on the field without forcing their types, which can be investigated though.
• these fields are optional (thus of Type Maybe) so users would have to artificially add a Just whenever they want to inhabit a field.
• this will lead to lengthy payment definitions with records in records. (this is only my personal opinion, but I already find it annoying that we have to open up another record when we want to override some options).

Another solution would be to just remove the priorities of the operators I defined (forcing users to parenthesis their payments, which is sensible) and maybe have a function instead of an operator to combine payable elements that could be used infix if needed, in which case we have to find a sensible name for it. I argue that in any case users have to remember something: either the function name or the operator name. This is unavoidable but, admittedly, a function name is usually easier to remember because it is made of actual meaningful words.

To summarize, I feel like this contribution is essential in our journey to improving cooked validator API, but I'm very open to suggestions as to how to overcome the limitations of my current attempt, which is not without challenges.

[yannham]

The issue is that you cannot really have an instance of Monoid on things that can implement an interface, which is why I had to add a custom operator. As the monoid operator is widely used in the community, this would have been very easily accepted by users.

I agree - a Monoid instance would have been reasonable. By the way, I haven't checked the types, but do you really need to compose payments in practice, or just to pay several things in a row? That is, could we currently replace most usage of foo `receives` bar &> baz by foo `receives` bar `receives` baz ? I don't find that too bad, and that wouldn't need any new operator or function name. Of course if the main use case is composing stand-alone payments without a receiver at hand this won't work. But I suspect this is not the case.

We can also reconsider later if this first simple approach has too much friction.

I argue that in any case users have to remember something: either the function name or the operator name. This is unavoidable but, admittedly, a function name is usually easier to remember because it is made of actual meaningful words.

On this point, it seems that you are mostly considering writing code 🙂 my personal reservations rather revolve around reading (indeed, when writing, you need to look for the proper function/operator to use anyway). IMHO a casual reader will have a harder time making sense of an arbitrary operator with an arbitrary precedence rather than a named function (if we can pick a good name).

[mmontin]

Following the comments this PR had and the various discussions / questions it raised, I have crafted another version of this contribution, which, I believe, should be the final one. The idea is to rely on small type families to ensure that payments are built in a sensible way. I took inspiration from @0xd34df00d's blog post to encode the payments requirements at type level.

The code is actually pretty small in the end. We have 3 small type families:

1. The first one defines list non-membership:

type family (∉) (el :: a) (els :: [a]) :: Constraint where
  x ∉ '[] = ()
  x ∉ (x ': xs) = TypeError ('Text "Unable to assign twice the following output feature: " ':<>: 'ShowType x)
  x ∉ (_ ': xs) = x ∉ xs

2. The second one defines disjoint lists:

type family (⩀) (els :: [a]) (els' :: [a]) :: Constraint where
  '[] ⩀ _ = ()
  (x ': xs) ⩀ ys = (x ∉ ys, xs ⩀ ys)

3. The last one is list union:

type family (∪) (xs :: [a]) (ys :: [a]) :: [a] where
  '[] ∪ ys = ys
  (x ': xs) ∪ ys = x ': (xs ∪ ys)

From these, we can build a GADT of payments. The idea is that each leaf populates a specific portion of the payment (either a datum, ref script and so on) and the composition only allows to combine payable elements with disjoint populated portions:

data Payable :: [Symbol] -> Type where
  -- | Hashed datums visible in the transaction are payable
  VisibleHashedDatum :: (TxSkelOutDatumConstrs a) => a -> Payable '["Datum"]
  -- | Inline datums are payable
  InlineDatum :: (TxSkelOutDatumConstrs a) => a -> Payable '["Datum"]
  -- | Hashed datums hidden from the transaction are payable
  HiddenHashedDatum :: (TxSkelOutDatumConstrs a) => a -> Payable '["Datum"]
  -- | Reference scripts are payable
  ReferenceScript :: (ToVersionedScript s) => s -> Payable '["Reference Script"]
  -- | Values are payable
  Value :: (ToValue a) => a -> Payable '["Value"]
  -- | Staking credentials are payable
  StakingCredential :: (ToMaybeStakingCredential cred) => cred -> Payable '["Staking Credential"]
  -- | Payables can be combined as long as their list of tags are disjoint
  PayableAnd :: (els ⩀ els') => Payable els -> Payable els' -> Payable (els ∪ els')

How this solves most (if not all) the concerns / comments / questions raised by this work:

1. It gives a proper semantics for composition. There is no longer a discrepancy between composition of values and composition of the rest, nor there is an arbitrary choice between various instance of similar payable elements. Indeed, it is no longer possible to combine payable elements of the same nature. Thus, at most, a payable structure will be composed of 4 elements composed together, one of each kind.
2. It is not longer possible to pay "nothing". A payment has to have at least one of the payable element present. Since no constructor of Payable gives an element of type Payable '[] this requirement is now ensured by construction.
3. There is no longer any significant overhead in building payments and users will no longer be incentivized to build their own helpers to build payments.
4. Reciprocally, there is no cryptic name and hidden instances of things being payable, as each payable element will be referred to and defined with a dedicated constructor. Even payment composition is done through the use of PayableAnd.
5. There is still the opportunity to combine payments using the infix operator (<&&>) however: it is totally optional and basically an alias for PayableAnd and it is easy to remember and symmetrical (which is made possible by the well defined composition of payments).

Overall, this solution seems to be satisfactory. Additionally, the type families defined are only used internally and are not visible to users, which will not have to explicitly deal with them. In particular, the function building TxSkelOut from Payable that should always be used by users is pretty straightforward and devoid of type complexity:

receives :: (Show owner, Typeable owner, IsTxSkelOutAllowedOwner owner, ToCredential owner) => owner -> Payable els -> TxSkelOut
receives owner = go $ Pays $ ConcreteOutput owner Nothing TxSkelOutNoDatum mempty $ Nothing @(Script.Versioned Script.Script)
  where
    go :: TxSkelOut -> Payable els -> TxSkelOut
    go (Pays output) (VisibleHashedDatum dat) = Pays $ setDatum output $ TxSkelOutDatum dat
    go (Pays output) (InlineDatum dat) = Pays $ setDatum output $ TxSkelOutInlineDatum dat
    go (Pays output) (HiddenHashedDatum dat) = Pays $ setDatum output $ TxSkelOutDatumHash dat
    go (Pays output) (Value v) = Pays $ setValue output $ toValue v
    go (Pays output) (ReferenceScript script) = Pays $ setReferenceScript output $ toVersionedScript script
    go (Pays output) (StakingCredential (toMaybeStakingCredential -> Just stCred)) = Pays $ setStakingCredential output stCred
    go pays (StakingCredential _) = pays
    go pays (PayableAnd p1 p2) = go (go pays p1) p2

Some additional thoughts / comments:

1. I got fed up of working in Skeleton.hs which became far too large, so I split it into subfiles. The content of this PR is located in Cooked.Skeleton.Payable and Cooked.Skeleton.Output.
2. In file Cooked.Skeleton.Mint, we now get the orphan instance warning for the semigroup instance of TxSkelMints. Does anybody have an idea why, as they are both defined together?

[mmontin]

Additionally, this PR already implements a (partial?) fix for #460 by having 3 distinct constructor with (what I believe is) good names: VisibleHashedDatum, HiddenHashedDatum and InlineDatum when issuing payments.

[nc6]

I took a quick scan through. I agree that this API seems pretty nice and makes a nicer interface for the user to work with

[yannham]

Side-note: I think splitting modules should be done in separate PRs in the future as much as possible. Splitting a file is very not diff-friendly, and it's now hard to know what's new and related to payable stuff, and what is just code that is coming verbatim from the original monolith.

FWIW, I find the current solution fine; it's another point in the design space. We have to use constructors again, and there is still a custom operator, but I guess the few added constructors are still better than paysFooBarBazWithQuxGlorb, and I find <&&> to be much better than &> (even if I still dislike custom operators 🙃 ). As any good compromise, It'll probably leave everyone slightly dissatisfied, but that shouldn't stop you 😛

[yannham]

Is this equivalent to:

wallet 1 `receives` (Value (Script.assetClassValue ac1 1) <&&> Value (Script.assetClassValue ac2 2))

[mmontin]

Not anymore. The composition no longer allows to combine the content of values. It was controversial and for good reasons. Now you can only assign each component once at most (thanks to the type families constraining the types).

[yannham]

Same: do we now have to ways of doing the same thing (put differently, is Value an homomorphism from (_, <>) to (_, <&&>)) ? Not that it's a problem per se, I'm just wondering.

[mmontin]

In short: no it's not (see above).

[mmontin]

@yannham thanks for your review. I agree I should have waited for another PR to split files. Thats why I emphasized where the actual changes were in my message. But still, it was bad practice in action !