[traits] digest traits are not misuse resistant

[robinhundt]

The current digest traits APIs are not misuse resistant. The following scenarios are possible:

Update after finish

Because the finish methods take &mut self and not self, it is possible to call update after finish. For algorithms such as SHA-3, this is not defined¹, as you can't go back to absorbing when you're already squeezing the state. If in an implementation of a the traits you want to guard against this misuse, there are no perfect solutions (see #1493), only different compromises.

Calling finish twice

It is possible call finish twice (again because of it taking &mut). It is unclear what the expected behavior should be. Should the same digest be returned? Should the state be reset internally before computing the second digest?

Solution

I think the solution is obvious: finish must take self by value instead of &mut self. This solves both problems described above. Additionally, we might want a finish_reset(&mut self) -> Digest API, that gives you a digest and resets the state in the same operation².

Footnotes

1. One exception being BLAKE-3, where it is defined to call update -> finalize -> update -> finalize ... ↩

2. Prior work is the digest::Digest::finalize_reset method. ↩