[MRESOLVER-269] [MRESOLVER-275] Trusted checksums source and more compact format backed source

[cstamas]

This PR implements several improvement issues:

• introduces "trusted" checksums source (and adapts them for provided transport checksums)
• pushed/moved existing implementation as "trusted" source and introduces "compact" (file) based source
• cleanup re naming: before it was "file" source, now there is "file-sparse" (old) and "file-compact" (new)
• also adds minor "cleanup" in AetherModule (stylistic, renames provider private vars and makes them unmodifiable). This does not affects consumer of this library that use sisu (like Maven).

Reason: existing ProvidedChecksumSource is all about transport (see API). Uses transport related classes and is meant -- as it's name and package says -- to provide expected checksums for transport related checks (it provides ChecksumKind#PROVIDED, uses RemoteRepository and transfer related classes). OTOH, there may be requirement to "provide" checksums for operations totally unrelated to transport. Hence, the introduced trusted checksums (using non transport related API) is exactly that, provides checksums for given Artifact (optionally factoring in origin as well in form of ArtifactRepository). This clearly separates "transport realm" and rest of the things.

Along with existing (moved) source, new trusted checksums implementation added that uses more compact format: a "summary" file that contains list of Artifact IDs and checksum per one line. This format is more VCS friendly, and also easier to handle then sparse directories.

By default, new "trusted" checksum sources are adapted to "provided" checksum sources (see TrustedToProvidedChecksumsSourceAdapter), so no functionality loss happens.

Perfomed cleanup around trusted checksum sources as well, old one was in wrong place and wrongly named, dropped it (as it was final class), and now we have two sources:

• sparse-directory -- behaves exactly same as dropped one, expects provided checksums in "local repo"-like sparse layout
• summary-file -- is the new format, where one file checksums.${checksumExt} is expected to contain Artifact ID and checksum for given algorithm per line (separated by space)

Both source are able to be "origin aware" when it factors in origin repository ID as well (so one could get checksums-central.sha1 with all the known trusted checksums for use).

Sources are DISABLED by default, as even if file is present (check possible only for file-compact) it does not mean user want to use it in every project. Enabling them is possible via usual means (-D... or by config in .mvn directory to make it per-project persistent). All configuration is sourced from repo system session, no system properties used.

Based on work done in #192

Co-authored-by: @raphw [email protected]

---

https://issues.apache.org/jira/browse/MRESOLVER-275 -- Introduce trusted checksums source
https://issues.apache.org/jira/browse/MRESOLVER-269 -- Allow more compact storage of provided checksums

[cstamas]

This PR obsoletes #192 as it provides 100% same change as in that PR but this PR also cleans up in some areas.

[raphw]

This looks really good, thanks!

[michael-o]

• Since the LocalPathComposer is used throughout the docs of the sources should mention that this substructure is also reflected in .checksums/
• As for the file-compact. I like the idea. Does it make sense to reuse rather a well-known format like BSD checksum file? See also Contents should include checksum and filename in standard format nicoulaj/checksum-maven-plugin#127

[cstamas]

• Since the LocalPathComposer is used throughout the docs of the sources should mention that this substructure is also reflected in .checksums/

The javadoc of sparse source says:

 * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
 * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
 * coordinates solely to form path from basedir, pretty much as Maven local repository does.

• As for the file-compact. I like the idea. Does it make sense to reuse rather a well-known format like BSD checksum file? See also nicoulaj/checksum-maven-plugin#127

IMHO no, as BSD checksum file contains filename, here, we operate with ArtifactId ("file format is artifact ID and checksum separated by space per line"), see https://github.com/apache/maven-resolver/blob/master/maven-resolver-util/src/main/java/org/eclipse/aether/util/artifact/ArtifactIdUtils.java#L45

[michael-o]

• Since the LocalPathComposer is used throughout the docs of the sources should mention that this substructure is also reflected in .checksums/

The javadoc of sparse source says:

 * Sparse file {@link FileTrustedChecksumsSourceSupport} implementation that use specified directory as base
 * directory, where it expects artifacts checksums on standard Maven2 "local" layout. This implementation uses Artifact
 * coordinates solely to form path from basedir, pretty much as Maven local repository does.

I don't see how this answers my question... ?!!

• As for the file-compact. I like the idea. Does it make sense to reuse rather a well-known format like BSD checksum file? See also nicoulaj/checksum-maven-plugin#127

IMHO no, as BSD checksum file contains filename, here, we operate with ArtifactId ("file format is artifact ID and checksum separated by space per line"), see https://github.com/apache/maven-resolver/blob/master/maven-resolver-util/src/main/java/org/eclipse/aether/util/artifact/ArtifactIdUtils.java#L45

hm...we basically use our current format, but compile multiple files and key them by artifact id. Agree.

[cstamas]

I don't see how this answers my question... ?!!

I might missed the real intent of your question then...

[michael-o]

Going through again...

[michael-o]

Re: the origin aware. We have now aether.enhancedLocalRepository.splitRemoteRepository. To make it consistent, why not name it instead of originalAware splitRepository?
This may be obsolete if the LocalPathPrefixComposer is used.

[michael-o]

I don't see how this answers my question... ?!!

I might missed the real intent of your question then...

Forget it, I understand now. No further explanation necessary.

[michael-o]

Yet another question: It is worthwhile to enable it by default and add checksums on the fly when they are present?

[michael-o]

Yet another issue with the .enabled: I think this is again inconsistent with our existing boolean properties: https://maven.apache.org/resolver/configuration.html
They use the name as boolean property with a default value. So to apply this appoach .enabled just can be dropped and its parent used.

[cstamas]

Yet another question: It is worthwhile to enable it by default and add checksums on the fly when they are present?

This could be similar as "record" functionality is for remote repository filter feature. Yes, it could be possible, but unsure is it worth it: the whole idea of this is to get checksums for other trusted source than remote...

[cstamas]

Yet another issue with the .enabled: I think this is again inconsistent with our existing boolean properties: https://maven.apache.org/resolver/configuration.html They use the name as boolean property with a default value. So to apply this appoach .enabled just can be dropped and its parent used.

This is fixed now.

[michael-o]

Yet another question: It is worthwhile to enable it by default and add checksums on the fly when they are present?

This could be similar as "record" functionality is for remote repository filter feature. Yes, it could be possible, but unsure is it worth it: the whole idea of this is to get checksums for other trusted source than remote...

ОК, let's put this aside.

[michael-o]

This is basically good/done, but https://github.com/apache/maven-resolver/blob/master/src/site/markdown/configuration.md needs to be updated for this.

[michael-o]

All good to merge.