CLI: sigstore verify pypi?

[woodruffw]

Right now, there's a bit of confusion from users about how to manually verify attestations that come from PyPI. Users also (very understandably!) find the distinction between Sigstore bundles and PEP 740 provenance/attestations confusing, and are (reasonably!) frustrated when the provenance JSON that comes from PyPI can't be immediately shoved into sigstore verify.

To ameliorate these sources of confusion, we could add a sigstore verify pypi or similar subcommand. Some rough sketches:

# fetch the latest version of foopkg from PyPI along with its provenance
sigstore verify pypi foopkg --repository https://github.com/foo/foopkg

# fetch a specific version
sigstore verify pypi foopkg==1.2.3 --repository https://github.com/foo/foopkg

Per sigstore/gh-action-sigstore-python#158 (comment).

Pros:

• Significantly simplifies the user story/reduces the amount of different moving parts users need to think about as part of consuming PEP 740 attestations
• Potentially allows us to get rid of pypi-attestations as a separate repo/project and fold everything into sigstore-python

Cons:

• sigstore-python needs to learn how to parse/handle the PEP 740 attestation and provenance models
• Technically outside of the scope of a Sigstore client, since these aren't Sigstore bundles?

CC @sethmlarson @di @jku for thoughts 🙂

[di]

I agree with making the overall functionality available to users but I'm not sure it should live inside a Sigstore client (would we expect all other Sigstore clients to support this? probably not).

Should this exist as part of the yet-to-be-created plugin for pip? I could see it offering both standalone and plugin functionality, and it seems like we'd need to implement the same verification logic there as well.

[woodruffw]

but I'm not sure it should live inside a Sigstore client (would we expect all other Sigstore clients to support this? probably not).

True -- it's definitely a mismatch IMO and would result in more confusion/feature disjunction between Sigstore clients.

Should this exist as part of the yet-to-be-created plugin for pip? I could see it offering both standalone and plugin functionality, and it seems like we'd need to implement the same verification logic there as well.

That makes sense to me -- right now we have the purely plugin aspects in https://github.com/trailofbits/pip-plugin-pep740, but we could easily have standalone functionality as well! CC @facutuesca for thoughts on that 🙂

[jku]

pip plugin sounds like the right direction in general. I think there's value in having some solution available in the short term though... I don't have strong opinions on whether that should happen through the plugin being standalone usable or providing something in sigstore-python: both seem like reasonable directions

On sigstore client feature parity: while it's good to have all sigstore clients support the same basic features, I don't think strictly limiting clients to that feature set is needed or ultimately useful. I think the question for sigstore-python is rather is this pypi specific feature something we want to keep supporting in future (or one we are prepared deprecate once better solutions are available)?

Maybe standalone plugin is the best option to look at first though.

[facutuesca]

Should this exist as part of the yet-to-be-created plugin for pip? I could see it offering both standalone and plugin functionality, and it seems like we'd need to implement the same verification logic there as well.

I don't know if I would add it to the pip plugin, since I think there's an advantage to leaving it as simple and stable as possible.

We already have some CLI functionality on pypi-attestations to verify the PEP740 attestations that are uploaded to PyPI, maybe we could add another verify command for the ones downloaded from PyPI.

[di]

We already have some CLI functionality on pypi-attestations to verify the PEP740 attestations that are uploaded to PyPI, maybe we could add another verify command for the ones downloaded from PyPI.

This seems like a logical place to put it to me!

[facutuesca]

# fetch the latest version of foopkg from PyPI along with its provenance
sigstore verify pypi foopkg --repository https://github.com/foo/foopkg

# fetch a specific version
sigstore verify pypi foopkg==1.2.3 --repository https://github.com/foo/foopkg

@woodruffw How do you think the CLI should handle verification for a given package and version? Should it download all distributions (wheels+sdist) and their corresponding provenances, and verify all of them?

[facutuesca]

PR with implementation: trailofbits/pypi-attestations#82