Improve framework for validation

[msschwartz21]

Proposed Change

This PR works towards the validation framework described here: #152 (comment). It will ultimately incorporate some of the validation functions contributed in #180.

New checks that have been implemented so far

• Check node id dtype, starting on Node IDs should be unsigned integers and this should be validated #176 (still need to do the uint part in another PR)
• Checks that properties specified in axes are actually present in the zarr without missing data

Types of Changes

What types of changes does your code introduce? Delete those that do not apply.

• New feature or enhancement

Which topics does your change affect? Delete those that do not apply.

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• I have read the developer/contributing docs.
• I have added tests that prove that my feature works in various situations or tests the bugfix (if appropriate).
• I have checked that I maintained or improved code coverage.
• I have written docstrings and checked that they render correctly.

Further Comments

[TeunHuijben]

Note: add validate to every possible test

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[msschwartz21]

Todos

• Pull out shape validation into separate functions and test
• Test other optional data types, e.g. segmentation? -> Only check the seg ids because all other segmentation related functions require loading the array which we aren't going to handle in automated validation. Not doing this because the function needs to be refactor to just take the seg id values.
• Check that the validation options are exposed at the level of the readers (and writers?)

[codspeed-hq]

CodSpeed Performance Report

Merging #204 will degrade performances by 23.19%

_{Comparing validator-framework (b8f2887) with main (edf9cbe)}

Summary

❌ 1 (👁 1) regressions
✅ 6 untouched benchmarks

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
👁	test_bench_validate[500]	25.8 ms	33.6 ms	-23.19%

[msschwartz21]

As discussed in #309, typing is failing due to an issue in python-typeshed, which should be resolved when python/typeshed#14597 is merged and released. I don't know that we need to hold this PR up for that though.

[cmalinmayor]

Thoughts on adding nodes/edges to this config and then only passing one argument to the read/write functions? I assume you considered it and rejected, and I don't have a strong opinion, but bringing it up just in case

[msschwartz21]

Ben and I had created a separation in our brains between the graph (as mandatory data) and optional attribute data. We hadn't considered putting the graph into the validation config, but I like it. I'll make that change.

[cmalinmayor]

As discussed elsewhere, could consider rolling the structure into the same argument but up to you

[msschwartz21]

I decided to leave structure out, but consolidated all the data once into the validation config

[cmalinmayor]

Just a few minor things about the API, but I love it in general! 🦖

[cmalinmayor]

🌴 🎋 🎄 🌳 🌲