A better gallery

[dcherian]

What is your issue?

Our current gallery is not great; it contains a fairly arbitrary selection of things you can do from simple to wildly complex (apply_ufunc hehe).

There has been some parallel work:

1. For SciPy, a bunch of us (@benbovy @keewis @scottyhq and I) built https://xarray-indexes.readthedocs.io/ (repo at https://github.com/xarray-contrib/xarray-indexes). We built it externally to move quickly and also because the environment on that repo is quite heavy.
2. Presumably, @ianhi's work as part of the CZI grant will also involve some sort of gallery (at https://xarray-contrib.github.io/xarray-for-bio/intro.html) with a different set of packages.

It would be nice to figure out a solution where visitors to our docs can discover all available galleries, AND for us to run a CI test that ensures we aren't breaking them.

I don't have any good ideas for how to do this in a sustainable manner at present.

[VeckoTheGecko]

AND for us to run a CI test that ensures we aren't breaking them

One part of the puzzle could be re-usable workflows that live in the source repo (xarray-indexes, xarray for bio) that (a) build the environment, and (b) run the notebooks of interest (e.g., via pytest/testing/docs build). In the context of this repo those workflows would be run (e.g., on commit to main), but overwriting the version of xarray that is installed. If installed via pip, this would be a matter of replacing xarray with git+https://github.com/pydata/xarray@main, if installing via conda or pixi I don't have any solutions (cant find any info about installing from a git repo for anything except pypi - might have to create a build or something).

do this in a sustainable manner at present

In terms of maintainability? What particular aspects are you concerned about?

[TomNicholas]

It would be nice to figure out a solution where visitors to our docs can discover all available galleries, AND for us to run a CI test that ensures we aren't breaking them.

The pythia people might have advice - see their blog post on moving to MyST.

[dcherian]

Good idea! cc @jukent @kafitzgerald @ktyle

[ianhi]

AND for us to run a CI test that ensures we aren't breaking them.

One thought on this front is to mimic the Zarr regression tests where each test is defined with pep-723 metadata installed via uv or juv (and my sense is that it is hopefully soon to be supported by pixi!) and replacing the xarray in that per example. Makes it much simpler to manage the docs envinroment.

I also don't think it's unreasonable to enforce a requirement that all galleries linked to somehow from the xarray docs regularly run upstream tests

[jukent]

Hey @dcherian we demonstrate using the MyST architecture for a gallery of external cards (each separate Cookbook is treated as external in this way). Let me know if you want to pursue this and chat more. More functionality is coming from the MyST MD and JupyterBook2 team.