-
Notifications
You must be signed in to change notification settings - Fork 71
Description
Following #2064 we noticed some published scripts are not covered in our CI/CD pipeline (https://github.com/spacetx/starfish/blob/master/.github/workflows/starfish-prod-ci.yml), although they appear on our documentations (https://spacetx-starfish.readthedocs.io/en/latest/gallery/index.html).
For example:
- https://spacetx-starfish.readthedocs.io/en/latest/gallery/how_to/assess_spotfindingresults.html#sphx-glr-gallery-how-to-assess-spotfindingresults-py
- https://spacetx-starfish.readthedocs.io/en/latest/gallery/how_to/blob_detector.html#sphx-glr-gallery-how-to-blob-detector-py
Looking into the Makefile, it seems that we only check locally examples/pipelines when running make test-examples (as part of "slow" testing prior to a new release).
include examples/pipelines/subdir.mk
test-examples: export TESTING=1
test-examples: run-examples
The corresponding examples/pipelines/subdir.mk file runs all the python (using ipython) and bash scripts within that folder.
path := examples/pipelines
py_files := $(wildcard $(path)/*.py)
sh_files := $(wildcard $(path)/*.sh)
py_run_targets := $(patsubst $(path)/%,%,$(py_files))
sh_run_targets := $(patsubst $(path)/%,%,$(sh_files))
PYTHON := ipython
run-examples: $(py_run_targets) $(sh_run_targets)
$(py_run_targets): % :
[ -e $(path)/$*.skip ] || $(PYTHON) $(path)/$*
$(sh_run_targets): % :
[ -e $(path)/$*.skip ] || $(SHELL) $(path)/$*
Furthermore, our Github Actions workflow only tests two pipelines from the examples folder (3d_smFISH & ISS) out of the 7 present. Because they are the only two that don't use interactive python. Not to worry, the others are being tested via their notebooks/py/ counterparts (there is also some redundancy like for ISS pipeline).
Moving forward, we need to make sure all published examples are covered in tests through our Github Actions workflow.
A possible solution is to move the file subdir.mk to examples/ and add all the python scripts in the subdirectories. Running all and skipping interactive ones.
Interactive scripts (most of the examples/pipelines) should be bundled with other napari tests as they need to run headless or they need to be modified.