Skip to content

Missing test coverage for some scripts published on Starfish ReadTheDocs #2065

@shachafl

Description

@shachafl

Following #2064 we noticed some published scripts are not covered in our CI/CD pipeline (https://github.com/spacetx/starfish/blob/master/.github/workflows/starfish-prod-ci.yml), although they appear on our documentations (https://spacetx-starfish.readthedocs.io/en/latest/gallery/index.html).
For example:

Looking into the Makefile, it seems that we only check locally examples/pipelines when running make test-examples (as part of "slow" testing prior to a new release).

include examples/pipelines/subdir.mk
test-examples: export TESTING=1
test-examples: run-examples

The corresponding examples/pipelines/subdir.mk file runs all the python (using ipython) and bash scripts within that folder.

path := examples/pipelines
py_files := $(wildcard $(path)/*.py)
sh_files := $(wildcard $(path)/*.sh)
py_run_targets := $(patsubst $(path)/%,%,$(py_files))
sh_run_targets := $(patsubst $(path)/%,%,$(sh_files))

PYTHON := ipython

run-examples: $(py_run_targets) $(sh_run_targets)

$(py_run_targets): % :
        [ -e $(path)/$*.skip ] || $(PYTHON) $(path)/$*

$(sh_run_targets): % :
        [ -e $(path)/$*.skip ] || $(SHELL) $(path)/$*

Furthermore, our Github Actions workflow only tests two pipelines from the examples folder (3d_smFISH & ISS) out of the 7 present. Because they are the only two that don't use interactive python. Not to worry, the others are being tested via their notebooks/py/ counterparts (there is also some redundancy like for ISS pipeline).

Moving forward, we need to make sure all published examples are covered in tests through our Github Actions workflow.
A possible solution is to move the file subdir.mk to examples/ and add all the python scripts in the subdirectories. Running all and skipping interactive ones.
Interactive scripts (most of the examples/pipelines) should be bundled with other napari tests as they need to run headless or they need to be modified.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions