diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index a33012a1..9295db3d 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -17,6 +17,8 @@ assignees: '' **Screenshots or steps for reproduction (using napari GUI)** +**Include relevant logs which are created next to the output dir, name of the dataset, yaml file(s) if encountering reconstruction errors.** + **Expected behavior** A clear and concise description of what you expected to happen. diff --git a/.github/workflows/pytests.yml b/.github/workflows/pytests.yml index 2c8c7cd7..fd929634 100644 --- a/.github/workflows/pytests.yml +++ b/.github/workflows/pytests.yml @@ -24,7 +24,7 @@ jobs: - name: Install dependencies run: | python -m pip install --upgrade pip - pip install ".[acq,dev]" + pip install ".[all,dev]" # - name: Lint with flake8 # run: | diff --git a/README.md b/README.md index 9acb5181..42b9dfee 100644 --- a/README.md +++ b/README.md @@ -48,10 +48,16 @@ conda activate recOrder ``` Install `recOrder-napari` with acquisition dependencies -(napari and pycro-manager): +(napari with PyQt6 and pycro-manager): ```sh -pip install recOrder-napari[acq] +pip install recOrder-napari[all] +``` + +Install `recOrder-napari` without napari, QtBindings (PyQt/PySide) and pycro-manager dependencies: + +```sh +pip install recOrder-napari ``` Open `napari` with `recOrder-napari`: diff --git a/docs/development-guide.md b/docs/development-guide.md index 78c03584..ac84fb74 100644 --- a/docs/development-guide.md +++ b/docs/development-guide.md @@ -19,7 +19,7 @@ ```sh cd recOrder - pip install -e ".[acq,dev]" + pip install -e ".[all,dev]" ``` 4. Optionally, for the co-development of [`waveorder`](https://github.com/mehta-lab/waveorder) and `recOrder`: diff --git a/docs/images/reconstruction_birefriengence.png b/docs/images/reconstruction_birefriengence.png new file mode 100644 index 00000000..20ac93b4 Binary files /dev/null and b/docs/images/reconstruction_birefriengence.png differ diff --git a/docs/images/reconstruction_data.png b/docs/images/reconstruction_data.png new file mode 100644 index 00000000..5a85e570 Binary files /dev/null and b/docs/images/reconstruction_data.png differ diff --git a/docs/images/reconstruction_data_info.png b/docs/images/reconstruction_data_info.png new file mode 100644 index 00000000..9a38b441 Binary files /dev/null and b/docs/images/reconstruction_data_info.png differ diff --git a/docs/images/reconstruction_models.png b/docs/images/reconstruction_models.png new file mode 100644 index 00000000..400812f9 Binary files /dev/null and b/docs/images/reconstruction_models.png differ diff --git a/docs/images/reconstruction_queue.png b/docs/images/reconstruction_queue.png new file mode 100644 index 00000000..2a221241 Binary files /dev/null and b/docs/images/reconstruction_queue.png differ diff --git a/docs/microscope-installation-guide.md b/docs/microscope-installation-guide.md index 2aef5189..da69e013 100644 --- a/docs/microscope-installation-guide.md +++ b/docs/microscope-installation-guide.md @@ -40,7 +40,7 @@ conda activate recOrder Install `recOrder` with acquisition dependencies (napari and pycro-manager): ``` -pip install recOrder-napari[acq] +pip install recOrder-napari[all] ``` Check your installation: ``` diff --git a/docs/napari-plugin-guide.md b/docs/napari-plugin-guide.md index 4a8ef6bf..04c8bd63 100644 --- a/docs/napari-plugin-guide.md +++ b/docs/napari-plugin-guide.md @@ -160,6 +160,36 @@ Examples of acquiring 2D birefringence data (kidney tissue) with this snap metho See the [reconstruction guide](./reconstruction-guide.md) for CLI usage instructions. +## Reconstruction Tab +The **Reconstruction** tab is designed to reconstruct `birefriengence, phase, birefrignence with phase, and flurescenece` datasets that have been either acquired or coverted to `.zarr` store as well as acquisitions that are in progress. + +![](./images/reconstruction_data.png) + +The **Input Store** and **Output Directory** point to the input and output `.zarr` data locations. Once an Input Store is selected some metadata parameters can be viewed by hovering the cursor over the `info label` ⓘ. + +![](./images/reconstruction_models.png) + +A `Model` defines the reconstruction parameters. Multiple models can be run against a dataset with varying parameters. The model generates a configuration file `.yml`, then uses the CLI to reconstruct the data with the configuration file, which makes all reconstructions exactly reproducible via a CLI. +* **New**: Builds a model based on the `Checkbox` selection. +* **Load**: Allows a model to be imported using a previous reconstruction `.yml` file. +* **Clear**: This will clear all defined models. + +![](./images/reconstruction_birefriengence.png) + +Once a `New` model is built, it is pre-populated with default values that can be accessed by clicking on the ► icon and the parameters can be changed as required. +See the [reconstruction guide](./reconstruction-guide.md) for further information on the parameters. + +![](./images/reconstruction_queue.png) + +Once the **RUN** button is triggered, the reconstruction will proceed based on the defined model(s) concurrently. + +> [!CAUTION] +> Since the models run concurrently, it is the users responsibility to manage compute resources accordingly on a local or SLURM system. + +The `Reconstruction Queue` section will display the progress of the reconstruction in the form of text output. Once a reconstruction finishes the queue will self clear. Only in the case of any issues or error that are encountered the entry will remain. + +Once the reconstruction processing finishes, based on the option `Show after Reconstruction` the reconstructed images will show up in the napari viewer. + ## Visualizations When an **Orientation*** layer appears at the top of the layers list, `recOrder` will automatically color it with an HSV color map that indicates the orientation. diff --git a/recOrder/acq/acq_functions.py b/recOrder/acq/acq_functions.py index 2db5d84e..8a60a46c 100644 --- a/recOrder/acq/acq_functions.py +++ b/recOrder/acq/acq_functions.py @@ -5,8 +5,9 @@ import numpy as np from iohub import read_micromanager -from pycromanager import Studio - +try: + from pycromanager import Studio +except:pass def generate_acq_settings( mm, diff --git a/recOrder/cli/gui_widget.py b/recOrder/cli/gui_widget.py index 5d2b2873..dfc3a195 100644 --- a/recOrder/cli/gui_widget.py +++ b/recOrder/cli/gui_widget.py @@ -1,7 +1,13 @@ import sys -from PyQt6.QtWidgets import QApplication, QWidget, QVBoxLayout, QStyle import click -from recOrder.plugin import tab_recon + +try: + from recOrder.plugin import tab_recon +except:pass + +try: + from qtpy.QtWidgets import QApplication, QWidget, QVBoxLayout, QStyle +except:pass try: import qdarktheme @@ -38,4 +44,4 @@ def __init__(self): layout.addWidget(recon_tab.recon_tab_mainScrollArea) if __name__ == "__main__": - gui() \ No newline at end of file + gui() diff --git a/recOrder/cli/main.py b/recOrder/cli/main.py index 2568d5ec..c124e82f 100644 --- a/recOrder/cli/main.py +++ b/recOrder/cli/main.py @@ -3,7 +3,9 @@ from recOrder.cli.apply_inverse_transfer_function import apply_inv_tf from recOrder.cli.compute_transfer_function import compute_tf from recOrder.cli.reconstruct import reconstruct -from recOrder.cli.gui_widget import gui +try: + from recOrder.cli.gui_widget import gui +except:pass CONTEXT = {"help_option_names": ["-h", "--help"]} @@ -23,7 +25,9 @@ def cli(): cli.add_command(reconstruct) cli.add_command(compute_tf) cli.add_command(apply_inv_tf) -cli.add_command(gui) +try: + cli.add_command(gui) +except:pass if __name__ == "__main__": - cli() \ No newline at end of file + cli() diff --git a/recOrder/plugin/__init__.py b/recOrder/plugin/__init__.py index 8d207a9f..72559f03 100644 --- a/recOrder/plugin/__init__.py +++ b/recOrder/plugin/__init__.py @@ -1,11 +1,9 @@ # qtpy defaults to PyQt5/PySide2 which can be present in upgraded environments -import qtpy +try: + import qtpy -if qtpy.PYQT5: - raise RuntimeError( - "Please remove PyQt5 from your environment with `pip uninstall PyQt5`" - ) -elif qtpy.PYSIDE2: - raise RuntimeError( - "Please remove PySide2 from your environment with `pip uninstall PySide2`" - ) + qtpy.API_NAME # check qtpy API name - one is required for GUI + +except RuntimeError as error: + if type(error).__name__ == "QtBindingsNotFoundError": + print("WARNING: QtBindings (PyQT or PySide) was not found for GUI") diff --git a/recOrder/plugin/main_widget.py b/recOrder/plugin/main_widget.py index 1f2b5ff9..121ea058 100644 --- a/recOrder/plugin/main_widget.py +++ b/recOrder/plugin/main_widget.py @@ -15,25 +15,33 @@ import numpy as np import yaml from dask import delayed -from napari import Viewer -from napari.components import LayerList -from napari.qt.threading import create_worker -from napari.utils.events import Event -from napari.utils.notifications import show_info, show_warning from numpy.typing import NDArray from numpydoc.docscrape import NumpyDocString from packaging import version -from pycromanager import Core, Studio, zmq_bridge from qtpy.QtCore import Qt, Signal, Slot from qtpy.QtGui import QColor, QPixmap from qtpy.QtWidgets import QFileDialog, QSizePolicy, QSlider, QWidget from superqt import QDoubleRangeSlider, QRangeSlider from waveorder.waveorder_reconstructor import waveorder_microscopy -from recOrder.acq.acquisition_workers import ( - BFAcquisitionWorker, - PolarizationAcquisitionWorker, -) +try: + from pycromanager import Core, Studio, zmq_bridge +except:pass + +try: + from napari import Viewer + from napari.components import LayerList + from napari.qt.threading import create_worker + from napari.utils.events import Event + from napari.utils.notifications import show_info, show_warning +except:pass + +try: + from recOrder.acq.acquisition_workers import ( + BFAcquisitionWorker, + PolarizationAcquisitionWorker, + ) +except:pass from recOrder.calib import Calibration from recOrder.calib.Calibration import LC_DEVICE_NAME, QLIPP_Calibration from recOrder.calib.calibration_workers import ( @@ -825,14 +833,23 @@ def connect_to_mm(self): # Order is important: If the bridge is created before Core, Core will not work self.bridge = zmq_bridge._bridge._Bridge() logging.debug("Established ZMQ Bridge and found Core and Studio") - except: + except NameError: + print("Is pycromanager package installed?") + except Exception as ex: + print( + "Could not establish pycromanager bridge.\n" + "Is Micro-Manager open?\n" + "Is Tools > Options > Run server on port 4827 checked?\n" + f"Are you using nightly build {RECOMMENDED_MM}?\n" + ) + template = "An exception of type {0} occurred. Arguments:\n{1!r}" + message = template.format(type(ex).__name__, ", ".join(ex.args)) + print(message) raise EnvironmentError( - ( "Could not establish pycromanager bridge.\n" "Is Micro-Manager open?\n" "Is Tools > Options > Run server on port 4827 checked?\n" f"Are you using nightly build {RECOMMENDED_MM}?" - ) ) # Warn the user if there is a Micro-Manager/ZMQ version mismatch diff --git a/recOrder/plugin/tab_recon.py b/recOrder/plugin/tab_recon.py index 6a14b526..26080e45 100644 --- a/recOrder/plugin/tab_recon.py +++ b/recOrder/plugin/tab_recon.py @@ -3,10 +3,9 @@ from pathlib import Path from qtpy import QtCore -from qtpy.QtCore import Qt, QEvent, QThread +from qtpy.QtCore import Qt, QEvent, QThread, Signal from qtpy.QtWidgets import * from magicgui.widgets import * -from PyQt6.QtCore import pyqtSignal from iohub.ngff import open_ome_zarr @@ -15,11 +14,13 @@ from magicgui.type_map import get_widget_class import warnings -from napari import Viewer +try: + from napari import Viewer + from napari.utils import notifications +except:pass from recOrder.io import utils from recOrder.cli import settings, jobs_mgmt -from napari.utils import notifications import concurrent.futures @@ -533,25 +534,31 @@ def validate_input_data( print(exc.args) try: - for _, pos in dataset.positions(): - axes = pos.zgroup.attrs["multiscales"][0]["axes"] - string_array_n = [str(x["name"]) for x in axes] - string_array = [ - str(x) - for x in pos.zgroup.attrs["multiscales"][0][ - "datasets" - ][0]["coordinateTransformations"][0]["scale"] - ] - string_scale = [] - for i in range(len(string_array_n)): - string_scale.append( - "{n}={d}".format( - n=string_array_n[i], d=string_array[i] + string_pos = [] + i = 0 + for pos_paths, pos in dataset.positions(): + string_pos.append(pos_paths) + if i == 0: + axes = pos.zgroup.attrs["multiscales"][0]["axes"] + string_array_n = [str(x["name"]) for x in axes] + string_array = [ + str(x) + for x in pos.zgroup.attrs["multiscales"][0][ + "datasets" + ][0]["coordinateTransformations"][0]["scale"] + ] + string_scale = [] + for i in range(len(string_array_n)): + string_scale.append( + "{n}={d}".format( + n=string_array_n[i], d=string_array[i] + ) ) - ) - txt = "\n\nScale: " + ", ".join(string_scale) - self.data_input_Label.tooltip += txt - break + txt = "\n\nScale: " + ", ".join(string_scale) + self.data_input_Label.tooltip += txt + i += 1 + txt = "\n\nFOV: " + ", ".join(string_pos) + self.data_input_Label.tooltip += txt except Exception as exc: print(exc.args) @@ -3363,7 +3370,7 @@ def table_update_and_cleaup_thread( # this is the only case where row deleting occurs # we cant delete the row directly from this thread # we will use the exp_id to identify and delete the row - # using pyqtSignal + # using Signal # break - based on status elif JOB_TRIGGERED_EXC in jobTXT: params["status"] = STATUS_errored_job @@ -3643,7 +3650,7 @@ class ShowDataWorkerThread(QThread): """Worker thread for sending signal for adding component when request comes from a different thread""" - show_data_signal = pyqtSignal(str) + show_data_signal = Signal(str) def __init__(self, path): super().__init__() @@ -3657,7 +3664,7 @@ class AddOTFTableEntryWorkerThread(QThread): """Worker thread for sending signal for adding component when request comes from a different thread""" - add_tableOTFentry_signal = pyqtSignal(str, bool, bool) + add_tableOTFentry_signal = Signal(str, bool, bool) def __init__(self, OTF_dir_path, bool_msg, doCheck=False): super().__init__() @@ -3675,7 +3682,7 @@ class AddTableEntryWorkerThread(QThread): """Worker thread for sending signal for adding component when request comes from a different thread""" - add_tableentry_signal = pyqtSignal(str, str, dict) + add_tableentry_signal = Signal(str, str, dict) def __init__(self, expID, desc, params): super().__init__() @@ -3691,7 +3698,7 @@ class AddWidgetWorkerThread(QThread): """Worker thread for sending signal for adding component when request comes from a different thread""" - add_widget_signal = pyqtSignal(QVBoxLayout, str, str, str, str) + add_widget_signal = Signal(QVBoxLayout, str, str, str, str) def __init__(self, layout, expID, jID, desc, wellName): super().__init__() @@ -3711,7 +3718,7 @@ class RowDeletionWorkerThread(QThread): """Searches for a row based on its ID and then emits a signal to QFormLayout on the main thread for deletion""" - removeRowSignal = pyqtSignal(int, str) + removeRowSignal = Signal(int, str) def __init__(self, formLayout): super().__init__() @@ -3818,7 +3825,7 @@ def setText(self, text): self.label.setText(text) class MyWidget(QWidget): - resized = pyqtSignal() + resized = Signal() def __init__(self): super().__init__() diff --git a/setup.cfg b/setup.cfg index 9ca61f10..70462d18 100644 --- a/setup.cfg +++ b/setup.cfg @@ -45,8 +45,16 @@ install_requires = psutil submitit pydantic==1.10.19 - + ome-zarr==0.8.3 # unpin when resolved: https://github.com/ome/napari-ome-zarr/issues/111 + qtpy + pyqtgraph>=0.12.3 + [options.extras_require] +all = + napari[pyqt6] + napari-ome-zarr>=0.3.2 # drag and drop convenience + pycromanager==0.27.2 + dev = pytest>=5.0.0 pytest-cov @@ -56,13 +64,6 @@ dev = black hypothesis -acq = - pycromanager==0.27.2 - pyqtgraph>=0.12.3 - napari-ome-zarr>=0.3.2 # drag and drop convenience - ome-zarr==0.8.3 # unpin when resolved: https://github.com/ome/napari-ome-zarr/issues/111 - napari[pyqt6] - [options.package_data] * = *.yaml @@ -70,6 +71,6 @@ acq = console_scripts = recorder = recOrder.cli.main:cli recOrder = recOrder.cli.main:cli - + napari.manifest = recOrder = recOrder:napari.yaml diff --git a/tox.ini b/tox.ini index 3628e989..67c53b18 100644 --- a/tox.ini +++ b/tox.ini @@ -29,5 +29,5 @@ passenv = PYVISTA_OFF_SCREEN extras = dev - acq + all commands = pytest -v --color=yes --cov=recOrder --cov-report=xml \ No newline at end of file