Make some dependencies optional

[maxnoe]

• matplotlib
• Bokeh
• eventio
• iminuit

[maxnoe]

People will get an OptionalDependencyMisisng error now for these:

In [3]: s = SimTelEventSource("dataset://gamma_prod5.simtel.zst")
---------------------------------------------------------------------------
OptionalDependencyMissing                 Traceback (most recent call last)
Cell In[3], line 1
----> 1 s = SimTelEventSource("dataset://gamma_prod5.simtel.zst")

File ~/Uni/CTA/ctapipe/src/ctapipe/io/simteleventsource.py:539, in SimTelEventSource.__init__(self, input_url, config, parent, **kwargs)
    537 super().__init__(input_url=input_url, config=config, parent=parent, **kwargs)
    538 if SimTelFile is None:
--> 539     raise OptionalDependencyMissing("eventio")
    541 self.file_ = SimTelFile(
    542     self.input_url.expanduser(),
    543     allowed_telescopes=self.allowed_tels,
    544     skip_calibration=self.skip_calibration_events,
    545     zcat=not self.back_seekable,
    546 )
    547 if self.back_seekable and self.is_stream:

OptionalDependencyMissing: 'eventio' is required for this functionality of ctapipe

[maxnoe]

And for using EventSource I added special handling for OptionalDependencyMissing:

ValueError: Could not find compatible EventSource for input_url: PosixPath('/home/maxnoe/.cache/ctapipe/cccta-dataserver.in2p3.fr/data/ctapipe-test-data/v1.1.0/gamma_prod5.simtel.zst')
in available EventSources: ['HDF5EventSource', 'PluginEventSource']
EventSources that could not be used due to missing optional dependencies:
	SimTelEventSource: eventio

[Tobychev]

Why strip out matplotlib? Is this to save time on the cluster setup? My first reaction is that this will make dev more annoying as I have to manually install it to make plots whenever I recreate the conda environment from the latest environment file.

Also, dropping ImPACT as a reco method available by default seems a bit drastic and something we're very likely to revert once we actually have a sensitivity curve for a tuned analysis.

Is it possible to make this stripped down version gated behind some special install instead of being the default case?

[maxnoe]

Tests (including the new one only testing the strictly required dependencies) are now passing.

@kosack @Tobychev please have a look.

[maxnoe]

Why strip out matplotlib? Is this to save time on the cluster setup? My first reaction is that this will make dev more annoying as I have to manually install it to make plots whenever I recreate the conda environment from the latest environment file.

This was explicitly requested by the ACADA SAG group. And I agree, matplotlib is a heavy dependency that we shouldn't install in circumstances where only processing events is required.

Also, dropping ImPACT as a reco method available by default seems a bit drastic and something we're very likely to revert once we actually have a sensitivity curve for a tuned analysis.

I think you are confusing DataPipe and ctapipe requirements here, DataPipe is free to depend on ctapipe and iminuit to support ImPACT. Not all users of ctapipe want or need impact though.

Is it possible to make this stripped down version gated behind some special install instead of being the default case?

No, not at the moment, at least not for the PyPI package. You can only add optional dependencies, not provide a default that has more and an extra that strips it down.

conda-forge doesn't do optional dependencies at all, so what most larger projects do is provide a -base package that only contains the strictly required depdencies and the nominal package name that comes with all recommended depdendencies, e.g. matplotlib-base vs. matplotlib and astropy-base vs astropy (the latter one is planned for v7, not yet published).

[maxnoe]

Why strip out matplotlib? Is this to save time on the cluster setup? My first reaction is that this will make dev more annoying as I have to manually install it to make plots whenever I recreate the conda environment from the latest environment file.

Also note that the environment.yaml file hasn't changed. It still includes all optional dependencies. The dev extra has also been changed so that it includes all dev optional dependencies.

So

$ pip install -e '.[dev]'

will still give your everything

[maxnoe]

Astropy is currently preparing a PEP for making it possible to have more dependencies by default pip install astropy but being able to select a minimal installation pip install astropy[minimal], but the time scale is quite long I guess.

https://github.com/astrofrog/peps/pull/1/files

[ctao-dpps-sonarqube]

• 78.60% Coverage on New Code (is less than 80.00%)

Analysis Details

1 Issue

• 0 Bugs
• 0 Vulnerabilities
• 1 Code Smell

Coverage and Duplications

• 78.60% Coverage (94.10% Estimated after merge)
• 0.00% Duplicated Code (0.70% Estimated after merge)

Project ID: cta-observatory_ctapipe_AY52EYhuvuGcMFidNyUs

View in SonarQube