Add no-optionals and full-optionals test runs (Qiskit#8967)

* Add no-optionals and full-optionals test runs

This restructures the CI slightly to perform a complete "no-optionals"
run, and a complete "all optionals" run (for the optionals that are
readily accessible as Python packages, without complex additional
setup).  Previously, we only did a partial test with some of the oldest
optional components, which could have allowed for behaviour to
accidentally require optional components without us noticing.

This splits the `requirements-dev.txt` file into two; lines that remain
in the file are what is actually _required_ to run the test suite,
run the style checks, and do the documentation build.  The rest (and
everything that was missing) is added to a new
`requirements-optional.txt` file, which can be used to pull in (almost)
all of the packages that Terra can use to provide additional /
accelerated functionality.

Several tests needed to gain additional skips to account for this
change.  There is a good chance that some tests are missing skips for
some libraries that are not the first point of failure, but it's hard to
test explicitly for these in one go.

* Fix typo in coverage workflow

* Try relaxing ipython constraints

* Squash newly exposed lint failures

* Fix typo in tutorials pipeline

* Update the 'also update' comments

* Remove unneeded qiskit-toqm dependency

* Section requirements-optional.txt

* Test all optionals on min not max Python version

Optionals are generally more likely to have been made available on the
older Pythons, and some may take excessively long to provide wheels for
the latest versions of Python.

* Add missing test skip

* Fix optional call

* Use correct boolean syntax

* Fix tests relying on Jupyter

* Install ipykernel in tutorials

* Remove HAS_PDFLATEX skip from quantum_info tests

For simple LaTeX tests, IPython/Jupyter can handle the compilation
internally using MathJax, and doesn't actually need a `pdflatex`
installation.  We only need that when we're depending on LaTeX libraries
that are beyond what MathJax can handle natively.

* Include additional tutorials dependencies

* Install all of Terra's optionals in tutorial run

* Do not install all optionals in docs build

* Use class-level skips where appropriate

* Do not install ibmq-provider in tutorials run

* Include matplotlib in docs requirements

* Remove unnecessary whitespace

* Split long pip line

* Only install graphviz when installOptionals

Co-authored-by: Eric Arellano <[email protected]>

* Install visualization extras in docs build

* Don't `--upgrade` when installing optionals

This is to prevent any optionals that have a dependency on Terra from
potentially upgrading it to some PyPI version during their installation.
This shouldn't happen with the current development model of having only
one supported stable branch and the main branch, but the `-U` is
unnecessary, and not having it is safer for the future.

* Update secondary installation of Terra in docs build

* Install all optionals during the docs build

I yoyo-ed on this, not wanting it to be too onerous to build the
documentation, but in the end, we need to have the optional features
installed in order to build most of the documentation of those features,
and some unrelated parts of the documentation may use these features to
enhance their own output.

* Fix test setup job

* Remove duplication of matplotlib in requirements files

* Update image test installation command

* Restore editable build

* Move `pip check` to `pip` section

* Remove redundant "post-install" description

* Expand comment on first-stage choices

* pytohn lol

---------

Co-authored-by: Eric Arellano <[email protected]>

Author: jakelishman

.azure/test-linux.yml

@@ -12,6 +12,10 @@ parameters:
   - name: "testImages"
     type: boolean
 
+  - name: "installOptionals"
+    type: boolean
+    default: false
+
   - name: "installFromSdist"
     type: boolean
     default: false
@@ -84,27 +88,27 @@ jobs:
           env:
             SETUPTOOLS_ENABLE_FEATURES: "legacy-editable"
 
+      - ${{ if eq(parameters.installOptionals, true) }}:
+        - bash: |
+            set -e
+            source test-job/bin/activate
+            python -m pip install -r requirements-optional.txt -c constraints.txt
+            python -m pip check
+          displayName: "Install optional packages"
+
+        - bash: |
+            set -e
+            sudo apt-get update
+            sudo apt-get install -y graphviz
+          displayName: 'Install optional non-Python dependencies'
+
       - bash: |
           set -e
           source test-job/bin/activate
-          python -m pip install -U \
-            -c constraints.txt \
-            "cplex ; python_version < '3.11'" \
-            "qiskit-aer" \
-            "tweedledum ; python_version < '3.11'" \
-            "z3-solver"
           mkdir -p /tmp/terra-tests
           cp -r test /tmp/terra-tests/.
           cp .stestr.conf /tmp/terra-tests/.
           cp -r .stestr /tmp/terra-tests/. || :
-          sudo apt-get update
-          sudo apt-get install -y graphviz
-          pip check
-        displayName: 'Install post-install optional dependencies'
-
-      - bash: |
-          set -e
-          source test-job/bin/activate
           pushd /tmp/terra-tests
           export PYTHONHASHSEED=$(python -S -c "import random; print(random.randint(1, 4294967295))")
           echo "PYTHONHASHSEED=$PYTHONHASHSEED"
@@ -178,7 +182,8 @@ jobs:
               -c constraints.txt \
               -r requirements.txt \
               -r requirements-dev.txt \
-              -e ".[visualization]"
+              -r requirements-optional.txt \
+              -e .
             sudo apt-get update
             sudo apt-get install -y graphviz pandoc
             image_tests/bin/pip check

.azure/test-macos.yml

@@ -3,6 +3,10 @@ parameters:
     type: string
     displayName: "Version of Python to test"
 
+  - name: "installOptionals"
+    type: boolean
+    default: false
+
 jobs:
   - job: "MacOS_Tests_Python${{ replace(parameters.pythonVersion, '.', '') }}"
     displayName: "Test macOS Python ${{ parameters.pythonVersion }}"
@@ -45,6 +49,13 @@ jobs:
         env:
           SETUPTOOLS_ENABLE_FEATURES: "legacy-editable"
 
+      - ${{ if eq(parameters.installOptionals, true) }}:
+        - bash: |
+            set -e
+            source test-job/bin/activate
+            pip install -r requirements-optional.txt -c constraints.txt
+          displayName: "Install optional packages"
+
       - bash: |
           set -e
           source test-job/bin/activate

.azure/test-windows.yml

@@ -3,6 +3,10 @@ parameters:
     type: string
     displayName: "Versions of Python to test"
 
+  - name: "installOptionals"
+    type: boolean
+    default: false
+
 jobs:
   - job: "Windows_Tests_Python${{ replace(parameters.pythonVersion, '.', '') }}"
     displayName: "Test Windows Python ${{ parameters.pythonVersion }}"
@@ -38,13 +42,20 @@ jobs:
             -c constraints.txt \
             -r requirements.txt \
             -r requirements-dev.txt \
-            "z3-solver" \
             -e .
           pip check
         displayName: 'Install dependencies'
         env:
           SETUPTOOLS_ENABLE_FEATURES: "legacy-editable"
 
+      - ${{ if eq(parameters.installOptionals, true) }}:
+        - bash: |
+            set -e
+            source test-job/Scripts/activate
+            pip install -c constraints.txt -r requirements-optional.txt
+            pip check
+          displayName: "Install optional packages"
+
       - bash: |
           set -e
           chcp.com 65001

.azure/tutorials-linux.yml

@@ -26,16 +26,8 @@ jobs:
             -c constraints.txt \
             -r requirements.txt \
             -r requirements-dev.txt \
-            "qiskit-ibmq-provider" \
-            "qiskit-aer" \
-            "z3-solver" \
-            "networkx" \
-            "matplotlib>=3.3.0" \
-            sphinx \
-            nbsphinx \
-            sphinx_rtd_theme \
-            "tweedledum ; python_version < '3.11'" \
-            cvxpy \
+            -r requirements-optional.txt \
+            -r requirements-tutorials.txt \
             -e .
           sudo apt-get update
           sudo apt-get install -y graphviz pandoc

.github/workflows/coverage.yml

@@ -52,7 +52,7 @@ jobs:
       - name: Generate unittest coverage report
         run: |
           set -e
-          python -m pip install -c constraints.txt -r requirements-dev.txt qiskit-aer
+          python -m pip install -c constraints.txt -r requirements-dev.txt -r requirements-optional.txt
           stestr run
           # We set the --source-dir to '.' because we want all paths to appear relative to the repo
           # root (we need to combine them with the Python ones), but we only care about `grcov`

azure-pipelines.yml

@@ -103,14 +103,17 @@ stages:
               testQPY: false
               testImages: false
               testRust: false
+              installOptionals: true
 
           - template: ".azure/test-macos.yml"
             parameters:
               pythonVersion: ${{ version }}
+              installOptionals: true
 
           - template: ".azure/test-windows.yml"
             parameters:
               pythonVersion: ${{ version }}
+              installOptionals: true
 
     - stage: "Nightly_Failure"
       displayName: "Comment on nightly failure"
@@ -150,9 +153,15 @@ stages:
         - template: ".azure/test-linux.yml"
           parameters:
             pythonVersion: ${{ parameters.minimumPythonVersion }}
+            # A PR is more likely to fail CI because it introduces a logic error
+            # into an existing test than because it adds a new test / optional
+            # dependency that isn't accounted for in the test-skipping logic
+            # (and such a failure would need fewer iterations to fix).  We want
+            # to fail fast in the first CI stage.
+            installOptionals: true
             testRust: true
             testQPY: true
-            testImages: false
+            testImages: true
 
     # The rest of the PR pipeline is to test the oldest and newest supported
     # versions of Python, along with the integration tests (via the tutorials).
@@ -172,24 +181,29 @@ stages:
             pythonVersion: ${{ parameters.maximumPythonVersion }}
             testRust: false
             testQPY: false
-            testImages: true
+            testImages: false
             installFromSdist: true
+            installOptionals: false
 
         - template: ".azure/test-macos.yml"
           parameters:
             pythonVersion: ${{ parameters.minimumPythonVersion }}
+            installOptionals: true
 
         - template: ".azure/test-macos.yml"
           parameters:
             pythonVersion: ${{ parameters.maximumPythonVersion }}
+            installOptionals: false
 
         - template: ".azure/test-windows.yml"
           parameters:
             pythonVersion: ${{ parameters.minimumPythonVersion }}
+            installOptionals: true
 
         - template: ".azure/test-windows.yml"
           parameters:
             pythonVersion: ${{ parameters.maximumPythonVersion }}
+            installOptionals: false
 
   # Push to main or the stable branches.  The triggering branches also need to
   # be in the triggers at the top of this file.
@@ -202,6 +216,7 @@ stages:
             testRust: true
             testQPY: true
             testImages: true
+            installOptionals: false
 
   # Push to a tag.  The triggering tags are set in the triggers at the top of
   # this file.

qiskit/transpiler/passes/layout/_csp_custom_solver.py

@@ -25,6 +25,8 @@
     class CustomSolver(RecursiveBacktrackingSolver):
         """A wrap to RecursiveBacktrackingSolver to support ``call_limit``"""
 
+        # pylint: disable=invalid-name
+
         def __init__(self, call_limit=None, time_limit=None):
             self.call_limit = call_limit
             self.time_limit = time_limit

qiskit/utils/optionals.py

@@ -87,6 +87,10 @@
       - Some methods of gradient calculation within :mod:`.opflow.gradients` require `JAX
         <https://github.com/google/jax>`__ for autodifferentiation.
 
+    * - .. py:data:: HAS_JUPYTER
+      - Some of the tests require a complete `Jupyter <https://jupyter.org/>`__ installation to test
+        interactivity features.
+
     * - .. py:data:: HAS_MATPLOTLIB
       - Qiskit Terra provides several visualisation tools in the :mod:`.visualization` module.
         Almost all of these are built using `Matplotlib <https://matplotlib.org/>`__, which must
@@ -205,6 +209,9 @@
 .. autoclass:: qiskit.utils.LazySubprocessTester
 """
 
+# NOTE: If you're changing this file, sync it with `requirements-optional.txt` and potentially
+# `setup.py` as well.
+
 import logging as _logging
 
 from .lazy_tester import (
@@ -256,6 +263,7 @@
     name="jax",
     install="pip install jax",
 )
+HAS_JUPYTER = _LazyImportTester(["jupyter", "nbformat", "nbconvert"], install="pip install jupyter")
 HAS_MATPLOTLIB = _LazyImportTester(
     ("matplotlib.patches", "matplotlib.pyplot"),
     name="matplotlib",

qiskit/visualization/bloch.py

@@ -62,6 +62,8 @@
 class Arrow3D(Patch3D, FancyArrowPatch):
     """Makes a fancy arrow"""
 
+    # pylint: disable=missing-function-docstring,invalid-name
+
     # Nasty hack around a poorly implemented deprecation warning in Matplotlib 3.5 that issues two
     # deprecation warnings if an artist's module does not claim to be part of the below module.
     # This revolves around the method `Patch3D.do_3d_projection(self, renderer=None)`.  The

requirements-dev.txt

@@ -1,30 +1,39 @@
-coverage>=4.4.0,<7.0
-hypothesis>=4.24.3
-python-constraint>=1.4
-ipython<7.22.0
-ipykernel<5.5.2
-ipywidgets>=7.3.0
-jupyter
-matplotlib>=3.3
-pillow>=4.2.1
+# Requirements to develop Terra, and the minimum needed to run its CI.  All
+# optional requirements should go in `requirements-optionals.txt` instead.
+#
+# Version requirements here can be more restrictive than elsewhere, because they
+# never become actual package requirements, but still try to be as relaxed as
+# possible so it's easy to develop multiple packages from the same venv.
+
+# Style
 black[jupyter]~=22.0
-pydot
+
+
+# Lint
+#
+# These versions are pinned precisely because pylint frequently includes new
+# on-by-default lint failures in new versions, which breaks our CI.
 astroid==2.14.2
 pylint==2.16.2
 ruff==0.0.267
+
+
+# Tests
+coverage>=4.4.0
+hypothesis>=4.24.3
 stestr>=2.0.0,!=4.0.0
-pylatexenc>=1.4
 ddt>=1.2.0,!=1.4.0,!=1.4.3
-seaborn>=0.9.0
+
+
+# Documentation tooling.
+#
+# This alone is not sufficient to fully build the documentation, because several
+# components of Terra use some of its optional dependencies in order to document
+# themselves.  These are the requirements that are _only_ required for the docs
+# build, and are not used by Terra itself.
+
 # TODO: switch to stable release when 4.1 is released
 reno @ git+https://github.com/openstack/reno.git@81587f616f17904336cdc431e25c42b46cd75b8f
 Sphinx>=5.0
 qiskit-sphinx-theme~=1.11.0
 sphinx-design>=0.2.0
-pygments>=2.4
-scikit-learn>=0.20.0
-scikit-quant<=0.7;platform_system != 'Windows'
-jax;platform_system != 'Windows'
-jaxlib;platform_system != 'Windows'
-docplex
-qiskit-qasm3-import

requirements-optional.txt

@@ -0,0 +1,40 @@
+# Optional dependencies of Terra that can (mostly) reliably be installed with
+# `pip`.  This file is still called `requirements-optional.txt` just to match
+# standard pip conventions, even though none of these are required.
+#
+# If updating this, you probably want to update `qiskit.utils.optionals` and
+# maybe `setup.py` too.
+
+# Test-runner enhancements.
+fixtures
+testtools
+jupyter
+
+# Interactivity.
+ipykernel
+ipython
+ipywidgets>=7.3.0
+matplotlib>=3.3
+pillow>=4.2.1
+pydot
+pygments>=2.4
+pylatexenc>=1.4
+seaborn>=0.9.0
+
+# Functionality and accelerators.
+qiskit-aer
+qiskit-qasm3-import
+python-constraint>=1.4
+cplex; python_version < '3.11'
+cvxpy
+docplex
+jax; platform_system != 'Windows'
+jaxlib; platform_system != 'Windows'
+scikit-learn>=0.20.0
+scikit-quant<=0.7; platform_system != 'Windows'
+SQSnobFit
+z3-solver>=4.7
+# Tweedledum is unmaintained and its existing Mac wheels are unreliable. If you
+# manage to get a working install on a Mac the functionality should still work,
+# but as a convenience this file won't attempt the install itself.
+tweedledum; python_version<'3.11' and platform_system!="Darwin"

requirements-tutorials.txt

@@ -0,0 +1,10 @@
+# Requirements for the tutorials CI run that go beyond the others in `requirements-optional.txt`.
+# This may also include some requirements that are only in `requirements-dev.txt`, since those
+# aren't runtime dependencies or optionals of Terra.
+
+networkx>=2.2
+jupyter
+Sphinx
+nbsphinx
+qiskit_sphinx_theme
+pyscf

setup.py

@@ -31,12 +31,12 @@
         flags=re.S | re.M,
     )
 
-
 # If RUST_DEBUG is set, force compiling in debug mode. Else, use the default behavior of whether
 # it's an editable installation.
 rust_debug = True if os.getenv("RUST_DEBUG") == "1" else None
 
-
+# If modifying these optional extras, make sure to sync with `requirements-optional.txt` and
+# `qiskit.utils.optionals` as well.
 qasm3_import_extras = [
     "qiskit-qasm3-import>=0.1.0",
 ]