Cython3 migration

.github/workflows/ci-autowrap.yaml

@@ -15,10 +15,6 @@ jobs:
       fail-fast: false
       matrix:
         include:
-          - CYTHON: "<=0.29.21"
-            python-version: "3.9" # Cython < 0.29.21 not compatible with 3.10, so neither are we
-          - CYTHON: ">0.29.21"
-            python-version: "3.10"
           - CYTHON: "==3.0.0"
             python-version: "3.10"
           - CYTHON: "==3.0.0"

CHANGELOG.md

@@ -8,3 +8,5 @@ autowrap 0.24.0 (unreleased)
 autowrap 0.23.0
 
 Support for Cython 3.1! This means the removal of some py2 compatibility code, no more python distinction between long and int, some fixes to multiline comment processing.
+
+- Dropped support for Cython versions older than 3.0; autowrap now requires Cython ≥ 3.0 and Python ≥ 3.9.

README.md

@@ -14,6 +14,12 @@ writing a wrapper can be learned easily. Further [Cython][] prevents you from
 many typical errors which might get in your way if you write such a wrapper in
 C++.
 
+Requirements
+------------
+
+- Python ≥ 3.9
+- Cython ≥ 3.0
+
 
 This wrapping process typically consist of four steps:
 

README_DEVELOP

@@ -1,9 +1,34 @@
-Activate the project in "development mode" first:
+Development setup
+=================
 
-   $ python setup.py develop
+The recommended way to work on `autowrap` is to use a virtual environment,
+install the package in editable mode, and run the tests from the project root.
 
-For testing run
+1. Create and activate a virtual environment (optional but recommended):
 
-   $ py.test tests
+   $ python -m venv .venv
+   $ source .venv/bin/activate         # on Windows: .venv\Scripts\activate
+
+2. Install `autowrap` with development dependencies:
+
+   $ python -m pip install -U pip
+   $ python -m pip install -e .[dev]
+
+   (Use `python3` instead of `python` if needed on your system.)
+
+3. Run the test suite from the project root:
+
+   $ pytest tests/ -v
+
+   For local coverage, you can also use:
+
+   $ ./run_test_coverage.sh
+
+4. Verify the CLI is available:
+
+   $ autowrap --help
+
+These commands have been verified to work with the current `pyproject.toml`
+configuration and are the preferred starting point for developing on `autowrap`.
 
 from the projects directory.

TODO

@@ -1,66 +1,36 @@
-- mehr klassen wrappen
+Autowrap TODO
+=============
 
-- attribut zugriff ! ( ->ProteinIdentification::ProteinGroup z.b.)
+- Internal code generator refactor
 
-- wrap-attach auch für klassen !
+  Consider splitting `CodeGenerator` into smaller, focused classes
+  (e.g. for class generation, iterator support, reference handling)
+  to improve maintainability and make future extensions easier.
 
-- codegen refak: 
-  extra klasse fürs eigentliche codegen.
-  ziel: via vererbung auch iteratoren udn ref-zugriffen wrappen
+- Improved iterator support
 
-- iteratoren und referenzen (siehe ungen)
+  Support parameterized iterator ranges (e.g. methods like
+  `beginRT(rtmin)` / `endRT(rtmax)` annotated with `wrap-iter-begin`
+  / `wrap-iter-end`) and expose them as dedicated Python iterator or
+  range methods instead of only supporting parameterless `begin()` /
+  `end()`.
 
-- DPosition(mit numerischen template argument)
+- Reference semantics for methods returning references
 
-- statische methoden direkt an die klassen hängen, bevor "wrap-inherits"
-  aufgelöst wird !
+  For methods that return references to other wrapped objects, explore
+  using holder / proxy objects (using the existing
+  `AutowrapRefHolder` / `AutowrapPtrHolder` infrastructure) to model
+  true reference semantics, rather than always copying values into new
+  wrapper instances. The goals are:
 
-- config extra cimport statments
+  - Preserve the lifetime of the referred-to object safely.
+  - Allow attribute/method access on the referenced object to affect
+    the original C++ object where appropriate.
 
-- begin(), end() mit param, eg
-     beginRT(rtmin)  # wrap-iter-begin rtrange
-     endRT(rtmax)    # wrap-iter-end   rtrange
+- Safe wrapping of non-const iterators
 
-  -> methode  rtrange(rtmin, rtmax)
-
-- neu: ClassGenerator (zum code erzeugen), 
-       RefProxyClassGenerator, IterProxyClassGenerator
-
-ideen:
-
-  1) wenn methode M von X eine Ref auf Y zurückgibt:
-
-     idee: reference holder objekt als pointer sollte mit cython
-     (cython schiebt variable dekl im c++ code nach vorne, z.b
-      T & x;
-      was so nicht geht, da referenz ja initalisierte werdne muss
-      statt desen:
-      holder[T] * X;
-
-      und später:
-
-        X.setReference(wrapped.inst.method_which_returns_ref(..))
-
-     als holder im wrappeb object.
-     original objekt als shared_ptr mitspeichern damit die gespeicherte
-     refernz (hoffentlich) erhalten bleibt.
-
-     -----------
-
-     - alle wrapped objekte haben shared_ptr
-     - methode M returns Proxy mit shared_ptr to X und name von M
-     - proxy wrapped methoden von R durch impliziten aufruf der methode M
-
-       der einfach heit halber: proxy objekt hat keine methoden die
-       referenzen auf zu wrappende objekte zurückliefern
-       ansonsten: --- chaining !?
-
-
-  2) sicheres wrappen von non const iteratoren:
-
-     - iterobject speichert shared_ptr auf das orignal objekt
-
-     - iterobject speicher iterator selbst, damit kann man
-       methoden des referenzierten objekts deim iterobjekt anhängen
-       und so bleiben inplace modifiationen erhalten
+  Investigate iterator wrappers that store both a reference to the
+  original container and the iterator state, so that in-place
+  modifications through iterator access are reflected in the
+  underlying C++ container (instead of always iterating over copies).
 

autowrap/Main.py

@@ -172,20 +172,14 @@ def register_converters(converters):
 def run_cython(inc_dirs, extra_opts, out, warn_level=1):
     from Cython.Compiler.Main import compile, CompilationOptions
     import Cython.Compiler.Errors
+    import Cython.Compiler.Options as CythonOptions
 
     Cython.Compiler.Errors.LEVEL = warn_level
 
-    # Try to get directive_defaults (API differs from 0.25 on)
-    try:
-        from Cython.Compiler.Options import directive_defaults
-    except ImportError:
-        # Cython 0.25
-        import Cython.Compiler.Options
-
-        directive_defaults = Cython.Compiler.Options.get_directive_defaults()
-
+    # Start from Cython's default compiler directives
+    directive_defaults = CythonOptions.get_directive_defaults()
     # TODO merge these options, if compiler_directives is given in extra_opts? Otherwise they are overwritten
-    directive_defaults["binding"] = False  # For backwards-compat to Cython 0.X
+    directive_defaults["binding"] = False
     directive_defaults["boundscheck"] = False
     directive_defaults["wraparound"] = False
     directive_defaults["language_level"] = sys.version_info.major

autowrap/PXDParser.py

@@ -411,9 +411,7 @@ def parseTree(cls, node: Cython.Compiler.Nodes.CppClassNode, lines: Collection[s
             and isinstance(template_parameters, list)
             and isinstance(template_parameters[0], tuple)
         ):
-            # Cython 0.24 uses [(string, bool)] to indicate name and whether
-            # template argument is required or optional.
-            # For now, convert to pre-0.24 format
+            # Convert Cython's (name, required_flag) tuples to just the name
             template_parameters = [t[0] for t in template_parameters]
         try:
             class_annotations = parse_class_annotations(node, lines)
@@ -624,10 +622,7 @@ def parse_pxd_file(path, warn_level=1):
     source = CompilationSource(source_desc, name, os.getcwd())
     result = create_default_resultobj(source, options)
 
-    try:  # Cython 0.X
-        context = options.create_context()
-    except:  # Cython 3.X
-        context = Context.from_options(options)
+    context = Context.from_options(options)
     pipeline = Pipeline.create_pyx_pipeline(context, options, result)
     context.setup_errors(options, result)
     root = pipeline[0](source)  # only parser

autowrap/__init__.py

@@ -40,6 +40,21 @@
 logger = L.getLogger(__name__)
 logger.setLevel(L.INFO)
 
+try:
+    from Cython.Compiler.Version import version as _cython_version
+except ImportError:
+    _cython_version = "0"
+else:
+    try:
+        _major_str = str(_cython_version).split(".")[0]
+        if int(_major_str) < 3:
+            raise RuntimeError(
+                "autowrap requires Cython >= 3.0; found Cython %s" % _cython_version
+            )
+    except Exception:
+        # If parsing the version fails for some reason, do not block import here.
+        pass
+
 """
 The autowrap process consists of two steps:
     i) parsing of files (done by DeclResolver, which in turn uses the PXDParser

docs/README.md

@@ -9,6 +9,12 @@ annotations in the header files to generate correct Cython code. For an
 example, please have a look at `examples/int_holder.h` and
 `example/int_holder.pxd` which together form the input to the program.
 
+Requirements
+------------
+
+- Python ≥ 3.9
+- Cython ≥ 3.0
+
 Simple example
 ---------------------
 
@@ -84,12 +90,12 @@ You can build the final Python extension module by running
 And you can use the final module running
 
 ```python
-    >>> import py_int_holder
-    >>> ih = py_int_holder.IntHolder(42)
-    >>> print ih.i_
-    42
-    >>> print ih.add(ih)
-    84
+>>> import py_int_holder
+>>> ih = py_int_holder.IntHolder(42)
+>>> print(ih.i_)
+42
+>>> print(ih.add(ih))
+84
 ```
 
 To get some insight how `autowrap` works, you can inspect files

pyproject.toml

@@ -29,7 +29,7 @@ classifiers = [
 ]
 requires-python = ">=3.9"
 dependencies = [
-    "Cython>=0.19",
+    "Cython>=3.0",
     "setuptools",
 ]
 

tests/test_code_generator.py

@@ -41,7 +41,6 @@
 import autowrap.Utils
 import autowrap.Code
 import autowrap
-from Cython.Compiler.Version import version as cython_version
 import os
 import math
 import sys
@@ -73,8 +72,6 @@ def test_enums():
 
     See tests/test_files/enums.pxd for the full example.
     """
-    if int(cython_version[0]) < 3:
-        return
     target = os.path.join(test_files, "enums.pyx")
 
     include_dirs = autowrap.parse_and_generate_code(
@@ -287,9 +284,6 @@ def test_templated():
     assert templated_o.xi[1].get() == 12
 
     # Test (wrap-attached) free functions = old way to wrap static functions (can only be called with class)
-    if int(str(cython_version).split(".")[0]) < 3:
-        assert templated.computeEight() == 8
-        assert templated_o.computeEight() == 8
     assert twrapped.Templated.computeEight() == 8
     assert twrapped.Templated_other.computeEight() == 8
 

tests/test_cython_version.py

@@ -0,0 +1,7 @@
+from Cython.Compiler.Version import version as cython_version
+
+
+def test_cython_major_version_at_least_3() -> None:
+    major_str = str(cython_version).split(".")[0]
+    assert int(major_str) >= 3
+

tests/test_files/A.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 
 cdef extern from "A.h":
 

tests/test_files/B.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 
 cdef extern from "B.h":
 

tests/test_files/C.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 
 cdef extern from "C.h":
 

tests/test_files/Cycle0.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 
 cdef extern from "Cycle0.h":
 

tests/test_files/Cycle1.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 
 cdef extern from "Cycle1.h":
 

tests/test_files/Cycle2.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 
 cdef extern from "Cycle2.h":
 

tests/test_files/D.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 
 cdef extern from "D.h":
 

tests/test_files/base.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 cdef extern from "abc.hpp":
 
     cdef cppclass Base[X]:

tests/test_files/base0.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 cdef extern from "abc.hpp":
 
     cdef cppclass Base0:  # wrap-ignore

tests/test_files/enums.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 #
 # =============================================================================
 # Example: Wrapping C++ Enums in Different Namespaces

tests/test_files/gil_testing.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 from libc.string cimport const_char
 
 cdef extern from "gil_testing.hpp":

tests/test_files/inherited.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 
 cdef extern from "inherited.hpp":
 
@@ -46,4 +46,4 @@ cdef extern from "inherited.hpp":
         InheritedTwo()
         A getBase()
         B getBaseB()
-        int getBaseZ()
+        int getBaseZ()

tests/test_files/int_container_class.pxd

@@ -1,4 +1,4 @@
-# cython: language_level=2
+# cython: language_level=3
 from libcpp.list cimport *
 
 cdef extern from "int_container_class.hpp":

tests/test_files/int_container_class.pyx

@@ -1,4 +1,4 @@
-#cython: language_level=2
+#cython: language_level=3
 from int_container_class cimport X as X_, XContainer as XContainer_
 from cython.operator import dereference as deref