Skip to content

Conversation

@timosachsenberg
Copy link
Collaborator

@timosachsenberg timosachsenberg commented Nov 28, 2025

Add converters for modern STL containers:

  • std::unordered_map<K,V> -> Python dict
  • std::unordered_set -> Python set
  • std::deque -> Python list
  • std::list -> Python list
  • std::optional -> Python T | None
  • std::string_view -> Python bytes/str

Update default C++ standard from C++11 to C++17 for compilation. Fix converter architecture to handle None values for std::optional.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • C++17 STL support: unordered_map, unordered_set, deque, list, optional, and string_view mapped to Python types (dict, set, list, Optional/None, bytes/str).
  • Improvements

    • Default C++ standard bumped to C++17.
    • Optional input handling fixed to accept and propagate None correctly.
  • Bug Fixes

    • Removed outdated note about arbitrary key-type support in indexing.
  • Tests

    • Added end-to-end tests validating generation, conversion, and runtime behavior for new STL support.
  • Documentation

    • Updated requirements to Cython ≥ 3.1.
  • Chores

    • .gitignore updated.

✏️ Tip: You can customize this high-level summary in your review settings.

Add converters for modern STL containers:
- std::unordered_map<K,V> -> Python dict
- std::unordered_set<T> -> Python set
- std::deque<T> -> Python list
- std::list<T> -> Python list
- std::optional<T> -> Python T | None
- std::string_view -> Python bytes/str

Update default C++ standard from C++11 to C++17 for compilation.
Fix converter architecture to handle None values for std::optional.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 28, 2025

Walkthrough

Adds C++17 STL container support and converters (unordered_map, unordered_set, deque, list, optional, string_view); registers converters; updates codegen imports and default C++ standard to C++17; fixes std::optional None handling; and adds tests and C++ test headers exercising the new mappings.

Changes

Cohort / File(s) Summary
Changelog
CHANGELOG.md
Documents New STL Container Support (C++17), default C++ standard change to C++17, and std::optional None handling fix.
Conversion system
autowrap/ConversionProvider.py
Adds converters: StdUnorderedMapConverter, StdUnorderedSetConverter, StdDequeConverter, StdListConverter, StdOptionalConverter, StdStringViewConverter; implements matching, Python type mapping, type checks, input/output conversions, and registers them in setup_converter_registry().
Code generation & imports
autowrap/CodeGenerator.py
Expands default C imports to include libcpp.unordered_map, libcpp.unordered_set, libcpp.deque, libcpp.list, libcpp.optional, libcpp.string_view; adds preincrement as inc and address as address to cython.operator imports.
PXD parsing
autowrap/PXDParser.py
Adjusts template argument extraction logic in _extract_type to handle nested/complex template args via an explicit loop and branching.
Build flags / utils
autowrap/Utils.py
Changes platform compile args from -std=c++11 to -std=c++17 for Darwin and Linux.
Tests — new STL integration
tests/test_code_generator_new_stl.py, tests/test_files/new_stl_test.pxd, tests/test_files/new_stl_test.hpp, tests/test_files/new_stl_test.pyx, tests/test_files/inherited.pyx
Adds C++ test class NewSTLTest, .pxd declarations, generated .pyx wrapper, updates generated imports, and an integration test that generates, compiles, imports, and validates Python-facing behavior for unordered_map, unordered_set, deque, list, optional, and string_view.
Wrapped containers tests
tests/test_files/wrapped_container_test.hpp, tests/test_files/wrapped_container_test.pxd, tests/test_wrapped_containers.py
Adds Item and WrappedContainerTest C++/pxd fixtures and tests covering many container combinations (vector/set/map/unordered_map/list/deque/unordered_set) with wrapped types.
Generated/test artifacts
tests/test_files/wrapped_container_wrapper.pyx, tests/test_files/new_stl_test.pyx (generated)
New/updated generated Cython wrappers exposing the C++ tests to Python.
Misc
.gitignore
Adds entries for generated test files and _codeql_detected_source_root.
Docs
docs/README.md
Bumps Cython requirement to ≥ 3.1.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

  • Review focus:
    • autowrap/ConversionProvider.py — correctness of input/output conversion code, reference vs. value semantics, wrapped-key handling in maps, nested-template cases, and std::optional None handling.
    • tests/test_files/* and tests/* — ensure tests exercise edge cases (by-ref mutation, None handling, bytes/str vs string_view) and that generated code assertions match intended types.
    • autowrap/CodeGenerator.py — confirm added imports and cython.operator aliases align with generated code.
    • autowrap/PXDParser.py — validate template-arg extraction changes for nested/complex templates.
    • autowrap/Utils.py — C++17 flag impact on supported platforms and CI/build configs.

Poem

🐰 I hopped through headers, petals and strings,
Maps and sets and deque-y things.
Optionals winked, None tucked away,
String views danced in C++17's day.
Nibbles of code — compile, test, hooray!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 42.36% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add C++17 STL container converters' clearly summarizes the main change: adding support for multiple C++17 STL containers. It is concise, specific, and directly reflects the primary purpose of the changeset.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch stl17_support

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@timosachsenberg
Copy link
Collaborator Author

about the string_view I was not 100% sure

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
tests/test_code_generator_new_stl.py (1)

82-137: Consider adding test for empty containers and additional edge cases.

The runtime tests are thorough. A few optional additions for robustness:

  • Empty container tests (empty dict, empty set, empty list)
  • Empty string for string_view

These are nice-to-haves and the current coverage is adequate for the initial implementation.

tests/test_files/new_stl_test.hpp (1)

6-14: Remove unused <vector> include.

The <vector> header is included but not used in this file.

 #include <unordered_map>
 #include <unordered_set>
 #include <deque>
 #include <list>
 #include <optional>
 #include <string_view>
 #include <string>
-#include <vector>
autowrap/ConversionProvider.py (1)

2408-2435: Cython does support automatic conversion for std::unordered_set with primitive types. The web search confirms this capability exists in Cython's standard library.

The original review comment is valid: StdSetConverter uses Cython's automatic conversion (cdef libcpp_set[$inner] $temp_var = $argument_var), while StdUnorderedSetConverter uses explicit manual iteration. Since Cython handles both containers the same way, the explicit iteration approach in lines 2408-2435 is unnecessarily verbose and potentially less efficient than leveraging Cython's automatic conversion.

This is a legitimate optional refactoring opportunity to align StdUnorderedSetConverter with the simpler pattern used in StdSetConverter.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between aba2a69 and 530e887.

📒 Files selected for processing (7)
  • CHANGELOG.md (1 hunks)
  • autowrap/CodeGenerator.py (1 hunks)
  • autowrap/ConversionProvider.py (2 hunks)
  • autowrap/Utils.py (1 hunks)
  • tests/test_code_generator_new_stl.py (1 hunks)
  • tests/test_files/new_stl_test.hpp (1 hunks)
  • tests/test_files/new_stl_test.pxd (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
tests/test_code_generator_new_stl.py (2)
autowrap/__init__.py (1)
  • parse_and_generate_code (92-106)
autowrap/Utils.py (1)
  • compile_and_import (65-147)
autowrap/ConversionProvider.py (2)
autowrap/Types.py (1)
  • CppType (42-289)
autowrap/Code.py (3)
  • Code (41-90)
  • add (56-75)
  • render (88-90)
🪛 Clang (14.0.6)
tests/test_files/new_stl_test.hpp

[error] 6-6: 'unordered_map' file not found

(clang-diagnostic-error)

🪛 Ruff (0.14.6)
autowrap/ConversionProvider.py

2163-2163: Unused method argument: cpp_type

(ARG002)


2166-2166: Unused method argument: cpp_type

(ARG002)


2259-2259: Unused method argument: res_type

(ARG002)


2259-2259: Unused method argument: with_const

(ARG002)


2309-2309: Unused method argument: cpp_type

(ARG002)


2312-2312: Unused method argument: cpp_type

(ARG002)


2437-2437: Unused method argument: res_type

(ARG002)


2437-2437: Unused method argument: with_const

(ARG002)


2517-2517: Unused method argument: cpp_type

(ARG002)


2520-2520: Unused method argument: cpp_type

(ARG002)


2612-2612: Unused method argument: res_type

(ARG002)


2612-2612: Unused method argument: with_const

(ARG002)


2681-2681: Unused method argument: cpp_type

(ARG002)


2684-2684: Unused method argument: cpp_type

(ARG002)


2782-2782: Unused method argument: res_type

(ARG002)


2782-2782: Unused method argument: with_const

(ARG002)


2865-2865: Unused method argument: cpp_type

(ARG002)


2868-2868: Unused method argument: cpp_type

(ARG002)


2913-2913: Unused method argument: res_type

(ARG002)


2913-2913: Unused method argument: with_const

(ARG002)


2976-2976: Unused method argument: cpp_type

(ARG002)


2979-2979: Unused method argument: cpp_type

(ARG002)


2982-2982: Unused method argument: cpp_type

(ARG002)


3002-3002: Unused method argument: res_type

(ARG002)


3002-3002: Unused method argument: with_const

(ARG002)


3006-3006: Unused method argument: cpp_type

(ARG002)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: test (>0.29.21, 3.10)
  • GitHub Check: test (==3.1.0, 3.13)
  • GitHub Check: test (==3.1.0, 3.10)
  • GitHub Check: test (==3.1.0, 3.11)
  • GitHub Check: test (==3.1.0, 3.12)
🔇 Additional comments (10)
autowrap/CodeGenerator.py (1)

2027-2036: LGTM! New STL imports and operator aliases are correctly added.

The new libcpp imports follow the existing aliasing pattern and the additional operator imports (preincrement as inc, address as address) are required by the new converter iteration code in ConversionProvider.py.

CHANGELOG.md (1)

1-18: LGTM! Changelog entry is comprehensive and well-structured.

The entry clearly documents all new STL container support, the C++ standard upgrade requirement, and the None handling fix. Good practice to note the breaking change (C++17 requirement) prominently.

tests/test_code_generator_new_stl.py (1)

16-77: LGTM! Comprehensive test coverage for new STL converters.

The test covers:

  • Code generation verification (import statements, method signatures, has_value pattern)
  • Runtime behavior for all 6 new container types
  • Edge cases like None for std::optional

The test structure follows existing patterns in the codebase.

tests/test_files/new_stl_test.pxd (1)

1-56: LGTM! Well-documented PXD example for new STL containers.

The file serves as both a test fixture and documentation showing users how to declare C++17 STL types in their PXD files. The comments explaining each container type's Python mapping are helpful.

tests/test_files/new_stl_test.hpp (1)

15-113: LGTM! Clean C++17 test implementation.

The implementation is straightforward and correctly exercises all the new STL container types. The static analysis error about missing headers is a false positive from the sandbox environment lacking C++ standard library headers.

autowrap/ConversionProvider.py (4)

2140-2164: LGTM! StdUnorderedMapConverter follows established patterns.

The converter correctly mirrors the existing StdMapConverter implementation while being appropriately simplified for the initial release. The static analysis warnings about unused cpp_type arguments in matches() and matching_python_type() are false positives—these match the required TypeConverterBase interface.


2835-2942: LGTM! StdOptionalConverter correctly handles None values.

The design is thoughtful:

  • Uses object as Python type to allow None to pass through
  • type_check_expression validates the value is either None or the correct inner type
  • Input conversion correctly handles both None (empty optional) and values
  • Output conversion properly checks has_value() before accessing

2985-3009: Verify string_view lifetime safety in input conversion.

The input conversion creates a Python bytes object that the string_view references. This is safe as long as the bytes object outlives the C++ function call, which it should since it's a local variable in the generated wrapper.

However, the docstring (lines 2954-2959) correctly notes this lifetime concern. For safety, ensure users understand that string_view should only be used for input parameters, not stored in C++ objects.


3112-3118: LGTM! Converters registered in correct order.

The new converters are registered after the existing ones, which is the correct pattern. The comment accurately identifies the C++ standard versions.

autowrap/Utils.py (1)

92-100: C++17 flag missing for Windows, but Windows support not currently tested.

The review comment correctly identifies that Windows lacks an explicit C++17 flag despite the codebase using C++17 features (std::optional, std::string_view in tests/test_files/new_stl_test.hpp). However, the CI pipeline only tests on Linux (ubuntu-latest), suggesting Windows support is not currently a priority.

If Windows support is intended in the future, adding /std:c++17 to the MSVC compile flags would be appropriate since MSVC doesn't enable all C++17 features by default in all versions. For now, this appears to be a valid forward-looking concern rather than a blocking issue.

@poshul
Copy link
Contributor

poshul commented Nov 28, 2025

It looks like string_view was only available from cython 3.1 on: cython/cython#6651

@timosachsenberg
Copy link
Collaborator Author

@copilot try to add and test the cleanup code. if not remove the deque support

Copy link
Contributor

Copilot AI commented Nov 28, 2025

@timosachsenberg I've opened a new pull request, #207, to work on those changes. Once the pull request is ready, I'll request review from you.

@poshul
Copy link
Contributor

poshul commented Nov 28, 2025

It looks like string_view was only available from cython 3.1 on: cython/cython#6651

Since cython 3.2 also works, I'd be amenable to just dropping 3.0 support and testing on 3.1 and 3.2.

@timosachsenberg
Copy link
Collaborator Author

Good idea!

timosachsenberg and others added 3 commits November 28, 2025 13:57
…#207)

* Initial plan

* Add tests for std::deque and std::list mutable reference cleanup code

Co-authored-by: timosachsenberg <[email protected]>

* Run code review and codeql checks

Co-authored-by: timosachsenberg <[email protected]>

* Remove CodeQL artifact and update gitignore

Co-authored-by: timosachsenberg <[email protected]>

---------

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: timosachsenberg <[email protected]>
Co-authored-by: Timo Sachsenberg <[email protected]>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
tests/test_code_generator_new_stl.py (1)

147-153: Consider adding explicit str cases for string_view coverage

Right now the string_view runtime checks only exercise bytes inputs. If you intend to support both bytes and str on the Python side, consider adding analogous assertions using plain str arguments to getStringViewLength/stringViewToString so regressions there are caught.

tests/test_files/new_stl_test.pxd (1)

19-58: PXD declarations match header semantics; consider const‑ref if you don’t want in/out behavior

The _NewSTLTest declarations correctly mirror the C++ header in terms of container types and return values. Using non‑const references for the sum* methods (libcpp_*&) is ABI‑safe against the current const & definitions and pairs with the wrappers’ “copy back” behavior to allow future mutating implementations.

If you’re certain these methods should remain read‑only, you could tighten them to const libcpp_*& to document that intent more clearly, but it isn’t required for correctness.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bb5fe19 and 1a2874e.

📒 Files selected for processing (6)
  • .gitignore (1 hunks)
  • tests/test_code_generator_new_stl.py (1 hunks)
  • tests/test_files/inherited.pyx (2 hunks)
  • tests/test_files/new_stl_test.hpp (1 hunks)
  • tests/test_files/new_stl_test.pxd (1 hunks)
  • tests/test_files/new_stl_test.pyx (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
tests/test_code_generator_new_stl.py (2)
autowrap/__init__.py (1)
  • parse_and_generate_code (92-106)
autowrap/Utils.py (1)
  • compile_and_import (65-147)
🪛 Clang (14.0.6)
tests/test_files/new_stl_test.hpp

[error] 6-6: 'unordered_map' file not found

(clang-diagnostic-error)

🪛 Ruff (0.14.6)
tests/test_code_generator_new_stl.py

130-130: Loop control variable i not used within loop body

(B007)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: test (==3.2.0, 3.13)
  • GitHub Check: test (==3.2.0, 3.12)
  • GitHub Check: test (==3.1.0, 3.10)
  • GitHub Check: test (==3.1.0, 3.12)
🔇 Additional comments (5)
.gitignore (1)

179-179: Verify the purpose and necessity of the CodeQL artifact entry.

The entry _codeql_detected_source_root is added without a clarifying comment. While gitignore entries don't strictly require documentation, this artifact name is not immediately recognizable as a standard build or code-generation artifact.

Please confirm:

  1. Is _codeql_detected_source_root a documented artifact generated by the autowrap code generation pipeline (e.g., as part of the new C++17 STL support tooling)?
  2. Should this entry have an associated comment for maintainability (e.g., # CodeQL artifacts or # Build-generated files)?
  3. If this is unrelated to PR#202 objectives, consider separating it into a dedicated commit.
tests/test_files/inherited.pyx (1)

1-1: New STL libcpp cimports look consistent with converter additions

The bumped generator header and added libcpp_* cimports for unordered_map/unordered_set/deque/list/optional/string_view are consistent with the new C++17 container support. Having them present in this generated file even if unused here is fine.

Also applies to: 14-19

tests/test_code_generator_new_stl.py (1)

16-82: End‑to‑end generation + runtime test is well structured

This test exercises both code generation (imports, wrappers present) and runtime behavior through compile_and_import, which gives good coverage for the new STL converters.

tests/test_files/new_stl_test.hpp (1)

6-126: C++ test fixture cleanly exercises all new STL container shapes

The NewSTLTest class methods cover read‑only and mutating scenarios for unordered_map/set, deque, list, optional, and string_view, matching the Python wrappers and tests. Signatures (by‑value vs const & vs mutable &) are well chosen for the behaviors you test.

tests/test_files/new_stl_test.pyx (1)

49-188: Container converters (unordered_map/set, deque, list, optional) look correct and memory‑safe

The wrappers for unordered_map/unordered_set, deque, list, and optional correctly:

  • Validate Python argument types.
  • Build corresponding C++ containers (using stack objects or new+del where appropriate).
  • Call into the NewSTLTest methods.
  • Propagate potential C++‑side mutations back into the Python dict/set/list via in‑place updates.
  • Map std::optional<int> to int | None cleanly, including the None‑to‑empty‑optional case via default construction and value_or(-1).

I don’t see correctness or lifetime issues in these sections.

Copy link
Contributor

Copilot AI commented Nov 28, 2025

@timosachsenberg I've opened a new pull request, #210, to work on those changes. Once the pull request is ready, I'll request review from you.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
CHANGELOG.md (2)

4-12: Verify that all documented STL converters are fully implemented and tested.

The new converters section documents six new C++17 STL types, but several PR discussion items suggest unresolved concerns that should be reflected or verified:

  1. std::deque support: PR comments indicate deque cleanup code needs to be added and tested, or the feature should be removed if cleanup cannot be implemented. Confirm this requirement has been addressed.
  2. std::string_view support: PR comments note that string_view support in Cython became available in version 3.1 and suggest dropping Cython 3.0 support in favor of 3.1+. Verify this constraint is reflected in CI/test configuration and documented if there are version-specific limitations.

Consider adding clarifying notes if any converters have platform, version, or implementation constraints (e.g., "std::string_view requires Cython ≥ 3.1").


13-18: Ensure C++ standard change and CI/tests are aligned.

The CHANGELOG documents updating the default C++ standard from C++11 to C++17, which enables optional and string_view support. Per PR discussions, CI and tests should also be updated to target Cython ≥ 3.1 (dropping 3.0 support). Verify that:

  1. CI configuration has been updated to test against Cython 3.1 and 3.2 (as suggested in PR comments)
  2. Build/compilation documentation reflects the new C++17 requirement
  3. Any breaking changes for users still on older Cython versions are documented

If CI/test updates are not yet complete, I can help verify the changes or generate relevant configuration updates.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1a2874e and 8386ae9.

📒 Files selected for processing (1)
  • CHANGELOG.md (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
  • GitHub Check: test (==3.2.0, 3.10)
  • GitHub Check: test (==3.1.0, 3.11)
  • GitHub Check: test (==3.2.0, 3.13)
  • GitHub Check: test (==3.1.0, 3.12)
  • GitHub Check: test (==3.2.0, 3.12)
  • GitHub Check: test (==3.2.0, 3.11)
  • GitHub Check: test (==3.1.0, 3.10)
  • GitHub Check: test (==3.1.0, 3.13)
🔇 Additional comments (1)
CHANGELOG.md (1)

21-22: Clarify status of "arbitrary key types" support.

The enriched summary indicates that a previous bullet about arbitrary key types support was removed, but lines 21–22 still document this feature in the final CHANGELOG. Please clarify:

  1. Is this feature included in the current PR, or should it be removed from this CHANGELOG entry?
  2. If retained, does it relate to or depend on the C++17/STL converter changes?

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
tests/test_files/new_stl_test.pyx (2)

113-187: Deque/list wrappers correctly mirror Python list semantics

Using stack-allocated libcpp_deque[int]/libcpp_list[double] and then re-slicing the original Python lists (d[:] / l[:]) cleanly exposes the C++ operations while avoiding any manual cleanup issues, and the O(n) list rebuilds are fine for test code. If you ever want to reduce boilerplate, these loops could be replaced by direct STL↔Python coercions, but that’s purely an optional refactor and not required for correctness here.


189-212: Optional[int] ↔ None mapping is implemented correctly; bool typing check is redundant

getOptionalValue correctly maps an engaged libcpp_optional[int] to an int and a disengaged one to None, and unwrapOptional builds a libcpp_optional[int] only when opt is not None, so the Optional[int]/None semantics look right. The combination of bool hasValue plus assert isinstance(hasValue, pybool_t) is effectively redundant because Cython will already coerce truthy Python values to C++ bool; it’s harmless, but you could simplify to an untyped Python parameter with an internal cdef bool if you want a more idiomatic interface.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8a417c7 and 7d9278e.

📒 Files selected for processing (1)
  • tests/test_files/new_stl_test.pyx (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
  • GitHub Check: test (==3.1.0, 3.13)
  • GitHub Check: test (==3.1.0, 3.12)
  • GitHub Check: test (==3.1.0, 3.10)
  • GitHub Check: test (==3.2.0, 3.11)
  • GitHub Check: test (==3.2.0, 3.12)
  • GitHub Check: test (==3.2.0, 3.13)
  • GitHub Check: test (==3.1.0, 3.11)
  • GitHub Check: test (==3.2.0, 3.10)
🔇 Additional comments (2)
tests/test_files/new_stl_test.pyx (2)

49-111: STL unordered_map/unordered_set conversions are correct and memory-safe

The dict/set round-trips via libcpp_unordered_map/libcpp_unordered_set look good: keys/values are type-checked, C++ containers are built explicitly, mutations are propagated back to the Python inputs, and new allocations are paired with del so there’s no leak. This relies on Cython’s standard STL↔Python conversions and the c_string_encoding=ascii directive for string keys, which matches the intended bytes-based mapping here.


214-240: string_view wrappers now accept both bytes and str as intended and depend on Cython’s string_view support

Changing getStringViewLength/stringViewToString to take an untyped sv and handling bytes/str via isinstance and UTF‑8 encoding fixes the earlier issue where str handling was unreachable behind a bytes-typed parameter and aligns with the Union[bytes, str] docstrings. Casting the normalised bytes buffer to libcpp_string_view for the C++ call is sound as long as the C++ side doesn’t store the view, and it matches Cython’s std::string_view from‑Py conversions that landed in the 3.1+ series; just ensure CI is actually running against the agreed Cython ≥3.1 versions.

@jpfeuffer
Copy link
Contributor

The tests do not cover containers of wrapped classes or if you want to support it, nested containers.

@timosachsenberg
Copy link
Collaborator Author

The tests do not cover containers of wrapped classes or if you want to support it, nested containers.

good point. I also wasn't sure about #210 which I closed.
What is your take on that?

So in general we should increase coverage of wrapped classes and ideally nested container (but this was the stuff with recursively parsing template parameter?)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
autowrap/ConversionProvider.py (2)

2750-2765: Reference cleanup gap for wrapper classes acknowledged.

The past review comment correctly identified that wrapper class references in deque don't have cleanup code to reflect C++ modifications back to Python. This is acceptable if by-ref wrapper class deques are not a priority use case for this initial implementation.


2918-2933: Same reference cleanup gap as StdDequeConverter.

Already noted in past review comments.

🧹 Nitpick comments (4)
autowrap/ConversionProvider.py (1)

2265-2268: Minor: Use not in for membership test.

Python style prefers not in over not ... in for readability.

             if (
                 tt_key.base_type in self.converters.names_of_wrapper_classes
-                and not tt_value.base_type in self.converters.names_of_wrapper_classes
+                and tt_value.base_type not in self.converters.names_of_wrapper_classes
             ):

Same applies to line 2291.

tests/test_files/wrapped_container_test.hpp (3)

16-37: Consider consistency between comparison operators and marking single-arg constructor explicit.

Two observations:

  1. operator< compares only value_, while operator== compares both value_ and name_. This inconsistency could lead to confusing behavior where two Items are considered equivalent by ordered containers (std::set, std::map) but not equal by operator==.

  2. The single-argument constructor Item(int v) allows implicit conversions from int to Item, which may be unintended.

For a test file this may be acceptable, but for robustness:

-    Item(int v) : value_(v), name_("item") {}
+    explicit Item(int v) : value_(v), name_("item") {}

And consider aligning operator< with operator==:

     bool operator<(const Item& other) const {
-        return value_ < other.value_;
+        if (value_ != other.value_) return value_ < other.value_;
+        return name_ < other.name_;
     }

39-47: Consider including name_ in the hash for better distribution.

The hash only uses value_, while operator== compares both value_ and name_. This is functionally correct but may cause unnecessary hash collisions when items differ only by name.

     struct hash<Item> {
         size_t operator()(const Item& item) const {
-            return std::hash<int>()(item.value_);
+            size_t h1 = std::hash<int>()(item.value_);
+            size_t h2 = std::hash<std::string>()(item.name_);
+            return h1 ^ (h2 << 1);
         }
     };

58-64: Consider using const references for read-only container parameters.

Multiple sum* methods take non-const references but only read from the containers. Using const& would be more idiomatic and allow passing temporary containers.

This applies to: sumVectorItems, sumSetItems, sumMapValues, sumMapKeys, sumNestedVector, sumMapItemToVecInt, sumMapBoth, sumUnorderedMapKeys, sumUnorderedMapValues, sumUnorderedMapBoth, sumUnorderedMapItemToVecInt, sumListItems, sumDequeItems, sumUnorderedSetItems, sumListOfVectors, sumDequeOfVectors.

Example fix for this method:

-    int sumVectorItems(std::vector<Item>& items) {
+    int sumVectorItems(const std::vector<Item>& items) {
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0950c4d and d0d5887.

📒 Files selected for processing (6)
  • .gitignore (1 hunks)
  • autowrap/ConversionProvider.py (3 hunks)
  • autowrap/PXDParser.py (1 hunks)
  • tests/test_files/wrapped_container_test.hpp (1 hunks)
  • tests/test_files/wrapped_container_test.pxd (1 hunks)
  • tests/test_wrapped_containers.py (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
autowrap/PXDParser.py (1)
autowrap/Types.py (1)
  • CppType (42-289)
autowrap/ConversionProvider.py (2)
autowrap/Code.py (3)
  • Code (41-90)
  • add (56-75)
  • render (88-90)
autowrap/Types.py (1)
  • CppType (42-289)
🪛 Clang (14.0.6)
tests/test_files/wrapped_container_test.hpp

[error] 4-4: 'vector' file not found

(clang-diagnostic-error)

🪛 Ruff (0.14.6)
autowrap/PXDParser.py

284-284: NameNode may be undefined, or defined from star imports

(F405)


287-287: IndexNode may be undefined, or defined from star imports

(F405)

autowrap/ConversionProvider.py

2184-2184: Unused method argument: cpp_type

(ARG002)


2187-2187: Unused method argument: cpp_type

(ARG002)


2267-2267: Test for membership should be not in

Convert to not in

(E713)


2291-2291: Test for membership should be not in

Convert to not in

(E713)


2359-2359: Unused method argument: res_type

(ARG002)


2359-2359: Unused method argument: with_const

(ARG002)


2483-2483: Unused method argument: cpp_type

(ARG002)


2486-2486: Unused method argument: cpp_type

(ARG002)


2612-2612: Unused method argument: res_type

(ARG002)


2612-2612: Unused method argument: with_const

(ARG002)


2692-2692: Unused method argument: cpp_type

(ARG002)


2695-2695: Unused method argument: cpp_type

(ARG002)


2787-2787: Unused method argument: res_type

(ARG002)


2787-2787: Unused method argument: with_const

(ARG002)


2856-2856: Unused method argument: cpp_type

(ARG002)


2859-2859: Unused method argument: cpp_type

(ARG002)


2957-2957: Unused method argument: res_type

(ARG002)


2957-2957: Unused method argument: with_const

(ARG002)


3040-3040: Unused method argument: cpp_type

(ARG002)


3043-3043: Unused method argument: cpp_type

(ARG002)


3088-3088: Unused method argument: res_type

(ARG002)


3088-3088: Unused method argument: with_const

(ARG002)


3151-3151: Unused method argument: cpp_type

(ARG002)


3154-3154: Unused method argument: cpp_type

(ARG002)


3157-3157: Unused method argument: cpp_type

(ARG002)


3177-3177: Unused method argument: res_type

(ARG002)


3177-3177: Unused method argument: with_const

(ARG002)


3181-3181: Unused method argument: cpp_type

(ARG002)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
  • GitHub Check: test (==3.2.0, 3.12)
  • GitHub Check: test (==3.2.0, 3.11)
  • GitHub Check: test (==3.1.0, 3.13)
  • GitHub Check: test (==3.2.0, 3.10)
  • GitHub Check: test (==3.2.0, 3.13)
  • GitHub Check: test (==3.1.0, 3.12)
  • GitHub Check: test (==3.1.0, 3.10)
  • GitHub Check: test (==3.1.0, 3.11)
🔇 Additional comments (11)
.gitignore (1)

175-180: LGTM!

The additions correctly ignore the generated wrapper file for the new wrapped container tests and the CodeQL artifact.

autowrap/PXDParser.py (1)

282-293: LGTM - Improved nested template handling.

The expanded loop correctly handles the three cases for template arguments:

  • NameNode: Simple type names (e.g., Item in map[int, Item])
  • IndexNode: Nested templates (e.g., vector[int] in map[int, vector[int]])
  • CComplexBaseTypeNode: Types with declarators (pointers, references)

The static analysis warnings about NameNode and IndexNode being undefined are false positives—they're imported via the star import from Cython.Compiler.ExprNodes on line 45.

autowrap/ConversionProvider.py (4)

1106-1127: LGTM - Proper handling of maps with both wrapped key and value.

The new output conversion correctly wraps both keys and values when they are wrapped classes, creating new Python wrapper objects with proper shared_ptr management.


3010-3058: LGTM - Well-designed Optional handling.

The design correctly uses object as the Python type to allow None while validating the actual type at runtime. The docstring clearly documents this intentional choice.


3180-3184: Verify string_view output safety in C++ implementations.

The output conversion correctly creates a Python bytes copy from the string_view. However, if C++ returns a string_view pointing to stack-allocated or freed memory, this will crash or produce garbage. This is inherent to string_view semantics, but consider adding a note in documentation that C++ functions returning string_view must ensure the underlying data remains valid.


3287-3293: LGTM - Correct converter registration.

All six new STL converters are properly registered after the existing converters and before class-specific converters, matching the established pattern.

tests/test_wrapped_containers.py (2)

1-42: LGTM - Comprehensive test fixture and setup.

The module-scoped fixture correctly compiles the test wrapper, and the naming convention (wrapped_container_wrapper.pyx) avoids Cython redeclaration issues as documented in the comment.


45-509: LGTM - Thorough container test coverage.

Excellent coverage of:

  • All container types (vector, set, map, list, deque, unordered variants)
  • Wrapped classes as keys, values, and both
  • Nested containers (vector<vector>, list<vector>)
  • Edge cases (empty containers)
  • Item class construction, methods, and copy semantics

The tests validate both input conversion (Python → C++) and output conversion (C++ → Python).

tests/test_files/wrapped_container_test.pxd (1)

1-130: LGTM - Cython declarations properly paired with C++ header.

The verification confirms wrapped_container_test.hpp exists at tests/test_files/wrapped_container_test.hpp, matching the extern declaration on line 13 of the pxd file. The declarations are well-structured with comprehensive coverage of container scenarios and clear sectional comments documenting test coverage.

tests/test_files/wrapped_container_test.hpp (2)

1-14: LGTM - Well-structured header with appropriate includes.

The include guard follows standard conventions, and all necessary STL headers are included for the container types used throughout the file.

Note: The static analysis error about 'vector' file not found is a false positive - this is a standard C++ header that requires a properly configured C++ toolchain.


50-355: Well-structured test class with comprehensive container coverage.

The WrappedContainerTest class provides excellent coverage of STL container types with wrapped class elements:

  • Ordered containers: vector, set, map
  • Unordered containers: unordered_map, unordered_set
  • Sequential containers: list, deque
  • Nested containers: vector<vector<Item>>, list<vector<int>>, deque<vector<int>>
  • Wrapped class as key, value, or both

The clear section headers and consistent naming conventions (sum*, create*, appendTo*) make the test scenarios easy to follow.

@timosachsenberg
Copy link
Collaborator Author

@jpfeuffer added

@timosachsenberg timosachsenberg merged commit 9021f4c into master Nov 28, 2025
11 checks passed
@jpfeuffer
Copy link
Contributor

jpfeuffer commented Nov 28, 2025

Yes nested containers will trigger the recursive part of the if cases in our conversion providers. We only support it for vectors currently.

@jpfeuffer
Copy link
Contributor

@timosachsenberg I think this still does not test hashing or any kind of lookups for dicts.
It always iterates through all items of the dict. Not very useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants