Skip to content

removing useless modules - #1516

Merged
phenobarbital merged 2 commits into
masterfrom
dev
Jun 19, 2026
Merged

removing useless modules#1516
phenobarbital merged 2 commits into
masterfrom
dev

Conversation

@phenobarbital

@phenobarbital phenobarbital commented Jun 19, 2026

Copy link
Copy Markdown
Owner

Summary by Sourcery

Document accepted security risk for the optional MySQL driver, update dependencies and package version, harden the release build to ensure compiled extensions are shipped, and remove unused conversion modules and related files.

Enhancements:

  • Document the security posture and usage constraints around the vulnerable asyncmy MySQL driver.
  • Loosen and bump several dependency versions (pyarrow, aiomysql, pyjwt) and update asyncmy within the vulnerable range with explicit risk acceptance.
  • Remove unused conversion modules and associated Rust-based converter code from the package.

Build:

  • Add a release-time check in the CI build to fail if wheels are produced without the compiled Cython extension.
  • Bump the library version to 2.15.9.

CI:

  • Enforce presence of the compiled Cython extension in built wheels via cibuildwheel test configuration.

Documentation:

  • Extend SECURITY.md with details of a known accepted risk and recommended mitigations for asyncmy usage.

@sourcery-ai

sourcery-ai Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Updates security documentation and dependency constraints, tightens release CI by verifying the compiled Cython extension is present in built wheels, bumps the library version, and removes unused conversion modules and Rust-based rst_convert tooling from the package.

Flow diagram for release CI Cython extension check

flowchart TD
    A[GitHub Actions release workflow] --> B[Run cibuildwheel]
    B --> C[Built wheel environment]
    C --> D[Execute CIBW_TEST_COMMAND Python check]
    D --> E[Locate asyncdb/utils directory]
    E --> F[Search for types.*.so or types.*.pyd]
    F -->|found| G[Print path and continue release]
    F -->|not found| H[AssertionError: compiled Cython extension missing]
    H --> I[Fail workflow and prevent publish]
Loading

File-Level Changes

Change Details Files
Document and constrain usage of vulnerable asyncmy dependency while updating related database and dataframe dependency versions.
  • Add a "Known Accepted Risks" section documenting the asyncmy SQL injection advisory and usage constraints.
  • Update dataframe extra to depend on pyarrow>=23.0.1 instead of a strict 19.0.1 pin.
  • Bump asyncmy in the mysql extra to 0.2.11 and add an inline security note referencing the advisory and SECURITY.md.
  • Update mariadb extra to aiomysql==0.3.2.
  • Relax arangodb extra pyjwt pin to a minimum version >=2.13.0.
SECURITY.md
pyproject.toml
Harden release CI to prevent publishing wheels missing the compiled Cython extension.
  • Configure CIBW_TEST_COMMAND in the release workflow to import the installed package, locate asyncdb/utils/types.*.so or .pyd in platlib, assert its presence, and print a confirmation path.
  • Add comments explaining that the test command is meant to fail builds that lack the compiled Cython extension to avoid broken releases.
.github/workflows/release.yml
Bump package version for a new release and remove unused conversion modules and Rust rst_convert implementation from the source tree and distribution.
  • Increment asyncdb package version from 2.15.8 to 2.15.9.
  • Drop the pgrecords conversion module and the Rust-based rst_convert crate (Cargo.toml, Cargo.lock, src/lib.rs) from the codebase and distribution metadata files.
asyncdb/version.py
MANIFEST.in
docs/requirements-dev.txt
asyncdb/conversions/pgrecords.py
asyncdb/conversions/rst_convert/Cargo.lock
asyncdb/conversions/rst_convert/Cargo.toml
asyncdb/conversions/rst_convert/src/lib.rs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@phenobarbital
phenobarbital merged commit e7d5456 into master Jun 19, 2026
2 of 3 checks passed
@phenobarbital
phenobarbital deleted the dev branch June 19, 2026 17:57

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The CIBW_TEST_COMMAND inline Python one-liner in the release workflow is fairly dense and tightly couples the check to the current layout; consider moving this into a small test script in the repo so it’s easier to maintain and adjust if the package structure changes.
  • For the dependency pins changed from exact versions to lower bounds (e.g., pyarrow>=23.0.1, pyjwt>=2.13.0, aiomysql>=0.3.2), consider whether you want to add an upper bound or compatible release specifier to reduce the risk of future breaking changes in those libraries.
  • With the removal of pgrecords and the rst_convert Rust code, double-check that no public APIs or documented features relied on these modules; if they were user-facing, you may want to provide a deprecation path or clear migration note in the changelog.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The CIBW_TEST_COMMAND inline Python one-liner in the release workflow is fairly dense and tightly couples the check to the current layout; consider moving this into a small test script in the repo so it’s easier to maintain and adjust if the package structure changes.
- For the dependency pins changed from exact versions to lower bounds (e.g., pyarrow>=23.0.1, pyjwt>=2.13.0, aiomysql>=0.3.2), consider whether you want to add an upper bound or compatible release specifier to reduce the risk of future breaking changes in those libraries.
- With the removal of pgrecords and the rst_convert Rust code, double-check that no public APIs or documented features relied on these modules; if they were user-facing, you may want to provide a deprecation path or clear migration note in the changelog.

## Individual Comments

### Comment 1
<location path="pyproject.toml" line_range="89" />
<code_context>
     "dask==2024.8.2",
     "polars>=1.12.0",
-    "pyarrow==19.0.1",
+    "pyarrow>=23.0.1",
     "connectorx==0.4.2",
     "deltalake==0.19.2",
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Consider adding an upper bound for pyarrow to avoid unexpected breaking changes.

Relaxing the pin to `pyarrow>=23.0.1` can expose users to breaking changes introduced in later `pyarrow` releases. Please add an upper bound (for example `<24` or another tested maximum) to limit this risk.

```suggestion
    "pyarrow>=23.0.1,<24",
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread pyproject.toml
"dask==2024.8.2",
"polars>=1.12.0",
"pyarrow==19.0.1",
"pyarrow>=23.0.1",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Consider adding an upper bound for pyarrow to avoid unexpected breaking changes.

Relaxing the pin to pyarrow>=23.0.1 can expose users to breaking changes introduced in later pyarrow releases. Please add an upper bound (for example <24 or another tested maximum) to limit this risk.

Suggested change
"pyarrow>=23.0.1",
"pyarrow>=23.0.1,<24",

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant