Refactoring/404 drop pyodbc support (#649)

* Remove pyodbc and odbc comments from documents

* Remove pyodbc from code part 1

* Make comment general after removing pyodbc

* Remove skip related to pyodbc; as tests should be ok to run now

* Remove override due to pyodbc

* Remove EXAODBC from connection string for websocket

* Add to documentation per:
Implicit autocommit was removed in SQLAlchemy 2.0. Use the .begin() method of Engine or Connection in order to use an explicit transaction for DML and DDL statements. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)

* Modify regression test to websocket, the remaining dialect variant

* Drop support for pyodbc with changelog entry

* With 2.0, we do not need the plugin

* Add typing hint for colspec to resolve base.py mypy error

* Remove older comment as not restricted anymore to pytest <6

* Remove odbc installation instructions for Ubuntu and ODBC usage

* Switch to websocket, as last remaining dialect

* Remove ODBC skip as no longer relevant; restore default test as override not needed; mark added test

* With pyodbc removal, linting dropped to 4.9. Should create ticket to bring back up

* Update slow tests to separately run sqla, exasol, & regression tests to make debugging easier in future

* Fix test to reduced state with only websocket still

* Without ODBC dialects, there is no longer a use_sql_fallback argument. Thus, test_compare_get_schema_names_for_sql_and_odbc is obsolete, as covered by test_get_schema_names

* Without ODBC dialects, there is no longer a use_sql_fallback argument. Thus, test_compare_get_table_names_for_sql_and_odbc is obsolete, as covered by test_get_table_names

* Without ODBC dialects, there is no longer a use_sql_fallback argument.
Thus, test_has_table_table_exists and test_has_table_table_exists_not can be simplified
and test_compare_has_table_for_sql_and_odbc can be removed.

* Without ODBC dialects, there is no longer a use_sql_fallback argument. Thus, these tests can be simplified

* Without ODBC dialects, there is no longer a use_sql_fallback argument. Thus, these tests can be removed for test_get_view_names and test_get_view_definition already cover

* Without ODBC dialects, there is no longer a use_sql_fallback argument. Thus, these tests can be simplified.

* Without ODBC dialects, there is no longer a use_sql_fallback argument.
Thus, these tests can be dropped as covered by test_get_columns,
test_get_foreign_keys, test_get_pk_constraint

* Remove comment with ODBC mention

* Reduce tests as sql_fallback does not exist now that ODBC-based dialects are gone

* Switch from skip in requirements.py to xfail in the tests

* Start collecting recurring issues in XfailRationale class

- Switch to Xfail with strict=True so that changes in state are obvious, i.e. failed test suite
- Stop overriding DifficultParametersTest as found all tests pass now

* Collect more recurring issues in XfailRationale class

- Switch from EXPLICIT_INDEX to MANUAL_INDEX
- ExpandingBoundInTest.test_null_in_empty_set_is_false is not a valid test anymore; this highlights the importance of using super() and XFAIL

* Finish moving skips to xfails

* Re-add test with modifications as only 1 still fails for <= 7.1.30

* Turn off autocommit in the connection for rollback to work

* Add change to README.rst per review comment

* Add changes to test_suite.py per review comments

* Add change to unreleased.md per review comment

* Add change to developer_guide.rst per review comment

* Add change to user_guide.rst per review comment

* Move user_guide into its own directory as will expand in future and rename

Author: ArBridgeman

.github/workflows/slow-checks.yml

@@ -11,7 +11,7 @@ jobs:
       contents: read
 
   tests:
-    name: Integration-Tests (${{matrix.connector}}, Python-${{ matrix.python-version }}, Exasol-${{ matrix.exasol-version}})
+    name: Integration-Tests (Python-${{ matrix.python-version }}, test:${{ matrix.integration-group }}, Exasol-${{ matrix.exasol-version}}, ${{matrix.connector}})
     needs: [ build-matrix ]
     runs-on: ubuntu-24.04
     permissions:
@@ -31,17 +31,20 @@ jobs:
         with:
           python-version: ${{ matrix.python-version }}
 
-      - name: Install via apt
-        run: sudo apt-get install unixodbc unixodbc-dev libboost-date-time-dev libboost-locale-dev libboost-system-dev
+      - name: Start test db
+        run: poetry run -- nox -s db:start -- --db-version ${{ matrix.exasol-version }}
 
       - name: Run Tests and Collect Coverage
         run: |
           export SQLALCHEMY_WARN_20=1
-          poetry run -- nox -s test:integration -- --coverage --db-version ${{ matrix.exasol-version }}
+          poetry run -- nox -s test:${{ matrix.integration-group }}
 
       - name: Upload Artifacts
         uses: actions/upload-artifact@v5
         with:
-          name: coverage-python${{ matrix.python-version }}-${{matrix.connector}}-exasol${{ matrix.exasol-version }}slow
+          name: coverage-python${{ matrix.python-version }}-${{ matrix.integration-group }}-exasol${{ matrix.exasol-version }}-${{matrix.connector}}-slow
           path: .coverage
           include-hidden-files: true
+
+      - name: Stop test db
+        run: poetry run -- nox -s db:stop

.gitignore

@@ -1,7 +1,6 @@
 .lint*
 .test-results.db
 .html-documentation
-odbcconfig/odbcinst.ini
 
 _build/
 

README.rst

@@ -42,32 +42,15 @@ SQLAlchemy Dialect for EXASOL DB
     :target: https://pypi.org/project/sqlalchemy_exasol
     :alt: PyPI - Downloads
 
-
 Getting Started with SQLAlchemy-Exasol
 --------------------------------------
-SQLAlchemy-Exasol supports multiple dialects, primarily differentiated by whether they are ODBC or Websocket-based.
-
-Choosing a Dialect
-++++++++++++++++++
-
-We recommend using the Websocket-based dialect due to its simplicity.
-ODBC-based dialects demand a thorough understanding of (Unix)ODBC, and the setup is considerably more complex.
-
-.. warning::
-
-    The maintenance of pyodbc support is currently paused, and it is planned to be phased out in future versions.
-
+SQLAlchemy-Exasol supports a Websocket-based dialect.
 
 System Requirements
 -------------------
-- Exasol >= 7.1 (e.g. `docker-db <test_docker_image_>`_ or a `cloud instance <test_drive_>`_)
+- Exasol >= 7.1 (e.g. `docker-db <test_docker_image_>`_ or a `Exasol SaaS instance <test_drive_>`_)
 - Python >= 3.10
 
-.. note::
-
-   For ODBC-Based Dialects, additional libraries required for ODBC are necessary
-   (for further details, checkout the `developer guide`_).
-
 Features
 --------
 
@@ -80,5 +63,4 @@ Check out sqlalchemy-exasols's [User Guide(https://exasol.github.io/sqlalchemy-e
 
 .. _developer guide: https://github.com/exasol/sqlalchemy-exasol/blob/master/doc/developer_guide/developer_guide.rst
 .. _test_docker_image: https://github.com/exasol/docker-db
-.. _odbc_driver: https://docs.exasol.com/db/latest/connect_exasol/drivers/odbc/odbc_linux.htm
 .. _test_drive: https://cloud.exasol.com/signup

doc/changes/unreleased.md

@@ -1,6 +1,8 @@
 # Unreleased
 
-In this release, the ODBC-dialect Turbodbc was dropped.
+In this release, the ODBC-based dialects pyodbc and Turbodbc were dropped. Please
+switch over to using the websocket dialect. Connection
+strings should be altered to start with `exa+websocket://`.
 
 ## Refactoring
 
@@ -17,3 +19,4 @@ In this release, the ODBC-dialect Turbodbc was dropped.
   - `ReturningGuardsTest` are used to indicate that the Exasol dialect, which does not natively support the [RETURNING clause](https://docs.sqlalchemy.org/en/20/glossary.html#term-RETURNING), is set up per the API specifications
   - `ComponentReflectionTest.test_not_existing_table` is used to indicate that specific `EXADialect` methods (i.e. `get_columns`) check to see if the requested table/view exists and if not, they will now toss a `NoSuchTableError` exception
 - #403: Dropped support for Turbodbc
+- #404: Dropped support for pyodbc

doc/developer_guide/developer_guide.rst

@@ -18,25 +18,9 @@ Tools
     * Prerequisites_
 
 
-Libraries
-+++++++++
-* unixodbc
-* unixodbc-dev
-* libboost-date-time-dev
-* libboost-locale-dev
-* libboost-system-dev
-
-
-Example: Install of required system libraries on Ubuntu
-
-.. code-block::
-
-    sudo apt-get install unixodbc unixodbc-dev libboost-date-time-dev libboost-locale-dev libboost-system-dev
-
-
 Locale
 +++++++
-Make sure the local is setup appropriately.
+Make sure the locale is set up appropriately.
 
 Example: Setting up an english locale
 
@@ -125,23 +109,10 @@ Tests
 
     .. code-block::
 
-        # make sure you are using the virtual environment poetry has setup for this project
+        # make sure you are using the virtual environment poetry has set up for this project
         poetry shell
 
 
-#. Run all tests with `pyodbc` connector
-
-    .. code-block::
-
-        nox
-
-    or
-
-    .. code-block::
-
-        nox -s "verify(connector='pyodbc')"
-
-
 .. attention::
 
     If something still is not working or unclear, you may want to look into the CI/CD action_ files.

doc/index.rst

@@ -22,14 +22,13 @@ Documentation of SQLAlchemy-Exasol
         :link: faq
         :link-type: ref
 
-        Frequently asked questsions.
+        Frequently asked questions.
 
 .. toctree::
    :maxdepth: 1
    :hidden:
 
-   user_guide
+   user_guide/index
    developer_guide/index
    faq
    changes/changelog
-

doc/user_guide.rst renamed to doc/user_guide/index.rst

@@ -46,28 +46,20 @@ Getting Started
             # sqlalchemy 2.x, be passed through `sqlalchemy.sql.text`.
             result = con.execute(sql.text("select 42 from dual")).fetchall()
 
-        # engine.begin() is for DML & DDL, as we don't want to rely on autocommit
-        # but you can with `engine.connect()` commit as you go with `con.commit()`
+        # For easier usage of transactions, you can use engine.begin() for DML & DDL,
+        # if you don't want to rely on autocommit. However, you can with
+        # `engine.connect()` commit as you go with `con.commit()` or use, for more
+        # complex scenarios:
+        #   with engine.connect() as con:
+        #      with conn.begin():
+        #        ...
         with engine.begin() as con:
         ...
 
 
-.. warning::
-
-   To use an ODBC-based dialect, you must specify it as an extra during installation.
-   Maintenance for these ODBC-based dialects is paused, and it is planned to remove them
-   in future versions.
-
-   .. code-block:: shell
-
-      pip install "sqlalchemy-exasol[pydobc]"
-
 Further Examples
 ~~~~~~~~~~~~~~~~
 
-Websocket-based Dialect
-------------------------
-
 .. code-block:: python
 
     from sqlalchemy import create_engine, sql
@@ -116,30 +108,6 @@ Websocket-based Dialect
         ...
 
 
-.. note::
-   Validation with fingerprint is only supported in the Websocket-based dialect, and not
-   the ODBC-based dialect (Pyodbc).
-
-
-Pyodbc-based Dialect
----------------------
-
-.. code-block:: python
-
-    from sqlalchemy import create_engine, sql
-
-    user = "sys"
-    password = "exasol"
-    host = "127.0.0.1"
-    port = "8563
-    schema = "my_schema
-
-    url = f"exa+pyodbc://{user}:{password}@{host}:{port}/{schema}?CONNECTIONLCALL=en_US.UTF-8&driver=EXAODBC"
-    engine = create_engine(url)
-    query = "select 42 from dual"
-    with engine.connect() as con:
-        result = con.execute(sql.text(query)).fetchall()
-
 General Notes
 ~~~~~~~~~~~~~
 

driver/libexaodbc-uo2214lv2.so

@@ -51,7 +51,7 @@ class Config(BaseConfig):
     environment_name: str = "test"
     db_port: int = 8563
     bucketfs_port: int = 2580
-    connectors: list[str] = ["pyodbc", "websocket"]
+    connectors: list[str] = ["websocket"]
     plugins: list = [StartDB, StopDB]