chore: run pgai tests against extension from source #206
Conversation
Change pgai testcontainer to build the extension and copy the extension code to a timescaledb-ha container. This will make CI run the pgai python library against the latest changes made to the extension, and surface breaking changes sooner.
There was a problem hiding this comment.
This doesn't work for versions 0.1.0, 0.2.0, and 0.3.0 of the extension. Maybe that doesn't matter though for the pgai library.
| COPY requirements.txt . | ||
| COPY ai/__init__.py ai/ | ||
| RUN version=$(grep -oP '(?<=__version__ = ")[^"]*' ai/__init__.py) && \ | ||
| mkdir -p /usr/local/lib/pgai/$version && \ | ||
| pip3 install -v --no-deps --compile -t /usr/local/lib/pgai/$version -r requirements.txt |
There was a problem hiding this comment.
why are we doing this? make install does this already
There was a problem hiding this comment.
To cache de dependencies in a Docker layer. If we change the extension code we don't need to re-install the dependencies.
| mkdir -p /usr/local/lib/pgai/$version && \ | ||
| pip3 install -v --no-deps --compile -t /usr/local/lib/pgai/$version -r requirements.txt | ||
| COPY . . | ||
| RUN make install-py build-sql install-sql |
There was a problem hiding this comment.
why not just make build-install??
There was a problem hiding this comment.
Because build_install also calls install_prior_py, and I don't need to install those deps.
| ARG PG_MAJOR=16 | ||
| COPY --from=build-latest /usr/share/postgresql/${PG_MAJOR}/extension/ai--*.sql /usr/share/postgresql/${PG_MAJOR}/extension/ | ||
| COPY --from=build-latest /usr/share/postgresql/${PG_MAJOR}/extension/ai.control /usr/share/postgresql/${PG_MAJOR}/extension/ai.control | ||
| COPY --from=build-latest /usr/local/lib/pgai/ /usr/local/lib/pgai/ |
There was a problem hiding this comment.
for versions 0.1.0, 0.2.0, and 0.3.0, the dependencies are installed in the system path
There was a problem hiding this comment.
The pgai tests are not parametrize on the extension version, so it'll only use the default version. We'd need those version if we parametrize the extension versions.
I think if we do that, instead of doing it in the code, we can do it in the ci matrix.
There was a problem hiding this comment.
the vectorizer code will need to be backwards compatible with prior versions of the extension, and I think we'll want tests to that end eventually, but happy to postpone
Change pgai testcontainer to build the extension and copy the extension code to a timescaledb-ha container.
This will make CI run the pgai python library against the latest changes made to the extension, and surface breaking changes sooner.