Repository navigation
build: declare directly-imported deps, drop the dead root eval extra - #208
Conversation
Follow-up to StarTrail-org#197, from auditing every module-level third-party import in the shipped packages against what the extras declare. Three of them were not declared where their importer lives. **aiohttp — a real break, not tidying.** zim_server_async.py does `from aiohttp import web` at module scope, but aiohttp was declared only in the root `eval` extra. So the documented invocation in its own docstring (`python -m pixelrag_serve.zim_server_async --zim ... --port`) failed after `pip install 'pixelrag[kiwix]'`: File ".../pixelrag_serve/zim_server_async.py", line 16, in <module> from aiohttp import web ModuleNotFoundError: No module named 'aiohttp' It belongs with libzim in `kiwix`: that module imports both, and the sync zim_server.py needs only libzim, so `[kiwix]` now covers both servers. **botocore.** index/distributed.py imports botocore.exceptions directly while only boto3 was declared. boto3 always pulls botocore in, so nothing was broken — but a direct import belongs in the declaration. **pandas (eval/).** eval/lib/benchmarks.py and simpleqa_data.py import pandas directly; it reached eval's venv only transitively through datasets. Declared in eval/pyproject.toml, where those files live. **Dropped the root `eval` extra.** Nothing in the repo references `pixelrag[eval]` or `--extra eval`, and it could not run the harness anyway: eval imports requests and tqdm at module scope and the extra listed neither. eval/ is its own uv project (pixelrag-repro) and eval/README documents `uv sync --frozen` from inside it, so the extra was a stale duplicate — and the only place aiohttp was declared, which is how the gap above hid. It is a user-facing removal of an extra that never worked. Verified per extra in fresh `uv run --isolated` envs: `[kiwix]` now imports zim_server_async cleanly (it raised ModuleNotFoundError on main), `[distributed]` imports index/distributed.py, and the suite is 298 passed / 2 skipped. The other extras are backed by real imports and untouched — playwright (strategies/connection.py), pdf (pdf2image in backends/pdf.py), qdrant, embed, serve, gpu. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
@dex0shubham is attempting to deploy a commit to the andylizf's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
The The lockfile changes need another look before this lands. The diff drops platform markers from a lot of entries, for example: and the same for I checked whether this was just lock drift from a newer uv, and it isn't: running Could you re-run |
The pyproject edit here had grown a second subject: it moved aiohttp into the kiwix extra, added botocore to distributed, and deleted the root eval extra. Those are #208's change, not this one's, and carrying them here broke CI in a way that was easy to misread as flaky. Dependency edits invalidate uv.lock, so `uv sync --extra dev` re-resolved from scratch on every run instead of installing from the lock. That re-resolution fails on Linux: No solution found ... (python_full_version >= '3.14' ... sys_platform == 'linux') faiss-gpu-cu12{sys_platform == 'linux'}>=1.13.2 has no Linux-compatible wheels a latent gap in the gpu extra that main never hits, because main's lock matches main's pyproject and is installed as-is. Locally it looked fine only because macOS resolves that split differently. pyproject is now one line -- the ruff exclude -- which is what the title says. `uv sync` leaves uv.lock untouched, 308 passed / 2 skipped, ruff check and format clean over the wider 194-file scope. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Correcting my previous comment — I was wrong about the lockfile, and the PR is fine. I said the dropped platform markers looked like they came from locking with a different uv version, and asked you to re-lock. I tested that the wrong way round: I checked whether Doing that now, on top of Byte-identical. So the marker removal is exactly what uv produces for this dependency set, not drift — and it follows from the change itself: dropping the root Verified the rest on the merge with current
The One related note, since it bit elsewhere: #209 had quietly grown a copy of these same dependency edits without a matching lock update, which made |
* ci: lint and test demos/ and train/ instead of excluding them `[tool.ruff] exclude` listed "demos" and "train", and CI runs `pytest tests/` only, so everything under those two directories was covered by neither lint nor tests. A green CI badge on a PR that only touched demos/ therefore attested to nothing: #175's 4/4 covered its README edit and none of its code. Drop both from the exclude list. The hidden debt was small: 2 errors in demos/ and 53 in train/, 51 of them auto-fixable (41 unsorted-imports, 5 explicit-f-string-type-conversion, 3 for-loop-writes, 2 regex-flag-alias). The four that needed a decision: - train/.cursor/rules/code-critic.md is a dead symlink pointing at .claude/agents/code-critic.md, which is not in the repo. Tracked since the initial release and invisible while train/ was excluded; it makes `ruff format` fail to read the path outright. Removed. - train_contrastors.py appended in a loop; now extends. - clean_queries_simpleqa_style.py validates a model's JSON response, so ValueError is the right contract for callers already catching it. TRY004 suppressed on the raise with the reason. - demos/quickstart.ipynb used sorted(...)[0] to take a minimum. Add a `demo-tests` job. Each demo is its own uv project depending on the repo root through a path source, which is why `pytest tests/` never reached them. The job discovers every demos/*/ with tests and runs it in its own environment; demos/parallel_search is the only one today, and it is offline (monkeypatched httpx.AsyncClient, no keys) — 3 passed in 19s with a 1.2s install, so this costs almost nothing. `ruff check .` and `ruff format --check .` now pass across 194 files with nothing excluded but tmp/, docs/ and skill/. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * style: format the python block in train/README.md Removing the ruff exclusions brings train/ into scope for `ruff format`, which reformats the fenced python example in train/README.md. Applying it here so `ruff format --check .` passes with the wider scope. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * build: drop the dependency changes; this PR is the lint scope only The pyproject edit here had grown a second subject: it moved aiohttp into the kiwix extra, added botocore to distributed, and deleted the root eval extra. Those are #208's change, not this one's, and carrying them here broke CI in a way that was easy to misread as flaky. Dependency edits invalidate uv.lock, so `uv sync --extra dev` re-resolved from scratch on every run instead of installing from the lock. That re-resolution fails on Linux: No solution found ... (python_full_version >= '3.14' ... sys_platform == 'linux') faiss-gpu-cu12{sys_platform == 'linux'}>=1.13.2 has no Linux-compatible wheels a latent gap in the gpu extra that main never hits, because main's lock matches main's pyproject and is installed as-is. Locally it looked fine only because macOS resolves that split differently. pyproject is now one line -- the ruff exclude -- which is what the title says. `uv sync` leaves uv.lock untouched, 308 passed / 2 skipped, ruff check and format clean over the wider 194-file scope. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Follow-up to #197. That PR removed core dependencies nothing imported; this one
comes from the opposite half of the same audit — every module-level third-party
import in the shipped packages (
render/,embed/,index/,serve/,src/)checked against what the extras actually declare.
Three imports were not declared where their importer lives. One of them breaks a
documented command.
1.
aiohttp— a real breakserve/src/pixelrag_serve/zim_server_async.py:16doesfrom aiohttp import webat module scope.
aiohttpwas declared only in the rootevalextra, so theinvocation in that module's own docstring fails after
pip install 'pixelrag[kiwix]':(reproduced on
mainin a clean--extra kiwixenv; clean after this change.)It belongs beside
libziminkiwix: that module imports both, and the synczim_server.pyneeds onlylibzim, so[kiwix]now covers both ZIM servers.Whether a given install happened to work was down to luck —
aiohttpcan arrivetransitively through
fsspec's optional http extra — which is exactly why it wentunnoticed.
2.
botocoreindex/src/pixelrag_index/distributed.py:24importsbotocore.exceptionsdirectly while only
boto3was declared.boto3always pullsbotocorein, sonothing was broken here — but a direct import belongs in the declaration
rather than relying on a transitive one.
3.
pandas(ineval/)eval/lib/benchmarks.py:19andeval/lib/simpleqa_data.py:19importpandasdirectly; it reached eval's venv only transitively via
datasets(
eval/uv.lock:2171). Declared ineval/pyproject.toml, where those files live.4. Dropping the root
evalextrapixelrag[eval]or--extra eval— no docs, CI,scripts or code;
requestsandtqdmatmodule scope and the extra listed neither;
eval/is its own uv project (pixelrag-repro) with a complete dependency list,and
eval/README.mddocumentsuv sync --frozenfrom inside it;aiohttpwas declared, which is how the gap in §1 hid.So it is a stale duplicate — but removing an extra is user-facing, even one
that never worked. Flagging that explicitly rather than burying it: anyone with
pixelrag[eval]in a script would need to switch to the eval project, which isthe documented path and the only one that actually works. Say the word if you'd
rather keep it as an alias.
Verification
Per-extra, in fresh
uv run --isolatedenvironments:[kiwix]→ importzim_server_asyncModuleNotFoundErroron main, clean now[distributed]→ importindex/distributed.py--extra dev)Extras left alone because their imports check out:
playwright(
strategies/connection.py),pdf(pdf2image inbackends/pdf.py),qdrant,embed,serve,gpu. This PR changes no Python, so the suite count ismain's.