Repository navigation
feat(demos): Mixpeek managed backend for visual web search - #53
esteininger wants to merge 4 commits into
Conversation
Adds demos/mixpeek/ — a managed alternative to the local PixelRAG pipeline. Uses Mixpeek's web_scraper extractor to handle rendering, embedding, indexing, and serving without local GPU or FAISS. - web_search.py: end-to-end CLI script (namespace → bucket → collection → batch → retriever → search) with configurable crawl params - README.md: quickstart, pipeline mapping table, and when-to-use guide Tested against the live Mixpeek API — returns results with scores. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@esteininger is attempting to deploy a commit to the andylizf's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
Thanks for putting this together. The code is clean and self-contained (new That said, we want to think about whether adding a managed/commercial backend demo fits the project's positioning — PixelRAG's core thesis is self-hosted native-multimodal retrieval, and this demo replaces that stack with a third-party service. We'll discuss internally and get back to you. In the meantime: CI would need |
|
@esteininger — picking this up after far too long. Four months of silence here is on us, not you, and I want to be straight about where it actually stands. The hold was never the code. andylizf's review said the merge risk is near zero (new That question has since been answered in practice, which is the part that makes keeping this open unfair. #175 landed last week — Either this merges on the same basis #175 did, or there's a distinction between them that we should state plainly so you and the other contributors know the rule. I don't think the distinction can be "managed service" — #175 is one too. @andylizf @yichuan-w this needs a yes/no rather than another month. My read: the honest options are to merge it, or to write down a One technical note either way, so it's not a surprise: the Vercel deploy check is still unauthorized ("a member of the Team first needs to authorize it"), so this has never had a green run. That needs a maintainer click regardless of the decision. 🤖 Generated with Claude Code |
|
Housekeeping while the positioning question is still open, so this isn't sitting on stale information when it gets answered. Merged One thing to fix whichever way the decision goes. #209 removes the plus a @esteininger — the open question is still the one andylizf raised about where managed/commercial backends sit in the project, not your code. I'm drafting a written policy for that class of PR so this stops being decided ad hoc, and you'll get a straight answer against it rather than more silence. |
StarTrail-org#209 removed `demos` from the ruff exclude list, so this file is now linted. It was written while demos/ was excluded, so this is the goalposts moving, not a defect in the PR: demos/mixpeek/web_search.py:26:8: F401 `json` imported but unused demos/mixpeek/web_search.py:322:11: F541 f-string without any placeholders Both auto-fixed, plus a `ruff format` pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Status update — this is now green and mergeable, and the only thing left on it is a decision no amount of code review settles. What I changed: merged current What is left: the positioning question @andylizf raised — whether a managed/commercial backend demo belongs in this repo — has never been answered, and it is the only thing holding this. There is precedent in both directions: #175 merged a vendor demo into I have drafted one for @ASuresh0524 and @andylizf to settle, proposing the bar be about upkeep rather than commerce: the demo has to show something about PixelRAG rather than just the vendor, stay self-contained, pass lint, and carry a named owner and a You will get a yes or a no against that, not more silence. Apologies that it has taken this long to get to a straight answer. |
Summary
Adds
demos/mixpeek/— a managed alternative to the local PixelRAG pipeline using Mixpeek as the backend. No local GPU, no FAISS, no Playwright setup needed.Pipeline mapping
pixelshot(render)web_scraperextractor (render_strategy: auto)pixelrag chunkchunk_strategy: paragraphs)pixelrag embedpixelrag build-indexpixelrag serveWhat's included
web_search.py: End-to-end CLI script. Creates a throwaway namespace (auto-expires), uploads a seed URL, crawls with the web_scraper extractor, builds a retriever, and runs a search query. Supports configurable crawl depth, max pages, crawl mode (deterministic/semantic), and render strategy.README.md: Quickstart, full pipeline comparison, and guidance on when to use managed vs local.Usage
Tested
Ran against the live Mixpeek API — returns ranked results with scores and source URLs. Namespaces auto-expire so no cleanup needed.