Parse Gateway

Parse Gateway is a small web app that shows off a cost-aware document parsing pipeline. Upload a PDF, and it will:

1. Score every page for complexity (text density, OCR needs, layout noise, embedded images, etc.) using @llamaindex/liteparse.
2. Optionally factor in layout complexity (multi-column text, ruled tables, dense figures) via the "Include Layout Complexity" switch in the UI, which can escalate a page's tier even if it needs no OCR at all.
3. Route each page to the cheapest parser tier that can still handle it:
   • LiteParse (local) — free, in-process parsing for simple, text-heavy pages.
   • LlamaParse — Cost Effective — for pages that need OCR but are otherwise simple.
   • LlamaParse — Agentic — for scanned pages, sparse text, embedded images, or moderate layout complexity (e.g. two columns).
   • LlamaParse — Agentic Plus — for garbled text, heavy vector-text, or pages with dense tables/figures/columns that need the most capable tier.
4. Stream progress back to the browser over Server-Sent Events (SSE) as complexity estimation and parsing happen.
5. Show the resulting markdown per page, with a breakdown of which tier handled which pages, and let you copy or download the combined result.

Under the hood it's a TanStack Start app (React + TanStack Router) with a single server API route (src/routes/api/parse.ts) that does the complexity estimation, fans work out across LiteParse and the LlamaCloud parsing API, and streams results back as SSE.

How it works

• Complexity estimation — LiteParse.isComplex() inspects the PDF and returns per-page stats: OCR-related signals (needsOcr, reasons, text coverage, image coverage, garbled text, etc.) and, separately, layout signals (layout.isComplex, column count, ruled-table coverage, figure coverage). Pages are bucketed into one of four tiers by classifyPageTier() (src/routes/api/parse.ts):
  • Pages with no OCR reasons and no complex layout go straight to the cheapest tier (or local LiteParse, see below) without further scoring.
  • Otherwise, each OCR reason (no-text, scanned, sparse-text, embedded-images, garbled, vector-text) maps to a baseline tier, with a few reasons escalated further based on magnitude (e.g. very sparse text with no full-page image, or many small embedded images).
  • Compounding escalation bumps agentic up to agentic_plus once 3 or more OCR reasons fire on the same page, since multiple independent problems compound.
  • Layout escalation (only applied when "Include Layout Complexity" is enabled) independently pushes the tier up based on layout signals (heavy ruled-table coverage, 2+ column layouts, dense figure coverage, or multiple layout issues firing together) even on pages that don't need OCR at all.
  • The final tier is the max across all of these signals; layout escalation can only raise the tier, never lower it.
• Tiered parsing — Each bucket of pages is parsed in parallel:
  • Pages that don't need OCR are parsed locally with LiteParse.
  • Everything else is uploaded to LlamaCloud and parsed with the appropriate LlamaParse tier (cost_effective, agentic, or agentic_plus), scoped to just those page numbers.
• Streaming — The /api/parse route responds with a text/event-stream body, emitting complexity_estimation and parsing events as each stage starts, progresses, fails, or completes. The client (src/hooks/use-parse-gateway.ts) parses these events and drives the UI.
• Results — Parsed pages from every tier are merged, sorted by page number, and rendered as markdown in the results viewer, alongside a breakdown of which tier handled which pages.

The "Include Layout Complexity" switch in the UI is sent to /api/parse as an include_layout form field ("true"/"false") and controls whether layout escalation runs at all — leave it off to score pages purely on OCR signals.

Your LlamaParse Platform API key is only ever stored in your browser's local storage and sent as a bearer token on the /api/parse request — it is never persisted server-side.

Getting Started

Install dependencies and run the dev server:

pnpm install
pnpm dev

The app will be available at http://localhost:3000. You'll need a LlamaParse Platform API key to parse pages that escalate beyond local LiteParse — paste it into the API key field in the UI.

Building For Production

pnpm build
pnpm start

pnpm start serves the production build with vite preview on $PORT (defaults to 4173).

Project Structure

src/
  routes/
    index.tsx          # Main UI: upload, progress, results
    api/parse.ts        # SSE endpoint: complexity estimation + tiered parsing
  hooks/
    use-parse-gateway.ts # Client-side SSE consumer / state machine
    use-local-storage.ts
  components/parse-gateway/
    ApiKeyInput.tsx      # LlamaParse API key input (stored in localStorage)
    FileDropzone.tsx     # PDF upload
    PipelineStatus.tsx   # Live status of estimation/parsing stages
    ComplexityBreakdown.tsx # Per-tier page counts
    ResultsViewer.tsx    # Combined markdown viewer, copy/export
  lib/parse-gateway-types.ts # Shared types, tier labels/colors

Linting & Formatting

This project uses eslint and prettier. Eslint is configured using tanstack/eslint-config.

pnpm lint
pnpm format
pnpm check

Styling

Styled with Tailwind CSS and shadcn/ui components (src/components/ui). Add new shadcn components with:

pnpm dlx shadcn@latest add <component>