Skip to content

Commit bf31620

Browse files
subtleGradientsubtleGradient
authored
Add FileParserPlugin example for fetch API (#50)
Add FileParserPlugin example for fetch API - Add PDF fixtures (small, medium, large, xlarge) with verification codes - Create comprehensive file-parser example testing all PDF sizes - Uses mistral-ocr engine for PDF processing - Validates extraction of verification codes from PDFs Fix biome linting issues in file-parser example - Use block statements for if condition - Replace template literal with string literal where interpolation not needed Add PDF URL example and fix xlarge verification code - Add file-parser-pdf-url.ts demonstrating PDF URLs - Fix EXPECTED_CODES: xlarge code is W6H5V not F6H2V - All tests now pass (4/4) Add shared fixtures module with JSON metadata - Create shared/fixtures.ts with absolute path utilities - Read verification codes from JSON metadata (not hard-coded) - Add all PDF JSON metadata files from generate-pdfs.sh - Update fetch examples to use shared fixtures module - Add @ts-expect-error for missing 'provider' field in types - All examples now work from any directory using absolute paths - Tests: 4/4 passing for all-sizes example Update generate-pdfs.sh to version with JSON metadata generation - Replace old script with version from ai-sdk-provider - New version generates JSON metadata files with verification codes - Supports TODO about dynamic randomization Regenerate PDF fixtures with correct generate-pdfs.sh script - Remove old codes.txt and README.md files - Regenerate all PDFs using updated generate-pdfs.sh from ai-sdk-provider - All PDFs validated successfully - All JSON metadata files validated successfully Improve large.pdf OCR readability - Increase verification code font size from 46pt to 96pt - Fixes OCR misreading M9N3T as M5N9T or MNGT - Update generate-pdfs.sh script with larger font - Tests: 4/4 passing across all SDKs
1 parent af2521f commit bf31620

File tree

14 files changed

+599
-1
lines changed

14 files changed

+599
-1
lines changed

‎fixtures/pdfs/generate-pdfs.sh‎

Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
#!/bin/bash
2+
# Script to generate test PDFs with unique verification codes
3+
# Each PDF contains embedded text that can be used to verify end-to-end AI processing
4+
#
5+
# TODO: Dynamically randomize verification codes at generation time so fixtures can't be
6+
# guessed in advance. This would involve:
7+
# 1. Generating a random code for each PDF at runtime
8+
# 2. Writing both PDF and JSON metadata with the randomized code
9+
# 3. Reading the code from the JSON in tests instead of hard-coding
10+
# This ensures the "needle" in the PDF cannot be known before execution.
11+
12+
set -e
13+
14+
# Verification codes
15+
SMALL_CODE="SMALL-7X9Q2"
16+
MEDIUM_CODE="MEDIUM-K4P8R"
17+
LARGE_CODE="LARGE-M9N3T"
18+
XLARGE_CODE="XLARGE-W6H5V"
19+
20+
echo "Generating test PDFs with verification codes..."
21+
echo
22+
23+
# Small PDF (33KB) - Text only with minimal content
24+
echo "Creating small.pdf with code $SMALL_CODE..."
25+
convert -size 800x600 xc:white \
26+
-pointsize 36 -gravity center \
27+
-annotate +0-150 'SMALL PDF TEST' \
28+
-annotate +0-80 'Verification Code:' \
29+
-fill red -pointsize 42 \
30+
-annotate +0-20 "$SMALL_CODE" \
31+
-fill black -pointsize 18 \
32+
-annotate +0+60 'If you can read this code, PDF processing works.' \
33+
small_text.jpg
34+
35+
convert small_text.jpg small.pdf
36+
rm small_text.jpg
37+
38+
# Medium PDF (813KB) - Text + some image padding
39+
echo "Creating medium.pdf with code $MEDIUM_CODE..."
40+
magick -size 1200x900 xc:white \
41+
-pointsize 48 -gravity center \
42+
-annotate +0-250 'MEDIUM PDF TEST' \
43+
-pointsize 32 \
44+
-annotate +0-150 'Verification Code:' \
45+
-fill blue -pointsize 38 \
46+
-annotate +0-80 "$MEDIUM_CODE" \
47+
-fill black -pointsize 20 \
48+
-annotate +0+20 'This is a medium test document.' \
49+
-annotate +0+60 'Code confirms AI processed the PDF.' \
50+
medium_base.pdf
51+
52+
# Add filler images to increase size
53+
magick -size 1000x1000 plasma:fractal med_fill1.jpg
54+
magick -size 1000x1000 plasma:fractal med_fill2.jpg
55+
magick medium_base.pdf med_fill1.jpg med_fill2.jpg medium.pdf
56+
rm medium_base.pdf med_fill*.jpg
57+
58+
# Large PDF (3.4MB) - Text + more image padding
59+
echo "Creating large.pdf with code $LARGE_CODE..."
60+
magick -size 1600x1200 xc:white \
61+
-pointsize 60 -gravity center \
62+
-annotate +0-350 'LARGE PDF TEST' \
63+
-pointsize 40 \
64+
-annotate +0-250 'Verification Code:' \
65+
-fill green -pointsize 96 \
66+
-annotate +0-170 "$LARGE_CODE" \
67+
-fill black -pointsize 24 \
68+
-annotate +0-80 'Large test document for PDF uploads.' \
69+
-annotate +0-40 'Verification code confirms processing.' \
70+
large_base.pdf
71+
72+
# Add filler images to increase size
73+
magick -size 1500x1500 plasma:fractal lg_fill1.jpg
74+
magick -size 1500x1500 plasma:fractal lg_fill2.jpg
75+
magick -size 1500x1500 plasma:fractal lg_fill3.jpg
76+
magick -size 1500x1500 plasma:fractal lg_fill4.jpg
77+
magick large_base.pdf lg_fill1.jpg lg_fill2.jpg lg_fill3.jpg lg_fill4.jpg large.pdf
78+
rm large_base.pdf lg_fill*.jpg
79+
80+
# XLarge PDF (11MB) - Text + lots of image padding
81+
echo "Creating xlarge.pdf with code $XLARGE_CODE..."
82+
magick -size 2000x1500 xc:white \
83+
-pointsize 72 -gravity center \
84+
-annotate +0-450 'XLARGE PDF TEST' \
85+
-pointsize 48 \
86+
-annotate +0-330 'Verification Code:' \
87+
-fill purple -pointsize 56 \
88+
-annotate +0-240 "$XLARGE_CODE" \
89+
-fill black -pointsize 28 \
90+
-annotate +0-140 'Extra-large test document.' \
91+
-annotate +0-100 'Code confirms AI read the PDF.' \
92+
xlarge_base.pdf
93+
94+
# Add many filler images to increase size
95+
for i in {1..8}; do
96+
magick -size 2000x2000 plasma:fractal xl_fill$i.jpg
97+
done
98+
magick xlarge_base.pdf xl_fill*.jpg xlarge.pdf
99+
rm xlarge_base.pdf xl_fill*.jpg
100+
101+
echo
102+
echo "✓ PDF generation complete!"
103+
echo
104+
echo "Generating JSON metadata files..."
105+
echo
106+
107+
# Create JSON metadata for each PDF with verification code
108+
# Format: { "verificationCode": "CODE", "description": "...", "size": "...", "type": "test_fixture" }
109+
110+
cat > small.json << EOF
111+
{
112+
"verificationCode": "$SMALL_CODE",
113+
"description": "Small test PDF (33KB) with minimal content",
114+
"size": "small",
115+
"type": "test_fixture",
116+
"purpose": "Test basic PDF processing with FileParserPlugin"
117+
}
118+
EOF
119+
echo " ✓ small.json"
120+
121+
cat > medium.json << EOF
122+
{
123+
"verificationCode": "$MEDIUM_CODE",
124+
"description": "Medium test PDF (813KB) with text and image padding",
125+
"size": "medium",
126+
"type": "test_fixture",
127+
"purpose": "Test PDF processing with moderate file size"
128+
}
129+
EOF
130+
echo " ✓ medium.json"
131+
132+
cat > large.json << EOF
133+
{
134+
"verificationCode": "$LARGE_CODE",
135+
"description": "Large test PDF (3.4MB) for FileParserPlugin regression testing",
136+
"size": "large",
137+
"type": "test_fixture",
138+
"purpose": "Test large PDF handling and plugin activation",
139+
"regression": "Validates fix for FileParserPlugin large PDF issue"
140+
}
141+
EOF
142+
echo " ✓ large.json"
143+
144+
cat > xlarge.json << EOF
145+
{
146+
"verificationCode": "$XLARGE_CODE",
147+
"description": "Extra-large test PDF (11MB) with extensive content",
148+
"size": "xlarge",
149+
"type": "test_fixture",
150+
"purpose": "Test maximum file size handling with FileParserPlugin"
151+
}
152+
EOF
153+
echo " ✓ xlarge.json"
154+
155+
echo
156+
echo "Generated files:"
157+
ls -lh *.pdf *.json
158+
echo
159+
echo "Verification codes:"
160+
echo " small.pdf -> $SMALL_CODE (small.json)"
161+
echo " medium.pdf -> $MEDIUM_CODE (medium.json)"
162+
echo " large.pdf -> $LARGE_CODE (large.json)"
163+
echo " xlarge.pdf -> $XLARGE_CODE (xlarge.json)"
164+
echo
165+
echo "Validating PDFs..."
166+
for pdf in small.pdf medium.pdf large.pdf xlarge.pdf; do
167+
if qpdf --check "$pdf" &>/dev/null; then
168+
echo " ✓ $pdf is valid"
169+
else
170+
echo " ✗ $pdf has issues"
171+
fi
172+
done
173+
echo
174+
echo "Validating JSON metadata..."
175+
for json in small.json medium.json large.json xlarge.json; do
176+
if python3 -m json.tool "$json" > /dev/null 2>&1; then
177+
echo " ✓ $json is valid"
178+
else
179+
echo " ✗ $json has syntax errors"
180+
fi
181+
done

‎fixtures/pdfs/large.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"verificationCode": "LARGE-M9N3T",
3+
"description": "Large test PDF (3.4MB) for FileParserPlugin regression testing",
4+
"size": "large",
5+
"type": "test_fixture",
6+
"purpose": "Test large PDF handling and plugin activation",
7+
"regression": "Validates fix for FileParserPlugin large PDF issue"
8+
}

‎fixtures/pdfs/large.pdf‎

3.31 MB
Binary file not shown.

‎fixtures/pdfs/medium.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"verificationCode": "MEDIUM-K4P8R",
3+
"description": "Medium test PDF (813KB) with text and image padding",
4+
"size": "medium",
5+
"type": "test_fixture",
6+
"purpose": "Test PDF processing with moderate file size"
7+
}

‎fixtures/pdfs/medium.pdf‎

809 KB
Binary file not shown.

‎fixtures/pdfs/small.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"verificationCode": "SMALL-7X9Q2",
3+
"description": "Small test PDF (33KB) with minimal content",
4+
"size": "small",
5+
"type": "test_fixture",
6+
"purpose": "Test basic PDF processing with FileParserPlugin"
7+
}

‎fixtures/pdfs/small.pdf‎

32.3 KB
Binary file not shown.

‎fixtures/pdfs/xlarge.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"verificationCode": "XLARGE-W6H5V",
3+
"description": "Extra-large test PDF (11MB) with extensive content",
4+
"size": "xlarge",
5+
"type": "test_fixture",
6+
"purpose": "Test maximum file size handling with FileParserPlugin"
7+
}

‎fixtures/pdfs/xlarge.pdf‎

10.3 MB
Binary file not shown.

‎typescript/fetch/src/plugin-file-parser/README.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# OpenRouter FileParserPlugin Examples (Fetch)
2+
3+
Examples demonstrating OpenRouter's FileParserPlugin with raw fetch API.
4+
5+
## Overview
6+
7+
The FileParserPlugin enables PDF processing for models that don't natively support file inputs. The plugin:
8+
9+
- Accepts PDFs via base64-encoded data URLs
10+
- Extracts text using configurable engines
11+
- Returns parsed content to the model for processing
12+
13+
## Examples
14+
15+
See the TypeScript files in this directory for specific examples.

0 commit comments

Comments
 (0)