feat: migrate AI from Perplexity to OpenAI (gpt-5-search-api)

Remove Perplexity as an AI provider and consolidate all AI tasks on OpenAI.
Web-grounded tasks (artistSummary, albumDetail, genreSummary, artistSentence,
albumRecommendations) now use gpt-5-search-api via the Chat Completions API,
which reliably returns url_citation annotations for source attribution.

Key changes:
- Delete packages/services/ai/src/perplexity.ts (PerplexityClient removed)
- Switch 5 web search tasks from Perplexity sonar to gpt-5-search-api
- Route gpt-5-search-api through Chat Completions with web_search_options
- Add web_search_options support to chatCompletionViaChatCompletions()
- Add citation link-to-number replacement in parseResponsesResult()
- Remove PERPLEXITY_API_KEY from Bindings, Env, wrangler.toml (web + bot)
- Remove perplexity from AIProvider type, RATE_LIMITS, DB schema
- Remove SearchContextSize type and searchContextSize from prompt files
- Simplify AIService to OpenAI-only (remove perplexityApiKey, getClientForTask)
- Increase OpenAI rate limit from 60 to 90 req/min
- Add xhigh to ReasoningEffort type for GPT-5.2

Why gpt-5-search-api over GPT-5.2 Responses API web_search:
GPT-5.2 with reasoning returns empty annotations[] (agentic search).
Even with reasoning:'none', Responses API annotations are unreliable.
gpt-5-search-api via Chat Completions always returns url_citation annotations.

Docs updated: CLAUDE.md, README.md, ai-models.md, migration doc, about page.
Tests updated: removed Perplexity tests, updated mocks for Chat Completions.

Author: rianvdm

CLAUDE.md

@@ -93,8 +93,7 @@ Required secrets in `apps/web/wrangler.toml`:
 | `SPOTIFY_REFRESH_TOKEN` | Spotify API |
 | `LASTFM_API_KEY` | Last.fm API (read-only) |
 | `LASTFM_SHARED_SECRET` | Last.fm API (for authentication) |
-| `OPENAI_API_KEY` | GPT models |
-| `PERPLEXITY_API_KEY` | Web-grounded AI |
+| `OPENAI_API_KEY` | GPT models (including web search) |
 | `INTERNAL_API_SECRET` | Internal API tokens |
 | `ADMIN_SECRET` | Admin endpoints |
 | `DISCORD_WEBHOOK_URL` | Discord notifications (user signups) |
@@ -110,8 +109,7 @@ Discord bot secrets in `apps/discord-bot/wrangler.toml`:
 | `SPOTIFY_CLIENT_SECRET` | Spotify API |
 | `SPOTIFY_REFRESH_TOKEN` | Spotify API |
 | `LASTFM_API_KEY` | Last.fm API |
-| `OPENAI_API_KEY` | AI summaries |
-| `PERPLEXITY_API_KEY` | AI summaries |
+| `OPENAI_API_KEY` | AI summaries (including web search) |
 | `APPLE_KEY_ID` | Apple MusicKit (for StreamingLinksService) |
 | `APPLE_TEAM_ID` | Apple MusicKit |
 | `APPLE_PRIVATE_KEY` | Apple MusicKit (PEM format) |
@@ -426,8 +424,7 @@ app.route('/', myInternalRoutes);  // Results in /api/internal/my-internal-data
 
 | Provider | Models | Best For |
 |----------|--------|----------|
-| Perplexity | `sonar` | Web-grounded responses with citations |
-| OpenAI | `gpt-5.1`, `gpt-5-mini`, `gpt-5-nano` | Reasoning, creative tasks, coding |
+| OpenAI | `gpt-5-search-api`, `gpt-5.2`, `gpt-5.1`, `gpt-5-mini` | All AI tasks including web search with citations |
 
 Key points:
 - Pass `internalToken` to Layout for pages using internal APIs
@@ -561,8 +558,7 @@ npx wrangler tail --format=json
 
 **AI responses failing:**
 - Check API keys in `.dev.vars` (local) or wrangler secrets (prod)
-- Perplexity rate limit: 30 req/min
-- OpenAI rate limit: 60 req/min
+- OpenAI rate limit: 90 req/min
 
 **Session/authentication issues:**
 - Session cookie not set - check `LASTFM_SHARED_SECRET` is configured

README.md

@@ -16,7 +16,7 @@ A music discovery platform that combines real-time listening data with AI-powere
 - **Album & Artist Search** - Search the Spotify catalog with instant results
 - **Album Detail Pages** - Rich album pages with release info, track listings, genres, and AI-generated summaries with citations
 - **Artist Detail Pages** - Artist profiles with biography, top albums, similar artists, and genre connections
-- **AI-Powered Summaries** - Get rich, contextual information about any artist, album, or genre powered by Perplexity AI (with source citations)
+- **AI-Powered Summaries** - Get rich, contextual information about any artist, album, or genre powered by OpenAI GPT-5.2 with web search (with source citations)
 - **Genre Exploration** - Discover music by genre with AI-generated descriptions, history, and key artists
 - **Cross-Platform Streaming Links** - Every album includes direct links to Spotify, Apple Music, and Songlink (for all other services)
 - **Album Recommendations** - AI-generated "if you like this, try these" recommendations on album pages
@@ -79,7 +79,7 @@ listentomore/
 │   ├── services/               # Backend service modules
 │   │   ├── spotify/            # Spotify Web API client
 │   │   ├── lastfm/             # Last.fm API client
-│   │   ├── ai/                 # OpenAI + Perplexity clients
+│   │   ├── ai/                # OpenAI client
 │   │   ├── songlink/           # Odesli/Songlink API client
 │   │   └── streaming-links/    # Cross-platform link service (Apple Music, URL parsing)
 │   ├── db/                     # D1 schema, migrations, queries
@@ -122,7 +122,6 @@ SPOTIFY_REFRESH_TOKEN=your_spotify_refresh_token
 LASTFM_API_KEY=your_lastfm_api_key
 LASTFM_USERNAME=your_lastfm_username
 OPENAI_API_KEY=your_openai_api_key
-PERPLEXITY_API_KEY=your_perplexity_api_key
 INTERNAL_API_SECRET=your_random_secret_for_internal_apis
 
 # Optional - for admin features
@@ -157,8 +156,7 @@ pnpm deploy
 
 - **[Spotify Web API](https://developer.spotify.com/documentation/web-api)** - Music catalog data
 - **[Last.fm API](https://www.last.fm/api)** - Listening history and scrobbles
-- **[OpenAI API](https://platform.openai.com)** - GPT-5 for chatbot and fact generation
-- **[Perplexity API](https://docs.perplexity.ai)** - Sonar model for grounded, cited summaries
+- **[OpenAI API](https://platform.openai.com)** - GPT-5.2 for AI summaries, web search with citations, chatbot, and fact generation
 - **[Apple MusicKit API](https://developer.apple.com/musickit/)** - Cross-platform streaming links via UPC matching
 - **[Songlink/Odesli](https://odesli.co)** - Cross-platform streaming links for all services
 

apps/discord-bot/src/index.ts

@@ -33,7 +33,6 @@ interface Env {
   SPOTIFY_REFRESH_TOKEN: string;
   LASTFM_API_KEY: string;
   OPENAI_API_KEY: string;
-  PERPLEXITY_API_KEY: string;
   // Apple MusicKit credentials (for streaming links)
   APPLE_KEY_ID: string;
   APPLE_TEAM_ID: string;
@@ -75,7 +74,6 @@ function createServices(env: Env): Services {
     }),
     ai: new AIService({
       openaiApiKey: env.OPENAI_API_KEY,
-      perplexityApiKey: env.PERPLEXITY_API_KEY,
       cache: env.CACHE,
     }),
   };

apps/discord-bot/wrangler.toml

@@ -37,7 +37,6 @@ vars = { ENVIRONMENT = "production" }
 # - SPOTIFY_REFRESH_TOKEN
 # - LASTFM_API_KEY
 # - OPENAI_API_KEY
-# - PERPLEXITY_API_KEY
 # Apple MusicKit (for streaming links):
 # - APPLE_KEY_ID
 # - APPLE_TEAM_ID

apps/web/src/__tests__/services/ai.test.ts

@@ -1,7 +1,7 @@
 // AIService integration tests
 
 import { describe, it, expect, vi, beforeEach } from 'vitest';
-import { PerplexityClient, OpenAIClient, AICache, generateArtistSummary } from '@listentomore/ai';
+import { OpenAIClient, AICache, generateArtistSummary } from '@listentomore/ai';
 import { createMockKV, setupFetchMock } from '../utils/mocks';
 
 describe('OpenAIClient', () => {
@@ -159,79 +159,6 @@ describe('OpenAIClient', () => {
   });
 });
 
-describe('PerplexityClient', () => {
-  let client: PerplexityClient;
-
-  beforeEach(() => {
-    client = new PerplexityClient('test-api-key');
-  });
-
-  describe('chatCompletion', () => {
-    it('sends chat completion request and returns response', async () => {
-      const response = {
-        choices: [{ message: { content: 'This is a response about Radiohead.' } }],
-        citations: ['https://en.wikipedia.org/wiki/Radiohead'],
-      };
-      setupFetchMock([{ pattern: /api\.perplexity\.ai/, response }]);
-
-      const result = await client.chatCompletion({
-        model: 'sonar',
-        messages: [{ role: 'user', content: 'Tell me about Radiohead' }],
-      });
-
-      expect(result.content).toBe('This is a response about Radiohead.');
-      expect(result.citations).toEqual(['https://en.wikipedia.org/wiki/Radiohead']);
-    });
-
-    it('preserves citation markers in response', async () => {
-      const response = {
-        choices: [{ message: { content: 'Radiohead [1] formed in 1985 [2] in Oxford.' } }],
-        citations: ['https://example.com', 'https://example2.com'],
-      };
-      setupFetchMock([{ pattern: /api\.perplexity\.ai/, response }]);
-
-      const result = await client.chatCompletion({
-        model: 'sonar',
-        messages: [{ role: 'user', content: 'test' }],
-      });
-
-      // Citation markers are preserved for client-side transformation to superscript links
-      expect(result.content).toBe('Radiohead [1] formed in 1985 [2] in Oxford.');
-      expect(result.citations).toEqual(['https://example.com', 'https://example2.com']);
-    });
-
-    it('throws error on API failure', async () => {
-      setupFetchMock([
-        {
-          pattern: /api\.perplexity\.ai/,
-          response: { error: 'Unauthorized' },
-          options: { status: 401, ok: false },
-        },
-      ]);
-
-      await expect(
-        client.chatCompletion({
-          model: 'sonar',
-          messages: [{ role: 'user', content: 'test' }],
-        })
-      ).rejects.toThrow('Perplexity API error 401');
-    });
-
-    it('handles empty citations', async () => {
-      const response = {
-        choices: [{ message: { content: 'Response without citations' } }],
-      };
-      setupFetchMock([{ pattern: /api\.perplexity\.ai/, response }]);
-
-      const result = await client.chatCompletion({
-        model: 'sonar',
-        messages: [{ role: 'user', content: 'test' }],
-      });
-
-      expect(result.citations).toEqual([]);
-    });
-  });
-});
 
 describe('AICache', () => {
   let mockKV: KVNamespace;
@@ -303,28 +230,37 @@ describe('AICache', () => {
 
 describe('generateArtistSummary', () => {
   let mockKV: KVNamespace;
-  let mockClient: PerplexityClient;
+  let mockClient: OpenAIClient;
   let cache: AICache;
 
   beforeEach(() => {
     mockKV = createMockKV();
     cache = new AICache(mockKV);
-    mockClient = new PerplexityClient('test-key');
+    mockClient = new OpenAIClient('test-key');
   });
 
   it('generates artist summary and caches result', async () => {
     const response = {
+      model: 'gpt-5-search-api',
       choices: [
         {
           message: {
-            content:
-              'Radiohead is an English rock band. Their album {{OK Computer}} is considered a masterpiece. Similar artists include [[Portishead]].',
+            content: 'Radiohead is an English rock band. Their album {{OK Computer}} is considered a masterpiece. Similar artists include [[Portishead]].',
+            annotations: [
+              {
+                type: 'url_citation',
+                url_citation: {
+                  url: 'https://en.wikipedia.org/wiki/Radiohead',
+                  title: 'Radiohead - Wikipedia',
+                },
+              },
+            ],
           },
         },
       ],
-      citations: ['https://en.wikipedia.org/wiki/Radiohead'],
+      usage: { prompt_tokens: 100, completion_tokens: 50, total_tokens: 150 },
     };
-    setupFetchMock([{ pattern: /api\.perplexity\.ai/, response }]);
+    setupFetchMock([{ pattern: /api\.openai\.com/, response }]);
 
     const result = await generateArtistSummary('Radiohead', mockClient, cache);
 

apps/web/src/index.tsx

@@ -121,7 +121,6 @@ app.use('*', async (c, next) => {
     'ai',
     new AIService({
       openaiApiKey: c.env.OPENAI_API_KEY,
-      perplexityApiKey: c.env.PERPLEXITY_API_KEY,
       cache: c.env.CACHE,
     })
   );
@@ -749,7 +748,6 @@ async function scheduled(
   if (minute < 5) {
     const ai = new AIService({
       openaiApiKey: env.OPENAI_API_KEY,
-      perplexityApiKey: env.PERPLEXITY_API_KEY,
       cache: env.CACHE,
     });
 

apps/web/src/pages/about/index.tsx

@@ -30,7 +30,7 @@ export function AboutPage({ currentUser }: AboutPageProps) {
 
       <h2>Nerdy details</h2>
       <p>
-        The site uses APIs from Last.fm, Spotify, OpenAI, and Perplexity to get album and artist
+        The site uses APIs from Last.fm, Spotify, and OpenAI to get album and artist
         data and generate some interesting facts about it all. It is built on{' '}
         <a href="https://cloudflare.com/" target="_blank" rel="noopener noreferrer">
           Cloudflare

apps/web/src/types.ts

@@ -25,7 +25,6 @@ export type Bindings = {
   LASTFM_SHARED_SECRET?: string;
   LASTFM_USERNAME: string;
   OPENAI_API_KEY: string;
-  PERPLEXITY_API_KEY: string;
   YOUTUBE_API_KEY?: string;
   APPLE_TEAM_ID?: string;
   APPLE_KEY_ID?: string;

apps/web/wrangler.toml

@@ -51,7 +51,6 @@ crons = ["*/5 * * * *"]  # Run every 5 minutes (random fact only at top of hour)
 # - SPOTIFY_STREAMING_REFRESH_TOKEN
 # - LASTFM_API_KEY
 # - OPENAI_API_KEY
-# - PERPLEXITY_API_KEY
 # - DISCOGS_API_TOKEN
 # - YOUTUBE_API_KEY (for streaming links service)
 # - INTERNAL_API_SECRET (for signing internal API tokens)

docs/PERPLEXITY_TO_OPENAI_MIGRATION.md

@@ -1,21 +1,25 @@
 # Perplexity to OpenAI Migration Plan
 
-Replace all Perplexity API calls with OpenAI GPT-5.2 (web search + citations) to consolidate on a single AI provider.
+Replace all Perplexity API calls with OpenAI to consolidate on a single AI provider.
+
+> **Status: COMPLETED.** Web search tasks use `gpt-5-search-api` via Chat Completions API (not GPT-5.2 Responses API) for reliable citation annotations.
 
 ## Background
 
-Currently, ListenToMore uses **Perplexity** (`sonar` model) for 5 web-grounded tasks that need citations, and **OpenAI** for everything else. Both providers already normalize to the same `{ content: string, citations: string[] }` interface via our `ChatClient` abstraction, making this migration straightforward.
+ListenToMore previously used **Perplexity** (`sonar` model) for 5 web-grounded tasks that need citations, and **OpenAI** for everything else. Both providers normalized to the same `{ content: string, citations: string[] }` interface via our `ChatClient` abstraction, making this migration straightforward.
 
-### Target Model: GPT-5.2
+### Target Model: gpt-5-search-api
 
-We're migrating to **GPT-5.2** (`gpt-5.2`), OpenAI's flagship model with:
-- **Web search support** via the Responses API
-- **Reasoning effort levels:** `none` (default), `low`, `medium`, `high`, `xhigh`
-- **400K context window**, 128K max output tokens
-- **Pricing:** $1.75/1M input, $14/1M output + $10/1K web search calls
-- **Knowledge cutoff:** August 31, 2025
+We migrated to **`gpt-5-search-api`** via the Chat Completions API:
+- **Always performs web search** and returns `url_citation` annotations reliably
+- **Uses Chat Completions API** with `web_search_options: {}` (not Responses API)
+- **Pricing:** Same as GPT-5 family + search call costs
 
-GPT-5.2 with `reasoning: 'none'` provides fast, low-latency responses. For factual web searches where accuracy matters, `reasoning: 'low'` is recommended.
+**Why not GPT-5.2 with Responses API `web_search` tool:**
+- GPT-5.2 with any reasoning level uses "agentic search" which returns **empty** `annotations: []`
+- Even with `reasoning: 'none'`, Responses API annotations were unreliable
+- This is a known issue in the OpenAI developer community
+- `gpt-5-search-api` via Chat Completions is the reliable path for citations
 
 ### Tasks Currently on Perplexity