migrate to v2 for cohere embed and chat

[narengogi]

cohere v2 spec https://docs.cohere.com/reference/chat

ran tests with chat and embed requests

[matter-code-review]

Description

Summary By MatterAI

🔄 What Changed

Migrated Cohere integration to v2 API for both chat and embed endpoints. Updated request/response handling to support new schema, including streaming logic, finish reason mapping, and embedding format normalization.

🔍 Impact of the Change

Ensures compatibility with Cohere’s latest API version, improves response consistency with OpenAI standards, and enhances token usage reporting. No breaking changes to external interfaces.

📁 Total Files Changed

File	ChangeLog
Embed Config src/providers/bedrock/embed.ts	Added v2 embedding response type with dual-format support (float array or nested object)
API Endpoints src/providers/cohere/api.ts	Updated all endpoints to use /v2/chat, /v2/embed, and batch job routes
Chat Logic src/providers/cohere/chatComplete.ts	Implemented v2 chat config, response transform, and streaming with state tracking
Embed Logic src/providers/cohere/embed.ts	Added v2 embed config, response parsing, and token usage extraction
Types src/providers/cohere/types.ts	Introduced v2 stream events, stop reasons, usage metadata, and chat response models
Utils src/providers/types.ts	Integrated COHERE_STOP_REASON into global finish reason union
Finish Map src/providers/utils/finishReasonMap.ts	Mapped Cohere v2 stop reasons to OpenAI equivalents
Stream Split src/utils.ts	Adjusted stream delimiter based on /chat presence in URL

🧪 Test Added/Recommended

Recommended

• Add unit tests for CohereChatCompleteStreamChunkTransform with all stream event types
• Add integration test for v2 embed API with float and non-float responses
• Validate transformFinishReason with all Cohere v2 stop reasons

🔒Security Vulnerabilities

N/A

⏳ Estimated code review effort

HIGH (~35 minutes)

Tip

Quality Recommendations

1. Add validation for gatewayRequest.model in chat and embed transforms to prevent empty model field

2. Implement error boundary in CohereChatCompleteStreamChunkTransform for malformed JSON chunks

3. Add unit test coverage for embeddings.float and array fallback paths in embed response parsing

4. Enforce non-empty texts array in embed request via config transform

5. Log warning when response.meta.cached_tokens is present but non-zero for audit purposes

♫ Tanka Poem

New API takes flight, 🚀
Streams flow with clean design, 💬
Tokens counted right, ✅
Cohere v2 now aligned, 🔗
Code evolves with grace. 🌿

Sequence Diagram

sequenceDiagram
    participant Client
    participant Gateway
    participant CohereAPI

    Note over Gateway: Chat Complete - v2 Migration

    Client->>Gateway: POST /chat (model, messages)
    Gateway->>CohereAPI: POST https://api.cohere.ai/v2/chat
    CohereAPI-->>Gateway: stream of event chunks

    alt Stream Response
        Gateway->>Gateway: parse event: message-start
        Gateway->>Gateway: parse event: content-delta
        Gateway->>Gateway: parse event: tool-call-delta
        Gateway->>Gateway: parse event: message-end
        Gateway->>Client: SSE with OpenAI-compatible chunks
    else Error
        Gateway->>Client: 4xx/5xx with error message
    end

    Note over Gateway: Embed - v2 Migration

    Client->>Gateway: POST /embed (texts)
    Gateway->>CohereAPI: POST https://api.cohere.ai/v2/embed
    CohereAPI-->>Gateway: { embeddings: { float: [...] }, meta: { tokens: {} } }
    Gateway->>Gateway: normalize to OpenAI format
    Gateway-->>Client: { data: [{ embedding: [...] }], usage: {} }

Loading

[matter-code-review]

✅ Reviewed the changes: Cohere API v2 migration with updated endpoints and response handling