Skip to content
This repository was archived by the owner on May 25, 2026. It is now read-only.
This repository was archived by the owner on May 25, 2026. It is now read-only.

feat: Add vision/image support for multimodal models #38

Closed

Description

Summary

Add support for sending images to multimodal models like Claude, GPT-4o, and Gemini.

Current State

The plugin declares vision support in model modalities but the actual image handling may not be fully implemented.

Proposed Implementation

1. Support image input formats

// Base64 encoded images
{ type: 'image', data: 'base64...', mimeType: 'image/png' }

// URL references
{ type: 'image', url: 'https://...' }

// File paths (converted to base64)
{ type: 'image', path: '/path/to/image.png' }

2. Models with vision support

  • claude-opus-4-5 / claude-sonnet-4-5
  • gpt-4o / gpt-5.2
  • google/gemini-2.5-pro / google/gemini-2.5-flash
  • openrouter:google/gemma-3-27b-it:free

3. Image preprocessing

  • Resize large images to fit model limits
  • Convert formats as needed
  • Validate image size/dimensions

Technical Notes

  • Update PuterChatLanguageModel.doGenerate() to handle image parts
  • Add image validation utilities
  • Test with various image formats (PNG, JPEG, WebP, GIF)

Priority: Medium

Activity

  1. chindris-mihai-alexandru commented on Feb 15, 2026

    @chindris-mihai-alexandru
    OwnerAuthor

    Consolidated into #48 to keep roadmap tracking in one place.

  2. chindris-mihai-alexandru commented on Feb 15, 2026

    @chindris-mihai-alexandru
    OwnerAuthor

    Reopening to track this item as a standalone issue while also keeping #48 as the roadmap index.

  3. chindris-mihai-alexandru commented on Mar 13, 2026

    @chindris-mihai-alexandru
    OwnerAuthor

    Implemented: vision/image input support (AI provider handles image file parts + vision flag, puter-chat tool supports image_url). See commit fbb2428.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestroadmapTracked in roadmap/meta issuesstatus:backlogPlanned but not in progress

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions