Using gemini-2.0-flash-exp with Image Generation in Firebase Vertex AI SDK for iOS

[tom-eberle]

I'm trying to implement Gemini 2.0 Flash with image generation capabilities in my iOS app using the Firebase Vertex AI SDK. Specifically, I want to use the gemini-2.0-flash-exp model with response_modalities=["Text", "Image"] to enable multimodal responses.

While I can see this feature is available in the direct Vertex AI API, I'm encountering issues implementing it in the Firebase SDK for iOS.

Questions:

1. Is the gemini-2.0-flash-exp model with image generation capabilities (response_modalities=["Text", "Image"]) supported in the Firebase Vertex AI SDK for iOS?
2. If yes, what's the correct implementation approach? Any code examples would be helpful.
3. If not yet supported, is there a timeline for when this feature will be available in the Firebase SDK?
4. I've reviewed the documentation but haven't found clear guidance on implementing this specific functionality in the iOS SDK.

Thank you for any assistance you can provide!

[andrewheard]

Hi @tom-eberle, the gemini-2.0-flash-exp image generation capabilities are unfortunately not yet supported in the SDK but I'm planning to work on it very soon. I can't make any guarantees that I'll get it done today but, if I don't run into any issues, would you be interested in trying it out on a branch?

Alternatively, we do support image generation using the Imagen 3 model (documentation). Although it's from a Google AI page, rather than Vertex AI, this section provides a nice description of when to use Gemini 2.0 vs. Imagen 3 for image generation: https://ai.google.dev/gemini-api/docs/image-generation#choose-a-model

I'll ping this discussion with any progress to keep you updated. Should soon be able to use either model for image generation in the SDK 🤞

[andrewheard]

@tom-eberle This is now available in the branch ah/vertex-multimodal-output if you want to try it out with Swift Package Manager. The documentation is still quite lacking but I put a usage example in #14658 that may help somewhat.