Skip to content

Conversation

@apascal07
Copy link
Collaborator

@apascal07 apascal07 commented Dec 11, 2025

This PR adds Go iterator-based streaming support for model generation and prompt execution, enabling real-time consumption and parsing of model output chunks.

Usage:

GenerateStream / GenerateDataStream

Stream model responses using Go's range-over-func pattern:

// Untyped streaming - chunks are *ModelResponseChunk
for result, err := range genkit.GenerateStream(ctx, g,
    ai.WithPrompt("Tell me a joke"),
) {
    if err != nil { return err }
    if result.Done {
        fmt.Println("Final:", result.Response.Text())
    } else {
        fmt.Print(result.Chunk.Text())
    }
}

// Typed streaming - chunks are parsed to the output type
for result, err := range genkit.GenerateDataStream[*Recipe](ctx, g,
    ai.WithPrompt("Create a pasta recipe"),
) {
    if err != nil { return err }
    if result.Done {
        fmt.Printf("Recipe: %+v\n", result.Output)
    } else {
        fmt.Printf("Ingredients so far: %v\n", result.Chunk.Ingredients)
    }
}

DataPrompt

Strongly-typed prompts with Execute and ExecuteStream:

// Define a typed prompt
storyPrompt := genkit.DefineDataPrompt[StoryInput, *Story](g, "story",
    ai.WithPrompt("Write a story about {{topic}}"),
)

// Non-streaming execution returns typed output directly
output, resp, err := storyPrompt.Execute(ctx, StoryInput{Topic: "dragons"})
fmt.Printf("Title: %s\n", output.Title)

// Streaming execution
for result, err := range storyPrompt.ExecuteStream(ctx, StoryInput{Topic: "dragons"}) {
    if err != nil { return err }
    if result.Done {
        fmt.Printf("Title: %s\n", result.Output.Title)
    }
}

You can also wrap .prompt files with type information using LookupDataPrompt:

storyPrompt := genkit.LookupDataPrompt[StoryInput, *Story](g, "story")
output, resp, err := storyPrompt.Execute(ctx, StoryInput{Topic: "dragons"})

@apascal07 apascal07 changed the base branch from main to ap/go-structured-streaming December 13, 2025 15:25
@apascal07 apascal07 requested a review from pavelgj December 18, 2025 18:26
@apascal07 apascal07 marked this pull request as ready for review December 18, 2025 18:26
@apascal07 apascal07 changed the title feat(go): added iterator-based streaming functions and typed prompts feat(go): added DataPrompt and iterator-based stream support Dec 18, 2025
Base automatically changed from ap/go-structured-streaming to main December 18, 2025 20:14
@apascal07 apascal07 requested a review from huangjeff5 December 19, 2025 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants