Assistant (v2) streaming returns an empty stream in v0.11.2

[joostbr]

tag 0.11.2:

When using the Assistant API (v2) in streaming mode, it seems to always return an empty stream.

Digging into the code revealed this :

com.openai.core.handlers.SseHandler.kt line 42:
if (message.event == null) {
...
yield message;
...
}

Because of this test the SSE streams related to streaming Assistant (v2) runs return nothing because every entry in the stream is decorated by an 'event' tag.
It does work for 'completions' however since they are plain 'data' objects without an event-tag attached to it.

Not sure why 'event' needs to be null to return (yield) the message in the stream

Example code snippet below, it never prints the "Got something" line, for whatever message content that has been passed to the thread.

// streaming
        try (StreamResponse<AssistantStreamEvent> responseStream = client.beta().threads().runs().createStreaming(BetaThreadRunCreateParams.builder()
                .assistantId(assistant.id())
                .threadId(thread.id())
                .build())) {

            responseStream.stream().forEach(x->System.out.println("Got something "+x));

        } catch (Exception e) {
            System.out.println(e.getMessage());
        }

[TomerAberbach]

Thanks for the report!

This is a misconfiguration in our streaming code generator config. The configuration is set up to support completions, which requires that only the data of events with an event tag are yielded. Clearly that's not suitable for the Assistants API, so we'll fix it :)

[joostbr]

Perfect thanks.
As a workaround to get this particular case working I patched 2 lines and it seems to do the job now.
(It is just a quick patch, not a thoroughly tested general fix)

(1) One in SseMessage.kt:

 private val jsonNode by lazy {
        try {
            jsonMapper.readTree(data)
        } catch (e: Exception) {
            throw OpenAIException("Error deserializing json", e)
        }
    }

to

 private val jsonNode by lazy {
        try {
            jsonMapper.readTree(if(event == null) data else "{\"event\":\"$event\",\"data\":$data}")
        } catch (e: Exception) {
            throw OpenAIException("Error deserializing json", e)
        }
    }

(2) in SseHandler.kt within the handle() method I changed:

if (message.event == null) {
   ...
}

to

if (!message.data.isEmpty()) {
   ...
}

[TomerAberbach]

Should be fixed in v0.12.0