WebSocket channel cannot replace webhooks for proactive message delivery (cron, heartbeat, agent-initiated sends in multi-tenant environments)

[ivelin]

Problem: WebSocket Does Not Replace Webhooks for Proactive Message Delivery in Multi-Tenant Environments

Background

Our previous issue (#2602) proposed an HTTP Streaming Channel with webhook-based proactive message delivery. The issue was closed with the maintainer noting "Now we have official websocket channel 💖" (PR #2946/#2964).

After weeks of production testing migrating our multi-tenant PirinAI infrastructure from HTTP streaming to the official WebSocket channel, we've confirmed that the WebSocket channel does not solve the proactive messaging problem for managed multi-tenant environments.

What We Tried

1. Full Migration to WebSocket Channel

We moved our entire infrastructure — Discord bot gateway managing per-user nanobot containers — from a custom HTTP streaming plugin to the official WebSocket channel. The WS channel handles interactive request-response streaming beautifully (token-by-token delta events work great).

But it fundamentally breaks for proactive messages:

• Cron jobs fire inside the nanobot container with no active WebSocket subscriber. The message() tool looks up self._subs[chat_id], finds nothing, and the message is silently dropped.
• Heartbeat evaluations have the same problem — agent-initiated sends hit an empty subscriber list.
• The agent's evaluate_response() is the only fallback, but it uses a local LLM that aggressively classifies most outputs as "routine" and suppresses them. Even when it decides to send, there's no WS subscriber to deliver to.

2. OpenAI API Channel Test

Before WS migration, we also tested nanobot's OpenAI-compatible API server:

• Purely request-response — no server-initiated push mechanism
• Session management is one-sided; cron/heartbeat sessions can't push results back to the gateway

The Core Problem

Both the WebSocket and OpenAI API channels are designed around a pull / request-response model:

Delivery Pattern	Proactive Send	Response Streaming
WebSocket	❌ No subscriber = no delivery	✅ Great for active connections
OpenAI API	❌ No push mechanism	✅ Per-request streaming
HTTP Streaming + Webhook	✅ Webhook POST to gateway	✅ SSE per request

In our multi-tenant setup, the gateway owns the Discord/Telegram connections. Nanobot containers are isolated agents. When a cron job fires:

1. The container needs to push results to the gateway
2. The gateway delivers to the platform channel

WebSocket only works with a persistent connection, which doesn't exist for automated cron/heartbeat sessions.

Current Hack Workaround

We implemented a fragile polling mechanism in our gateway:

• On WebSocket connect events, scan the container's cron session JSONL files for recent assistant messages without corresponding message() tool calls
• Extract and forward those to Discord
• Downsides: Only triggers on new WS connections. Idle periods = missed messages. Adds latency and complexity.

The Cleaner Approach (Our Proposal)

An HTTP streaming channel with webhook delivery that:

1. Exposes POST + SSE endpoints for interactive streaming
2. Adds a configurable webhook URL — when cron/heartbeat/agent-initiated sends fire and no WS subscriber exists, the response is POSTed to the webhook URL
3. Falls back gracefully — WS subscriber present = stream directly. No subscriber = use webhook. The gateway handles both.

Request

Could the WebSocket channel (or a companion plugin) be extended to support:

• Webhook fallback — when send() finds no subscribers, POST to a configured webhook URL instead of silently dropping?
• Or exposing a hook that external gateways can register for cron/heartbeat/unsolicited send events?

We have a production-tested external plugin and are happy to contribute a clean PR.

Related

• feat: HTTP Streaming Channel for Real-Time Multi-Tenant Architectures #2602 — Original HTTP Streaming Channel feature request
• Bug: Cron reminders are not delivered in agent mode #2601 — Cron reminders not delivered in agent mode
• Heartbeat repeatedly executes and sends non-actionable status messages #2582 — Heartbeat non-actionable messages (evaluator suppressing output)
• feat: base64 media upload for QQ & WeCom, streaming & proactive messaging #2303 — Proactive messaging for QQ/WeCom channels (addressed non-WS channels specifically)

Happy to collaborate on implementation.

[ivelin]

Clarification: #2602 was an issue/feature request, not a pull request. It proposed an HTTP streaming channel with webhook-based proactive delivery. The maintainers closed it noting the official WebSocket channel (PR #2946/#2964) and OpenAI API server had been added to nanobot, implying these solved the underlying need.

After extensive testing of both additions in production, the core gap remains: neither the WebSocket channel nor the OpenAI API server solves the proactive (server-initiated) messaging problem for multi-tenant environments.

The WebSocket channel handles streaming to active connections well. The OpenAI API server handles request-response well. Neither provides a push mechanism for cron/heartbeat/agent-initiated sends when there's no active subscriber — which is always the case for automated sessions.

[ivelin]

Update — Root cause identified on our end.

After extensive tracing, the cron delivery failure was NOT a limitation of the WebSocket channel itself. The nanobot's WebSocket channel correctly delivered the message (logged "websocket send 1 connections for cid=..."). The problem was in our gateway's dispatcher code:

The bug:

1. Cron fires → nanobot agent runs → calls message() tool → WS channel sends event="message" to the gateway's persistent connection
2. Gateway's WS reader publishes an OutboundMessage with metadata={"_stream_end": True} (because the message arrived via WS as a complete message, not streaming chunks)
3. Gateway's dispatcher routes _stream_end → calls channel.edit_stream_message() — which expects a pending "Thinking..." message to edit
4. For cron jobs there's no pending thinking message → edit_stream_message() returns False
5. The dispatcher never fell through to send() — message silently dropped

The fix (in our gateway code): Check the return value of edit_stream_message() and fall through to channel.send() when it returns False.

This was a bug in our dispatcher's message routing, not in nanobot's WebSocket channel. The core issue from #2602 (webhook-based proactive delivery) remains architecturally valid for cases where there's no persistent WS connection, but our immediate cron delivery problem was local to our code.

Apologies for the noise on this issue — the original feature request (#2602) stands as a valid architectural discussion, but this specific problem was on our side.