A0 CLI / WebSocket Remote Execution Disconnects After Update — Reproducible Build Blocker

[esxenterprise]

We have frozen the current A0 environment and will not make further changes until advised, so the failing state can be preserved for diagnosis.

Agent Zero Support Request — Persistent CLI / Remote Execution Disconnect After Update & Clean Container Rebuild
To: Agent Zero Support Team
Date: September 3, 2026
Subject: Persistent A0 CLI Host / WebSocket Remote-Execution Disconnect — Survives Update, Proxy Repair, Port Migration and Fresh Container
Severity: High / Development Blocker
Troubleshooting period covered: Approximately August 28–September 3, 2026
Environment: Ubuntu server / Docker / PM2 / Agent Zero WebUI + A0 CLI remote execution
Hello Agent Zero Support Team,
We are requesting assistance with a persistent Agent Zero CLI/remote-execution connectivity problem that has consumed approximately a week of troubleshooting.
We have reached the point where we do not want to make additional changes to the working environment without guidance from the Agent Zero team, because the problem has survived multiple repairs, an Agent Zero update, a fresh side-by-side container, connector/proxy changes, and direct HTTP/Socket.IO testing.
The key symptom is:
Agent Zero's WebUI remains available and the agent/model can reason normally, but access to the connected host through the A0 CLI intermittently disappears. Agent Zero subsequently reports that the A0 CLI host must be reconnected and that the CLI process may have stopped or the WebSocket dropped.
The failure has repeatedly interrupted actual application builds.

1. Environment
   Primary development server:
   Host: ESX TWO CLEAN
   User used for AUTUMN-OS / PM2: autumn
   OS: Ubuntu
   Agent Zero deployment: Docker
   A0 CLI: Host-side CLI used to give Agent Zero remote execution access to the development server
   Process management: PM2
   Model currently used: GLM 5.2 through Fireworks
   A0 model configuration: Other OpenAI compatible
   Application project: AUTUMN-OS / It's Me V2
   The development application and Agent Zero are not the same component. Agent Zero is being used as the coding agent and accesses the host/project through A0 CLI remote execution.
2. Original Behavior
   Before the current problem became persistent, this architecture was capable of performing real development work.
   Agent Zero could:
   reason about a task;
   access the host through the CLI;
   inspect the project;
   modify files;
   execute shell commands;
   run TypeScript/ESLint validation;
   return build reports.
   The problem evolved into an intermittent pattern where Agent Zero would work normally for a period and then lose remote execution.
   At various points the CLI appeared to work for approximately 5–10 minutes before another stall/disconnect.
   The Agent Zero Docker container itself did not necessarily crash when this occurred.
   Earlier diagnostics showed:
   Agent Zero container still running;
   container restart count remained 0;
   A0 CLI process could remain alive;
   TCP/WebSocket connectivity could initially exist;
   failure occurred later at or around the remote-execution communication boundary.
   This made the problem substantially harder to isolate because "container running" did not mean "remote execution working."
3. Initial Connector / Authentication Problem
   One problem discovered during the investigation involved the connector path reaching Agent Zero but receiving an authentication redirect.
   The A0 connector was reaching Agent Zero, but Agent Zero responded with:
   HTTP 302 → /login
   because the connector did not have an authenticated Agent Zero WebUI session.
   We therefore investigated the connector authentication/session path rather than assuming the Agent Zero container itself was dead.
   A proxy implementation was developed to authenticate against Agent Zero and reuse the resulting session cookie for subsequent requests.
   The current proxy implementation is:
   /home/autumn/handshake/a0-connector-async-proxy.js
   The proxy contains configurable values for:
   listen host;
   listen port;
   Agent Zero target host;
   Agent Zero target port;
   authenticated login/session handling.
4. Agent Zero Update / Repair Attempts
   After repeated CLI failures, we moved through a controlled repair/escalation sequence rather than continuously modifying the same installation.
   The intended escalation ladder was:
   repair existing installation → update Agent Zero → if necessary create/rebuild clean Agent Zero container → reconnect integrations → test again.
   Before major changes, preservation/backups were considered, including the persistent Agent Zero Docker volume and the existing container state.
   A side-by-side/new Agent Zero container was subsequently created rather than blindly destroying the previous environment.
   During this phase we discovered evidence of two Agent Zero/container paths using different addresses/ports, which was investigated as a possible source of the intermittent connection behavior.
   The newer Agent Zero instance was ultimately made reachable through:
   127.0.0.1:8081
   This new 8081 container/path initially worked correctly.
5. Issues Seen During the New-Container Investigation
   During the clean/new-container testing we encountered several distinct errors. We are listing them because we do not know which, if any, are related to the persistent CLI failure.
   OpenRouter / Clerk authentication
   At one stage logs from the newer Agent Zero container showed an OpenRouter/Clerk HTTP 401 condition.
   WebSocket behavior
   Logs also showed WebSocket connections being:
   accepted;
   established;
   subsequently disconnected.
   LiteLLM
   A stack trace was observed involving Agent Zero's LiteLLM integration, including:
   litellm_transport.py
   These model/provider errors were investigated separately from the host remote-execution problem.
6. Fireworks / GLM Configuration
   Agent Zero is presently configured with GLM through the OpenAI-compatible provider interface.
   The Agent Zero model screen currently shows approximately:
   Main:
   Other OpenAI compatible / accounts/fireworks/models/glm-5p2
   Utility:
   Other OpenAI compatible / accounts/fireworks/models/glm-5p2
   Embedding uses a HuggingFace sentence-transformer configuration.
   This configuration does successfully invoke GLM.
   Therefore "OpenAI compatible" in the Agent Zero UI is being used as the API compatibility layer; we are not claiming that OpenAI is the current inference provider for these requests.
7. Fireworks Billing Error Encountered September 3
   During final testing on September 3, Agent Zero produced a LiteLLM/API error indicating the Fireworks account was suspended / unable to make the model request.
   The Fireworks billing page showed:
   Suspended
   and:
   Credits depleted, please purchase credits or add a payment method to resume usage.
   At the time the billing interface also displayed prepaid credits, but the account was still reporting suspension.
   Additional credit was supplied / service resumed.
   This is important because after resolving the Fireworks problem, Agent Zero successfully began reasoning again.
   We therefore consider that billing incident a separate failure from the continuing A0 CLI/WebSocket problem.
   After Fireworks service resumed:
   GLM reasoning worked.
   But host remote execution subsequently failed again.
8. Current PM2 Environment
   On September 3 we verified the existing PM2 processes.
   The visible PM2 set contained seven processes:
   a0-connector-proxy
   autumn-telegram
   bambi-dashboard
   handshake-bambi
   handshake-bambi-return
   handshake-scanner
   handshake-summer
   They were shown as online.
   This is a partially reconstructed AUTUMN-OS environment; additional historical PM2 processes are being recovered separately.
   We do not presently believe recovery of those unrelated PM2 services should be necessary for the A0 CLI transport itself.
9. Current Connector Configuration
   Inspection of:
   /home/autumn/handshake/a0-connector-async-proxy.js
   showed the following architecture:
   Listen host:
   127.0.0.1
   Listen port default:
   9001
   through:
   A0_CONNECTOR_PROXY_PORT
   Target host default:
   127.0.0.1
   through:
   A0_CONNECTOR_TARGET_HOST
   Target port:
   8081
   through:
   A0_CONNECTOR_TARGET_PORT
   The proxy therefore currently represents approximately:
   connector/client → 127.0.0.1:9001 → async proxy → Agent Zero at 127.0.0.1:8081
   The proxy also contains login/session-cookie handling for Agent Zero authentication.
10. Important September 3 Direct Connectivity Test
    Immediately before the latest failure, we tested the new Agent Zero endpoint directly.
    The command tested:
    127.0.0.1:8081
    and returned:
    A0_8081 HTTP 200
    This established that the Agent Zero HTTP service was alive and responding at the expected local endpoint.
11. Socket.IO / WebSocket Handshake Test
    We then tested Agent Zero's Socket.IO endpoint directly against the same 8081 service.
    The request was made to approximately:
    http://127.0.0.1:8081/socket.io/?EIO=4&transport=polling
    The response was:
    HTTP/1.1 200 OK
    Headers included:
    server: uvicorn
    Content-Type: text/plain; charset=UTF-8
    Access-Control-Allow-Credentials: true
    Transfer-Encoding: chunked
    Most importantly, the Engine.IO response returned a valid session ID and advertised:
    "upgrades":["websocket"]
    It also returned ping configuration including approximately:
    "pingTimeout":120000
    and a ping interval.
    Therefore, immediately before the final failure:
    Agent Zero HTTP endpoint = responding
    Socket.IO endpoint = responding
    Engine.IO session = created
    WebSocket upgrade = advertised
12. Actual Build Test After Those Checks
    After obtaining the successful HTTP and Socket.IO results, we intentionally stopped troubleshooting and attempted a real, tightly controlled application build.
    Agent Zero/GLM began normally.
    The WebUI displayed:
    A0: Reasoning...
    GLM inference was functioning.
    The task instructed Agent Zero to work on the existing It's Me project using the established AUTUMN-OS/Stack Library components and to perform normal validation including:
    npx tsc --noEmit
    The agent successfully reasoned about the task.
    However, when it needed live access to the development host/project, remote execution was unavailable again.
13. Final Failure — September 3, 2026
    Agent Zero ultimately reported:
    "What's Needed to Unblock"
    Please reconnect the A0 CLI host:
    Agent Zero then instructed us to:
    ensure the A0 CLI is running and connected;
    press F4 in the CLI to enable remote execution if necessary;
    confirm the connection appears active in the WebUI;
    resend the instruction.
    Agent Zero explicitly stated that:
    the CLI process may have stopped or the WebSocket dropped
    It was unable to execute:
    npx tsc --noEmit
    because it no longer had access to the project directory on the host.
    Agent Zero correctly remained read-only rather than fabricating execution results.
    This final failure occurred after the direct 8081 HTTP test and Socket.IO handshake test succeeded.
    That sequence is the primary reason for this support request.
14. Important Failure Pattern
    Our observed pattern is approximately:
    A0 WebUI alive
    ↓
    Model inference works
    ↓
    Host A0 CLI connects / remote execution initially works
    ↓
    HTTP endpoint remains reachable
    ↓
    Socket.IO handshake can succeed
    ↓
    Agent Zero begins real work
    ↓
    remote execution later disappears
    ↓
    A0 reports CLI host disconnected / possible WebSocket drop
    In previous occurrences, restarting/reconnecting could temporarily restore functionality.
    The failure subsequently returned.
15. Things We Have Already Investigated or Changed
    Over approximately the last week we have investigated or attempted, at various stages:
    verifying the Agent Zero Docker container remained running;
    checking container restart behavior;
    verifying the A0 CLI process;
    checking TCP/Socket.IO/WebSocket connectivity;
    testing whether device sleep was responsible;
    checking for duplicate/parallel connections;
    investigating two Agent Zero containers/endpoints;
    inspecting local port mappings;
    investigating the original localhost/port-80 deployment;
    updating Agent Zero;
    preserving the existing environment before major changes;
    creating/testing a newer side-by-side Agent Zero container;
    moving/testing the new Agent Zero endpoint on port 8081;
    repairing the connector target;
    implementing an asynchronous connector proxy;
    correcting connector target host/port;
    addressing Agent Zero's /login HTTP 302 authentication behavior;
    adding authenticated session/cookie handling to the proxy;
    inspecting PM2 process health;
    testing Agent Zero's HTTP endpoint directly;
    testing Engine.IO/Socket.IO directly;
    confirming WebSocket upgrade capability;
    testing GLM/Fireworks independently through Agent Zero;
    resolving a separate Fireworks billing suspension;
    performing a fresh real-world build test after transport checks passed.
    Despite all of the above, the CLI/remote-execution disconnect returned.
16. What We Have NOT Established
    We do not want to incorrectly diagnose the issue for you.
    We have not established whether the root cause is:
    the A0 CLI itself;
    WebSocket lifecycle/heartbeat behavior;
    Engine.IO/Socket.IO session handling;
    the Agent Zero server;
    the async connector/proxy;
    stale or duplicate client/session state;
    interaction between old/new containers;
    authentication/session expiration;
    an A0 update/version compatibility issue;
    or another component entirely.
    Earlier behavior suggested a possible queue/backpressure or connection-lifecycle problem, but that remains only a hypothesis.
17. Current State — Environment Frozen for Support
    As of the final failure on September 3, 2026, we have intentionally stopped modifying the Agent Zero infrastructure.
    We would prefer to preserve the failing state for diagnosis rather than continue changing variables.
    The latest known-good transport tests immediately before failure were:
    Agent Zero 127.0.0.1:8081: HTTP 200
    Socket.IO/Engine.IO endpoint: HTTP 200
    Valid Engine.IO SID returned
    upgrades:["websocket"] advertised
    GLM 5.2 inference: working
    Agent Zero reasoning: working
    followed by:
    A0 host remote execution unavailable
    and Agent Zero's instruction:
    "Please reconnect the A0 CLI host."
18. Questions for Agent Zero Engineering / Support
    Could you please advise:
    What is the best diagnostic procedure for determining why an A0 CLI host disappears from an otherwise-running Agent Zero session?
    Which Agent Zero server and A0 CLI logs should we capture immediately before and after the disconnect?
    Is there a known issue in the current Agent Zero/A0 CLI versions involving long-lived Socket.IO/WebSocket connections, reconnects, heartbeats, duplicate sessions, or remote-execution state?
    Is there a supported health-check that verifies the complete chain—not merely HTTP/Socket.IO connectivity, but that the WebUI still considers the host connected and capable of remote execution?
    Could our localhost reverse/async proxy interfere with the WebSocket lifecycle even though the initial Socket.IO handshake succeeds?
    Is port 8081 / the current Docker deployment architecture supported for the CLI connection, or does A0 assume a particular endpoint/port elsewhere in its CLI/WebUI connection logic?
    What diagnostic information can we collect from the CLI itself when the WebUI transitions from connected to disconnected?
    Should we capture WebSocket frames, server debug logs, CLI debug output, container logs, or another specific trace before reproducing the issue again?
    We are happy to provide:
    Agent Zero version;
    A0 CLI version;
    Docker inspect output;
    container logs;
    PM2 configuration;
    connector source;
    sanitized environment configuration;
    screenshots;
    timestamps;
    Socket.IO response;
    network/socket state;
    reproduction steps.
    Please tell us specifically what you would like captured before we alter or restart the environment again.
19. Reproduction Summary
    The shortest reproduction we currently have is:
20. Start/verify Agent Zero.
21. Verify A0 CLI host connection and remote execution.
22. Verify Agent Zero on 127.0.0.1:8081.
    Result:
    HTTP 200
23. Verify /socket.io/?EIO=4&transport=polling.
    Result:
    HTTP 200 + valid SID + WebSocket upgrade advertised
24. Start a normal Agent Zero development task.
25. GLM begins reasoning normally.
26. Agent attempts to access the host/project.
27. Remote execution is unavailable.
28. Agent Zero reports:
    Please reconnect the A0 CLI host
    and indicates:
    the CLI process may have stopped or the WebSocket dropped.
    Requested Outcome
    We are looking for a root-cause diagnosis and durable fix, rather than another temporary reconnect.
    This environment is used for active software development, and repeated A0 CLI disconnections are preventing Agent Zero from reliably completing builds.
    We have spent approximately one week isolating the issue and would greatly appreciate guidance from someone familiar with the internals of the A0 CLI/WebSocket/remote-execution path.
    Thank you very much for taking a look.
    — AUTUMN-OS / It's Me Development

[Junaid-PK]

The successful GET /socket.io/?EIO=4&transport=polling is useful, but it does not verify the remote-execution chain. It proves only that Engine.IO created a transport session and advertised that WebSocket upgrade is possible.

For code_execution_remote to work, all of these must still be true:

1. The client connects to the authenticated Socket.IO namespace /ws.
2. Its connect payload activates plugins/_a0_connector/ws_connector.
3. Its web session cookie plus Origin/Referer survive the polling and WebSocket-upgrade paths.
4. connector_hello advertises remote_exec.enabled=true (F4) and file write access (F3) when the command may modify files.
5. That socket is subscribed to the active Agent Zero context.

Those are separate application-level steps in the current client (namespace, handler, and events, cookie/origin headers, real namespace connection). The backend removes the remote-exec metadata immediately when that /ws SID disconnects (source).

That makes the custom a0-connector-async-proxy.js the first thing I would isolate. A proxy can return the valid polling handshake you captured while still failing later because it does not forward Upgrade/Connection, loses the session cookie, rewrites Host/Origin, caches polling, or sends the Engine.IO SID to a different upstream. The official CLI already signs in through /login, retains the resulting web session cookie, and sends it to /ws; an unauthenticated 302 /login from a manual request is therefore expected and does not require a separate login proxy (documented flow).

Before changing the frozen setup, capture these read-only facts:

date -u +'%Y-%m-%dT%H:%M:%SZ'
a0 --version
curl -sS -X POST http://127.0.0.1:8081/api/plugins/_a0_connector/v1/capabilities \
  -H 'Content-Type: application/json' -d '{}' | jq '{protocol,agent_zero_version,auth_required,websocket_namespace,websocket_handlers}'
curl -sS -X POST http://127.0.0.1:9001/api/plugins/_a0_connector/v1/capabilities \
  -H 'Content-Type: application/json' -d '{}' | jq '{protocol,agent_zero_version,auth_required,websocket_namespace,websocket_handlers}'
docker logs --since 15m <agent-zero-container> 2>&1
pm2 logs a0-connector-proxy --lines 300 --nostream

Also copy the exact code_execution_remote tool result, not only the model's summary. Current Core intentionally emits three different diagnoses: no CLI socket, CLI connected but F4/remote exec disabled, or execution enabled but F3/write access disabled (source). That one line identifies which state disappeared.

Then do one controlled A/B test, after saving the logs and stopping the existing CLI so there is no duplicate client:

a0 --host http://127.0.0.1:8081 --no-docker-discovery --connect

Let the official CLI perform the protected-instance login, enable F3/F4, and run a harmless end-to-end command such as pwd through Agent Zero. Keep it connected longer than the usual 5–10 minute failure window. If the direct path remains stable, the proxy is the fault domain; compare its WebSocket-upgrade request with the direct request and ensure /socket.io and /api/plugins/ pass through unchanged. This matches the project's own proxy guidance (troubleshooting).

Finally, verify both Agent Zero Core and a0 are 2.11, not merely “updated.” The current public releases are Agent Zero 2.11 and A0 CLI 2.11. CLI 2.11 specifically changed recovery from five attempts (about 38 seconds) to indefinite retries every 30 seconds after the initial backoff (implementation, recovery loop). A mixed or pre-2.11 CLI can explain why a transient disconnect becomes a permanent “reconnect the host” blocker.

So the next useful evidence is: exact Core/CLI versions, exact remote-tool error, /ws connect/disconnect timestamps, and a direct-8081 A/B run. HTTP 200 plus an Engine.IO SID is not yet a connector health check.

[esxenterprise]

Thank you Junaid-PK very much, for a very fast response and so very well thought out, that I think we are now back on our way : }}

Below are the tests we ran after your response to our being lost in the woods...lol

We completed additional isolation testing and found an important distinction.

The Agent Zero CLI had previously been launched directly inside an interactive SSH/Termius session. When that SSH session disconnected, the A0 CLI process also disappeared, causing Remote Code Execution to become unavailable.

We reproduced this behavior and then changed the test setup by launching the A0 CLI inside a persistent tmux session.

Test results:

• A0 CLI connected successfully.
• Remote Code Execution was enabled.
• Remote execution of date -u succeeded.
• We deliberately closed the SSH/Termius connection completely.
• After reconnecting to the server, the tmux session was still running.
• We reattached to the same A0 CLI session.
• Remote Code Execution remained enabled.
• A second date -u remote execution succeeded immediately.

This confirms that the A0 CLI and Remote Code Execution remain operational when the CLI process is kept alive independently of the SSH client.

We are therefore separating two issues:

1. SSH/Termius sessions occasionally disconnect, sometimes quite frequently.
2. Previously, those SSH disconnects also terminated the A0 CLI process because it was attached directly to the SSH session.

Running A0 inside tmux prevents an SSH disconnect from taking down the CLI and has successfully survived our disconnect/reconnect test.

We will use the persistent tmux configuration going forward. We would still appreciate clarification on whether Agent Zero CLI is expected to survive terminal/SSH disconnects on its own, or whether running it under tmux/screen or another persistent service is the recommended deployment method.

Thank you again my friend
kindly,
Jeff & Jenn