Connecting over pipes

[OrKoN]

Currently, WebDriver BiDi spec defines that communication happens over the WebSocket protocol. It would be great to support local communication transports such as named pipes or some way to allow negotiate a different transport when launching a browser/driver. Pipes offer better security out of the box and potentially better performance.

[OrKoN]

Related feedback: https://news.ycombinator.com/item?id=41186834

[jgraham]

I agree that this makes sense to have as an option for BiDi-only sessions (maybe also for BiDi connections to mixed sessions, but if you're already running a local HTTP server the value proposition of the BiDi part not being a TCP server seems weaker; I don't think we want to allow different transports for classic).

In terms of selecting the transport, I assume it would just have to be an option when starting the BiDi server (e.g. the browser). I don't quite see how you could have in-band negotiation of the transport layer.

I think the main question that needs a proposal is how this actually works in detail; with websockets we get a lot "for free" in terms of the on-the-wire data format and semantics. I don't know quite what CDP is doing here, but maybe that's a useful starting point?

I'll also note that I think this should be an orthogonal question to allowing a non-JSON message format.

[OrKoN]

We send CDP messages as strings using \0 as the delimiter. We can already run WebDriver BiDi over a pipe with Chrome (due to how our implementation is based on CDP) but it would be great to have this cross-browser. I guess the first step could be to define an abstract transport interface in the specification and allow for custom transports?

[jgraham]

So, having said that this is orthogonal to the message format: I thought you also supported binary messages? How does that work? I was really really hoping for a length-framed wire format like WebSockets rather than something null separated :)

I agree we can probably start by refactoring to an abstract interface that defines a "read a protocol message" operation, and a "write a protocol message" operation (and probably "create a connection" and "close a connection" or something) and depends on the specific transport to actually implement those operations. But I think we do need to define how this actually works in detail for each transport; I don't want clients to have to implement slightly different wire formats per browser.

Another slight concern is that we need to continue to allow multiple WebSocket connections, whereas that might not be supported for other transports (or if it is, it requires designing a message format that distinguishes where messages are coming from and doesn't make it trivial to spoof, which seems potentially challenging).

[OrKoN]

In Puppeteer we just send strings (which are JSON.stringified CDP messages) https://github.com/puppeteer/puppeteer/blob/main/packages/puppeteer-core/src/node/PipeTransport.ts#L15 But Chrome also supports cbor https://cbor.io/ / https://source.chromium.org/chromium/chromium/src/+/main:content/browser/devtools/devtools_pipe_handler.cc;l=404;drc=5203366d4cf174b9331ee33563e0520f10b60828

we currently do not support multiple connections over pipes (I am not sure if it is even possible).

[jgraham]

Chrome also supports cbor

Which turns out to use a mixture of length encoding (for primitives, sometimes) "structural" length encoding ("a map of N items") and separators ("a map of unknown length, delimited by a break item"). Exciting!

Anyway I guess we really don't want to use CBOR directly for the case of JSON messages because it's uncommon to have an encoder / decoder for it, and "messages are CBOR, but actually they're just CBOR-encoded strings containing JSON" is a lot of complexity (not to be confused with actually using CBOR as the protocol-level message format, as Chrome is doing).

we currently do not support multiple connections over pipes (I am not sure if it is even possible).

I'm not sure, but I imagine you could make it work with named pipes just like the TCP case, with one pipe that responds to an incoming request by creating a new pipe and passing back a handle to that to use for future messages. I don't know the technical details though!

[OrKoN]

I started a draft to refactor the spec to allow supporting multiple transports #996 Not sure if the approach I took is good enough. PTAL if you have a chance.