zig-websocket

The WebSocket Protocol — RFC 6455 — for Zig servers, on top of zig-http.

One handler serves WebSocket over HTTP/1.1 and over HTTP/2 without knowing which it is doing:

fn chat(req: *http.server.Request, res: *http.server.Response) anyerror!void {
    var buffer: [64 * 1024]u8 = undefined;
    var conn = ws.accept(req, res, .{ .buffer = &buffer }) catch |err| {
        return ws.refuse(res, err);
    };
    while (try conn.receive()) |message| {
        try conn.send(message.kind, message.data);
    }
}

The two opening handshakes are different documents — RFC 6455 §4.2 and RFC 8441 §5 — and what makes them one function is that zig-http's Request describes HTTP semantics rather than a wire format. "Did the client offer to switch to this protocol" has one answer and two implementations: an Upgrade field on one version, a :protocol pseudo-header on the other.

Status

Early. The framing is exercised by 60-odd tests, four property-based fuzz targets and a real Python websockets client over a real socket, and the rules are the ones RFC 6455 gives — but this has not served anything in anger and there is no permessage-deflate. The client handshake is newer still: it is covered by tests of its own, but it has not yet been run against another implementation's server the way the framing has against another implementation's client.

Why this is not part of zig-http

After the opening handshake there is no HTTP left. A masked, fragmented, opcode-tagged frame protocol shares not one rule with RFC 9110, and putting it in an HTTP library would make that library's claim about itself false. The dependency points one way and only one way, which is the usual sign that two things are two things.

What is in zig-http is the seam: Response.upgrade stops speaking HTTP and hands back a reader and a writer, positioned exactly where the handshake ended, so a client that pipelined its first frames behind the handshake does not lose them. Everything from there on is here.

Where this lives

The repository lives in three places that carry the same history. The Forgejo instance at https://git.jcollie.dev/jeff/zig-websocket is the web-visible one:

$ git clone https://git.jcollie.dev/jeff/zig-websocket.git

it is mirrored on Tangled at https://tangled.org/jcollie.dev/zig-websocket, and it is also on the Radicle network, where the repository's identifier is

rad:zgU6MCJUmKJEZF5r8xepUk7tqGuK

and rad clone rad:zgU6MCJUmKJEZF5r8xepUk7tqGuK fetches it from any node that seeds it. Any of the three is the whole project.

Adding it to a project

$ zig fetch --save git+https://git.jcollie.dev/jeff/zig-websocket.git

const websocket = b.dependency("zig_websocket", .{});
exe.root_module.addImport("websocket", websocket.module("websocket"));

Two modules, split at the only place an HTTP implementation is needed:

Module	Depends on	What it is
websocket	nothing	The framing, the connection, and connect — the opening handshake as the client.
websocket_server	websocket, zig-http	accept and refuse: the handshake as the server.

A server wants both. A client wants only the first, and that is the reason for the split — accept is written against zig-http because a server has a request in hand and an HTTP version to worry about, and a client that had to take an HTTP server library along to open a connection would be paying for something it cannot use.

The API

accept performs the handshake as the server and connect as the client; both give back a Connection, and everything else is on that.

var conn = try ws.accept(req, res, .{
    .buffer = &buffer,                        // the largest message accepted
    .subprotocols = &.{ "chat.v2", "chat.v1" }, // most preferred first
});

while (try conn.receive()) |message| {   // null once the connection closes
    switch (message.kind) {              // .text or .binary
        .text => try conn.sendText(message.data),
        .binary => try conn.sendBinary(message.data),
    }
}

receive answers a Ping with a Pong, ignores a Pong, completes the closing handshake, and reassembles fragments, so the loop above is conformant as it stands. conn.ping, conn.pong and conn.close are there for a caller that wants to start those itself.

A message points into the buffer you supplied and is valid until the next receive. The buffer's length is the largest message the connection will accept: a bigger one is refused with 1009 Message Too Big rather than grown into, so how much memory a connection costs is a number chosen by the server and not by whoever connects to it.

Over HTTP/2

Nothing changes in the handler, and three things change underneath.

RFC 8441 replaces the Upgrade mechanism — which HTTP/2 does not have, and whose header fields §8.2.2 forbids outright — with an extended CONNECT carrying a :protocol pseudo-header. zig-http advertises SETTINGS_ENABLE_CONNECT_PROTOCOL, req.offersUpgrade("websocket") answers from :protocol, and res.upgrade sends 200 where HTTP/1.1 sends 101.

There is no Sec-WebSocket-Key exchange. RFC 8441 §5 says so outright, and it is not a simplification. The Sec- prefix exists so that an XMLHttpRequest cannot be talked into producing a handshake — it names a header a browser will not let script set. A pseudo-header is the stronger version of the same argument: script cannot create one at all, because it is not a header, it is part of the frame layer.

A tunnel is one stream, not the connection. Over HTTP/1.1 an upgraded connection is spent — every request that would have been multiplexed with it has to find another socket. Over HTTP/2 the other streams carry on.

What is not here

• No interop test for the client. connect performs the opening handshake as the client, and Connection has always implemented the client role of the framing — masking what it sends, refusing masked frames from the server. What is missing is the other half of the standard this project holds itself to: the framing is judged by Python's websockets acting as a client, and nothing yet judges the handshake by anybody's server.
• No permessage-deflate (RFC 7692). No extension is negotiated at all, and the handshake never answers with Sec-WebSocket-Extensions: agreeing to an extension whose framing is not implemented would corrupt every message after the handshake, which is worse than not having it.
• No ping timer and no read deadline. Both want an Io and a task of their own, and a handler that wants them knows better than this library what they should be.
• No Origin checking. RFC 6454's header is passed through to the handler as an ordinary field, because whether a given origin may connect is a question about the application and not about the protocol.

What is refused, and with what

RFC 6455 calls all of these "failing the WebSocket connection": a Close frame carrying the code, and then nothing further is read.

Input	Answer
A frame from a client that is not masked (§5.1)	1002 Protocol Error
A reserved bit set with no extension negotiated (§5.2)	1002
An opcode in the ranges reserved for extensions (§5.2)	1002
A fragmented control frame, or one over 125 octets (§5.5)	1002
A continuation frame with nothing to continue (§5.4)	1002
A new data frame while a message is in progress (§5.4)	1002
A length encoded in more octets than it needed (§5.2)	1002
A Close frame carrying one octet (§5.5.1)	1002
A Close frame carrying a code §7.4.1 forbids sending	1002
A text message that is not valid UTF-8 (§8.1)	1007 Invalid Payload
A Close reason that is not valid UTF-8 (§5.5.1)	1007
A message larger than the buffer	1009 Message Too Big

Two of those are stricter than they had to be and are worth arguing.

A length encoded in more octets than it needed is a sender-side MUST in §5.2 that a lenient reader could let through. It is refused because two encodings of one length is the same shape as the two framing fields that make HTTP request smuggling work: it lets two implementations that both believe they are conformant disagree about where a frame ends.

UTF-8 is validated on the assembled message, not on each frame, which is the only answer that can be right: a code point split across a fragment boundary is valid in the message and invalid in both halves on its own. There is a test for exactly that, and another for the overlong encoding of /, which is the one that gets past a naive path check.

Building and testing

$ nix develop
$ zig build test --summary all
$ zig build            # the demonstration server, into zig-out/bin
$ zig build check      # compile the things no test builds

src/** holds the unit tests next to what they test, including RFC 6455 §5.7's frame examples and the §1.3 worked example of the handshake — the two places the document does the arithmetic for you.

tests/e2e.zig drives whole conversations through the server over zig-http's memory transport and asserts on the octets that come back, byte for byte wherever a frame is small enough to read. There is no socket and no port, and the Io is std.Io.failing: a library that reached for I/O it had not been given would fail loudly there.

The client's frames in that file are built by code that shares nothing with src/frame.zig. A frame writer checked against its own reader agrees with itself by construction; those two agree only because both were written from §5.2, and the test that anchors them is the masked "Hello" the RFC prints.

Fuzzing

tests/fuzz.zig holds four targets, each a property rather than an example:

• frames — a header the parser accepted is one it would write out the same way. The length has three encodings and only one is legal for any given length, so this is exactly the statement that the parser refused the other two, checked against every input rather than four examples.
• masking — §5.3's transformation is its own inverse, for any key, any offset and any division into chunks. The offset is the part that can be wrong, and a test that used one chunk would never reach it.
• conversations — whole conversations through the server, after which the output is parsed back: it must be a whole number of well-formed frames, none masked, none with a reserved bit, no data frame after a Close, and nothing left over. A server that can be made to emit a frame whose declared length does not match the octets after it has desynchronised the stream, and everything the peer reads afterwards is something the server never sent.
• handshakes — 101 is only ever answered to a request that really carried a well-formed key, and Sec-WebSocket-Accept is the token that key determines. That header is the only thing a client checks before it starts speaking a different protocol on the same socket.

The oracle the conversation target rests on has tests of its own, because an oracle that never fails is a test that never ran.

$ zig build fuzz-run -- --seconds 60
$ zig build fuzz-run -- --seconds 600 --target conversations

Zig 0.16.0 cannot build a test executable in fuzz mode, and leaves the fuzzer's coverage table empty once patched so that it can; flake.nix carries the one-word patch with the whole story in a comment, and tools/fuzz.zig is a loop written down honestly instead.

The virtual machine test

$ nix flake check --print-build-logs
$ nix build --print-build-logs .#checks.x86_64-linux.interop

tests/nixos/interop.nix runs the demonstration server on one machine and Python's websockets on another. That library is the implementation to be judged by: strict, widely deployed, and it reports a framing disagreement as an exception rather than quietly recovering from it.

It checks text and binary, every boundary of the three payload length forms, fragmentation, a code point split across a fragment boundary, ping and pong, a control frame interleaved with a fragmented message, the closing handshake, and subprotocol negotiation in all three of its outcomes.

Then it does the whole thing again over HTTP/2, using Python's h2 — a separate implementation with its own framing and its own HPACK. That half checks itself in a way worth knowing about: h2 refuses to send a :protocol pseudo-header unless the peer's SETTINGS actually carried ENABLE_CONNECT_PROTOCOL, so getting a tunnel at all is a statement that the server advertised it properly.

Finally it drops to a raw socket for the frames a conformant client will not send — an unmasked one, invalid UTF-8, a reserved opcode, a set reserved bit, and a message larger than the server's buffer — and checks that each is refused with the code the RFC names and that the server is still running afterwards.

API documentation

Generated from the doc comments, which carry most of the explanation in this library:

$ zig build docs
$ zig build docs-serve      # then read it at http://127.0.0.1:8000/

The server that serves it is zig-http, which makes reading the documentation the smallest honest test of the dependency.

References cited

Fette, I., and A. Melnikov. 2011. The WebSocket Protocol. RFC 6455. RFC Editor. https://www.rfc-editor.org/info/rfc6455. The framing of §5, the closing handshake of §7, and the worked examples of §1.3 and §5.7, which run as tests.

Barth, A. 2011. The Web Origin Concept. RFC 6454. RFC Editor. https://www.rfc-editor.org/info/rfc6454. The Origin header, which is passed through rather than judged.

McManus, P. 2018. Bootstrapping WebSockets with HTTP/2. RFC 8441. RFC Editor. https://www.rfc-editor.org/info/rfc8441. The extended CONNECT that this library reaches through zig-http's Response.upgrade.

Yoshino, T. 2015. Compression Extensions for WebSocket. RFC 7692. RFC Editor. https://www.rfc-editor.org/info/rfc7692. Not implemented, and cited because the handshake deliberately never agrees to it.

Yergeau, F. 2003. UTF-8, a Transformation Format of ISO 10646. RFC 3629. RFC Editor. https://www.rfc-editor.org/info/rfc3629. What §8.1 requires a text message to be.

Licence

MIT. See LICENSES/MIT.txt; the project follows the REUSE specification, so every file says so itself.