modem

XMODEM, YMODEM, ZMODEM and Kermit in Zig 0.16, built to sit between two std.Io streams as a filter.

Bytes are passed through unchanged until a ZMODEM frame header or a Kermit S packet appears in the remote stream, at which point the filter stops forwarding and receives the file. Sending works the other way: offer a file and the filter starts a transfer at the next opportunity.

lrzsz and C-Kermit are the references. Every ambiguity in Chuck Forsberg's and Frank da Cruz's specifications was settled by capturing what those programs actually do, and zig build oracle checks both directions of all four protocols against them.

The four protocols

They do the same job and agree on almost nothing about how.

	XMODEM	YMODEM	ZMODEM	Kermit
Files per session	one	many	many	many
Carries a file name	no	yes	yes	yes
Carries a length	no	yes	yes	yes
Receiver can decline	no	no	yes	yes
Resume part way	no	no	yes	no
Sender announces	no	no	yes	yes
Block check	sum, CRC16	CRC16	CRC16, CRC32	3 kinds
Sends 8-bit data raw	yes	yes	escaped	quoted
Packets in flight	1, or all	1, or all	all	1 to 31

Each has its own Receiver and Sender with the options its wire format actually needs, under modem.xmodem, modem.ymodem, modem.zmodem and modem.kermit. AnyReceiver and AnySender sit over the top for callers that pick a protocol at run time.

What "the sender announces" costs is worth spelling out, because it decides what a filter can do. An XMODEM or YMODEM sender says nothing at all until a receiver prompts it, so an incoming transfer cannot be recognised: somebody has to decide to go and fetch. ZMODEM and Kermit senders speak first, so those transfers start by themselves.

Using the filter

const modem = @import("modem");

var storage: [1 << 14]u8 = undefined;
var pipe_buffer: [8192]u8 = undefined;
var pipe: modem.Pipe = .init(io, &storage, &pipe_buffer);

var receive_buffer: [modem.receive_buffer_size]u8 = undefined;
var send_buffer: [modem.send_buffer_size]u8 = undefined;

var filter: modem.Filter = .init(
    io,
    .{ .in = &remote_in, .out = &remote_out },
    .{ .in = &local_in, .out = &local_out },
    my_sink,
    .{},
    &pipe,
    &receive_buffer,
    &send_buffer,
);

try filter.run(); // forwards both directions until `filter.stop()`

Received files go wherever the caller's Sink puts them. The library never touches the filesystem:

fn open(context: *anyopaque, info: modem.file.Info, offset: *u64)
    modem.Sink.OpenError!?*std.Io.Writer
{
    const self: *MySink = @ptrCast(@alignCast(context));
    // `info.name` comes from the peer. `baseName` strips any directory
    // prefix, so a sender cannot ask for `../../etc/passwd`. It is empty
    // for XMODEM, which carries no name at all.
    const name = info.baseName();
    if (name.len == 0) return null; // returning null declines the file

    self.file = try self.dir.createFile(self.io, name, .{});
    self.writer = self.file.writer(self.io, &self.buffer);
    return &self.writer.interface;
}

To resume an interrupted transfer, set offset.* to the number of bytes already held; the sender is asked to restart there. Only ZMODEM can do this.

Sending is queued from any task and awaited separately, so a user interface can offer a file while the filter is busy forwarding:

var upload: modem.Filter.Offer = .{
    .info = info,
    .source = source,
    .protocol = .ymodem,
};
try filter.offer(&upload);
switch (try filter.awaitOffer(&upload)) {
    .sent => {},
    .skipped => {},                 // the peer declined it
    .failed => return upload.err.?,
    .pending, .sending => unreachable,
}

Consecutive offers naming the same protocol are sent as one session.

An XMODEM or YMODEM download has to be asked for, since neither sender will speak first:

var download: modem.Filter.Request = .{
    .protocol = .xmodem,
    // XMODEM carries no name, so the receiving end picks one.
    .info = .{ .name = "download.bin" },
};
try filter.request(&download);
_ = try filter.awaitRequest(&download);

Either half of any protocol can also be driven on its own, without the filter, against any pair of std.Io streams: see each protocol's Receiver and Sender.

Command line

The modem binary exists mostly so the library can be tested against lrzsz and C-Kermit, but it is usable on its own:

modem recv [-p PROTO] [-d DIR] [-n NAME]   receive over stdin/stdout
modem send [-p PROTO] FILE...              send over stdin/stdout
modem filter [options]                     pass stdin/stdout through

-p takes zmodem (the default), xmodem, ymodem or kermit. -w sets how many Kermit packets may be in flight at once. -n names a file received over XMODEM, which carries no name of its own; the filter takes --send FILE to offer a file and --receive to fetch one.

Keeping the line busy

Only ZMODEM streams unconditionally: its sender talks until the receiver interrupts, so a round trip costs nothing. The other three wait to be acknowledged, and on a link with any real delay that leaves the line idle most of the time.

XMODEM and YMODEM have one answer between them, YMODEM-G, which drops acknowledgements entirely and gives up on the first error. Kermit has a better one: a window of up to 31 packets, with the receiver holding back anything that arrives early and the sender resending only what actually went missing. A single lost packet then costs one retransmission rather than a window's worth.

var receive_buffer: [modem.receiveBufferSize(16)]u8 = undefined;
var send_buffer: [modem.sendBufferSize(16)]u8 = undefined;

var receiver: modem.kermit.Receiver = .init(
    io, &pipe, &out, my_sink, .{ .window = 16 }, &receive_buffer,
);

The window costs a packet's worth of memory per slot at each end, so it is the buffer the caller hands over that decides how wide it can be: asking for more slots than the buffer holds narrows the window rather than failing, and so does a peer that will not go as wide. modem.receiveBufferSize and modem.sendBufferSize work out what a given window needs.

Why a Pipe sits in the middle

All four protocols are built on timeouts: a ZMODEM receiver that hears nothing for ten seconds prods the sender with a ZRPOS rather than waiting forever, an XMODEM receiver re-prompts every three, a Kermit receiver re-sends its NAK. std.Io.Reader has no notion of a deadline, and cancelling a read already in flight risks losing bytes the transport has handed over.

So a feeder task owns the blocking read and pushes whatever it gets into a ring buffer. The protocol engine consumes from the ring with a deadline it can change between frames, and giving up on a read costs nothing because the bytes stay buffered for whoever asks next. Pipe is that ring; it is the reason Filter.run needs real concurrency rather than Io.async.

What is implemented

XMODEM. Both block checks, 128 and 1024 byte blocks, the C/NAK handshake with a fallback from CRC to checksum, duplicate block detection and retransmission, and the two-CAN abort. Optionally strips the trailing SUB padding a sender adds to the final block, which recovers the original length of a text file and corrupts a binary one that genuinely ends in SUB.

YMODEM. Everything XMODEM has, plus block zero carrying the name, length, timestamp and mode; batches; exact truncation to the stated length; and YMODEM-G, which streams without acknowledgements and abandons the transfer on the first error, because that is all it can do.

ZMODEM. Binary and hex headers, 16- and 32-bit frame checks, ZDLE escaping including the control-character and carriage-return-after-@ rules, streaming with ZCRCG, windowed flow control with ZCRCW/ZACK for receivers that advertise a bounded buffer, ZRPOS error recovery, batch transfers, ZSKIP, resume at an offset, and the five-CAN abort.

Kermit. The S/F/A/D/Z/B/E/N/Y packets, all three block checks, control quoting, the 8th-bit prefix, run-length encoding with the repeat prefix, the full parameter exchange, extended packet lengths up to four kilobytes, attribute packets carrying the length and timestamp, and sliding windows of up to 31 packets.

What is not

ESC8, the ZCOMPRESS transport options, and ZCOMMAND — the last deliberately, since honouring it would hand the peer arbitrary code execution. It is always answered with ZNAK.

Kermit's locking shifts, RESEND and server mode.

Kermit's timestamps are written and read as UTC, while C-Kermit writes local time, so a timestamp exchanged with it is offset by the local zone. The value is handed to the caller's sink and nothing here acts on it; the alternative was carrying a timezone database for a field that is only ever displayed.

ZMODEM and YMODEM carry file positions in 32 bits, so files of 4 GiB and beyond cannot be described. That is a limit of the protocols.

Building

zig build              # the CLI
zig build test         # unit tests and in-process sender/receiver sessions
zig build oracle       # interoperability (cases skip if their tool is absent)

A flake.nix provides Zig 0.16, lrzsz and C-Kermit.