zig-mime

Email messages for Zig 0.17: reading them, writing them, and building them from nothing. RFC 5322 and the five MIME documents, with the thirty years of other people's software that came after them taken into account.

$ zig build
$ ./zig-out/bin/zig-mime tree message.eml
multipart/mixed 2 parts
  multipart/related 2 parts
    multipart/alternative 2 parts
      text/plain [quoted-printable] 55 bytes
      text/html [quoted-printable] 159 bytes
    image/png [base64] "logo.png" cid:[email protected] 38 bytes
  application/pdf [base64] "café.pdf" (attachment) 26 bytes

Three layers, and you can stop at any of them:

the codecs	base64, quoted-printable, encoded words — pure functions over bytes, no allocation you did not ask for
the headers	addresses, dates, media types, parameters — each header's own grammar, parsed and written
mime.Message	the whole message as a tree of parts, and the builder that makes one
zig-mime	a command-line tool, which exists to show what all that looks like from outside

The API documentation is generated from the doc comments, which are most of the explanation of why mail is the shape it is.

Where this lives

The repository lives in three places that carry the same history. The Forgejo instance at https://git.jcollie.dev/jeff/zig-mime is the web-visible one, and is where the continuous integration and the published documentation are:

$ git clone https://git.jcollie.dev/jeff/zig-mime.git

It is mirrored on Tangled at https://tangled.org/jcollie.dev/zig-mime, a forge built on the AT Protocol, where a repository is addressed by its owner's identity rather than by a server name.

It is also on Radicle, a peer-to-peer forge that needs no account on anything. A Radicle repository is findable only by its repository ID, so this is that ID:

rad:z4WSVdMPhsgpLHM5xaWeqpAMi33YC

and rad clone rad:z4WSVdMPhsgpLHM5xaWeqpAMi33YC fetches it from any node that seeds it. rad clone finds seeds through your local node's routing table, so the node has to be running first, and cloning seeds the repository in turn, which helps keep it available:

$ rad node start
$ rad clone rad:z4WSVdMPhsgpLHM5xaWeqpAMi33YC

If you already have the repository and only want to help host it:

$ rad seed rad:z4WSVdMPhsgpLHM5xaWeqpAMi33YC

Any of the three is the whole project.

Adding it to a project

It needs Zig 0.17.0. The development shell and the Nix package take it from nixpkgs unstable, as zig_0_17.

$ zig fetch --save git+https://git.jcollie.dev/jeff/zig-mime.git

For Zig 0.16.0, use the zig-0.16 branch, which holds the last of this library to build with it:

$ zig fetch --save git+https://git.jcollie.dev/jeff/zig-mime.git#zig-0.16

The v0.1.0 tag also predates the move to 0.17 and builds with 0.16.0.

const mime = b.dependency("zig_mime", .{ .target = target }).module("mime");
exe.root_module.addImport("mime", mime);

Reading a message

const mime = @import("mime");

var message: mime.Message = try .parse(gpa, source, .{});
defer message.deinit();

std.debug.print("{s}\n", .{(try message.root.subject()) orelse "(no subject)"});

var from = try message.root.addresses(gpa, "from");
defer from.deinit(gpa);
for (from.addresses) |address| std.debug.print("{f}\n", .{address});

if (message.root.findText(&.{ "plain", "html" })) |part| {
    const text = try part.textAlloc(gpa);   // decoded, and UTF-8
    defer gpa.free(text);
    std.debug.print("{s}", .{text});
}

var attachments = message.root.attachments();
while (attachments.next()) |part| {
    const bytes = try part.decodeAlloc(gpa);
    defer gpa.free(bytes);
    std.debug.print("{s}: {d} bytes\n", .{ (try part.filename()).?, bytes.len });
}

A part will also say what it knows about the file it carries — RFC 2183's size, creation-date and modification-date — which is worth having when saving one to disk:

if (try part.modificationDate()) |when| { ... }
if (try part.declaredSize()) |size| { ... }

size is a claim: the sender wrote it, it is documented as approximate, and nothing checks it. Use it to decide whether to offer a download or how to draw a progress bar; a receiver that uses it to size a buffer it then fills from the body has written the bug this parameter is best known for. decodeAlloc is what actually knows. Writing them is DataOptions.size, .creation_date and .modification_date, none of which is filled in for you — size is a claim to make deliberately, and setDataStreaming could not know it in any case.

findText and attachments know two rules that find and walk do not, and that everybody writing this by hand gets wrong once. An attachment is skipped when looking for the body even when it is text/plain, because a notes.txt somebody attached is not the message. And a preference is asked for in full before the tree is searched again, so &.{ "plain", "html" } gives back plain text even when the HTML part comes first — which is the difference between plain text and a page of HTML.

A parsed message written back out is the message that went in, byte for byte. Bodies are kept in the transfer encoding they arrived in and header values in wire form, folds and all, so the round trip preserves the preamble, the epilogue, the exact position of every line break, the boundary as written, and even the line ending a truncated message was missing. That is what makes it safe to parse a signed message and forward it: the signature still verifies. Decoding happens when it is asked for.

The one thing not preserved is a message whose lines end in LF alone. Mail is CRLF; a message read out of a Unix file often is not, and its headers are written back out canonicalised. That is a repair rather than a loss, and it is the only one.

Building a message

var message: mime.Message = try .init(gpa);
defer message.deinit();

try message.root.setAddresses("From", &.{
    .{ .display_name = "Ollie, Jeff", .local_part = "jeff", .domain = "example.com" },
});
try message.root.setAddresses("To", &.{
    .{ .display_name = "José García", .local_part = "jose", .domain = "example.net" },
});
try message.root.setSubject("Grüße aus München");
try message.root.setDate(now);
try message.root.setMessageId("example.com", random);
try message.setText("Schöne Grüße.\n", .{});

_ = try message.attach(random, pdf_bytes, .{
    .filename = "Quartalsbericht für München.pdf",
});

try message.write(writer);

What comes out is US-ASCII, in lines no longer than the rule allows, with the display name quoted because it has a comma in it, the subject in RFC 2047 encoded words split so that no word exceeds seventy-five characters and no UTF-8 character is cut in half, the body in quoted-printable because it is mostly text, the attachment in base64 because it is not, and the filename in RFC 2231 continuations because it is neither short nor ASCII. None of those decisions is one the caller has to make.

The attachment's media type was not given and did not need to be: it is taken from the end of the filename, which is what the sender knew all along and what would otherwise have been thrown away at the last moment. A file that arrives as application/octet-stream gets no preview, no icon and no "open with". Pass content_type where the type is actually known — it always wins over the guess — and expect application/octet-stream for an extension the table has never heard of, which is the correct thing to say when you do not know.

attach turns a message that already has a body into a multipart/mixed, moving the body and every Content-* header that described it down into a part of their own — which is what those headers describe, and moving them is the difference between an attachment and a corrupted message.

A multipart built by hand — makeMultipart, makeRelated, makeReport — takes the caller's boundary and refuses one that RFC 2046 §5.1.1 does not allow with error.InvalidBoundary: empty, longer than seventy characters, ending in a space, or holding anything outside its short list of characters. Message.isValidBoundary asks the same question beforehand. The other half of the rule, that the boundary must not appear inside the parts, is the caller's to keep, since there are no parts yet when it is chosen. The *Random forms generate one with 128 random bits behind a =_ that encoded content cannot contain, and are what to use without a reason to choose.

HTML, and the images inside it

// The identifier has to exist before the HTML that refers to it does.
const logo = try message.newContentId("example.com", random);

var html: std.Io.Writer.Allocating = .init(gpa);
defer html.deinit();
try html.writer.print("<p>Grüße!</p><img src=\"cid:{s}\">", .{logo});

try message.setAlternative(random, "Grüße!\n\n[logo]\n", html.written(), .{});
_ = try message.embed(random, png_bytes, .{ .filename = "logo.png", .content_id = logo });
_ = try message.attach(random, csv_bytes, .{ .filename = "Bericht.csv" });

Four calls, and what comes out is the five-level structure that every mail client expects and that almost nobody assembles correctly by hand:

multipart/mixed
├── multipart/related              type="multipart/alternative"
│   ├── multipart/alternative
│   │   ├── text/plain
│   │   └── text/html
│   └── image/png                  inline, Content-ID: <[email protected]>
└── text/csv                       attachment

Each of those levels is load-bearing:

• The plain text comes first. RFC 2046 §5.1.4 puts alternatives in increasing order of preference and a reader shows the last one it can display. Reversed, every HTML client shows the plain text instead.
• The image is inline and inside the related, not beside it. A resource the body refers to is not an attachment, and a client that lists it as one shows a signature logo next to the file somebody actually meant to send.
• multipart/related carries a type parameter naming the media type of its root part. RFC 2387 §3.1 requires it, because a reader has to know which of the parts is the document before it has looked at any of them. embed fills it in from what the body already was.
• attach wraps rather than appends. Adding the spreadsheet to the alternative would claim it was a third rendering of the message — so a client showing the last alternative it understands would display the spreadsheet instead of the text.

Reading it back, findContentId resolves a reference in whichever form the HTML happens to hold it — cid:[email protected], <[email protected]> or bare:

if (message.root.findContentId(src_attribute)) |image| {
    const bytes = try image.decodeAlloc(gpa);
    ...
}

A message can also be forwarded whole rather than quoted, which is message/rfc822 and one call:

_ = try message.attachMessage(random, original, .{});

Replies, and the thread they belong to

try reply.root.setMessageId("example.com", random);
try reply.setInReplyTo(&parent, .{});

That writes In-Reply-To with the parent's Message-ID and References with the parent's whole ancestry followed by the parent itself, which is the entire mechanism behind every threaded mailbox view. Reading it back is an iterator over the identifiers, oldest first:

var it = try message.root.references();
while (it.next()) |id| ...

References grows by one identifier per reply and RFC 5322 §3.6.4 says nothing about what to do when it gets long. The convention, inherited from netnews, is specific about which entries to drop: the first is kept and the trimming is done from the second onwards. The first identifier is the message that started the thread and is what a reader groups the conversation by; the last few are the immediate ancestry and are what it nests replies with; everything between is the part nobody looks at. max_references is where that limit lives.

Identifiers are found by their angle brackets, so a References header carrying commas, comments or stray text still threads — and a value with no brackets at all falls back to taking the whitespace-separated tokens that look like addresses, because the client that writes In-Reply-To: [email protected] exists and its threading should still work.

Wrapped text that a reader may rewrap

try message.setText(paragraphs, .{ .format = .flowed });

text/plain; format=flowed — RFC 3676 — is the fix for the ragged long-short-long-short that plain text mail turns into when the reader's window is narrower than the sender's wrap width. A line that ends in a space is a line that was wrapped, and the reader may join it to the next and rewrap the result; a line that does not is a line the author ended. That is the whole format, and a reader that has never heard of it sees ordinary wrapped text.

Each line of the text handed to setText is then a paragraph — what the author typed, with the breaks the author made — because text that has already been hard-wrapped no longer says which of its breaks were the author's.

Reading is automatic: textAlloc joins a flowed body back into its paragraphs, since the line breaks it removes are ones the sender's composer put in rather than ones the sender typed. decodeAlloc is still the way to the body exactly as it arrived.

Three details hang off the one idea, and src/flowed.zig has them: a line that would otherwise begin with a space is given another one that the reader removes (space-stuffing, without which the format could not represent an indented line at all); two lines join only if they are quoted to the same depth; and delsp=yes says the trailing space is only a marker and is deleted on joining, which is what a script that does not separate its words with spaces needs. Plus one exception older than the format: a line that is exactly -- is the signature separator, ends in a space, and is not a wrapped line.

Writing flowed text upgrades an automatic 7bit to quoted-printable, because everything the format says it says with trailing whitespace and RFC 2045 warns that trailing whitespace is exactly what a transport may strip. Pass .encoding = .seven_bit to decide otherwise.

Large attachments

_ = try message.attachStreaming(random, &file_reader.interface, .{
    .filename = "recording.mp4",
});

and, reading, the other direction:

try part.decodeWriter(&file_writer.interface);

Neither holds the file. attachStreaming encodes as it reads, so the plain bytes are never all in memory; decodeWriter decodes as it writes, so a hundred megabyte attachment reaches a file with a sixty-four kilobyte buffer in between and nothing else. zig-mime extract is written that way, and is the worked example.

What attachStreaming still holds is the encoded body, because the message tree owns it: a body that stayed on disk until the message was written could only be written once, and a Message is something you can write twice. The exception is an explicit .quoted_printable, which buffers its input after all — whether a space may be written as itself depends on how far the rest of its run reaches and on what follows it, so an encoder fed a chunk at a time would have to buffer that anyway. The default for a file is base64, which streams.

A std.Random is passed in rather than reached for, because Zig 0.17 gets its randomness through std.Io and a library that went looking for a global would be reaching around the caller's back:

var source: std.Random.IoSource = .{ .io = io };
const random = source.interface();

Reports: bounces, receipts and complaints

Three quite different things arrive in the same envelope. A multipart/report — RFC 6522 — is two or three parts, of which only one is worth parsing:

multipart/report                report-type=delivery-status
├── text/plain                  the apology
├── message/delivery-status     what actually happened
└── message/rfc822              the message that failed

The apology is the part everybody reads and the part nobody can parse: every mail system words it differently and half of them word it badly. The second part is the same information in a format a program can act on, and its report-type says which of three it is:

report-type	what it is	what reads it
delivery-status	a bounce (RFC 3464)	Part.deliveryStatus
disposition-notification	a read receipt (RFC 8098)	Part.dispositionNotification
feedback-report	a spam complaint (RFC 5965)	Part.feedbackReport

All three write their contents the same way — groups of Name: value in header syntax, folded by the same rules — which is why the same code reads them and why Headers.parse was already most of it. A bounce is the one with more than one group, because it is about a message and then about each recipient of it in turn.

if (try message.root.deliveryStatus(gpa)) |found| {
    var status = found;
    defer status.deinit();

    for (status.recipients.items) |*recipient| {
        const who = (try recipient.finalRecipient()).?.value;
        switch (try recipient.action()) {
            .failed => if ((try recipient.status()).?.isPermanentFailure()) {
                // Stop mailing this address.
            },
            .delayed => {},   // Still being retried; a later report will say.
            else => {},
        }
        _ = who;
    }
}

Ask it of the message rather than of the report part: a bounce is a multipart/report at the top, but a bounce that has itself been forwarded is one somewhere further down, and the whole tree is searched either way.

Two fields carry the answer, and it is worth reading the right one. Action says what happened in a word, and only failed is permanent — delayed is a warning that a later report will resolve, and a great many of them resolve themselves. Status is an RFC 3463 code like 5.1.1, whose first number is the part to switch on: 2 succeeded, 4 is temporary, 5 is permanent. A program that matches 5.1.1 exactly will be surprised by the server that says 5.0.0 and means the same thing, and one that reads Diagnostic-Code is reading a remote server's own prose.

The report's body is header syntax — groups of Name: value separated by blank lines, folded the same way — which is why it is parsed by the same code as a header block, and why a Diagnostic-Code folded across three lines comes back as one value.

Part.report() gives the three parts by name rather than by position, so a mail system that puts its message/delivery-status third has still sent a bounce this can read. The original may be a whole message/rfc822 or just its headers as text/rfc822-headers, and Part.rfc822HeadersAlloc is the second case. Part.makeReport builds one.

A read receipt says what became of a message after it arrived, which is a different question from whether it arrived. It is a claim rather than evidence: it is generated by the recipient's software, sent only if that software chooses to, and displayed means a mail program put the message on a screen rather than that a person read it. The format says so itself — manual-action means somebody agreed to send the receipt and automatic-action means nobody was asked, which is the closest it comes to admitting how little it can promise.

A spam complaint arrives through a feedback loop when somebody presses "this is spam", and what a sender does with one is unsubscribe the complainant without asking. The complainant's address is usually not in it — RFC 6590 has providers redact it — so the envelope sender in Original-Mail-From, and the sender's own tracking identifier in the third part, are what identify the subscription. A report with no recipient in it is the normal case.

zig-mime headers prints the summary for all three, which is the shortest way to see what a report in a file actually says:

$ zig-mime headers bounce.eml
  reported by: mx.example.net
  [email protected]: failed 5.1.1
    550 5.1.1 <[email protected]>: User unknown

$ zig-mime headers receipt.eml
  disposition: displayed (nobody was asked)
  about: <[email protected]>

mailto: links

var message = try mime.mailto.parse(gpa, url, .{});
defer message.deinit();

RFC 6068. Everything after the ? is header fields, percent-encoded, with one name that is not a header — body, which is the text — and the addresses before it are recipients. mailto.writeUrl goes the other way.

Two details that are got wrong more often than they are got right:

• + is not a space. That rule belongs to HTML form encoding; a mailto: is a URI, where a space is %20 and a + is a plus sign — a character in a great many real addresses. Converting it silently mails somebody else.
• The header list is not open. A mailto: arrives from a web page and is attacker-supplied by construction: one that sets From is a forged sender, and one that sets Content-Type can make a reader display something other than what the URL appears to say. Options.allowed defaults to the fields a link plausibly needs and nothing else. Widening it is a decision, which is why it has to be made by the caller.

$ zig-mime mailto 'mailto:[email protected]?subject=Lunch&[email protected]'
Subject: Lunch
To: [email protected]
Content-Type: text/plain;
 charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0

Streaming

Message.parse takes a slice, so a message has to fit in memory to be opened. That is the right shape for nearly everything — a message is usually small, and a tree you can walk and write back out byte for byte is worth more than the memory it costs. mime.Parser is the other shape: a pull parser over an Io.Reader, which never holds the message at all.

var parser: mime.Parser = .init(gpa, reader, .{});
defer parser.deinit();

while (true) switch (try parser.next()) {
    .begin => |d| _ = d,
    .field => |f| std.debug.print("{s}: {s}\n", .{ f.name, f.value }),
    .headers_end => if (parser.mediaType().is("application", "pdf")) {
        // parser.transferEncoding() says how to decode what follows.
    },
    .body => |chunk| try out.writeAll(chunk),
    .end => |d| _ = d,
    .eof => break,
};

Nesting arrives flattened: a multipart/mixed produces begin, its headers, and then a begin for each of its parts in turn, each ended before the next begins. begin and end carry the depth, which is what a caller reconstructing a tree needs and what a caller looking for one part can ignore.

Bodies arrive in their transfer encoding, as everywhere else here, and transfer_encoding.Decoder is what turns the chunks back into bytes without holding any of them — it keeps the few bytes of state that span a chunk boundary, a partial base64 group or a half-finished quoted-printable escape:

var decoder: mime.transfer_encoding.Decoder = .init(parser.transferEncoding());
defer decoder.deinit(gpa);
// ...on each `.body` chunk:
try decoder.write(gpa, file_writer, chunk);
// ...and at `.end`:
try decoder.finish(gpa, file_writer);

Each codec has one decoder, and the slice forms are wrappers over it — base64.decode is base64.Decoder with the whole input as one chunk. Two decoders for one encoding drift apart the first time either is touched, and the drift shows up as an attachment that comes out right when it is read one way and wrong when it is read the other. Uuencode is the exception and buffers: it is line oriented and its begin line may sit anywhere after a covering note, so a decoder that could not look back would decode the note as data.

zig-mime tree --stream and zig-mime extract --stream are the worked examples, and the difference is measurable. On a 27 MB message with a 20 MB attachment, writing the same bytes out:

$ zig-mime extract          big.eml out    # 58 MB peak
$ zig-mime extract --stream big.eml out    #  5 MB peak

The second figure is the two 64 KB buffers and the process itself; it does not move when the message gets bigger.

A streaming read is not a round trip. It discards what a reader does not need: the preamble and epilogue of a multipart, the exact spelling of the line endings, the whitespace a fold was broken at. Use Message.parse when the bytes matter — when a signature has to keep verifying, or when the message is going out again. Header values also arrive unfolded, unlike Headers, which keeps them in wire form because it has to write them again; a streaming caller cannot ask later, so it is given the form it wants.

The one structural thing it will not do is open a message/rfc822 that is transfer encoded, because decoding it would mean holding it. RFC 2045 §6.4 forbids encoding one; a part that does it anyway arrives as a leaf, and its bytes are the caller's to decode and parse again.

Both parsers are checked against each other rather than separately — see the fuzzing section, where that is most of the point.

The codecs

const encoded = try mime.base64.encodeAlloc(gpa, bytes, .{});      // wrapped at 76
const decoded = try mime.base64.decodeAlloc(gpa, encoded);         // lenient
_ = try mime.base64.decodeStream(reader, writer);                  // no copy of either

const qp = try mime.quoted_printable.encodeAlloc(gpa, text, .{});
const subject = try mime.encoded_word.decodeUtf8Alloc(gpa, header_value);

Each of them is the MIME dialect rather than the textbook one, and the differences are the whole point:

• base64 wraps, and decodes leniently. RFC 2045 §6.8 says a body is broken into lines of at most 76 characters and that "any characters outside of the base64 alphabet are to be ignored" — which is how the line breaks get in, and how a decoder survives the stray > of a badly quoted forward. std.base64 does neither. decodeStrict is there for a caller who would rather be told the attachment is damaged than handed the damage.
• quoted-printable has to know whether the body is text. In .text a CRLF is a line break and stays one; in .binary it is two octets and becomes =0D=0A, so that a body survives a transport that rewrites line endings.
• An encoded word is a token, not a substring. =?utf-8?q?x?= inside a longer atom is not one, and a subject reading "look at =?this?=" must survive being written out and read back. The whitespace between two adjacent encoded words is not whitespace — it is how a long run was folded — and the whitespace next to ordinary text is.

Uuencode

Content-Transfer-Encoding: x-uuencode is not in RFC 2045 and never was, and it is why an archive of mail from before about 1996 has attachments a strict MIME reader cannot open. It is spelled five ways, all of them accepted here, and it is decoded through zuucode.

Nothing writes it by choice — transfer_encoding.choose never returns it, and base64 is what replaced it — but .encoding = .uuencode will, for a message being built to be read by something old.

Two things about it are worth knowing:

• The body names itself. The begin 644 picture.gif line carries a filename, and a message old enough to be uuencoded is usually old enough to predate Content-Disposition as well, so that line is frequently the only place the name appears. Part.filename falls back to it, after the two headers that should have had it.
• Decoding stops where the damage starts. Everything before the begin line is skipped, so a body with a covering note in front of it works; a line that will not decode ends the body, as does the empty line a transport leaves behind when it strips the terminator's single space. What decoded, decoded — the same decision the base64 decoder makes, for the same reason.

Mail from before MIME put uuencoded files straight into an ordinary text/plain body, with no Content-Transfer-Encoding to describe them, no Content-Disposition to name them, and usually a covering note in front. To a MIME reader the attachment is invisible: the body is text, and the text happens to contain the file. Those messages are still in everybody's archives.

Nothing here restructures such a message, because no header says it has happened and rebuilding a tree on a guess is worse than leaving it alone. What there is instead is the explicit question:

if (try part.uuencodedFile(gpa)) |file| {
    defer file.deinit(gpa);
    // file.name, file.mode, file.data
}

Only a text body is examined — or one that declares uuencode, so the same question answers both shapes — because a begin line found by chance inside a decoded JPEG is not a file. zig-mime extract asks it, which is how a pre-MIME message's attachment comes out of a body nothing else would look in:

$ zig-mime extract 1994.eml out
script.sh (3 bytes, uuencoded in the body)

zuucode is also re-exported as mime.zuucode, for anything this does not cover.

The headers

var list: mime.address.List = try .parse(gpa, "\"Ollie, Jeff\" <[email protected]>, [email protected]");
defer list.deinit(gpa);

var media: mime.MediaType = try .parse(gpa, "multipart/mixed; boundary=\"----=_Part_1\"");
defer media.deinit(gpa);

const sent = try mime.date.parse("Tue, 01 Jul 2003 10:52:37 +0200 (CEST)");

Addresses are the difficult one, and the comma is why: it separates addresses, and it is the most common character in a display name, and a group holds commas of its own between a : and a ;. The entries cannot be found by splitting — the parser has to know where the quotes, the angle brackets, the domain-literals, the comments and the groups begin and end first. Splitting on commas is why so much software shows a message from "Ollie addressed to Jeff" <jeff@…>.

Display names come back decoded, because that is the only form anything wants to display. The local part and the domain come back as written, because they are identifiers: an address is compared, stored and sent to. Mailbox.isValid is the question a program about to send something should ask, and it is also exactly the condition under which writing an address back out reproduces it byte for byte — the parser accepts far more than that, because a message that has already arrived has to be filed whatever is in its To:.

Internationalized domains

bücher.example and xn--bcher-kva.example are the same domain written two ways. The domain name system carries ASCII, so the second is what goes on the wire; the first is what a person should be shown.

const ascii = try mailbox.domainAsciiAlloc(gpa);    // xn--bcher-kva.example
const shown = try mailbox.domainUnicodeAlloc(gpa);  // bücher.example

Converting between them is not simply Punycode. UTS #46 is a mapping as well as an encoding — case folding, Normalization Form C, a table of code points that are disallowed outright, and rules about joiners and bidirectional text — so that Bücher.example and bücher.example produce one A-label and a domain nobody could register is refused rather than encoded. That is zig-uri's uri_unicode.idna, and mime.idn is the mail-shaped layer over it.

Punycode applies to the right of the @ and nowhere else. An address whose local part is not ASCII cannot be encoded at all and needs a transport that speaks SMTPUTF8 (RFC 6531) end to end, which is a different question from whether the domain needs converting:

if (recipients.requiresSmtpUtf8()) {
    // Nothing can be done about this one but find a transport that does it.
}

domainAsciiAlloc fails rather than guessing, because an address that cannot be encoded cannot be delivered to and silently sending something else would deliver it to somebody else. A domain that is already ASCII is copied rather than converted — putting an existing ASCII domain through IDNA would refuse some that already work.

Showing the U-label is a decision, not a default. раypal.com with a Cyrillic а and paypal.com with a Latin one look identical in most fonts and are different domains. Browsers decide with a script-mixing heuristic and show the A-label when a name looks suspicious; nothing here does that for you, and nothing here silently prefers one form over the other. Both are available, and which one a reader shows is the reader's problem to think about.

Dates come from zig-datetime, whose rfc822 module already speaks the syntax; mime.date is the layer between that and a header field, which removes the comments a header may carry anywhere and forgives the two things a sender's clock gets wrong — a day name that contradicts the date, and a missing zone.

Charsets

This library does not convert charsets. It reports the charset a message declared and hands over the bytes; turning Shift_JIS or windows-1252 into Unicode belongs beside whatever a program already uses for text rather than buried inside its mail parser. encoded_word.Iterator and Parameters.getParameter are where the name comes back together with the bytes, so a caller with a transcoder of its own has everything it needs.

The convenience functions — textAlloc, getUtf8, decodeUtf8Alloc — handle the part that needs no mapping at all: UTF-8 and US-ASCII, which are already the answer, and ISO-8859-1, whose 256 bytes are exactly the first 256 Unicode code points. Anything else is passed through with invalid sequences replaced, so the result is always valid UTF-8 and a Shift-JIS subject degrades to something legible-ish rather than to an error in the middle of a mailbox listing.

It used to be possible to put this more strongly — that the library carried no tables of any kind — and that is no longer true. Internationalized domain names need UTS #46, and UTS #46 needs Unicode character data, so building this library pulls in zig-uri's IDNA support and the character tables under it. Those are properties of code points rather than a mapping out of some other encoding, which is a real distinction and not a large one: the honest position is that Unicode data is now in the tree, and charset conversion is still absent because it is somebody else's job and not because it would be expensive.

If that trade is wrong for you, the line is in build.zig, where the module asks zig-uri for its uri_unicode module with unicodeModule. zig-uri keeps the tables in that separate module and fetches uucode only for a dependent that asks for it, so dropping the call leaves everything but mime.idn compiling — Zig analyses only what is referenced, so a program that never converts a domain never reaches the two functions that need the tables, and zig build succeeds without it even though zig build test does not.

Injection

A header value containing a bare CR or LF is how a program that interpolates user input into a header becomes a program that sends whatever headers an attacker likes — a Subject ending in \r\nBcc: everyone adds a recipient. Everything here that writes a value strips bare CR and LF, and the address writer drops every control character from a local part and a domain. It is not a configurable strictness: no legitimate value contains one, so nothing is lost, and the failure mode of getting it wrong is not a malformed message but a security hole.

A multipart boundary is the one value written into the body rather than a header, on a delimiter line of its own, and a line ending inside it would put whatever followed at the top of the next part — where it reads as that part's headers. That is why a boundary with anything outside RFC 2046's characters is refused rather than written.

The same care applies in the other direction. A filename out of a message is attacker-controlled, and zig-mime extract writes a part under its suggested name only if that name is a plain one — no separators, no .., no leading dot.

The command-line tool

$ zig-mime tree    message.eml     # the structure, one part per line
$ zig-mime tree --stream message.eml   # the same, never holding the message
$ zig-mime headers message.eml     # the headers, decoded
$ zig-mime text    message.eml     # the text, decoded to UTF-8
$ zig-mime extract message.eml out # every attachment, into a directory
$ zig-mime extract --stream message.eml out   # the same, holding nothing
$ zig-mime build   -                # a message assembled from the library
$ zig-mime mailto  '<url>'         # the message a mailto: link asks for

A file of - is read from standard input.

Trace fields, for a server taking delivery

A system that accepts a message adds a Received: saying so, and where it goes matters as much as what it says: each hop prepends its own, so the list reads newest first and the path the message took can be read down it.

try message.addReceived(.{
    .from = helo_name,                      // what the peer said in EHLO
    .from_info = "client.example.com [192.0.2.1]", // what we observed
    .by = "mx.example.net",
    .by_info = "zig-mime",
    .with = mime.received.protocolFor(.{ .tls = true, .authenticated = auth }),
    .id = queue_id,
    .for_recipient = single_recipient,      // only when there is exactly one
    .received_at = now,
});

Received: from client.example.com (client.example.com [192.0.2.1])
	by mx.example.net (zig-mime) with ESMTPSA id 3F2A1C0A
	for <[email protected]>; Tue, 01 Jul 2003 10:52:37 +0200

Three things this gets right that are easy to get wrong by hand:

• Nothing already in the message moves. RFC 5321 §4.4 does not let a system adding a trace field alter the ones already there — not reorder them, not refold them — and the whole value of a trace is that each line is exactly what the hop that wrote it wrote. The new field is inserted and nothing else is touched.
• A Return-Path: stays above it. RFC 5322 §3.6.7 gives a trace block as an optional Return-Path: followed by its Received: lines, and the Return-Path: is written by the system making final delivery. A Received: above one claims the message was received after it was delivered.
• The peer cannot end the field. Almost every clause comes from the client — the from name is whatever it said in EHLO — so a bare CRLF in it would otherwise start a header of the sender's choosing in every message the hop relays. Control characters and stray parentheses are dropped on the way in, and a fuzz target puts arbitrary bytes into every clause and checks that one field comes back out.

mime.received.protocolFor is there because ESMTPS, ESMTPA and ESMTPSA are the only part of a trace a later reader can use to tell an authenticated submission from an anonymous relay, and getting them the right way round by hand is a thing people get wrong. for_recipient takes one recipient, not a list, for the reason RFC 5321 §4.4 gives: a message delivered to several people with all of their addresses in the trace tells each of them who else received it, which is Bcc: broken by the transport rather than by the sender.

This is a header, not a protocol — see What is deliberately not here — but it is the header an SMTP server cannot avoid writing, so the placement rules live here rather than in every server.

Signing

DKIM is zig-dkim's, not this library's: it takes the raw bytes of a message, hashes a canonicalised form of them, and hands back the header field to put in front. The two compose at the byte level and neither depends on the other.

const raw = try message.toBytesAlloc(gpa);
defer gpa.free(raw);

var buffer: [dkim.signing.max_field_len]u8 = undefined;
const field = try dkim.sign(&buffer, raw, .{
    .domain = "example.com",
    .selector = "mail",
    .key = .{ .ed25519 = pair },
    .timestamp = now,
});
// The signed message is `field` followed by `raw`, unchanged.

What this library owes it is the byte-for-byte round trip, and that is what makes the pairing work: a signature covers the exact octets of a set of headers and a body, so a message that comes back from a parser one space different is a message whose signature no longer verifies. tests/dkim.zig checks exactly that — a message built here, signed, parsed back, written out again, and still verifying — and it is the only place the round trip is checked against something that would notice. zig build test -Ddkim runs it; the dependency is lazy and off by default, so a program that only reads mail does not fetch a crypto library to do it.

To keep the field in the tree rather than writing it out in front, use prependRaw and not prepend:

const colon = std.mem.indexOfScalar(u8, field, ':').?;
try message.root.headers.prependRaw(
    field[0..colon],
    std.mem.trim(u8, field[colon + 1 ..], " \t\r\n"),
);

prepend folds what it is given, which is right for a value a program composed and wrong for one that was composed somewhere else: the signature covers the field as it stands, and refolding it breaks verification under simple canonicalization.

Two rules follow from the same fact, and neither is enforceable here:

• Sign the bytes you are going to send, and do not touch the message afterwards. Adding a header is safe if it is not one the signature listed; changing one that it did, or re-encoding a body, is not.
• Verification needs the bytes as they arrived. Message.parse keeps them, so a message parsed from a slice can be handed straight to a verifier. mime.Parser does not — it holds one line at a time by design — so a streaming read cannot also verify unless the caller keeps a copy of what went past.

What is deliberately not here

• No SMTP, IMAP or POP. This library is about the message, not about moving it. Everything it produces is bytes, and bytes are what those protocols take. addReceived is the one concession, and it is still only a header: an SMTP server has to write a Received: in a particular place, and that placement is a property of the message format rather than of the protocol.
• No S/MIME or PGP, and no DKIM either — though DKIM is a different kind of absence, since zig-dkim does it over the bytes this library produces and the section above is how they fit together.

Fuzzing

tests/fuzz.zig holds nineteen targets, each a property rather than an example: whatever arrives, the parser terminates, stays inside its buffers, frees what it allocated, and — if it claims to have understood the input — writing it back out and reading it again gives the same answer.

$ zig build test                    # the properties, over the checked-in seeds
$ zig build fuzz --fuzz             # Zig's fuzzer, until interrupted
$ zig build fuzz-run                # a minute of each, with generated input
$ zig build fuzz-run -- --seconds 300 --target address
$ zig build fuzz-run -- --input fuzz-findings/x.bin --target address

There are two ways to run them. zig build fuzz --fuzz is Zig's own fuzzer, with coverage feedback, which needs the test binary compiled by LLVM rather than the self-hosted backend Debug uses by default; build.zig sees to that, and -Dfuzz-filter=address narrows it to one target. zig build fuzz-run is a loop of our own, described in tools/fuzz.zig: no coverage, but a corpus of real messages to mutate and a run that is the same on every machine for a given seed, which is what the workflow uses.

Two of those targets go the other way from the rest. build composes a message — a subject, a text body, an attachment with a name — writes it out, parses it back with this same library and asks for each of them again; it is the property a program that sends mail depends on, and nothing about the wire form is asserted, because the encoding chosen and the wrapping and the boundary are all free to change while the content is not. report does the same for the bodies of bounces, read receipts and spam complaints, which arrive written by a system the recipient has never met. chunks feeds a body to the streaming decoders split in every way the input asks for — one byte at a time included — and demands the same output each time, because the joins are where a decoder that streams differs from one that does not, and a reader whose buffer fills at the wrong byte is what finds the difference. encode-stream is its mirror: attaching a file without reading it whole reaches a different encoder from attaching one that was, and the fuzzer is what keeps the two writing the same bytes. headers-edit edits a block somebody else wrote — set, remove, prepend — which is what a program handling mail does constantly and what the reading targets never reach.

It has found thirteen things worth having found, most within the first few hundred thousand inputs:

• The quoted-printable encoder could lose a space. A run of whitespace that ended within three columns of the seventy-sixth got a soft line break inserted into the middle of it — and the decoder deletes the whitespace in front of a soft break, because that is what the rule says. Invisible in every small test; the round-trip property caught it.
• getUtf8 could return invalid UTF-8. A header with no encoded words in it was passed through unchanged, so a Subject: carrying a sender's raw ISO-8859-15 bytes came back as something a UTF-8 consumer three layers away would choke on.
• A multipart with no delimiter in it grew by two bytes every time it was opened and saved.
• A local part could carry a newline into the header. A malformed address parsed out a local part containing a CR, and the writer wrote it straight back — which is header injection, arrived at from the parsing side rather than the building side.
• Four scans over an address disagreed about which stretches were opaque. One skipped domain-literals and another did not, so a ( inside […] became a comment, a < inside one became an angle-addr, and a : inside one became the end of a source route. Each of those changed what the address was.
• A flowed wrap point could invent a signature separator. A paragraph beginning -- wrapped at a narrow width produced a line whose content was exactly -- , which a reader treats as fixed — so the paragraph stopped joining and everything after it became a signature.
• The two parsers disagreed three ways. mime.Parser and Message.parse are asked to describe the same message and the descriptions are compared, which is most of why a second parser is safe to have at all. It found that the tail of a header line too long for the reader's buffer was being read as a field of its own — and that a colon past the end of that buffer was missed entirely; that a lone CR at the end of a line was read as a fold, because the real line ending had already been taken away and the CR had come to sit beside the next line's whitespace; and that an empty message/rfc822 body produced a part in the tree that nothing had sent.
• Folding a header value could turn a tab into a space. A fold inserts a line ending before whitespace that is already there; this one replaced the whitespace with a single space, so a value with a tab in it — or with three spaces — came back from the unfolder as a different value. Inside a quoted string that whitespace is content, and a library whose claim is that what goes in comes out cannot quietly rewrite it.
• A bounce could grow a recipient that nobody sent to. A group of lines in a delivery-status report that held no field at all — a separator, or a line of prose some reporting systems put between the groups — was read as a recipient, which then reported Action.other and named nobody, and was counted by anything walking the list to say which addresses had failed. The same junk was treated two different ways depending on where it sat: as the per-message group when it came first, and as a phantom recipient when it did not.
• A report value with no type re-split at the wrong semicolon. RFC 3464 writes these as type; value, and a value with no type but a ; of its own was written back without the separator — so Final-Recipient: a;[email protected] came back as type a and recipient [email protected], which is a bounce reporting on an address nobody sent to.
• A uuencoded attachment with a newline in its filename arrived empty. The begin line carries the name and runs to the end of the line, so a CRLF in the name ended the header early and left a blank line straight after it — which is exactly what the decoder reads as the end of the body. The file was silently lost, and a filename is very often something that came from outside the program.
• And they disagreed a fourth way, about an empty attached message. A message/rfc822 with an empty body is not a message, and Message.parse refuses to open one — a text/plain that nothing sent is what find would then hand to a caller asking for the text. The streaming parser opened one anyway whenever the part was the last thing before a delimiter, because the line ending in front of a delimiter belongs to the delimiter: there is nothing between the header block and --boundary-- however it is spelled, and "is there another line" is not the same question. It took four independent campaigns and about fifteen million inputs to reach.
• An IDN conversion could produce a domain with a space in it. UTS #46 maps as well as validates, and with the STD3 rules off — they are off so that the underscores real mail hosts have survive — U+00A8 maps to a space and a combining mark. The conversion succeeded and handed back something that was not a domain. The output is now checked as well as the input.

The API documentation

https://jeff.jcollie.page/zig-mime/, rebuilt from main on every push. Zig writes it out of the doc comments, which in this project carry most of the explanation — why an encoded word cannot be found by searching, what -0000 means and +0000 does not, why a boundary must be checked against the whole line.

$ zig build docs-serve            # then open http://127.0.0.1:8000/
$ zig build docs                  # or just build it, into zig-out/docs

It has to be served rather than opened. What Zig emits is not a page but a program: a WebAssembly viewer that fetches the source of everything it shows out of a tar file beside it, and a browser refuses to fetch anything from a file:// page. That is the same reason zig std runs a server rather than opening a file. tools/docs_server.zig is that server — one directory, one person, the loopback interface, and nothing else.

References cited

The specifications this library implements or follows, each held with its full text in the zig-mime collection in Zotero.

• RFC 2045: Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies. RFC Editor, November 1996. Draft Standard.
• RFC 2046: Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types. RFC Editor, November 1996. Draft Standard.
• RFC 2047: MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text. RFC Editor, November 1996. Draft Standard.
• RFC 2183: Communicating Presentation Information in Internet Messages: The Content-Disposition Header Field. RFC Editor, August 1997. Proposed Standard.
• RFC 2231: MIME Parameter Value and Encoded Word Extensions: Character Sets, Languages, and Continuations. RFC Editor, November 1997. Proposed Standard.
• RFC 2387: The MIME Multipart/Related Content-type. RFC Editor, August 1998. Proposed Standard.
• RFC 2392: Content-ID and Message-ID Uniform Resource Locators. RFC Editor, August 1998. Proposed Standard.
• RFC 2606: Reserved Top Level DNS Names. RFC Editor, June 1999. Best Current Practice, BCP 32.
• RFC 3463: Enhanced Mail System Status Codes. RFC Editor, January 2003. Draft Standard.
• RFC 3464: An Extensible Message Format for Delivery Status Notifications. RFC Editor, January 2003. Draft Standard.
• RFC 3492: Punycode: A Bootstring encoding of Unicode for Internationalized Domain Names in Applications (IDNA). RFC Editor, March 2003. Proposed Standard.
• RFC 3676: The Text/Plain Format and DelSp Parameters. RFC Editor, February 2004. Proposed Standard.
• RFC 3848: ESMTP and LMTP Transmission Types Registration. RFC Editor, July 2004. Draft Standard.
• RFC 5321: Simple Mail Transfer Protocol. RFC Editor, October 2008. Draft Standard.
• RFC 5322: Internet Message Format. RFC Editor, October 2008. Draft Standard.
• RFC 5965: An Extensible Format for Email Feedback Reports. RFC Editor, August 2010. Proposed Standard.
• RFC 6068: The 'mailto' URI Scheme. RFC Editor, October 2010. Proposed Standard.
• RFC 6266: Use of the Content-Disposition Header Field in the Hypertext Transfer Protocol (HTTP). RFC Editor, June 2011. Proposed Standard.
• RFC 6522: The Multipart/Report Media Type for the Reporting of Mail System Administrative Messages. RFC Editor, January 2012. Internet Standard, STD 73.
• RFC 6531: SMTP Extension for Internationalized Email. RFC Editor, February 2012. Proposed Standard.
• RFC 6532: Internationalized Email Headers. RFC Editor, February 2012. Proposed Standard.
• RFC 6533: Internationalized Delivery Status and Disposition Notifications. RFC Editor, February 2012. Proposed Standard.
• RFC 6590: Redaction of Potentially Sensitive Data from Mail Abuse Reports. RFC Editor, April 2012. Proposed Standard.
• RFC 8098: Message Disposition Notification. RFC Editor, February 2017. Internet Standard, STD 85.
• RFC 8187: Indicating Character Encoding and Language for HTTP Header Field Parameters. RFC Editor, September 2017. Proposed Standard.
• UTS #46: Mark Davis and Markus Scherer, Unicode IDNA Compatibility Processing. Unicode Consortium, September 2025. Unicode Technical Standard, Revision 35; implemented by zig-uri's uri_unicode module, which mime.idn uses.

Licence

MIT. See LICENSES/MIT.txt.