- Zig 97.6%
- Nix 2.4%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
The streaming parser hands out chunks of a body still in its transfer encoding, and every decoder took a slice -- so decoding a part meant holding that part, and the message that was never in memory came back into it one attachment at a time. `transfer_encoding.Decoder` is the missing half. It keeps the few bytes of state that span a chunk boundary: a partial base64 group, and for quoted-printable a half-finished escape, a soft line break split across two chunks, and a run of whitespace that cannot be written until something proves whether it is content or padding a transport added. Each codec now has exactly one decoder and the slice forms are wrappers over it -- `base64.decode` is `base64.Decoder` with the whole input as one chunk. That is the point rather than a tidy-up: two decoders for one encoding drift apart the first time either is touched, and the drift shows up as an attachment that comes out right when it is read one way and wrong when it is read the other. The quoted-printable decoder in particular went from a loop with an index to a four-state machine, and what says it kept its behaviour is that its own tests and the round-trip fuzz target did not move. Uuencode is the exception and buffers, which is documented where it happens: it is line oriented and its `begin` line may sit anywhere after a covering note, so a decoder that could not look back would decode that note as though it were data. Its bodies are from before 1996 and are not the ones anybody streams. `zig-mime extract --stream` is the end of the path. On a 27 MB message with a 20 MB attachment it writes the same bytes as the tree version does, at 5 MB of peak memory rather than 58 -- and that figure is the two buffers and the process itself, so it does not move when the message gets bigger. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HJpiv2716fNr3NpUsFL39E |
||
| .forgejo/workflows | ||
| LICENSES | ||
| src | ||
| tests | ||
| tools | ||
| .gitignore | ||
| build.zig | ||
| build.zig.zon | ||
| build.zig.zon.nix | ||
| flake.lock | ||
| flake.nix | ||
| package.nix | ||
| README.md | ||
| REUSE.toml | ||
zig-mime
Email messages for Zig 0.16: reading them, writing them, and building them from nothing. RFC 5322 and the five MIME documents, with the thirty years of other people's software that came after them taken into account.
$ zig build
$ ./zig-out/bin/zig-mime tree message.eml
multipart/mixed 2 parts
multipart/related 2 parts
multipart/alternative 2 parts
text/plain [quoted-printable] 55 bytes
text/html [quoted-printable] 159 bytes
image/png [base64] "logo.png" cid:1f13baa8@example.com 38 bytes
application/pdf [base64] "café.pdf" (attachment) 26 bytes
Three layers, and you can stop at any of them:
| the codecs | base64, quoted-printable, encoded words — pure functions over bytes, no allocation you did not ask for |
| the headers | addresses, dates, media types, parameters — each header's own grammar, parsed and written |
mime.Message |
the whole message as a tree of parts, and the builder that makes one |
zig-mime |
a command-line tool, which exists to show what all that looks like from outside |
The API documentation is generated from the doc comments, which are most of the explanation of why mail is the shape it is.
Where this lives
The repository lives in three places that carry the same history. The Forgejo instance at https://git.jcollie.dev/jeff/zig-mime is the web-visible one, and is where the continuous integration and the published documentation are:
$ git clone https://git.jcollie.dev/jeff/zig-mime.git
It is mirrored on Tangled at https://tangled.org/jcollie.dev/zig-mime, a forge built on the AT Protocol, where a repository is addressed by its owner's identity rather than by a server name.
It is also on Radicle, a peer-to-peer forge that needs no account on anything. A Radicle repository is findable only by its repository ID, so this is that ID:
rad:z4WSVdMPhsgpLHM5xaWeqpAMi33YC
and rad clone rad:z4WSVdMPhsgpLHM5xaWeqpAMi33YC fetches it from any node that
seeds it. rad clone finds seeds through your local node's routing table, so
the node has to be running first, and cloning seeds the repository in turn,
which helps keep it available:
$ rad node start
$ rad clone rad:z4WSVdMPhsgpLHM5xaWeqpAMi33YC
If you already have the repository and only want to help host it:
$ rad seed rad:z4WSVdMPhsgpLHM5xaWeqpAMi33YC
Any of the three is the whole project.
Adding it to a project
$ zig fetch --save git+https://git.jcollie.dev/jeff/zig-mime.git
const mime = b.dependency("zig_mime", .{ .target = target }).module("mime");
exe.root_module.addImport("mime", mime);
Reading a message
const mime = @import("mime");
var message: mime.Message = try .parse(gpa, source, .{});
defer message.deinit();
std.debug.print("{s}\n", .{(try message.root.subject()) orelse "(no subject)"});
var from = try message.root.addresses(gpa, "from");
defer from.deinit(gpa);
for (from.addresses) |address| std.debug.print("{f}\n", .{address});
if (message.root.find("text", "plain")) |part| {
const text = try part.textAlloc(gpa); // decoded, and UTF-8
defer gpa.free(text);
std.debug.print("{s}", .{text});
}
var walker = message.root.walk();
while (walker.next()) |part| {
if (!part.isAttachment()) continue;
const bytes = try part.decodeAlloc(gpa);
defer gpa.free(bytes);
std.debug.print("{s}: {d} bytes\n", .{ (try part.filename()).?, bytes.len });
}
A parsed message written back out is the message that went in, byte for byte. Bodies are kept in the transfer encoding they arrived in and header values in wire form, folds and all, so the round trip preserves the preamble, the epilogue, the exact position of every line break, the boundary as written, and even the line ending a truncated message was missing. That is what makes it safe to parse a signed message and forward it: the signature still verifies. Decoding happens when it is asked for.
The one thing not preserved is a message whose lines end in LF alone. Mail is CRLF; a message read out of a Unix file often is not, and its headers are written back out canonicalised. That is a repair rather than a loss, and it is the only one.
Building a message
var message: mime.Message = try .init(gpa);
defer message.deinit();
try message.root.setAddresses("From", &.{
.{ .display_name = "Ollie, Jeff", .local_part = "jeff", .domain = "example.com" },
});
try message.root.setAddresses("To", &.{
.{ .display_name = "José García", .local_part = "jose", .domain = "example.net" },
});
try message.root.setSubject("Grüße aus München");
try message.root.setDate(now);
try message.root.setMessageId("example.com", random);
try message.setText("Schöne Grüße.\n", .{});
_ = try message.attach(random, pdf_bytes, .{
.filename = "Quartalsbericht für München.pdf",
});
try message.write(writer);
What comes out is US-ASCII, in lines no longer than the rule allows, with the display name quoted because it has a comma in it, the subject in RFC 2047 encoded words split so that no word exceeds seventy-five characters and no UTF-8 character is cut in half, the body in quoted-printable because it is mostly text, the attachment in base64 because it is not, and the filename in RFC 2231 continuations because it is neither short nor ASCII. None of those decisions is one the caller has to make.
The attachment's media type was not given and did not need to be: it is taken
from the end of the filename, which is what the sender knew all along and what
would otherwise have been thrown away at the last moment. A file that arrives as
application/octet-stream gets no preview, no icon and no "open with". Pass
content_type where the type is actually known — it always wins over the
guess — and expect application/octet-stream for an extension the table has
never heard of, which is the correct thing to say when you do not know.
attach turns a message that already has a body into a multipart/mixed,
moving the body and every Content-* header that described it down into a part
of their own — which is what those headers describe, and moving them is the
difference between an attachment and a corrupted message.
HTML, and the images inside it
// The identifier has to exist before the HTML that refers to it does.
const logo = try message.newContentId("example.com", random);
var html: std.Io.Writer.Allocating = .init(gpa);
defer html.deinit();
try html.writer.print("<p>Grüße!</p><img src=\"cid:{s}\">", .{logo});
try message.setAlternative(random, "Grüße!\n\n[logo]\n", html.written(), .{});
_ = try message.embed(random, png_bytes, .{ .filename = "logo.png", .content_id = logo });
_ = try message.attach(random, csv_bytes, .{ .filename = "Bericht.csv" });
Four calls, and what comes out is the five-level structure that every mail client expects and that almost nobody assembles correctly by hand:
multipart/mixed
├── multipart/related type="multipart/alternative"
│ ├── multipart/alternative
│ │ ├── text/plain
│ │ └── text/html
│ └── image/png inline, Content-ID: <logo@example.com>
└── text/csv attachment
Each of those levels is load-bearing:
- The plain text comes first. RFC 2046 §5.1.4 puts alternatives in increasing order of preference and a reader shows the last one it can display. Reversed, every HTML client shows the plain text instead.
- The image is
inlineand inside therelated, not beside it. A resource the body refers to is not an attachment, and a client that lists it as one shows a signature logo next to the file somebody actually meant to send. multipart/relatedcarries atypeparameter naming the media type of its root part. RFC 2387 §3.1 requires it, because a reader has to know which of the parts is the document before it has looked at any of them.embedfills it in from what the body already was.attachwraps rather than appends. Adding the spreadsheet to thealternativewould claim it was a third rendering of the message — so a client showing the last alternative it understands would display the spreadsheet instead of the text.
Reading it back, findContentId resolves a reference in whichever form the
HTML happens to hold it — cid:logo@example.com, <logo@example.com> or bare:
if (message.root.findContentId(src_attribute)) |image| {
const bytes = try image.decodeAlloc(gpa);
...
}
A message can also be forwarded whole rather than quoted, which is
message/rfc822 and one call:
_ = try message.attachMessage(random, original, .{});
Replies, and the thread they belong to
try reply.root.setMessageId("example.com", random);
try reply.setInReplyTo(&parent, .{});
That writes In-Reply-To with the parent's Message-ID and References with
the parent's whole ancestry followed by the parent itself, which is the entire
mechanism behind every threaded mailbox view. Reading it back is an iterator
over the identifiers, oldest first:
var it = try message.root.references();
while (it.next()) |id| ...
References grows by one identifier per reply and RFC 5322 §3.6.4 says
nothing about what to do when it gets long. The convention, inherited from
netnews, is specific about which entries to drop: the first is kept and the
trimming is done from the second onwards. The first identifier is the message
that started the thread and is what a reader groups the conversation by; the
last few are the immediate ancestry and are what it nests replies with;
everything between is the part nobody looks at. max_references is where that
limit lives.
Identifiers are found by their angle brackets, so a References header
carrying commas, comments or stray text still threads — and a value with no
brackets at all falls back to taking the whitespace-separated tokens that look
like addresses, because the client that writes In-Reply-To: 1a2b@example.com
exists and its threading should still work.
Wrapped text that a reader may rewrap
try message.setText(paragraphs, .{ .format = .flowed });
text/plain; format=flowed — RFC 3676 — is the fix for the ragged
long-short-long-short that plain text mail turns into when the reader's window
is narrower than the sender's wrap width. A line that ends in a space is a
line that was wrapped, and the reader may join it to the next and rewrap the
result; a line that does not is a line the author ended. That is the whole
format, and a reader that has never heard of it sees ordinary wrapped text.
Each line of the text handed to setText is then a paragraph — what the
author typed, with the breaks the author made — because text that has already
been hard-wrapped no longer says which of its breaks were the author's.
Reading is automatic: textAlloc joins a flowed body back into its
paragraphs, since the line breaks it removes are ones the sender's composer put
in rather than ones the sender typed. decodeAlloc is still the way to the
body exactly as it arrived.
Three details hang off the one idea, and src/flowed.zig has them: a line that
would otherwise begin with a space is given another one that the reader removes
(space-stuffing, without which the format could not represent an indented
line at all); two lines join only if they are quoted to the same depth; and
delsp=yes says the trailing space is only a marker and is deleted on
joining, which is what a script that does not separate its words with spaces
needs. Plus one exception older than the format: a line that is exactly -- is
the signature separator, ends in a space, and is not a wrapped line.
Writing flowed text upgrades an automatic 7bit to quoted-printable, because
everything the format says it says with trailing whitespace and RFC 2045 warns
that trailing whitespace is exactly what a transport may strip. Pass
.encoding = .seven_bit to decide otherwise.
Large attachments
_ = try message.attachStreaming(random, &file_reader.interface, .{
.filename = "recording.mp4",
});
and, reading, the other direction:
try part.decodeWriter(&file_writer.interface);
Neither holds the file. attachStreaming encodes as it reads, so the plain
bytes are never all in memory; decodeWriter decodes as it writes, so a
hundred megabyte attachment reaches a file with a sixty-four kilobyte buffer in
between and nothing else. zig-mime extract is written that way, and is the
worked example.
What attachStreaming still holds is the encoded body, because the message
tree owns it: a body that stayed on disk until the message was written could
only be written once, and a Message is something you can write twice. The
exception is an explicit .quoted_printable, which buffers its input after
all — whether a space may be written as itself depends on how far the rest of
its run reaches and on what follows it, so an encoder fed a chunk at a time
would have to buffer that anyway. The default for a file is base64, which
streams.
A std.Random is passed in rather than reached for, because Zig 0.16 gets
its randomness through std.Io and a library that went looking for a global
would be reaching around the caller's back:
var source: std.Random.IoSource = .{ .io = io };
const random = source.interface();
Bounces
A delivery status notification arrives as a multipart/report of two or three
parts — RFC 6522 — and only one of them is worth parsing:
multipart/report report-type=delivery-status
├── text/plain the apology
├── message/delivery-status what actually happened
└── message/rfc822 the message that failed
The apology is the part everybody reads and the part nobody can parse: every
mail system words it differently and half of them word it badly. The second
part is the same information in a format a program can act on — RFC 3464 — and
it is what deliveryStatus returns:
if (try message.root.deliveryStatus(gpa)) |found| {
var status = found;
defer status.deinit();
for (status.recipients.items) |*recipient| {
const who = (try recipient.finalRecipient()).?.value;
switch (try recipient.action()) {
.failed => if ((try recipient.status()).?.isPermanentFailure()) {
// Stop mailing this address.
},
.delayed => {}, // Still being retried; a later report will say.
else => {},
}
_ = who;
}
}
Ask it of the message rather than of the report part: a bounce is a
multipart/report at the top, but a bounce that has itself been forwarded is
one somewhere further down, and the whole tree is searched either way.
Two fields carry the answer, and it is worth reading the right one.
Action says what happened in a word, and only failed is permanent —
delayed is a warning that a later report will resolve, and a great many of
them resolve themselves. Status is an RFC 3463 code like 5.1.1, whose
first number is the part to switch on: 2 succeeded, 4 is temporary, 5 is
permanent. A program that matches 5.1.1 exactly will be surprised by the
server that says 5.0.0 and means the same thing, and one that reads
Diagnostic-Code is reading a remote server's own prose.
The report's body is header syntax — groups of Name: value separated by
blank lines, folded the same way — which is why it is parsed by the same code
as a header block, and why a Diagnostic-Code folded across three lines comes
back as one value.
Part.report() gives the three parts by name rather than by position, so a
mail system that puts its message/delivery-status third has still sent a
bounce this can read. The original may be a whole message/rfc822 or just its
headers as text/rfc822-headers, and Part.rfc822HeadersAlloc is the second
case. Part.makeReport builds one.
zig-mime headers prints the summary, which is the shortest way to see what a
bounce in a file actually says:
$ zig-mime headers bounce.eml
...
reported by: mx.example.net
nobody@example.com: failed 5.1.1
550 5.1.1 <nobody@example.com>: User unknown
Streaming
Message.parse takes a slice, so a message has to fit in memory to be opened.
That is the right shape for nearly everything — a message is usually small, and
a tree you can walk and write back out byte for byte is worth more than the
memory it costs. mime.Parser is the other shape: a pull parser over an
Io.Reader, which never holds the message at all.
var parser: mime.Parser = .init(gpa, reader, .{});
defer parser.deinit();
while (true) switch (try parser.next()) {
.begin => |d| _ = d,
.field => |f| std.debug.print("{s}: {s}\n", .{ f.name, f.value }),
.headers_end => if (parser.mediaType().is("application", "pdf")) {
// parser.transferEncoding() says how to decode what follows.
},
.body => |chunk| try out.writeAll(chunk),
.end => |d| _ = d,
.eof => break,
};
Nesting arrives flattened: a multipart/mixed produces begin, its headers,
and then a begin for each of its parts in turn, each ended before the next
begins. begin and end carry the depth, which is what a caller
reconstructing a tree needs and what a caller looking for one part can ignore.
Bodies arrive in their transfer encoding, as everywhere else here, and
transfer_encoding.Decoder is what turns the chunks back into bytes without
holding any of them — it keeps the few bytes of state that span a chunk
boundary, a partial base64 group or a half-finished quoted-printable escape:
var decoder: mime.transfer_encoding.Decoder = .init(parser.transferEncoding());
defer decoder.deinit(gpa);
// ...on each `.body` chunk:
try decoder.write(gpa, file_writer, chunk);
// ...and at `.end`:
try decoder.finish(gpa, file_writer);
Each codec has one decoder, and the slice forms are wrappers over it —
base64.decode is base64.Decoder with the whole input as one chunk. Two
decoders for one encoding drift apart the first time either is touched, and
the drift shows up as an attachment that comes out right when it is read one
way and wrong when it is read the other. Uuencode is the exception and buffers:
it is line oriented and its begin line may sit anywhere after a covering
note, so a decoder that could not look back would decode the note as data.
zig-mime tree --stream and zig-mime extract --stream are the worked
examples, and the difference is measurable. On a 27 MB message with a 20 MB
attachment, writing the same bytes out:
$ zig-mime extract big.eml out # 58 MB peak
$ zig-mime extract --stream big.eml out # 5 MB peak
The second figure is the two 64 KB buffers and the process itself; it does not move when the message gets bigger.
A streaming read is not a round trip. It discards what a reader does not
need: the preamble and epilogue of a multipart, the exact spelling of the line
endings, the whitespace a fold was broken at. Use Message.parse when the
bytes matter — when a signature has to keep verifying, or when the message is
going out again. Header values also arrive unfolded, unlike Headers, which
keeps them in wire form because it has to write them again; a streaming caller
cannot ask later, so it is given the form it wants.
The one structural thing it will not do is open a message/rfc822 that is
transfer encoded, because decoding it would mean holding it. RFC 2045 §6.4
forbids encoding one; a part that does it anyway arrives as a leaf, and its
bytes are the caller's to decode and parse again.
Both parsers are checked against each other rather than separately — see the fuzzing section, where that is most of the point.
The codecs
const encoded = try mime.base64.encodeAlloc(gpa, bytes, .{}); // wrapped at 76
const decoded = try mime.base64.decodeAlloc(gpa, encoded); // lenient
_ = try mime.base64.decodeStream(reader, writer); // no copy of either
const qp = try mime.quoted_printable.encodeAlloc(gpa, text, .{});
const subject = try mime.encoded_word.decodeUtf8Alloc(gpa, header_value);
Each of them is the MIME dialect rather than the textbook one, and the differences are the whole point:
- base64 wraps, and decodes leniently. RFC 2045 §6.8 says a body is broken
into lines of at most 76 characters and that "any characters outside of the
base64 alphabet are to be ignored" — which is how the line breaks get in, and
how a decoder survives the stray
>of a badly quoted forward.std.base64does neither.decodeStrictis there for a caller who would rather be told the attachment is damaged than handed the damage. - quoted-printable has to know whether the body is text. In
.texta CRLF is a line break and stays one; in.binaryit is two octets and becomes=0D=0A, so that a body survives a transport that rewrites line endings. - An encoded word is a token, not a substring.
=?utf-8?q?x?=inside a longer atom is not one, and a subject reading "look at=?this?=" must survive being written out and read back. The whitespace between two adjacent encoded words is not whitespace — it is how a long run was folded — and the whitespace next to ordinary text is.
Uuencode
Content-Transfer-Encoding: x-uuencode is not in RFC 2045 and never was, and
it is why an archive of mail from before about 1996 has attachments a strict
MIME reader cannot open. It is spelled five ways, all of them accepted here,
and it is decoded through zuucode.
Nothing writes it by choice — transfer_encoding.choose never returns it, and
base64 is what replaced it — but .encoding = .uuencode will, for a message
being built to be read by something old.
Two things about it are worth knowing:
- The body names itself. The
begin 644 picture.gifline carries a filename, and a message old enough to be uuencoded is usually old enough to predateContent-Dispositionas well, so that line is frequently the only place the name appears.Part.filenamefalls back to it, after the two headers that should have had it. - Decoding stops where the damage starts. Everything before the
beginline is skipped, so a body with a covering note in front of it works; a line that will not decode ends the body, as does the empty line a transport leaves behind when it strips the terminator's single space. What decoded, decoded — the same decision the base64 decoder makes, for the same reason.
Mail from before MIME put uuencoded files straight into an ordinary
text/plain body, with no Content-Transfer-Encoding to describe them, no
Content-Disposition to name them, and usually a covering note in front. To a
MIME reader the attachment is invisible: the body is text, and the text
happens to contain the file. Those messages are still in everybody's archives.
Nothing here restructures such a message, because no header says it has happened and rebuilding a tree on a guess is worse than leaving it alone. What there is instead is the explicit question:
if (try part.uuencodedFile(gpa)) |file| {
defer file.deinit(gpa);
// file.name, file.mode, file.data
}
Only a text body is examined — or one that declares uuencode, so the same
question answers both shapes — because a begin line found by chance inside a
decoded JPEG is not a file. zig-mime extract asks it, which is how a
pre-MIME message's attachment comes out of a body nothing else would look in:
$ zig-mime extract 1994.eml out
script.sh (3 bytes, uuencoded in the body)
zuucode is also re-exported as mime.zuucode, for anything this does not
cover.
The headers
var list: mime.address.List = try .parse(gpa, "\"Ollie, Jeff\" <jeff@example.com>, alice@example.net");
defer list.deinit(gpa);
var media: mime.MediaType = try .parse(gpa, "multipart/mixed; boundary=\"----=_Part_1\"");
defer media.deinit(gpa);
const sent = try mime.date.parse("Tue, 01 Jul 2003 10:52:37 +0200 (CEST)");
Addresses are the difficult one, and the comma is why: it separates addresses,
and it is the most common character in a display name, and a group holds
commas of its own between a : and a ;. The entries cannot be found by
splitting — the parser has to know where the quotes, the angle brackets, the
domain-literals, the comments and the groups begin and end first. Splitting on
commas is why so much software shows a message from "Ollie addressed to
Jeff" <jeff@…>.
Display names come back decoded, because that is the only form anything wants
to display. The local part and the domain come back as written, because
they are identifiers: an address is compared, stored and sent to.
Mailbox.isValid is the question a program about to send something should ask,
and it is also exactly the condition under which writing an address back out
reproduces it byte for byte — the parser accepts far more than that, because a
message that has already arrived has to be filed whatever is in its To:.
Internationalized domains
bücher.example and xn--bcher-kva.example are the same domain written two
ways. The domain name system carries ASCII, so the second is what goes on the
wire; the first is what a person should be shown.
const ascii = try mailbox.domainAsciiAlloc(gpa); // xn--bcher-kva.example
const shown = try mailbox.domainUnicodeAlloc(gpa); // bücher.example
Converting between them is not simply Punycode. UTS #46 is a mapping as well
as an encoding — case folding, Normalization Form C, a table of code points
that are disallowed outright, and rules about joiners and bidirectional text —
so that Bücher.example and bücher.example produce one A-label and a domain
nobody could register is refused rather than encoded. That is
zig-uri's idna, and mime.idn is
the mail-shaped layer over it.
Punycode applies to the right of the @ and nowhere else. An address whose
local part is not ASCII cannot be encoded at all and needs a transport that
speaks SMTPUTF8 (RFC 6531) end to end, which is a different question from
whether the domain needs converting:
if (recipients.requiresSmtpUtf8()) {
// Nothing can be done about this one but find a transport that does it.
}
domainAsciiAlloc fails rather than guessing, because an address that cannot
be encoded cannot be delivered to and silently sending something else would
deliver it to somebody else. A domain that is already ASCII is copied rather
than converted — putting an existing ASCII domain through IDNA would refuse
some that already work.
Showing the U-label is a decision, not a default. раypal.com with a
Cyrillic а and paypal.com with a Latin one look identical in most fonts and
are different domains. Browsers decide with a script-mixing heuristic and show
the A-label when a name looks suspicious; nothing here does that for you, and
nothing here silently prefers one form over the other. Both are available, and
which one a reader shows is the reader's problem to think about.
Dates come from zig-datetime,
whose rfc822 module already speaks the syntax; mime.date is the layer
between that and a header field, which removes the comments a header may carry
anywhere and forgives the two things a sender's clock gets wrong — a day name
that contradicts the date, and a missing zone.
Charsets
This library carries no charset tables. It reports the charset and hands
over the bytes; converting them belongs beside whatever a program already uses
for text, not buried inside its mail parser. encoded_word.Iterator and
Parameters.getParameter are where the name comes back with the bytes.
What the convenience functions — textAlloc, getUtf8, decodeUtf8Alloc —
handle is the part that needs no table: UTF-8 and US-ASCII, which are already
the answer, and ISO-8859-1, whose 256 bytes are exactly the first 256 Unicode
code points. Anything else is passed through with invalid sequences replaced,
so that the result is always valid UTF-8 and a Shift-JIS subject degrades to
something legible-ish rather than to an error in the middle of a mailbox
listing.
Injection
A header value containing a bare CR or LF is how a program that interpolates
user input into a header becomes a program that sends whatever headers an
attacker likes — a Subject ending in \r\nBcc: everyone adds a recipient.
Everything here that writes a value strips bare CR and LF, and the address
writer drops every control character from a local part and a domain. It is not
a configurable strictness: no legitimate value contains one, so nothing is
lost, and the failure mode of getting it wrong is not a malformed message but a
security hole.
The same care applies in the other direction. A filename out of a message is
attacker-controlled, and zig-mime extract writes a part under its suggested
name only if that name is a plain one — no separators, no .., no leading dot.
The command-line tool
$ zig-mime tree message.eml # the structure, one part per line
$ zig-mime tree --stream message.eml # the same, never holding the message
$ zig-mime headers message.eml # the headers, decoded
$ zig-mime text message.eml # the text, decoded to UTF-8
$ zig-mime extract message.eml out # every attachment, into a directory
$ zig-mime extract --stream message.eml out # the same, holding nothing
$ zig-mime build - # a message assembled from the library
A file of - is read from standard input.
What is deliberately not here
- No SMTP, IMAP or POP. This library is about the message, not about moving it. Everything it produces is bytes, and bytes are what those protocols take.
- No DKIM, S/MIME or PGP. What is here is the part those need: a parser that gives back the exact bytes it was handed.
Fuzzing
tests/fuzz.zig holds twelve targets, each a property rather than an example:
whatever arrives, the parser terminates, stays inside its buffers, frees what
it allocated, and — if it claims to have understood the input — writing it back
out and reading it again gives the same answer.
$ zig build test # the properties, over the checked-in seeds
$ zig build fuzz-run # a minute of each, with generated input
$ zig build fuzz-run -- --seconds 300 --target address
$ zig build fuzz-run -- --input fuzz-findings/x.bin --target address
tools/fuzz.zig is that loop and says at the top why it exists: Zig 0.16.0
cannot build a test executable in fuzz mode without a patched standard library
— flake.nix carries the patch — and leaves the fuzzer's coverage table empty
even then. A fuzzer with no coverage is a random number generator, so this is
one written down honestly, with a corpus of real messages to mutate instead.
It found five things worth having found, all within the first few hundred thousand inputs:
- The quoted-printable encoder could lose a space. A run of whitespace that ended within three columns of the seventy-sixth got a soft line break inserted into the middle of it — and the decoder deletes the whitespace in front of a soft break, because that is what the rule says. Invisible in every small test; the round-trip property caught it.
getUtf8could return invalid UTF-8. A header with no encoded words in it was passed through unchanged, so aSubject:carrying a sender's raw ISO-8859-15 bytes came back as something a UTF-8 consumer three layers away would choke on.- A multipart with no delimiter in it grew by two bytes every time it was opened and saved.
- A local part could carry a newline into the header. A malformed address parsed out a local part containing a CR, and the writer wrote it straight back — which is header injection, arrived at from the parsing side rather than the building side.
- Four scans over an address disagreed about which stretches were opaque.
One skipped domain-literals and another did not, so a
(inside[…]became a comment, a<inside one became an angle-addr, and a:inside one became the end of a source route. Each of those changed what the address was. - A flowed wrap point could invent a signature separator. A paragraph
beginning
--wrapped at a narrow width produced a line whose content was exactly--, which a reader treats as fixed — so the paragraph stopped joining and everything after it became a signature. - The two parsers disagreed three ways.
mime.ParserandMessage.parseare asked to describe the same message and the descriptions are compared, which is most of why a second parser is safe to have at all. It found that the tail of a header line too long for the reader's buffer was being read as a field of its own — and that a colon past the end of that buffer was missed entirely; that a lone CR at the end of a line was read as a fold, because the real line ending had already been taken away and the CR had come to sit beside the next line's whitespace; and that an emptymessage/rfc822body produced a part in the tree that nothing had sent. - An IDN conversion could produce a domain with a space in it. UTS #46 maps
as well as validates, and with the STD3 rules off — they are off so that the
underscores real mail hosts have survive —
U+00A8maps to a space and a combining mark. The conversion succeeded and handed back something that was not a domain. The output is now checked as well as the input.
The API documentation
https://jeff.jcollie.page/zig-mime/, rebuilt from main on every push. Zig
writes it out of the doc comments, which in this project carry most of the
explanation — why an encoded word cannot be found by searching, what -0000
means and +0000 does not, why a boundary must be checked against the whole
line.
$ zig build docs-serve # then open http://127.0.0.1:8000/
$ zig build docs # or just build it, into zig-out/docs
It has to be served rather than opened. What Zig emits is not a page but a
program: a WebAssembly viewer that fetches the source of everything it shows
out of a tar file beside it, and a browser refuses to fetch anything from a
file:// page. That is the same reason zig std runs a server rather than
opening a file. tools/docs_server.zig is that server — one directory, one
person, the loopback interface, and nothing else.
Licence
MIT. See LICENSES/MIT.txt.