zxml

A small XML pull parser for Zig 0.17. Its one dependency is zig-charset, which decodes a document that is not in UTF-8.

It is deliberately not a general-purpose XML implementation. What it covers is elements, attributes, text, CDATA, comments, processing instructions, and the predefined plus numeric entities — the subset that data formats such as NZB, RSS, and Atom are actually built from.

A document type declaration is an opaque token by default and declarations when you ask: initDtd reads the internal subset, resolves the general entities it declares, and supplies the attribute defaults it gives. Nothing external is ever fetched — a SYSTEM or PUBLIC identifier is recorded and never dereferenced, so a document still cannot make the parser open a file or reach the network, which is a feature rather than an omission.

Installation

$ zig fetch --save git+https://git.jcollie.dev/jeff/zxml.git

zxml needs Zig 0.17.0. nixpkgs does not package it yet, so the development shell takes it from zig-overlay.

For Zig 0.16, use the v0.1.0 tag, the last release built with it; the zig-0.16 branch is where it lives:

$ zig fetch --save git+https://git.jcollie.dev/jeff/zxml.git#v0.1.0

Repository

The repository's home is my Forgejo instance at git.jcollie.dev/jeff/zxml, which is where CI runs. It is mirrored on Tangled at tangled.org/jcollie.dev/zxml, and also published on Radicle, a peer-to-peer forge built on git, where the copy needs no account and no server anyone has to keep running.

$ git clone https://git.jcollie.dev/jeff/zxml.git

$ git clone https://tangled.org/jcollie.dev/zxml

$ rad clone rad:z2eX8o5SHYVJXkRZoQVhZSnUVP1nm

Any of them is the whole project, on the main branch, with the same history.

rad clone finds seeds through your local node's routing table rather than through a known host, so the node has to be running before it can find anything:

$ rad node start

If you already have the repository and only want to help host it, seeding it tells your node to carry a copy for others:

$ rad seed rad:z2eX8o5SHYVJXkRZoQVhZSnUVP1nm

Standards

What the parser itself implements:

Standard	Title	Support in zxml
XML 1.0 (5th ed.)	Extensible Markup Language	Elements, attributes, text, CDATA, comments, processing instructions, the XML declaration, and and the DOCTYPE declaration — opaque by default, and read into declarations by initDtd. Line endings are normalized, and attribute values are too when decoded with .attribute. No external subset or external entity is ever fetched
XML Namespaces 1.0	Namespaces in XML	Prefixes surfaced as text — localNameOf, prefixOf, nameIs — rather than resolved against their declarations
HTML 4.01 entities	Character entity references	The 248 named references XML does not define, under EntityPolicy.html
XML 1.0 Appendix F	Autodetection of Character Encodings	sniffEncoding: a byte order mark, then <? in UTF-16, then the declaration's encoding, then UTF-8
WHATWG Encoding	Encoding Standard	Every label and encoding it defines, decoded to UTF-8 by toUtf8 and StreamReader through zig-charset. A label it does not know leaves the document as it is

Vocabularies the parser has no opinion about, but which the test corpus and the --feed profile cover:

Standard	Title	Where it appears
RSS 2.0	Really Simple Syndication	--feed checks a channel's title, link, and description, and that an item has a title or a description
RFC 4287	The Atom Syndication Format	--feed checks a feed's and an entry's id, title, and updated
RFC 4685	Atom Threading Extensions	The thr: prefix in the corpus
OPML 2.0	Outline Processor Markup Language	--feed checks for head and body, and an outline's text attribute
RFC 822	Date and time specification	--feed reports a pubDate that is not in this form
NZB 1.1	newzBin NZB	Test corpus, and the format znzb decodes
D-Bus introspection	Object introspection format	Test corpus

Design

• Pull, not push. Reader.next returns one Event at a time. There are no callbacks to thread state through and no tree to allocate.

• Borrowed, not copied. Reader works over a complete in-memory document and hands out slices into it, so it never allocates unless you ask it to — decodeAlloc and textAlloc are the only entry points that take an allocator. A document that should not be in memory all at once goes through StreamReader instead, which borrows from a buffer rather than the document; see Streaming. Attribute.raw_value is deliberately raw for the same reason: normalization cannot be undone, so the distinction between a typed tab and a 	 only exists before decoding.

• Well-formedness is checked. Mismatched end tags, duplicate attributes, unquoted attribute values, and unterminated constructs are errors, and nesting is capped at max_depth so a hostile document hits an error rather than a stack overflow. A name a namespace cannot read — <:/>, <a:/>, <a:b:c/> — is an error too, because prefixOf and localNameOf have no honest answer for one and a consumer that writes <:/> back out produces <>.

• Whitespace is normalized, and the rule depends on where the text is. Character data has its line endings folded to LF. An attribute value has that and every literal tab, line feed and carriage return replaced by a space, which is what makes an attribute written across two lines mean the same thing as one written on a single line. A character reference is never normalized, so 	 survives where a typed tab does not — which is why decodeAlloc asks which of the two you want and Element.attrAlloc answers for you.

• Empty elements are not a special case. <foo/> reports a start_element with self_closing set and then a synthetic end_element, so a consumer walking the event stream never needs to branch on it.

• Encoding is handled. toUtf8 works out what a document is in — a byte order mark, UTF-16 by the shape of its first characters, or the declaration's encoding — and decodes it to UTF-8 with zig-charset, which covers the forty encodings of the WHATWG Encoding Standard. The labels are read as a browser reads them, so iso-8859-1 is decoded as windows-1252, which is what files labelled that way overwhelmingly really contain, and a byte that means nothing becomes U+FFFD rather than an error. It returns the input unchanged when no conversion is needed.

• A DTD is declarations, not a verdict. initDtd reads the internal subset: <!ELEMENT> content models are compiled to an automaton, <!ATTLIST>, <!ENTITY> and <!NOTATION> are indexed by name. What the parser then does with them is the part XML asks of a processor that does not validate — resolve the declared entities, supply the declared attribute defaults — and it stops there. Checking a document against the declarations is somebody else's job, and the compiled models are exposed so that somebody else can do it.

• Entities are policy. Documents assembled by string concatenation contain bare & characters often enough that rejecting them would reject documents every other client accepts. .lenient passes an unknown reference through untouched; .strict reports error.UnknownEntity.

  .html additionally resolves the 248 HTML 4.01 named references — —,  , ’ and the rest — which XML does not define but RSS and Atom feeds use anyway.

  Reach for it only where the text really is HTML-adjacent. In the two Podcasting 2.0 feeds in the test corpus every one of the 2,261 named references sits inside a CDATA section, and CDATA is literal by definition — no policy touches it, because those entities belong to the HTML payload and to whatever renders it. .html changes nothing for those feeds. It earns its keep on the feeds that put   in a bare <title>.

Usage

const std = @import("std");
const xml = @import("zxml");

var reader: xml.Reader = .init(document);
while (true) {
    switch (try reader.next()) {
        .start_element => |e| {
            // `attributes` borrows storage inside the reader and dies at the
            // next call to `next`, so copy anything you keep.
            if (xml.nameIs(e.name, "segment")) {
                const number = e.attr("number") orelse continue;
                const id = try reader.trimmedTextAlloc(gpa, .lenient);
                defer gpa.free(id);
                std.debug.print("{s}: {s}\n", .{ number, id });
            } else {
                try reader.skipElement();
            }
        },
        .eof => break,
        else => {},
    }
}

reader.depth is the nesting level after the most recent event, which is what makes subtree-scoped loops straightforward: a subtree opened at depth d is closed by the end_element that brings depth back below d. skipElement and textAlloc are built on exactly that.

Streaming

StreamReader reads from a std.Io.Reader — a file, a socket, an HTTP body, a decompressor — so a document of any size is parsed through a buffer of the size you choose. Its API is Reader's: next, depth(), skipElement, textAlloc, trimmedTextAlloc, entityPolicy, and dtd() for what a <!DOCTYPE> declared.

var buffer: [64 * 1024]u8 = undefined;
var file_reader = file.reader(io, &buffer);
var reader: xml.StreamReader = .init(gpa, &file_reader.interface, .{});
defer reader.deinit();
while (true) {
    switch (try reader.next()) {
        .start_element => |e| if (xml.nameIs(e.name, "title")) {
            const title = try reader.trimmedTextAlloc(gpa, .lenient);
            defer gpa.free(title);
            std.debug.print("{s}\n", .{title});
        },
        .eof => break,
        else => {},
    }
}

It runs the same tokenizer as Reader, over whatever the Io.Reader has buffered, asking for more when a token runs off the end. So the two cannot disagree about a document: a test reads every sample document both ways, through buffers from 64 bytes up and with the input arriving a byte at a time, and a fuzz target does the same with arbitrary input. Three things differ:

• An event's slices last until the next call to next, because they point into the buffer and the next call reuses it. Copy anything you keep. (With Reader only Element.attributes is that short-lived.)
• Character data longer than the buffer arrives in pieces, as consecutive text or cdata events. A piece never ends inside an entity reference, a CRLF or a UTF-8 sequence, so each decodes on its own to exactly its share of the whole, and textAlloc joins them without being asked.
• Every other token has to fit in the buffer — a start tag with all its attributes, a comment, a processing instruction, the document type declaration with its internal subset — or next returns error.TokenTooLong. The buffer must be at least StreamReader.min_buffer_len (64) bytes; 64 KiB is plenty for anything that is not deliberately hostile.

The names of the open elements, which end tags are checked against, and the document type declaration, which the declarations borrow from, are copied with the allocator init takes; deinit frees them. A document in any encoding but UTF-8 is decoded as it is read, after the same detection toUtf8 does, so a stream yields what Reader yields for toUtf8 of the same bytes. reader.pos counts the bytes read so far, as Reader.pos does, for placing an error.

Namespaces

Prefixes are surfaced as text rather than resolved against their declarations, which is enough for formats that use a fixed, known vocabulary. localNameOf, prefixOf, and nameIs do the splitting, and Element.attrLocal matches an attribute regardless of prefix:

xml.nameIs("nzb:segment", "segment")        // true
xml.prefixOf("newznab:attr").?              // "newznab"

Document type declarations

Reader.init leaves a <!DOCTYPE> as the opaque token it has always been. Reader.initDtd reads it instead, which costs an allocator and is the only thing in the library that takes one at construction:

var reader: xml.Reader = .initDtd(gpa, document, .{});
defer reader.deinit();

Two things then happen on their own, and both are what XML asks of a processor that does not validate:

• Declared entities resolve. &company; gets the replacement text the internal subset gave it, recursively, in element text and attribute values alike. Go on passing .lenient; the reader adds its own declarations to whatever policy you name.
• Attribute defaults are supplied. An <!ATTLIST> default or #FIXED value appears on the Element even when the start tag omits it, and a value the start tag did write is left alone. Attribute.defaulted says which happened, since a default belongs to the document type rather than to the element and anything writing the document back out needs to know.

Either can be turned off with .{ .entities = false, .defaults = false }, which still reads the declarations and simply does not apply them.

What it will not do

It fetches nothing. An external subset, an external entity and a notation are all recorded and never dereferenced, so the guarantee that a document cannot make this parser open a file or reach the network survives the DTD intact. A reference to an external entity passes through untouched under .lenient and is error.ExternalEntityReference under .strict — refused rather than silently resolved to nothing.

Because an external subset is named and not read, Dtd.incomplete says so, and the declarations are then only part of the story. It is the honest answer, which matters to anything counting on them being complete.

Replacement text is expanded as character data and not re-parsed as markup, so an entity whose text contains a tag yields the tag as characters. .strict reports error.MarkupInEntity rather than making that substitution quietly.

Expansion is bounded in two ways, because the attacks are two: an entity that names itself, directly or through others, is error.RecursiveEntity, and one that merely multiplies — the billion laughs, where nothing is recursive and nothing is deeply nested — runs into error.EntityExpansionLimit.

Validating is somebody else's job

The declarations are read and exposed; no verdict is reached about them. A <!ELEMENT> content model is compiled to a position automaton, and Cursor walks it one child at a time using storage you provide:

const model = &reader.dtd.?.element("chapter").?.content.children;
var scratch: [8]u64 = undefined;
var cursor = model.cursor(scratch[0..model.scratchWords()]);

cursor.accept("title");   // true: the model allows it here
cursor.complete();        // false: `para+` still wants one

accept reports whether a name may appear at this point and complete whether the model may stop here. Turning either "no" into a validity error, along with ID uniqueness and IDREF resolution — questions a stream cannot answer, since an IDREF may point forward — belongs to something holding the whole document. ztree is where that lives.

Events

Event	Payload
declaration	version, encoding, standalone from <?xml … ?>
doctype	the declaration verbatim, internal subset included; reader.dtd holds what it declared, when it was read
start_element	name, attributes, self_closing
end_element	the name being closed, synthesized for <foo/>
text	character data, entities still in place
cdata	the contents of a CDATA section, literal
comment	the contents of a comment
pi	target and data of a processing instruction
eof	the document is exhausted, with nothing left open

Reaching the end of input with elements still open is error.UnexpectedEndOfDocument, not an eof.

Command line

The package also builds a zxml binary that dumps the event stream, one event per line, indented by nesting depth. The usual question about a misbehaving document is not "what does the tree look like" but "where did the parser stop agreeing with me", which a flat event stream answers directly. It doubles as a worked example of the API.

$ zig build run -- example.xml
declaration version="1.0" encoding="UTF-8"
<rss> xmlns:podcast="https://podcastindex.org/namespace/1.0" version="2.0"
  <channel>
    <title>
      text "Podcasting 2.0 Namespace Example"
    </title>
    <podcast:locked> owner="[email protected]"
      text "yes"
    </podcast:locked>

Whitespace-only text nodes are elided unless -w asks for them, since a pretty-printed document is mostly indentation. -c prints a tally by event kind instead of the stream, -q checks well-formedness and prints nothing, and -r shows text before entity decoding. -s reads the document through a 64 KiB buffer with StreamReader instead of loading it whole, so text longer than that is shown in pieces and an error is placed by byte offset rather than line and column. -d reads the internal DTD subset, so that declared entities resolve and defaulted attributes appear, marked (dtd) so they are not mistaken for something the document wrote:

$ zxml --dtd catalog.xml
declaration version="1.0" encoding="UTF-8"
doctype "catalog [\n<!ELEMENT catalog (product+)>\n<!ELEMENT product "…
  dtd root="catalog"
  dtd 5 elements
  dtd 1 attribute list
  dtd 3 entities
  dtd 1 notation
<catalog>
  <product> id="p1" currency(dtd)="USD" status(dtd)="new"
    <name>
      text "Example & Co. Widget"

A malformed document reports the line and column where the parser stopped, and exits non-zero:

$ zxml broken.xml
error: broken.xml:12:5: MismatchedEndTag

A file of - reads standard input, so a feed can be piped straight in.

Validating

--check reports on a document instead of dumping it: whether it is well-formed, what encoding it claims and whether it really is that, whether every text node and attribute decodes to valid UTF-8, and how much headroom it leaves against the parser's limits.

$ zxml --check podnews.xml
podnews.xml: ok
  well-formed        yes
  encoding           utf8
  elements           3796  (depth 5 / 256)
  attributes         max 7 / 64 on <rss>
  text nodes         5064, all valid UTF-8
  cdata              301 sections
  decoding           .lenient and .html agree

--feed adds a vocabulary profile on top, recognising RSS 2.0, Atom, and OPML from the root element and checking only what those specifications actually require — a channel's title, link, and description; an Atom entry's id, title, and updated; an outline's text attribute; an item having at least one of title or description; and that a pubDate is RFC 822 rather than the ISO 8601 that generators sometimes emit:

$ zxml --feed broken.xml
broken.xml: problems
  ...
  document kind      RSS 2.0
  items              1
  item 1: has neither <title> nor <description>  (line 1)
  channel: missing <link>  (line 1)
  2 problems

It reports nothing for optional elements, since a feed that omits one is not broken and crying wolf about it would make the tool useless on the real web. A document that is not a feed gets no opinion beyond the XML battery. Exit status is 0 when clean and 1 when not, and -q suppresses the report and leaves only that status, which is what to use in a script.

The feed vocabulary lives in the command-line tool, in src/feeds.zig. The library is a general XML parser and has no opinion about RSS.

Compatibility

The test suite covers the NZB specification's example document and a Podcasting 2.0 feed condensed from the podcast-namespace specification's own example — namespace prefixes, self-nesting itunes:category, nested podcast:value/podcast:valueRecipient, <content:encoded> CDATA, an xml-stylesheet processing instruction, and numeric character references.

Beyond the suite, the parser has been run over production feeds:

Feed	Size	Elements	Prefixes	Parse
podcast-namespace example.xml	17 KB	172	2	clean
Daily Tech News Show	113 KB	152	4	clean
Twenty Thousand Hertz	975 KB	272	3	49 ms
Security Now (audio and video)	121 KB	407 / 417	9	clean
Podnews Daily	2.3 MB	3,796	4	61 ms
Accidental Tech Podcast	3.0 MB	8,500	2	77 ms
Linux Unplugged	10 MB	15,622	5	47 ms

Podcast RSS is only one dialect, so the corpus also covers two unrelated vocabularies:

Document	Kind	Elements	Notes
Daring Fireball, Rust Blog, xkcd, GitHub releases	Atom	29–582	default namespace, no prefixes
Whatever (John Scalzi)	Atom and RSS	150 / 144	same content, both dialects
OPML 2.0 specification examples (all five)	OPML	22–77	attribute-only elements, depth 6
ISBN International RangeMessage	data	6,796	internal DTD subset, dotted names

Atom is a useful contrast because it puts its whole vocabulary in a default namespace, so nothing carries a prefix except extensions such as thr:; the podcast feeds are the opposite, prefixing almost everything. Scalzi's blog publishes the same posts as both Atom and RSS, which isolates the dialect from the content. OPML inverts the usual shape again: an <outline> has no text at all, carrying its data entirely in attributes, and nests deeper than any feed here — the specification's states.opml reaches depth 6.

The ISBN RangeMessage is the one document in the corpus that is not a feed at all, and it exercises three things none of the others do: a declaration quoted with apostrophes, element names containing dots (EAN.UCC), and a real internal DTD subset — 604 bytes of <!ELEMENT> declarations, which --dtd reads into four content models and init still hands over as one verbatim token. Parsing it yields 287 registration groups and 1,878 range rules in 23 ms, and reading 978 back as International ISBN Agency and 978-0 as English language confirms the content survives the trip.

Between them they cover eleven distinct namespace prefixes — podcast:, itunes:, media:, rawvoice:, googleplay:, creativeCommons:, sy:, content:, atom:, dc:, and a publisher's own jb: — along with 331 CDATA sections, 146,000 escaped </> pairs, and 2,402 astral-plane code points arriving as numeric character references. Every text node and attribute value decodes to valid UTF-8. As the parser counts them — which excludes the markup inside CDATA sections, where a grep would find far more — they peak at 13 attributes on an element and 5 levels of nesting, against caps of max_attributes (64) and max_depth (256).

They also disagree with each other in useful ways. Most are pretty-printed, so most of their text nodes are indentation; ATP is written compact, with 2 whitespace-only nodes out of 6,375, which is why the dumper elides such nodes by default rather than assuming they exist. ATP is likewise 85% CDATA by volume, where Linux Unplugged escapes the same kind of HTML into text instead. DTNS, generated by WordPress, is 91% CDATA and writes its <rss> start tag across four lines with tabs between the attributes and a newline before the closing bracket — the only feed here that does. Twenty Thousand Hertz, from Squarespace, is the only one to put a comment in the prolog, between the declaration and the root element, and it is 98% CDATA.

The two large feeds take opposite approaches to embedded HTML — one wraps it in CDATA, the other escapes it into text — and between them they establish that neither is mangled. In particular, decoding all 29,627 text nodes of the escaping feed produces byte-identical output under .lenient and .html, because decoding is a single left-to-right pass and cannot re-resolve the — that an — just produced.

Testing

$ zig build test

That includes three std.testing.fuzz targets, whose corpora replay on every run: one over arbitrary documents; one over arbitrary document types, which is given a <!DOCTYPE to work with because random bytes essentially never form one and the subset scanner is otherwise never reached at all; and one that reads arbitrary input both whole and as a stream, through a buffer and in chunks of sizes it chooses, and fails if the two disagree. Most of the time it puts a declaration or a byte order mark in front of its input, so that the decoders are in the path.

To fuzz continuously, with coverage feedback:

$ zig build test --fuzz          # until interrupted, with a web interface
$ zig build test --fuzz=1M       # a bounded run, then a report

The library's test binary is compiled with LLVM even in Debug, because the self-hosted backend emits no coverage instrumentation and the fuzzer would run blind. The bounded report names only the first fuzz test in the binary, but all of them run and share its coverage figure. A finding is saved to .zig-cache/f/crash.

Used by

• znzb — NZB file decoder
• znewznab — newznab API response decoder

References cited

These are kept in the zxml collection in Zotero.

• Bray, T., Paoli, J., Sperberg-McQueen, C. M., Maler, E., & Yergeau, F. (2008). Extensible Markup Language (XML) 1.0 (Fifth Edition). W3C. https://www.w3.org/TR/2008/REC-xml-20081126/
• van Kesteren, A. (2026). Encoding Standard. WHATWG. https://encoding.spec.whatwg.org/

License

MIT.