zig-flac

An encoder and decoder for the Free Lossless Audio Codec, written in Zig against RFC 9639, the 2024 specification of the format. It is a library first: it has no filesystem, no threads and no dependencies, reads from a std.Io.Reader and writes to a std.Io.Writer, and leaves the question of where the bytes come from to whoever is calling it. A command-line tool comes with it, mostly to prove the library can be used.

$ zig-flac encode -8 song.wav song.flac
song.wav: 4410000 samples, 44100 Hz, 2 channels, 16-bit -> song.flac (6.0MB, 33.8% of 17.6MB)
$ zig-flac test song.flac
song.flac: OK, 4410000 samples in 1077 frames, MD5 verified
$ zig-flac decode song.flac back.wav
song.flac: 4410000 samples, 44100 Hz, 2 channels, 16-bit -> back.wav (MD5 verified)

What it does

Decoding is complete. Every subframe type, every predictor, both residual coding methods and the escape code, all three stereo layouts, wasted bits, variable block sizes, every bit depth from 4 to 32, up to 8 channels, and every metadata block the format defines. The decoder hashes what it produces and checks it against the MD5 in the streaminfo block, which is the format's own end-to-end proof that nothing was lost.

Encoding is complete and competitive. It searches the models the format offers -- a constant, the samples verbatim, the five fixed predictors, and a linear predictor it solves for -- costs each one exactly in bits, and writes the cheapest; then it does the same for the four ways a stereo block can be split between its two subframes. There are nine presets, -0 through -8, in the spirit of the reference encoder's.

Metadata is read and written for all seven block types, including the cuesheet and picture blocks that most implementations skip. A block whose bytes the parse cannot fully account for keeps them, so that reading a file and writing it back never loses anything.

An ID3v2 tag at the front of a file is stepped over, since taggers written for other formats put one there and a file with one plays everywhere else.

Not implemented: seeking within a stream (the seek table is read and written, but nothing acts on it yet), the Ogg, Matroska and MP4 mappings of Section 10, and any use of threads. A stream whose sample rate, channel count or bit depth changes part-way through is refused rather than decoded: the format permits such a stream and permits a decoder to refuse it, and nothing downstream -- the MD5 over the whole stream least of all -- has anywhere to put the change.

Using the library

Fetch it and import the module it exposes as flac:

$ zig fetch --save git+ssh://git@git.ocjtech.us/jeff/zig-flac.git

const flac = b.dependency("zig_flac", .{ .target = target, .optimize = optimize });
exe.root_module.addImport("flac", flac.module("flac"));

Decoding

Decoder.init reads the signature and every metadata block, leaving the stream on its first frame. next then yields one frame at a time until the stream ends, and finish checks what the streaminfo block promised about the whole of it -- the sample count, and the MD5 of the audio.

const flac = @import("flac");

var d: flac.Decoder = try .init(gpa, reader, .{});
defer d.deinit();

std.debug.print("{d} Hz, {d} channels, {d}-bit\n", .{
    d.streaminfo.sample_rate,
    d.streaminfo.channels,
    d.streaminfo.bits_per_sample,
});

while (try d.next()) |frame| {
    for (frame.channels) |channel| {
        for (channel[0..frame.header.block_size]) |sample| {
            // one i32 per sample, in the channel order the frame header names
        }
    }
}
try d.finish();

The slices a frame hands out belong to the decoder and are overwritten by the next call to next. Samples arrive as i32, which is what the format's bit depths need; stereo decorrelation has already been undone.

d.blocks holds every metadata block the stream carried, in order, with the streaminfo block first. d.vorbisComment() and d.seekTable() reach for the two of those a caller usually wants:

if (d.vorbisComment()) |comment| {
    if (comment.get("ARTIST")) |artist| { ... }  // matched case-insensitively
}

Encoding

var e: flac.Encoder = try .init(gpa, writer, .{
    .sample_rate = 44100,
    .channels = 2,
    .bits_per_sample = 16,
}, .{ .level = .@"5" });
defer e.deinit();

try e.writeInterleaved(samples);   // or writePlanar, a slice per channel
try e.finish();

An encoder cannot know a stream's MD5, its true frame sizes or its length until it has seen the whole of it, so the streaminfo block it writes first is a placeholder. After finish, e.streaminfo holds the real one. A caller writing to something it can seek in should put it back; it is always 34 bytes at offset 8, right after the signature and the block header:

try file_writer.seekTo(flac.streaminfo_body_offset);
try e.streaminfo.write(&file_writer.interface);

A stream written to a pipe keeps the placeholder, which is legal: the format has encodings for "not known" in exactly these fields, and every decoder accepts them. Only the MD5 check and the length are lost.

Metadata goes in through the options, and padding is reserved by default so that tags can be added later without rewriting the file:

const fields = [_][]const u8{ "TITLE=Ægir", "ARTIST=zig-flac" };
const blocks = [_]flac.Block{.{ .body = .{ .vorbis_comment = .{
    .vendor = "my encoder 1.0",
    .fields = &fields,
} } }};

var e: flac.Encoder = try .init(gpa, writer, config, .{
    .level = .@"8",
    .metadata = &blocks,
    .padding = 8192,
    .seek_points = 64,
});

seek_points needs Config.total_samples, since a seek table has to be written before the audio it points into. The encoder reserves the table, fills it in as it goes, and leaves it in e.seek_points for a caller that can seek back to e.seek_table_offset and write it down.

The options

Everything below is set by level and can be overridden one at a time.

Option	What it does
level	.@"0" to .@"8". Sets the defaults for everything else.
block_size	Interchannel samples per frame. Bigger blocks predict better and pay for their headers less often; smaller ones bound what a damaged frame costs and how far a decoder must read to start.
stereo	.independent or .adaptive. Adaptive costs all four layouts the format allows and writes the cheapest, which is worth about a tenth of a stereo file.
max_lpc_order	The highest linear predictor order to solve for. Zero uses only the five fixed predictors, which is quick and surprisingly close.
qlp_precision	Bits per stored predictor coefficient, 1 to 15.
lpc_orders_costed	How many predictor orders to cost in full, taken in the order the recursion's error estimate ranks them. Zero costs every order.
apodizations	Tukey window parameters to solve a predictor against. Each one weighs the block's edges differently and so proposes a different predictor.
min_partition_order, max_partition_order	The range of Rice partition orders to cost. The streamable subset caps this at 8.
metadata, padding, seek_points	What goes between the streaminfo block and the audio.

The presets:

	-0	-1	-2	-3	-4	-5	-6	-7	-8
block size	1152	1152	1152	4096	4096	4096	4096	4096	4096
stereo	independent	adaptive	adaptive	adaptive	adaptive	adaptive	adaptive	adaptive	adaptive
max LPC order	0	0	0	6	8	8	12	12	32
orders costed	--	--	--	1	1	1	1	3	6
windows	--	--	--	1	1	1	2	2	3
max partition order	3	3	4	4	4	5	6	6	6

-5 is the default, as it is everywhere else. -0 has no stereo decorrelation, which follows the reference encoder's preset of the same name and is why it can be much worse than -1 on material whose two channels are alike.

WAVE files

flac.wav is not part of the format and nothing else in the library depends on it. It is there because audio to be encoded is usually in a WAVE file, and writing the same two hundred lines again is nobody's idea of a good time. It understands WAVE_FORMAT_PCM and WAVE_FORMAT_EXTENSIBLE holding PCM, at 8, 16, 24 or 32 bits per sample, and skips chunks it does not know. A data chunk whose size is zero or all ones -- which is what a WAVE file written to a pipe says, since the size cannot be filled in afterwards -- is read to the end of the stream instead.

The command-line tool

zig-flac encode [options] <input.wav> <output.flac>
zig-flac decode [options] <input.flac> <output.wav>
zig-flac info <input.flac>
zig-flac test <input.flac>...

encode takes -0 through -8, --block-size, --padding, --seek-points, --tag NAME=VALUE (repeatable) and --no-tags. decode takes --no-verify. info prints the streaminfo block and everything after it; test decodes a file and checks its MD5, which is the whole of what verifying a FLAC file means.

Both encode and decode write to a file rather than a pipe, because both finish by going back to fix up a header.

$ zig-flac info song.flac
song.flac
  sample rate     44100 Hz
  channels        2
  bits per sample 16
  total samples   4410000 (100.00 s)
  block size      4096
  frame size      3464 to 6147 bytes
  MD5             0e9cb317b4371398383cb934ecd13e2d
  vendor          zig-flac 0.1.0
  tag             TITLE=Ægir
  seek table      64 points (0 placeholder)
  padding         8192 bytes

How it compares

Twenty seconds of synthetic music, 44.1 kHz 16-bit stereo, against ffmpeg's FLAC encoder on the same machine. Percentages are of the 3,528,044-byte WAVE file; seconds are wall clock for the whole encode.

level	zig-flac		ffmpeg
-0	55.57%	0.05 s	37.02%	0.06 s
-1	36.94%	0.08 s	36.95%	0.06 s
-2	36.93%	0.09 s	36.93%	0.06 s
-3	35.08%	0.12 s	34.87%	0.06 s
-4	34.88%	0.13 s	34.56%	0.07 s
-5	34.88%	0.14 s	34.55%	0.06 s
-6	34.65%	0.22 s	34.54%	0.09 s
-7	34.30%	0.32 s	34.54%	0.09 s
-8	33.91%	0.96 s	34.23%	0.10 s

Within a third of a percent from -1 to -6, and smaller than ffmpeg at -7 and -8, where the predictor order search earns its time. It is two to three times slower than ffmpeg, which has hand-written vector code for the inner loops and this does not; at -5 that is still around 155 times faster than real time, and decoding is around 830 times faster than real time with the MD5 check on, or 1100 without it.

Measuring it yourself

zig build benchmark times encoding and decoding in process and against memory, so what it reports is the codec rather than the filesystem. Each figure is the fastest of several runs.

$ zig build benchmark -- --seconds 30 --levels 0,5,8
$ zig build benchmark -- --file song.wav --ablate

--ablate is the useful one. Rather than sampling a profile it turns one part of the search off at a time and reports what that saved, which measures the thing a change would actually alter. On the file above, from -5:

	size	time
no linear prediction	+5.32%	-43.6%
no stereo search	+48.56%	-40.7%
one Rice partition	+0.20%	-9.6%
every LPC order	-1.69%	+95.7%

The last row is why the presets cost a handful of predictor orders rather than all of them: the recursion's own error estimate ranks the orders nearly right, so costing the best few in full finds most of what costing all of them would, for a fraction of the time.

Across a wider corpus at -5, as a percentage of the source WAVE file:

	raw	zig-flac	ffmpeg
synthetic music, 16-bit stereo	3.5 MB	34.88%	34.55%
white noise, 16-bit stereo	353 kB	51.85%	51.88%
white noise, 24-bit stereo	265 kB	52.79%	52.82%
digital silence, 16-bit stereo	176 kB	4.77%	4.78%
1 kHz sine, 24-bit 96 kHz stereo	576 kB	19.64%	19.35%
440 Hz sine, 16-bit stereo	353 kB	10.24%	9.40%
440 Hz sine, 16-bit 6-channel	576 kB	16.61%	15.15%

One thing that is not a comparison: ffmpeg's FLAC encoder is not lossless at 32 bits per sample. Given 32-bit PCM it truncates to 24 and says so in the stream; zig-flac encodes all 32 bits, and ffmpeg's decoder reads them back unchanged. So a 32-bit file from zig-flac is bigger than one from ffmpeg for the same reason a lossless file is bigger than a lossy one.

How it is tested

zig build test runs everything below except the last two.

Against the specification's own worked examples. Appendix D of RFC 9639 prints three complete FLAC files in hexadecimal and walks each one through field by field. All three are in src/tests/rfc9639/, transcribed byte for byte, and between them they cover every subframe type, both stereo layouts that turn up in such short files, wasted bits, an escaped Rice partition and a residual split across partitions. Where a test asserts a sample value, a CRC or a coded number, the number is one the RFC prints rather than one this implementation produced -- "the value is then 0b011000 11111101 00, or 25588" is a test.

Against a reference encoder. Every file the decoder is pointed at carries an MD5 of its own audio, so decoding one and comparing digests is a complete check with no reference decoder needed. The decoder has been run over FLAC files produced by ffmpeg at all thirteen of its compression levels, at 8, 16, 24 and 32 bits per sample, at sample rates from 8 kHz to 192 kHz, in 1 to 8 channels, including silence, white noise and a file of a single frame. All verify.

In the other direction, files this encoder produced have been decoded by ffmpeg and compared with the original PCM, byte for byte, at every level and across the same corpus.

Round trips. The unit tests encode and decode again at every level, every bit depth from 4 to 32, every channel count from 1 to 8, and block sizes chosen so that the last frame is short -- comparing every sample, and checking the MD5 the encoder computed against the audio the decoder produced.

Fuzzing. src/tests/fuzz.zig holds three targets, each stating a property that has to hold for every input rather than the ones somebody thought of: decoding anything terminates, reports and frees; reading metadata and writing it back changes nothing; and a round trip is lossless whatever the samples and whatever the settings. They run against a bounded allocator, so a stream that asks for more memory than the machine has is refused rather than obeyed -- which exercises the allocation-failure paths at the same time.

Zig 0.16.0 cannot build a test executable in fuzz mode without a one-word patch to its own standard library, and even with the patch nothing populates its coverage tables, so zig build fuzz --fuzz gets only half way. The devshell applies the patch anyway, and zig build fuzz-run is the loop around the same targets without the part that does not work: it mutates the corpus, hands the result to a target, and saves anything that comes back with an error.

Three real defects came out of it, which is the argument for having it:

• A double free in Decoder.init, on the path where a stream's streaminfo block parses but a later allocation fails.
• Metadata blocks whose declared length disagrees with what their structure implies -- a Vorbis comment block with bytes after its last field, a padding block whose padding is not zeroes -- losing those bytes on being written back.
• An integer overflow reconstructing a linear predictor. A residual sample is bounded by the format but a prediction is not, so a stream can be written whose samples grow without bound however wide the arithmetic. Each restored sample is now checked against the depth its subframe codes at, before the next one uses it, which is what keeps every multiplication inside 64 bits.

Getting the source

The repository has two homes, and they carry the same history.

The everyday one is a Forgejo instance:

git clone ssh://git@git.ocjtech.us/jeff/zig-flac.git

It is also published on Radicle, a peer-to-peer code forge built on git, where a repository is found by its Repository ID (RID) rather than by a server name:

rad:z3zwzMBNErfqXqj27E7aJFkuXL6HA

If you do not have Radicle yet, install it, create an identity, and start your node:

curl -sSf https://radicle.dev/install | sh
rad auth
rad node start

Then clone the repository:

rad clone rad:z3zwzMBNErfqXqj27E7aJFkuXL6HA

rad clone consults your node's routing table to find seeds that carry the repository, and drops the working copy into a zig-flac directory. Pass a target path as a second argument to put it somewhere else. If your node has not discovered a seed yet, name one directly with --seed <NID>.

The result is an ordinary git working copy on the main branch, with a rad remote wired up, so everyday work is just git. To pull in changes published since your clone:

rad sync --fetch
git pull rad main

Building

Zig is not on PATH; everything runs through the flake's devshell.

$ nix develop --command zig build test        # unit tests, and the fuzz corpus
$ nix develop --command zig build check       # compile everything, run nothing
$ nix develop --command zig build             # the command-line tool
$ nix develop --command zig build run -- info song.flac
$ nix develop --command zig build docs        # generated API documentation
$ nix develop --command zig build fuzz-run -- --seconds 300
$ nix develop --command zig build benchmark -- --ablate
$ nix develop --command zig fmt --check .
$ nix develop --command reuse lint            # every file needs an SPDX header

One test, by name -- the build has no filter option, so go at the module directly:

$ nix develop --command zig test src/root.zig --test-filter "wasted bits"

.forgejo/workflows/test.yaml runs reuse lint, zig fmt --check, zig build test, zig build check and twenty seconds of fuzz-run on every push -- all of it through the same devshell, so the toolchain that builds in CI is the toolchain that builds here.

What is where

src/root.zig	The public API, and what everything else is for.
src/bits.zig	The bit-level reader and writer a frame is built out of, carrying the two CRCs as bytes go past.
src/crc.zig	Which catalogued CRCs the format's two checks are.
src/format.zig	The format's limits, its enumerations, and the UTF-8-like coded number.
src/frame.zig	Frame and subframe headers.
src/metadata.zig	The blocks between the signature and the first frame.
src/decoder.zig	Frames in, samples out.
src/encoder.zig	Samples in, frames out, by way of a search over every model the format offers.
src/lpc.zig	Predictor analysis: windowing, autocorrelation, Levinson-Durbin, quantization.
src/wav.zig	Enough of RIFF/WAVE to get uncompressed samples in and out.
src/main.zig	The command-line tool.
src/tests/fuzz.zig	The fuzz targets and their corpus.
tools/fuzz.zig	The loop that drives them without Zig's fuzzer.
tools/benchmark.zig	How fast it is, and where the encoder's time goes.

A note on the arithmetic

The format is lossless, which means every step from sample to bitstream and back has to be exact. Everything that touches a sample here is integer arithmetic in i64, which is what the specification recommends when the alternative is proving that a narrower type cannot overflow (Appendix A). The one place floating point appears is lpc.zig, in finding a predictor -- and the specification is explicit that this is fine: "Use of floating-point representations in analysis (e.g., finding a good predictor or Rice parameter) is not a concern as long as the process of using the found predictor and Rice parameter to encode audio samples is implemented with only integer math." The coefficients that analysis proposes are quantized to integers before a single residual is computed from them, so what comes out is lossless whatever the analysis said.

The residual coder is exact in a stronger sense. The cost of a Rice partition at parameter k is n * (1 + k) plus the sum of the folded residuals shifted right by k, and that sum is additive: the sums at the finest partition order add up into the sums at every coarser one. So one pass over the residual yields the true cost of every parameter at every partition order, and the encoder picks the true best rather than one near it -- no estimate, no search window.

Licence

MIT. Every file carries an SPDX header and reuse lint is expected to pass; LICENSES/MIT.txt is the text. The three example files under src/tests/rfc9639/ are code components of RFC 9639 and carry the IETF Trust's Revised BSD licence instead, which REUSE.toml records.