zig-paperless

A client for the Paperless-ngx REST API, for Zig 0.17: finding documents, reading and changing what is known about them, downloading their files, uploading new ones and waiting for them to be consumed, and the tags, correspondents, document types, storage paths, custom fields, saved views, share links, users and groups they are filed under.

HTTP is zig-http's client, uploads stream from a reader through a multipart/form-data body that is never held in memory, and the media type table and the RFC 2231 decoding of download filenames come from zig-mime.

const paperless = @import("paperless");

var client: paperless.Client = try .init(io, gpa, .{
    .base_url = "https://paperless.example.com",
    .auth = .{ .token = token },
});
defer client.deinit();

const task_id = try client.documents().upload(.{
    .filename = "invoice.pdf",
    .content = .{ .bytes = pdf },
    .tags = &.{inbox_tag},
});
const task = try client.tasks().wait(&task_id, .{});
defer task.deinit();
const id = task.value.documentId() orelse return error.UploadFailed;

const doc = try client.documents().update(id, .{
    .title = "Invoice 42",
    .correspondent = .{ .set = acme },
    .document_type = .clear,
});
defer doc.deinit();

The API documentation is at https://jeff.jcollie.page/zig-paperless/. It is generated from the doc comments, which carry most of the explanation of how each call behaves.

Where this lives

The repository lives in three places that carry the same history. The Forgejo instance at https://git.jcollie.dev/jeff/zig-paperless is the web-visible one, and is where the continuous integration and the published documentation are:

$ git clone https://git.jcollie.dev/jeff/zig-paperless.git

It is mirrored on Tangled at https://tangled.org/jcollie.dev/zig-paperless, and it is also on the Radicle network, where a repository is found by its identifier and nothing else:

rad:z2Y41dMQk7mnpxNV2D7oPPSzU5wEx

rad clone rad:z2Y41dMQk7mnpxNV2D7oPPSzU5wEx fetches it from any node that seeds it. Any of the three is the whole project.

Adding it to a project

It needs Zig 0.17.0.

$ zig fetch --save git+https://git.jcollie.dev/jeff/zig-paperless.git

const paperless = b.dependency("paperless", .{
    .target = target,
    .optimize = optimize,
}).module("paperless");
exe.root_module.addImport("paperless", paperless);

What is covered

The models and the calls are written against API version 10, which is what Paperless-ngx 3 speaks by default, and are tested against Paperless-ngx 3.2.1. Client.Options.api_version asks for another; version 9 is also accepted by 3.x servers.

client.documents()	list and pages with the common filters as fields, get, update, delete, metadata, suggestions, notes, share links, nextAsn, autocomplete, download (archive, original, preview, thumbnail), upload, updateVersion, bulkEdit, deleteMany, reprocess, rotate, merge
client.tasks()	list, pages, get, byTaskId, acknowledge, and wait, which asks until a task has finished; Task.outcome says how an upload ended — a new document, a refused duplicate, stopped by a plugin, or failed — since status alone does not
client.tags(), correspondents(), documentTypes(), storagePaths(), customFields(), savedViews(), shareLinks(), users(), groups()	list, pages, get, create, update, delete, and findByName where the endpoint filters by name
client	statistics, status, remoteVersion, search (the global search box), trash, restoreFromTrash, emptyTrash
client.raw	any other endpoint, by path — mail accounts and rules, workflows, the application configuration — with the same authentication and the same error handling
paperless.obtainToken	trades a username and password for the user's API token

What comes back

A JSON response is a std.json.Parsed(T): it owns everything it points to and deinit frees it. Fields the server sends and the model does not name are ignored, so a server newer than the client does not break it, and nearly every field of a model has a default — what arrives depends on how it was asked for (?fields=id,title trims a document to two fields; ?full_perms=true swaps user_can_change for permissions), and anything absent reads as its default rather than failing the parse. Enumerations that grow from release to release, such as a task's type, read a value they do not know as unknown.

Dates and times are the ISO 8601 strings Paperless sent. Interpreting them is left to the caller rather than imposing a calendar library on everybody.

What is sent

Each object has a …Fields struct for creating and updating it. A field left null is not sent, so the same struct is a whole object to create and a partial one to update, which is a PATCH. A field Paperless allows to be null is a ?Clearable(T), so that leaving it alone and clearing it are different: .correspondent = .clear sends null.

List filters are fields named exactly as django-filter names them — title__icontains, tags__id__all — so the Paperless documentation, the browsable API and this library spell a filter the same way. Anything a query struct does not name goes in its extra.

When it goes wrong

A status outside 2xx is an error named for it — error.NotFound, error.BadRequest, error.Unauthorized — and the body that came with it is kept as client.last_error until the next request, since a 400 from Paperless explains itself field by field and an error code cannot carry that.

Lists are paged by number. The next link a page holds is built by the server from the request as it saw it, which behind a proxy can be a scheme, host or port the client cannot reach, so pages() counts instead of following it.

The command-line tool

zig-paperless is a small client that is also the worked example of the library. It reads the server from PAPERLESS_URL and the credential from PAPERLESS_TOKEN:

$ export PAPERLESS_URL=https://paperless.example.com
$ PAPERLESS_PASSWORD=... zig-paperless token jeff
$ export PAPERLESS_TOKEN=...
$ zig-paperless status
$ zig-paperless upload -t 4 --title "Invoice 42" invoice.pdf
invoice.pdf  consuming... document 812
$ zig-paperless list invoice
$ zig-paperless show 812
$ zig-paperless download --original 812

upload streams each file from disk, and download streams into a file of its own that is renamed into place only once the whole document has arrived.

Downloads stream in the library too: the body goes to the caller's writer as it arrives, and only when the status is 2xx, since zig-http says what the status is before any of the body is read. An error page never reaches the writer; it is kept as last_error like any other.

Where the schema and the server disagree

Paperless-ngx 3.2.1's OpenAPI schema says a tag's children are identifiers and a search hit's note_highlights is always a string; the server sends tag objects, and null. The models follow the server. A Paperless install generates the schema at /api/schema/, and with the package from nixpkgs paperless-ngx spectacular --format openapi-json --file schema.json writes it without a running server.

Testing

$ nix develop -c zig build test

runs three sets of tests:

• the unit tests in src/, for the JSON quirks, the query strings, the multipart writer and the Content-Disposition parsing;

• tests/client.zig, which runs zig-http's own server on a loopback socket as a fake Paperless — its JSON copied from a real one — and checks both what each request looked like when it arrived and how each response was read;

• and compiles tests/live.zig, which needs a real server and runs as zig build live-test when one is named:

  $ PAPERLESS_URL=http://127.0.0.1:8000 PAPERLESS_TOKEN=... nix develop -c zig build live-test
  

  It creates what it needs with a random suffix and removes it again.

nix flake check builds the package and boots Paperless-ngx from the NixOS module — Redis, Celery, the consumer — in a virtual machine, obtains a token with the command-line tool, runs the live tests against it, and uploads, lists and downloads a file with the tool. The live tests are what found the two places where the schema and the server disagree, and building this found two things in zig-http and one in zig-mime, each fixed there.

zig build docs writes the API documentation to zig-out/docs, and zig build docs-serve serves it at http://127.0.0.1:8000/.

License

MIT; the project follows REUSE, and reuse lint passes.

References cited

• Paperless-ngx contributors. “REST API.” Paperless-ngx documentation. https://docs.paperless-ngx.com/api/.
• Freed, N. and N. Borenstein. Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types. RFC 2046, RFC Editor, November 1996. https://www.rfc-editor.org/info/rfc2046.
• Freed, N. and K. Moore. MIME Parameter Value and Encoded Word Extensions: Character Sets, Languages, and Continuations. RFC 2231, RFC Editor, November 1997. https://www.rfc-editor.org/info/rfc2231.
• Berners-Lee, T., R. Fielding and L. Masinter. Uniform Resource Identifier (URI): Generic Syntax. RFC 3986 (STD 66), RFC Editor, January 2005. https://www.rfc-editor.org/info/rfc3986.
• Reschke, J. Use of the Content-Disposition Header Field in the Hypertext Transfer Protocol (HTTP). RFC 6266, RFC Editor, June 2011. https://www.rfc-editor.org/info/rfc6266.
• Masinter, L. Returning Values from Forms: multipart/form-data. RFC 7578, RFC Editor, July 2015. https://www.rfc-editor.org/info/rfc7578.
• Fielding, R., M. Nottingham and J. Reschke, editors. HTTP Semantics. RFC 9110 (STD 97), RFC Editor, June 2022. https://www.rfc-editor.org/info/rfc9110.