Investigate the index version 2 format and its MSZIP-compressed versionData #1

New issue

Open

opened 2026-10-13 19:51:51 -05:00 by jeff · 0 comments

jeff commented

2026-10-13 19:51:51 -05:00

Owner

The source publishes only source.msix with a schema 1.x index (1.7) and plain
YAML manifests. The current winget client also reads schema 2.0–2.1, served as
source2.msix, which is what Microsoft's own source uses.

Version 2 replaces the per-manifest paths in the index with one file per
package at packages/<PackageIdentifier>/<first 8 hex of hash>/versionData.mszyml.
That file is YAML compressed with the Windows Compression API's MSZIP framing,
which has no ready-made equivalent on Linux, and so is out of scope for the
first version of the source.

To find out:

The exact MSZIP framing the Compression API writes (block headers, the CK
signature, block size), and whether zlib's raw deflate plus that framing
reproduces a file the client accepts.
The keys in versionData (sV, vD, v, rP, s256H, …) and how the
client checks them against the index.
What a client gains from version 2 for a source this size, and whether it
ever stops reading 1.x.

Code to read in microsoft/winget-cli:

src/AppInstallerCommonCore/PackageVersionDataManifest.cpp
src/AppInstallerRepositoryCore/Microsoft/SQLiteIndexSourceV2.cpp
src/AppInstallerRepositoryCore/Microsoft/PreIndexed/RemotePackage.cpp

The source publishes only `source.msix` with a schema 1.x index (1.7) and plain YAML manifests. The current winget client also reads schema 2.0–2.1, served as `source2.msix`, which is what Microsoft's own source uses. Version 2 replaces the per-manifest paths in the index with one file per package at `packages/<PackageIdentifier>/<first 8 hex of hash>/versionData.mszyml`. That file is YAML compressed with the Windows Compression API's MSZIP framing, which has no ready-made equivalent on Linux, and so is out of scope for the first version of the source. To find out: - The exact MSZIP framing the Compression API writes (block headers, the `CK` signature, block size), and whether zlib's raw deflate plus that framing reproduces a file the client accepts. - The keys in versionData (`sV`, `vD`, `v`, `rP`, `s256H`, …) and how the client checks them against the index. - What a client gains from version 2 for a source this size, and whether it ever stops reading 1.x. Code to read in microsoft/winget-cli: - `src/AppInstallerCommonCore/PackageVersionDataManifest.cpp` - `src/AppInstallerRepositoryCore/Microsoft/SQLiteIndexSourceV2.cpp` - `src/AppInstallerRepositoryCore/Microsoft/PreIndexed/RemotePackage.cpp`