Codepages #14

New issue

Closed

opened 2026-10-14 01:26:57 -05:00 by jeff · 2 comments

jeff commented

2026-10-14 01:26:57 -05:00

Owner

Parent

Part of #1

What to build

Product/@Codepage and Package/@SummaryCodepage choose the Windows codepage the installer database's strings and the summary information are stored in, defaulting to 1252. Encoding goes through zig-charset. A character the codepage cannot encode is an error naming the attribute; codepage 65001 is accepted with a warning that Windows Installer does not support it officially.

Acceptance criteria

Seam 1 tests: a product name with characters outside ASCII stored in 1252, and in another single-byte codepage, round-trips through the installer database as the right bytes.
Seam 1 test: a character 1252 cannot encode is a diagnostic naming the attribute, file, line and column.
Seam 1 test: 65001 gives a warning and builds.
VM test: msiinfo reports the non-ASCII product name correctly, and the tables match the reference implementation's with generated columns masked.

## Parent Part of #1 ## What to build `Product/@Codepage` and `Package/@SummaryCodepage` choose the Windows codepage the installer database's strings and the summary information are stored in, defaulting to 1252. Encoding goes through `zig-charset`. A character the codepage cannot encode is an error naming the attribute; codepage 65001 is accepted with a warning that Windows Installer does not support it officially. ## Acceptance criteria - [ ] Seam 1 tests: a product name with characters outside ASCII stored in 1252, and in another single-byte codepage, round-trips through the installer database as the right bytes. - [ ] Seam 1 test: a character 1252 cannot encode is a diagnostic naming the attribute, file, line and column. - [ ] Seam 1 test: 65001 gives a warning and builds. - [ ] VM test: `msiinfo` reports the non-ASCII product name correctly, and the tables match the reference implementation's with generated columns masked.

jeff added the

ready-for-agent

label

2026-10-14 01:26:57 -05:00

jeff added a new dependency

2026-10-14 01:27:07 -05:00

#2 Tracer bullet: an empty installer package, end to end

jeff commented

2026-10-14 02:31:15 -05:00

Author

Owner

Found while doing #2: the reference implementation (msitools 0.106 wixl) ignores Product/@Codepage for the database. Given Codepage="1252" it still stores 0 (neutral) as the database code page, so msidump writes 0 in _ForceCodepage.idt. zixl stores the code page the attribute names, as WiX documents. This ticket needs to decide whether to follow WiX here (and teach the table comparison that this difference is expected) or follow the reference implementation.

Found while doing #2: the reference implementation (msitools 0.106 `wixl`) ignores `Product/@Codepage` for the database. Given `Codepage="1252"` it still stores 0 (neutral) as the database code page, so `msidump` writes `0` in `_ForceCodepage.idt`. zixl stores the code page the attribute names, as WiX documents. This ticket needs to decide whether to follow WiX here (and teach the table comparison that this difference is expected) or follow the reference implementation.

jeff referenced this issue

2026-10-14 02:59:09 -05:00

Tracer bullet: an empty installer package, end to end #2

jeff self-assigned this

2026-10-14 05:16:35 -05:00

jeff referenced this issue from a commit

2026-10-14 05:39:57 -05:00

Store strings in the code page the WiX source chooses (#14)

jeff commented

2026-10-14 05:55:47 -05:00

Author

Owner

Done in 749183f; CI run 11 is green.

The decision this issue left open: zixl follows WiX and Q22. The database's code page comes from Product/@Codepage and the summary's from Package/@SummaryCodepage, both defaulting to 1252.
Encoding: strings are encoded with zig-charset:
- 874 and 1250 to 1258;
- 0, ASCII only;
- 65001, accepted with a warning.
Any other code page is refused.
Diagnostics: every stored piece of text is recorded with its location and checked once the code pages are known. A character that won't fit names the attribute and its file, line and column.
Seam 1: the 1252 and 1251 bytes as written, the summary information in its own code page, the unencodable diagnostic, neutral holding ASCII alone, the 65001 warning, and an unsupported code page.
End to end:
- the harness corrects wixl's _ForceCodepage, always 0, to the code page the fixture asks for;
- the codepage fixture isn't compared with wixl, since .wixl-skip explains that wixl stores every string as UTF-8 in a neutral database;
- under Wine, that fixture's product name and publisher, "Café Zürich" and "Société Générale de Test", are registered exactly, on x86 and x64.

Done in 749183f; CI run 11 is green. - **The decision this issue left open:** zixl follows WiX and Q22. The database's code page comes from `Product/@Codepage` and the summary's from `Package/@SummaryCodepage`, both defaulting to 1252. - **Encoding:** strings are encoded with zig-charset: - 874 and 1250 to 1258; - 0, ASCII only; - 65001, accepted with a warning. Any other code page is refused. - **Diagnostics:** every stored piece of text is recorded with its location and checked once the code pages are known. A character that won't fit names the attribute and its file, line and column. - **Seam 1:** the 1252 and 1251 bytes as written, the summary information in its own code page, the unencodable diagnostic, neutral holding ASCII alone, the 65001 warning, and an unsupported code page. - **End to end:** - the harness corrects wixl's `_ForceCodepage`, always 0, to the code page the fixture asks for; - the `codepage` fixture isn't compared with wixl, since `.wixl-skip` explains that wixl stores every string as UTF-8 in a neutral database; - under Wine, that fixture's product name and publisher, "Café Zürich" and "Société Générale de Test", are registered exactly, on x86 and x64.

jeff closed this issue

2026-10-14 05:55:48 -05:00

jeff referenced this issue from a commit

2026-10-16 13:51:01 -05:00

Keep build-machine paths out of the product and package codes