Matterhorn 31-028 · PDF/UA-1 UA1:7.21.7-2
ToUnicode entries that map to U+0000, and who writes them
U+0000 is not a valid mapping target; it usually means the producer could not determine the character and wrote a placeholder.
The condition, as the protocol states it. “One or more Unicode values specified in the ToUnicode CMap are zero (0).”
Matterhorn Protocol 1.1, checkpoint 31, index 31-028, section UA1:7.21.7-2. Software can decide this one on its own.
What a screen reader actually does
Those characters extract as NUL and are dropped or announced as noise.
What the entry looks like
A ToUnicode CMap maps each character code a font uses to the Unicode text it stands for. Each target is written in UTF-16, and <0000> is U+0000, the NUL control character:
4 beginbfchar
<00> <0000> % code 0 → NUL
<01> <00E9> % code 1 → é
<02> <0000> % code 2 → NUL
<20> <0020> % code 32 → space
endbfchar
or, as a range, every code mapped to the Unicode value with the same number, starting from U+0000:
1 beginbfrange
<0000> <FFFF> <0000>
endbfrange
PDF/UA-1 clause 7.21.7 requires every value in the map to be greater than zero. Condition 31-028 is any entry that is not. A map like this passes the simplest check there is — the font has a /ToUnicode entry — which is why 31-027 does not catch it.
What the user hears
Where a code the text actually uses is mapped to U+0000, that character is gone. Readers drop NUL, or pass it to speech as noise: a word whose é was given a code mapped to zero reads as caf. Copy, paste and search lose it in the same way.
Where only unused codes map to zero, nobody hears anything wrong. The map is still non-conformant, and a checker that reads the map will say so.
Who writes it
Two producers we could test directly do this by default.
ReportLab, the Python PDF library. The first subset of every TrueType font it embeds gets a ToUnicode map with slots for codes 0 to 31, reserved for characters outside ASCII, and every slot not yet used is written as <0000>. A short English document made with ReportLab 5.0.1 and an embedded Arial produced a map with 31 entries pointing at U+0000, and Taggart reported:
The ToUnicode CMap of /AAAAAA+ArialMT maps 31 code(s) to U+0000, which is not a valid mapping target.
Tesseract, the OCR engine, when it writes a searchable PDF. Its invisible text layer uses a font called GlyphLessFont whose ToUnicode map is the single range shown above: every code maps to the Unicode value with the same number, starting from zero. Code 0 is not used for text, but it is in the map. Tesseract 5.5.2 on a one-line image gave:
The ToUnicode CMap of /GlyphLessFont maps 1 code(s) to U+0000, which is not a valid mapping target.
Tools built on either of them inherit the behaviour unless they post-process the map. Taggart’s own report renderer uses ReportLab and strips the zero entries before saving, for this reason.
How to check it free
Copy and paste catches used codes mapped to zero: missing characters where the page shows text. It cannot catch unused ones.
Read the CMap. Find the font’s /ToUnicode object number in its dictionary (qpdf --show-object=12 file.pdf, with the font’s number from pdffonts), then print the stream: qpdf --show-object=14 --filtered-stream-data file.pdf. Look at the second value on each bfchar line and the third on each bfrange line. <0000> in that position is the condition; <0000> as the first value is just code 0, which is fine.
veraPDF’s PDF/UA-1 profile tests the Unicode value of every glyph the document shows against zero, U+FEFF and U+FFFE. The test is per glyph shown, so map entries no text uses are outside it.
Acrobat Pro has an Accessibility Checker rule called Character encoding, and Adobe’s guidance for it in Create and verify PDF accessibility is to fix the source: change the font, re-create the PDF. Adobe does not say which mapping problems that rule detects, and says some character-encoding problems are not repairable within Acrobat. Preflight’s Options → Browse Internal PDF Structure shows the font dictionaries read-only, per Adobe’s advanced preflight inspections page; it documents no way to edit a CMap.
How to fix it free
Delete the zero entries. When they cover only codes the text never uses — the ReportLab and Tesseract cases — removing them changes nothing that is drawn or read. Open the file with qpdf in QDF mode (qpdf --qdf --object-streams=disable in.pdf edit.pdf), delete the <..> <0000> lines from the bfchar block, correct the count at the start of the block, and run fix-qdf edit.pdf out.pdf.
Tesseract’s single range needs rewriting rather than trimming, because a range may vary only in its last byte and <0000> <FFFF> breaks that rule as well. The replacement is one range per high byte, starting at <0001>: <0001> <00FF> <0001>, <0100> <01FF> <0100>, and so on, skipping the surrogate block from <D800> to <DFFF>. That is what Taggart’s own OCR writes when it runs Tesseract.
If a used code maps to zero, deleting the entry leaves the character unmapped, which is 10-001 instead of this. The real fix is to write the right character, and that needs to come from somewhere: the source document, or a re-export.
What Taggart does
Taggart parses the whole ToUnicode map, bfchar and bfrange blocks alike, and counts the targets that are zero. One finding per font, with the count and the font’s object number. It reads the map rather than the text, so it reports zero entries for codes nothing uses, as on ReportLab files.
The repair is tounicode-repair. It is not applied in bulk: it writes a statement about what characters a font contains, so it is proposed and waits for someone to accept it. It has three outcomes:
- The document says what the character is. Where a span around a single glyph carries
/ActualText, and every such span for that code agrees, the code is mapped to that text and the zero entries are dropped from the map. The change log names each code and the character it now maps to. - The font uses WinAnsiEncoding or MacRomanEncoding, or Differences names that spell out a code point, such as
uni00E9. The map is rebuilt from that published table and those names — replaced, not edited — and the change log says so: wrote a /ToUnicode CMap for 219 code(s), derived from /WinAnsiEncoding. - Anything else is refused, with this message, and the finding stays in the re-checked report:
cannot recover a Unicode mapping for: /AAAAAA+ArialMT. Re-export the source document with the fonts embedded rather than letting Taggart guess.
That last case includes both producers above. ReportLab’s subsets are symbolic TrueType fonts with no standard encoding, and Tesseract’s is a composite font, so neither qualifies for a rebuild, and Taggart does not special-case zero entries for unused codes — the one situation where deleting them would be safe. On those files the hand edit above is the fix, and the advice in the message to re-export with fonts embedded does not apply, since the fonts already are.
How Taggart handles it
Taggart checks for this condition automatically and reports it as a blocker, with where in the file it found it.
The fix is Repair ToUnicode CMaps. It changes what the document means, so Taggart proposes it and waits for your approval rather than applying it unattended.
Whatever Taggart changes, the page still looks identical. Fixes edit the PDF object model, not the content streams, and every remediation is re-rendered and compared against the original at structural similarity ≥ 0.999 before you get the file back. A tool that quietly reflows your document is worse than no tool.
Frequently asked questions
Why would a producer write U+0000 at all?
Usually as a placeholder. ReportLab gives every TrueType subset a map with slots for codes 0 to 31 and fills the unused ones with U+0000; Tesseract writes its identity map as one range starting at zero. Elsewhere it is what a producer writes when it could not work out the character.
If no text uses the code, does it matter?
Not to a listener, but the condition is about the map, and Taggart reads the map. veraPDF tests only the codes the text actually shows, so on a ReportLab file Taggart reports this condition and veraPDF may not.
Can Taggart repair it?
Sometimes. If the document's own /ActualText says what a code is, or the font uses WinAnsi or MacRoman encoding, it can write a correct map, and it asks before doing so. For anything else it refuses and says why, including ReportLab and Tesseract files where deleting the zero entries would have been enough.
Where this sits in the standards
| Standard | Reference |
|---|---|
| Matterhorn Protocol 1.1 | Checkpoint 31 (Fonts), index 31-028 |
| PDF/UA-1 (ISO 14289-1) | Clause 7.21.7-2 |
| WCAG 2.1 | 1.3.1 Info and Relationships — Level A |
| EN 301 549 / Section 508 / ADA Title II | All three point at WCAG 2.1 Level AA for non-web documents, so a Level A or AA criterion here is in scope for each of them. |
Related conditions
- 31-029 Unicode values in the ToUnicode CMap are U+FEFF or U+FFFEByte-order marks are not characters and must not appear as mapping targets.
- 31-027 A font dictionary does not contain the ToUnicode entry and no exemption appliesA ToUnicode CMap is required unless the font uses MacRoman/MacExpert/WinAnsi encoding, or is a Type 1/Type 3 whose glyph names are all in the Adobe Glyph List or the Symbol set, or is a Type 0 whose descendant uses a standard Adobe character collection, or is a non-symbolic TrueType.
- 10-001 Character code cannot be mapped to UnicodeA glyph is painted whose character code has no Unicode equivalent reachable through the font's ToUnicode CMap, encoding, or glyph names. The text renders correctly but cannot be extracted.
- 31-030 Characters used in text showing operators reference the .notdef glyphThe content stream paints glyph index 0, which is by definition "no glyph" — typically a hollow box or nothing at all.
Other conditions in checkpoint 31
Checkpoint 31 covers fonts. These are its other failure conditions; the ones without a link are in the rule set but do not have a written page yet.
- 31-001 CIDSystemInfo Registry values are not identicalThe CMap and the descendant CID font must agree on the character collection registry.
- 31-002 CIDSystemInfo Ordering values are not identicalThe CMap and the descendant CID font must agree on the character collection ordering.
- 31-003 CIDSystemInfo Supplement of the CID font is lower than that of the CMapThe font's character collection supplement must be at least as new as the CMap's, or the CMap may reference CIDs the font does not define.
- 31-004 A Type 2 CID font contains neither a stream nor the name Identity as CIDToGIDMapCIDToGIDMap must be either the name /Identity or a stream mapping CIDs to glyph indices.
- 31-005 A Type 2 CID font does not contain a CIDToGIDMap entryThe entry is absent entirely.
- 31-006 A CMap is neither listed in ISO 32000-1 Table 118 nor embeddedA predefined CMap must be one of the standard names; anything else must be embedded in the file.
- 31-007 The WMode entry in a CMap dictionary is not identical to the WMode value in the CMap streamWriting mode (horizontal or vertical) must be declared consistently in the dictionary and the stream.
- 31-008 A CMap references another CMap which is not listed in ISO 32000-1 Table 118A UseCMap chain must terminate at a predefined CMap.
- 31-009 For a font used by text intended to be rendered the font program is not embeddedPDF/UA-1 requires every font used for visible text to be embedded, so the document renders and extracts identically everywhere. A glyph counts as rendered if the text render mode is not 3.
- 31-010 A font program is embedded that is not legally embeddable for unlimited, universal renderingThe font's embedding permission bits forbid the kind of embedding used. Machines can read the flags but the flags may not reflect the actual licence.
- 31-011 The embedded font program does not contain glyphs for all referenced glyphsThe subset embedded in the file is missing glyphs the content stream asks for — very common when a document is edited after subsetting.
- 31-012 A glyph present in an embedded Type 1 font program is not listed in CharSetCharSet must accurately list the glyphs in the embedded subset.
- 31-013 A glyph listed in CharSet is not present in the Type 1 font programCharSet promises glyphs the font does not contain.
- 31-014 A glyph present in an embedded CID font program is not listed in CIDSetCIDSet must accurately list the CIDs in the embedded subset.
- 31-015 A glyph listed in CIDSet is not present in the CID font programCIDSet promises CIDs the font does not contain.
- 31-016 Glyph widths in the font dictionary and the font program differThe /Widths array must agree with the embedded font program.
- 31-017 A non-symbolic TrueType font has no non-symbolic cmapA font declared non-symbolic must actually carry a (3,1) or (1,0) cmap subtable.
- 31-018 A glyph cannot be looked up by any non-symbolic cmap entrySome rendered glyph is unreachable through the declared cmap tables.
- 31-019 The font dictionary for a non-symbolic TrueType font does not contain an Encoding entryNon-symbolic TrueType fonts must declare an /Encoding.
- 31-020 The Encoding dictionary for a non-symbolic TrueType font does not contain BaseEncodingIf /Encoding is a dictionary it must name a /BaseEncoding.
- 31-021 BaseEncoding is neither MacRomanEncoding nor WinAnsiEncodingOnly MacRomanEncoding and WinAnsiEncoding are permitted for non-symbolic TrueType fonts.
- 31-022 Differences contains glyph names not listed in the Adobe Glyph ListGlyph names in /Differences must be resolvable through the Adobe Glyph List so a processor can derive Unicode.
- 31-023 Differences present but the font has no (3,1) Microsoft Unicode cmapThe Differences array is resolved through the (3,1) cmap; without it the mapping is undefined.
- 31-024 The Encoding entry is present in the font dictionary for a symbolic TrueType fontSymbolic TrueType fonts must not carry an /Encoding; the font's own cmap governs.
- 31-025 The embedded font program for a symbolic TrueType font contains no cmapA symbolic TrueType font must contain at least one cmap subtable.
- 31-026 A symbolic TrueType font has multiple cmaps but none is a (3,0) Microsoft Symbol cmapWith multiple cmaps present, a (3,0) symbol cmap is required to disambiguate.
- 31-027 A font dictionary does not contain the ToUnicode entry and no exemption appliesA ToUnicode CMap is required unless the font uses MacRoman/MacExpert/WinAnsi encoding, or is a Type 1/Type 3 whose glyph names are all in the Adobe Glyph List or the Symbol set, or is a Type 0 whose descendant uses a standard Adobe character collection, or is a non-symbolic TrueType.
- 31-029 Unicode values in the ToUnicode CMap are U+FEFF or U+FFFEByte-order marks are not characters and must not appear as mapping targets.
- 31-030 Characters used in text showing operators reference the .notdef glyphThe content stream paints glyph index 0, which is by definition "no glyph" — typically a hollow box or nothing at all.
Check your own file. Taggart’s validator is free and unlimited — 75 of the 87 Matterhorn conditions software can decide, with the other 12 listed for you to confirm; no watermark, no expiry, and no account needed up to 60 pages.
Checked with Taggart on a generated file that fails this condition on . The Acrobat steps follow Adobe’s documentation and were not walked through by hand. Last updated .