PDF Encoding
A PDF Encoding object maps character codes in a font's glyph range to Unicode values and glyph names, enabling text extraction, searching, copy-paste, and accessibility. Without a correct encoding, a PDF may display correctly on screen but produce garbled text when extracted, searched, or read by a screen reader.
In PDF, encoding defines the mapping between character codes in a string and the glyph names or Unicode code points that those codes represent. When a PDF viewer encounters a string in a text-showing operator (like <code>Tj</code> or <code>TJ</code>), it uses the font's encoding to determine which character each byte represents and which glyph to draw.
Encoding is one of the most technically complex areas of the PDF specification — and one of the most common sources of text extraction problems. A PDF that looks correct visually may be completely unreadable when text is copied, searched, or extracted, if the encoding information is missing, incomplete, or incorrect.
The PDF specification defines multiple encoding mechanisms: named encodings (WinAnsiEncoding, MacRomanEncoding, MacExpertEncoding, StandardEncoding), glyph name-based /Differences arrays, and ToUnicode CMaps (which map directly to Unicode code points). Modern PDF workflows use ToUnicode for reliable text extraction; legacy PDFs frequently lack it.
Named encodings and the /Differences array
PDF's simplest encoding mechanism is the named encoding — a predefined mapping from byte values (0–255) to glyph names. The four named encodings in PDF are:
<strong>WinAnsiEncoding</strong> — The Windows-1252 code page, which is a superset of ISO-8859-1 (Latin-1). Values 0x20–0x7E match ASCII; additional characters (curly quotes, em dash, euro sign, etc.) are at specific positions in 0x80–0x9F and 0xA0–0xFF. This is the most common encoding for Latin-script PDFs created on Windows.
<strong>MacRomanEncoding</strong> — The Mac OS Roman encoding (classic Mac OS). Similar coverage to WinAnsiEncoding but with different byte positions for extended characters. Used by PDFs created on older Mac systems.
<strong>MacExpertEncoding</strong> — A specialised encoding for expert typographic characters (fractions, swash caps, subscript digits) historically used in high-end Mac typography.
<strong>StandardEncoding</strong> — The Adobe Standard Latin encoding, defined in the PostScript Language Reference Manual. Used historically as the default encoding for PostScript Type 1 fonts.
A font can customise a named encoding using the /Differences array — a list of character code positions and glyph names that override the base encoding at those positions:
<code>/Encoding <<</code> <code> /Type /Encoding</code> <code> /BaseEncoding /WinAnsiEncoding</code> <code> /Differences [128 /Euro /quotesinglbase /florin]</code> <code>>></code>
This example says: starting at position 128, replace the base encoding with glyphs named <code>/Euro</code>, <code>/quotesinglbase</code>, and <code>/florin</code>. Everything else inherits from WinAnsiEncoding.
The glyph name approach works for Latin-script fonts, but breaks down for non-Latin scripts (Arabic, Chinese, Korean, Japanese, Cyrillic in unusual encodings) — these require CID fonts and CMap-based encoding, not the simple byte-to-glyph-name model.
CID fonts and CMaps for multibyte text
For scripts beyond the Latin alphabet and for Unicode-aware text, PDF uses CID (Character Identifier) fonts with CMaps for encoding:
<strong>CID fonts</strong> — A CID font is a font containing a large number of glyphs identified by CID numbers (integers) rather than glyph names. CID fonts are used for CJK (Chinese, Japanese, Korean) scripts and other large character sets. The glyph set can be as large as 65,535 entries.
<strong>CMaps</strong> — A CMap (character map) maps from character codes in a PDF string to CID numbers. CMaps can handle multi-byte character codes (2 or 4 bytes per character, not just single bytes), enabling access to large glyph sets from compact string representations. Predefined CMaps for CJK scripts are specified in the PDF standard (e.g., UniGB-UTF16-H for Simplified Chinese, UniJIS-UTF16-H for Japanese).
<strong>Identity-H / Identity-V</strong> — A special predefined CMap that maps each 2-byte character code directly to the same-numbered CID. Used for fonts embedded with their full glyph set where the character codes in the string are already CID numbers. Common in PDFs generated by TeX/LaTeX systems. Identity-H (horizontal) and Identity-V (vertical) are distinguished by text direction.
The CID font approach separates the character encoding (the CMap, which maps codes to CIDs) from the visual representation (the CID font, which maps CIDs to glyph shapes). This layering allows: the same glyph set to be accessed with different character encodings; partial font embedding (only the CIDs used in the document); and consistent text extraction via ToUnicode.
ToUnicode CMaps and text extraction reliability
The /ToUnicode entry in a PDF font dictionary is the most important encoding element for text extraction, search, copy-paste, and accessibility. A ToUnicode CMap maps from character codes in the font's encoding to Unicode code points — providing a direct mapping from "byte value in the PDF string" to "Unicode character".
Without a ToUnicode CMap, text extraction software must: 1. Look up the character code in the font's encoding to get a glyph name 2. Map the glyph name to a Unicode code point using Adobe's glyph name to Unicode mapping table 3. Fall back to heuristics if the glyph name is not in the standard mapping
This process fails for: custom glyph names (many PDFs use names like <code>/G0001</code> that don't map to Unicode); ligature glyphs (a single glyph for "fi" needs to map to the two Unicode code points U+0066 and U+0069); fonts with non-standard encodings.
A ToUnicode CMap looks like:
<code>/CIDInit /ProcSet findresource begin</code> <code>12 dict begin</code> <code>begincmap</code> <code>/CIDSystemInfo << /Registry (Adobe) /Ordering (UCS) /Supplement 0 >> def</code> <code>/CMapName /Adobe-Identity-UCS def</code> <code>/CMapType 2 def</code> <code>4 beginbfchar</code> <code><0041> <0041> % code 0x41 → Unicode U+0041 (A)</code> <code><0066 0069> <0066 0069> % fi ligature code → U+0066 U+0069</code> <code>endbfchar</code> <code>endcmap</code>
<strong>When ToUnicode is missing</strong> — PDFs generated from: old LaTeX setups without Unicode-aware output, older versions of InDesign or Word, custom publishing systems, or some scanned+OCR workflows may lack ToUnicode entries. These PDFs look correct visually but produce garbled text when copied or searched. This is a major problem for AI document processing pipelines and for accessibility (screen readers need correct text extraction). See <a href="https://pdftheguide.com/glossary/text-layer" target="_blank" rel="noopener">Text Layer</a> for context.
Frequently asked questions
Why does my PDF look correct but copy-paste produces garbled text?
This is almost always caused by missing or incorrect font encoding information — specifically, absent or incorrect /ToUnicode CMaps in the embedded fonts. The PDF viewer renders glyphs from the embedded font using the font's outline data, so the visual rendering is correct regardless of encoding quality. But text extraction (copy-paste, search, accessibility, indexing) relies on the ToUnicode mapping to determine which Unicode characters the glyph codes represent. Without ToUnicode, the viewer falls back to glyph name lookups — which fail for custom glyph names, ligatures, and non-standard encodings. The fix requires regenerating the PDF with correct ToUnicode data from the source document. In some cases, Acrobat Pro's "Make Accessible" or OCR features can add a text layer, but this is an approximation rather than a repair of the original encoding.
What is the difference between character codes, glyph names, and Unicode code points in PDF?
These are three distinct levels of the encoding stack: Character code — the byte value (or multi-byte value for CID fonts) in a PDF string as written in the content stream. For example, the byte 0x41. Glyph name — a string name for a specific glyph shape in a font, following Adobe glyph naming conventions (e.g., /A for the Latin capital letter A, /fi for the fi ligature). The font's encoding maps character codes to glyph names. Unicode code point — a universal integer identifier for a character in the Unicode standard (e.g., U+0041 for LATIN CAPITAL LETTER A). The /ToUnicode CMap maps character codes directly to Unicode code points. Modern PDF workflows go: character code → Unicode (via ToUnicode), bypassing glyph names for extraction purposes. Legacy workflows go: character code → glyph name → Unicode (via Adobe's glyph name mapping), which is unreliable.
What is WinAnsiEncoding and which characters does it include?
WinAnsiEncoding (also called Windows-1252) is a single-byte encoding that maps byte values 0x20–0xFF to Latin characters. Values 0x20–0x7E correspond to standard ASCII (printable characters). Values 0x80–0x9F include typographic characters not in ISO-8859-1: the Euro sign (€ at 0x80/U+20AC), typographic quotes, em dash, en dash, ellipsis, dagger, double dagger, permille, and others. Values 0xA0–0xFF correspond to ISO-8859-1 extended Latin characters. WinAnsiEncoding does not include: non-Latin scripts; characters beyond U+02DC approximately; accented characters outside Western European languages. Most Latin-alphabet PDFs created on Windows use WinAnsiEncoding as their base encoding. For any character outside this range, a different encoding or a ToUnicode CMap is required.