summaryrefslogtreecommitdiff
path: root/Master/texmf-dist/tex/luatex/optex/base/pdfuni-string.opm
blob: 25781ddfb4f10f7dec8f6a8699c284be83196a01 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
%% This is part of the OpTeX project, see http://petr.olsak.net/optex

\_codedecl \pdfunidef {PDFunicode strings for outlines <2021-02-08>} % preloaded in format

   \_doc -----------------------------
   \`\_hexprint` is a command defined in Lua, that scans a number and expands
   to its UTF-16 Big Endian encoded form for use in PDF hexadecimal strings.
   \_cod -----------------------------

\bgroup
\_catcode`\%=12
\_gdef\_hexprint{\_directlua{
   local num = token.scan_int()
   if num < 0x10000 then
      tex.print(string.format("%04X", num))
   else
      num = num - 0x10000
      local high = bit32.rshift(num, 10) + 0xD800
      local low = bit32.band(num, 0x3FF) + 0xDC00
      tex.print(string.format("%04X%04X", high, low))
   end
}}
\egroup

   \_doc -----------------------------
   \`\pdfunidef``\macro{<text>}` does more things than only converting to hexadecimal PDF string.
   The <text> can be scanned in verbatim mode (it is true becuase \^`\_Xtoc`
   reads the <text> in verbatim mode). First `\edef` do
   `\_scantextokens\unexpanded` and second `\edef` expands the parameter
   according to current values on selected macros from `\_regoul`. Then
   \`\_removeoutmath` converts `..$x^2$..` to `..x^2..`, i.e removes dollars.
   Then \`\_removeoutbraces` converts `..{x}..` to `..x..`.
   Finally, the <text> is detokenized, spaces are preprocessed using \^`\replstring`
   and then the \`\_pdfunidefB` is repeated on each character. It calls the
   `\directlua` chunk to print hexadecimal numbers in the macro \^`\_hexprint`.\nl
   Characters for quotes (and separators for quotes) are activated by first
   `\_scatextokens` and they are defined as the same non-active characters.
   But `\_regoul` can change this definition.
   \_cod -----------------------------

\_def\_pdfunidef#1#2{%
   \_begingroup
      \_catcodetable\_optexcatcodes \_adef"{"}\_adef'{'}%
      \_the\_regoul \_relax % \_regmacro alternatives of logos etc.
      \_ifx\_savedttchar\_undefined \_def#1{\_scantextokens{\_unexpanded{#2}}}%
      \_else \_lccode`\;=\_savedttchar \_lowercase{\_prepinverb#1;}{#2}\fi
      \_edef#1{#1}%
      \_escapechar=-1
      \_edef#1{#1\_empty}%
      \_escapechar=`\\
      \_ea\_edef \_ea#1\_ea{\_ea\_removeoutmath   #1$\_end$}%  $x$ -> x
      \_ea\_edef \_ea#1\_ea{\_ea\_removeoutbraces #1{\_end}}%  {x} -> x
      \_edef#1{\_detokenize\_ea{#1}}%
      \_replstring#1{ }{{ }}%  text text -> text{ }text
      \_catcode`\\=12 \_let\\=\_bslash
      \_edef\_out{<FEFF}
      \_ea\_pdfunidefB#1^%  text -> \_out in octal
      \_ea
   \_endgroup
   \_ea\_def\_ea#1\_ea{\_out>}
}
\_def\_pdfunidefB#1{%
   \_ifx^#1\_else
      \_edef\_out{\_out \_hexprint `#1}
   \_ea\_pdfunidefB \_fi
}

\_def\_removeoutbraces #1#{#1\_removeoutbracesA}
\_def\_removeoutbracesA #1{\_ifx\_end#1\_else #1\_ea\_removeoutbraces\_fi}
\_def\_removeoutmath #1$#2${#1\_ifx\_end#2\_else #2\_ea\_removeoutmath\_fi}

   \_doc -----------------------------
   The \`\_prepinverb``<macro><separator>{<text>}`,
   e.g.\ `\_prepinverb\tmpb|{aaa |bbb| cccc |dd| ee}`
   does `\def\tmpb{<su>{aaa }bbb<su>{ cccc }dd<su>{ ee}}` where
   <su> is `\scantextokens\unexpanded`. It means that in-line verbatim
   are not argument of `\scantextoken`. First `\edef\tmpb` tokenizes again
   the <text> but not the parts which were in the the in-line verbatim.
   \_cod -----------------------------

\_def\_prepinverb#1#2#3{\_def#1{}%
   \_def\_dotmpb ##1#2##2{\_addto#1{\_scantextokens{\_unexpanded{##1}}}%
      \_ifx\_end##2\_else\_ea\_dotmpbA\_ea##2\_fi}%
   \_def\_dotmpbA ##1#2{\_addto#1{##1}\_dotmpb}%
   \_dotmpb#3#2\_end
}

   \_doc -----------------------------
   The \^`\regmacro` is used in order to sed the values of macros
   `\em`, `\rm`, `\bf`, `\it`, `\bi`, `\tt`, `\/` and `~` to values usable in
   PDF outlines.
   \_cod -----------------------------

\_regmacro {}{}{\_let\em=\_empty \_let\rm=\_empty \_let\bf=\_empty
    \_let\it=\_empty \_let\bi=\_empty \_let\tt=\_empty \_let\/=\_empty
    \_let~=\_space
}
\public \pdfunidef ;

\_endcode % --------------------------------

There are only two encodings for PDF strings (used in PDFoutlines, PDFinfo,
etc.). The first one is PDFDocEncoding which is single-byte encoding, but it
misses most international characters.

The second encoding is Big Endian UTF-16 which is implemented in this file. It
encodes a single character in either two or four bytes.
This encoding is \TeX/-discomfortable because it looks like

\begtt
<FEFF 0043 0076 0069 010D 0065 006E 00ED 0020 006A 0065 0020 007A 00E1 0074
011B 017E 0020 0061 0020 0078 2208 D835DD44>
\endtt

This example shows a hexadecimal PDF string (enclosed in \code{<>} as opposed
to the literal PDF string enclosed in `()`). In these strings each byte is
represented by two hexadecimal characters (`0-9`, `A-F`). You can tell the
encoding is UTF-16BE, becuase it starts with \"Byte order mark" `FEFF`. Each
unicode character is then encoded in one or two byte pairs. The example string
corresponds to the text \"Cvičení je zátěž a ${\rm x} ∈ 𝕄$". Notice the 4 bytes
for the last character, $𝕄$. (Even the whitespace would be OK in a PDF file,
because it should be ignored by PDF viewers, but \LuaTeX\ doesn't allow it.)

\_endinput

2021-02-08 \_octalprint -> \_hexprint
2020-03-12 Released