1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
|
%% This is part of the OpTeX project, see http://petr.olsak.net/optex
\_codedecl \hisyntax {Syntax highlithing of verbatim listings <2020-04-04>} % preloaded in format
\_doc -----------------------------
The following macros `\replfromto` and `\replthis` manipulate with the
verbatim text which has been read already and stored in the `\_tmpb` macro.
The \`\replfromto` `{<from>}{<to>}{<what>}` finds first `<from>` then the
first `<to>` following by `<from>` pattern and the
`<text>` between them is packed to `#1`.
Then `<from><text><to>` is replaced by `<what>`. The `<what>` parameter
can use `#1` which is replaced by the `<text>`.
The `\replfromto` continues by finding next `<from>`, then, next `<to>`
repeatedly over the whole verbatim text. If the verbatim text is ended by
opened `<from>` but not closing by `<to>` then `<to>` is appended to the
verbatim text automatically and the last part of the verbatim text is replaced too.
First two parameters are expanded before usage of `\replfromto`. You can
use `\csstring\%` or something else here.
\_cod -----------------------------
\_def\_replfromto #1#2{\_edef\_tmpa{{#1}{#2}}\_ea\_replfromtoE\_tmpa}
\_def\_replfromtoE#1#2#3{% #1=from #2=to #3=what to replace
\_def\_replfrom##1#1##2{\_addto\_tmpb{##1}%
\_ifx\_end##2\_ea\_replstop \_else \_afterfi{\_replto##2}\_fi}%
\_def\_replto##1#2##2{%
\_ifx\_end##2\_afterfi{\_replfin##1}\_else
\_addto\_tmpb{#3}%
\_afterfi{\_replfrom##2}\_fi}%
\_def\_replfin##1#1\_end{\_addto\_tmpb{#3}\_replstop}%
\_edef\_tmpb{\_ea}\_ea\_replfrom\_tmpb#1\_end#2\_end\_end\_relax
}
\_def\_replstop#1\_end\_relax{}
\_def\_finrepl{}
\_doc -----------------------------
The \`\replthis` `{<pattern>}{<what>}` replaces each `<pattern>` by `<what>`.
Both parameters of `\replthis` are expanded first.
\_cod -----------------------------
\_def\_replthis#1#2{\_edef\_tmpa{{#1}{#2}}\_ea\_replstring\_ea\_tmpb \_tmpa}
\_public \replfromto \replthis ;
\_doc -----------------------------
The patterns `<from>`, `<to>` and `<pattern>` are not found when they are
hidden in braces `{...}`. Example:
\begtt
\replfromto{/*}{*/}{\x C{/*#1/*}}
\endtt
replaces all C comments by `\x C{...}`. The patterns inside `{...}` are
not used by next usage of `\replfromto` or `\replthis` macros.
The \`\_xscan` macro does replacing `\x` by `\z` in the post-processing
phase. The `\x <letter>{<text>}` expands to `\_xscan {<letter>}<text>^^J^`.
If `#3` is `\_end` then it signals that something wrong happens, the
`<from>` was not terminated by legal `<to>` when `\replfromto` did work.
We must to fix it by the \`\_xscanR` macro.
\_cod -----------------------------
\_def\_xscan#1#2^^J#3{\_ifx\_end#3 \_ea\_xscanR\_fi
\z{#1}{#2}%
\_ifx^#3\_else ^^J\_afterfi{\_xscan{#1}#3}\_fi}
\_def\_xscanR#1\_fi#2^{^^J}
\_doc -----------------------------
The \`\hicolor` `<letter> <color>` defines `\_z<letter>{<text>}`
as `{<color><text>}`. It should be used in the context of
`\x <letter>{<text>}` macros.
\_cod -----------------------------
\_def\_hicolor #1#2{\_sdef{_z:#1}##1{{#2##1}}}
\_doc -----------------------------
The \`\hisyntax``{<name>}` re-defines default \^`\_prepareverbdata``<macro><verbtext>`
in order to it does more things:
It saves `<verbtext>` to `\_tmpb`, appends `\n` around spaces and
`^^J` characters in pre-processing phase, it opens `hisyntax-<name>.opm`
file if `\_hisyntax<name>` is not defined. Then `\_the\_isyntax<name>`
is processed. Finally, the post-processing phase is realized by setting
appropriate values to `\x` and `\y` macros and doing
`\_edef\_tmpb{\_tmpb}`.
\_cod -----------------------------
\_def\_hisyntax#1{\_def\_prepareverbdata##1##2{%
\_let\n=\_relax \_let\b=\_relax \_def\t{\n\_noexpand\t\n}\_let\_start=\_relax
\_adef{ }{\n\_noexpand\ \n}\_edef\_tmpb{\_start^^J##2\_end}%
\_replthis{^^J}{\n^^J\b\n}\_replthis{\b\n\_end}{\_end}%
\_let\x=\_relax \_let\y=\_relax \_let\z=\_relax \_let\t=\_relax
\_hicomments % keeps comments declared by \commentchars
\_endlinechar=`\^^M
\_lowercase{\_def\_tmpa{#1}}%
\_ifcsname _hialias:\_tmpa\_endcsname \_edef\_tmpa{\_cs{_hialias:\_tmpa}}\_fi
\_ifx\_tmpa\_empty \_else
\_unless \_ifcsname _hisyntax\_tmpa\_endcsname
\_isfile{hisyntax-\_tmpa.opm}\_iftrue \_opinput {hisyntax-\_tmpa.opm} \_fi\_fi
\_ifcsname _hisyntax\_tmpa\_endcsname
\_ifcsname hicolors\_tmpa\_endcsname
\_cs{_hicolors\_tmpa}=\_cs{hicolors\_tmpa}%
\_else
\_if^\_the\_hicolors^\_else
\_ifcsname _hicolors\_tmpa\_endcsname
\_global\_cs{_hicolors\_tmpa}=\_hicolors \_global\_hicolors={}%
\_fi\_fi\_fi
\_ea\_the \_csname _hisyntax\_tmpa\_endcsname % \_the\_hisyntax<name>
\_else\_opwarning{Syntax "\_tmpa" undeclared (no file hisyntax-\_tmpa.opm)}
\_fi\_fi
\_replthis{\_start\n^^J}{}\_replthis{^^J\_end}{^^J}%
\_def\n{}\_def\b{}\_adef{ }{\_dsp}%
\_bgroup \_lccode`\~=`\ \_lowercase{\_egroup\_def\ {\_noexpand~}}%
\_def\w####1{####1}\_def\x####1####2{\_xscan{####1}####2^^J^}%
\_def\y####1{\_ea \_noexpand \_csname ####1\_endcsname}%
\_edef\_tmpb{\_tmpb}%
\_def\z####1{\_cs{_z:####1}}%
\_def\t{\_hskip \_dimexpr\_tabspaces em/2\_relax}%
\_localcolor
}}
\_public \hisyntax \hicolor ;
\_doc -----------------------------
Aliases for languages can be declared like this.
When `\hisyntax{xml}` is used then this is the same as `\hisyntax{html}`.
\_cod -----------------------------
\_sdef{_hialias:xml}{html}
\_sdef{_hialias:json}{c}
\_endcode % -------------------------------------------
The user can write
\begtt \adef/{\_csstring\\}
\begtt \hisytnax{C}
...
/endtt
\endtt
and the code is colorized by C syntax.
The user can write `\everytt={\hisyntax{C}}` and all verbatim listings are
colorized.
The \^`\hisyntax``{<name>}` reads the file `hisyntax-<name>.opm` where the
colorization is declared. The parameter `<name>` is case insensitive and the
file name must include it in lowercase letters. For example, the file
`hisyntax-c.opm` looks like this:
\printdoc hisyntax-c.opm
\OpTeX/ provides `hisyntax-{c,python,tex,html}.opm` files.
You can take inspiration from these files and declare more languages.
User can re-declare colors by `\hicolors={...}` This value has precedence
before `\_hicolors<name>` values declared in the `hicolors-<name>.opm` file.
What exactly to do: copy `\_hicolors<name>={...}`
from `hicolors-<name>.opm` to your document,
rename it as `\hicolors={...}` and do you own colors modifications.
Another way to set non-default colors is to declare
`\newtoks\hicolors<name>` (without the `_` prefix) and set the colors palette here.
It has precedence before `\_hicolors<name>` (with the `_` prefix) declared in
the `hicolors-<name>.opm` file.
This is useful when there are more hi-syntax languages used in one document.
\secl4 Notes for hi-syntax macro writers
The file `hisyntax-<name>.opm` is read only once in the \TeX/ group.
If there are definitions then they must be declared as global.
The `hisyntax-<name>.opm` file must (globally) declare `\_hisyntax<name>` tokens string
where the action over verbatim text is declared typically by `\replfromto` or
`\replthis` macros.
The verbatim text is prepared by {\em pre-processing phase}, then the
`\_hisyntax<name>` is applied and then {\em post-processing phase} does final
corrections. Finally, the verbatim text is printed line by line.
The pre-processing phase does:
\begitems
* Each space is replaced by {\visiblesp`\n\ \n`}, so `\n<word>\n` should be a pattern to
finding whole words (no subwords). The `\n` control sequence is removed in
the post-processing phase.
* Each end of line is represented by `\n^^J\n`.
* The `\_start` control sequence is added before the verbatim text and the `\_end` control
sequence is appended to the end of the verbatim text. These control
sequences are removed in the post-processing phase.
\enditems
Special macros are working only in a group when processing the verbatim
text.
\begitems
* `\n` means noting but it should be used as a boundary of words as mentioned above.
* `\t` means a tabulator. It is prepared as `\n\t\n` because it can be at
the boundary of a word.
* `\x <letter>{<text>}` can be used as replacing text. Suppose the example
\begtt
\replfromto{/*}{*/}{\x C{/*#1*/}}
\endtt
This replaces all C comments `/*...*/` by `\x C{/*...*/}`. But the
C comments may span more lines, i.e. the `^^J` should be inside it.
The macro `\x <letter>{<text>}` is replaced by one or more
`\z <letter>{<text>}` in post-processing phase where each parameter `<text>` of
`\z` keeps inside one line. Inside-line parameters are represented
by `\x C{<text>}` and they are replaced to `\z C{<text>}` without any change.
But:
\begtt \catcode`\<=13
\x C{<text1>^^J<text3>^^J<text3>}
is replaced by
\z C{<text1>}^^J\z C{<text2>}^^J\z C{<text3>}
\endtt
The `\z <letter>{<text>}` is expanded to `\_z:<letter>{<text>}` and if
`\hicolor <letter> <color>` is declared then
`\_z:<letter>{<text>}` expands to `{<color><text>}`. So, required color is
activated at all lines (separately) where C comment spans.
* `\y {<text>}` is replaced by `\<text>` in the post processing phase. It should
be used for macros without a parameter. You cannot use unprotected macros
as replacement text before the post-processing phase, because the post-processing
phase is based on expansion whole verbatim text.
\enditems
|
