%% This is part of OpTeX project, see http://petr.olsak.net/optex \_codedecl \hisyntax {Syntax highlithing of verbatim listings <2020-04-04>} % preloaded in format \_doc ----------------------------- The following macros `\replfromto` and `\replthis` manipulate with the verbatim text which has been read already and stored in the `\_tmpb` macro. The \`\replfromto` `{}{}{}` finds first `` then the first `` following by `` pattern and the `` between them is packed to `#1`. Then `` is replaced by ``. The `` parameter can use `#1` which is replaced by the ``. The `\replfromto` continues by finding next ``, then, next `` repeatedly over the whole verbatim text. If the verbatim text is ended by opened `` but not closing by `` then `` is appended to the verbatim text automatically and the last part of verbatim text is replaced too. First two parameters are expanded before usage of `\replfromto`. You can use `\csstring\%` or something else here. \_cod ----------------------------- \_def\_replfromto #1#2{\_edef\_tmpa{{#1}{#2}}\_ea\_replfromtoE\_tmpa} \_def\_replfromtoE#1#2#3{% #1=from #2=to #3=what to replace \_def\_replfrom##1#1##2{\_addto\_tmpb{##1}% \_ifx\_end##2\_ea\_replstop \_else \_afterfi{\_replto##2}\_fi}% \_def\_replto##1#2##2{% \_ifx\_end##2\_afterfi{\_replfin##1}\_else \_addto\_tmpb{#3}% \_afterfi{\_replfrom##2}\_fi}% \_def\_replfin##1#1\_end{\_addto\_tmpb{#3}\_replstop}% \_edef\_tmpb{\_ea}\_ea\_replfrom\_tmpb#1\_end#2\_end\_end\_relax } \_def\_replstop#1\_end\_relax{} \_def\_finrepl{} \_doc ----------------------------- The \`\replthis` `{}{}` replaces each `` by ``. Both parameters of `\replthis` are expanded first. \_cod ----------------------------- \_def\_replthis#1#2{\_edef\_tmpa{{#1}{#2}}\_ea\_replstring\_ea\_tmpb \_tmpa} \_public \replfromto \replthis ; \_doc ----------------------------- The patterns ``, `` and `` are not found when they are hidden in braces `{...}`. Example: \begtt \replfromto{/*}{*/}{\x C{/*#1/*}} \endtt replaces all C comments by `\x C{...}`. The patterns inside `{...}` are not used by next usage of `\replfromto` or `\replthis` macros. The \`\_xscan` macro does replacing `\x` by `\z` in the post-processing phase. The `\x {}` expands to `\_xscan {}^^J^`. If `#3` is `\_end` then it signals that something wrong happens, the `` was not terminated by legal `` when `\replfromto` did work. We must to fix it by the \`\_xscanR` macro. \_cod ----------------------------- \_def\_xscan#1#2^^J#3{\_ifx\_end#3 \_ea\_xscanR\_fi \z{#1}{#2}% \_ifx^#3\_else ^^J\_afterfi{\_xscan{#1}#3}\_fi} \_def\_xscanR#1\_fi#2^{^^J} \_doc ----------------------------- The \`\hicolor` ` ` defines `\_z{}` as `{}`. It should be used in the context of `\x {}` macros. \_cod ----------------------------- \_def\_hicolor #1#2{\_sdef{_z:#1}##1{{#2##1}}} \_doc ----------------------------- The \`\hisyntax``{}` re-defines default \^`\_prepareverbdata``` in order to it does more things: It saves `` to `\_tmpb`, appends `\n` around spaces and `^^J` characters in pre-processing phase, it opens `hisyntax-.opm` file if `\_hisyntax` is not defined. Then `\_the\_isyntax` is processed. Finally, the post-processing phase is realized by setting appropriate values to `\x` and `\y` macros and doing `\_edef\_tmpb{\_tmpb}`. \_cod ----------------------------- \_def\_hisyntax#1{\_def\_prepareverbdata##1##2{% \_let\n=\relax \_def\t{\n\noexpand\t\n}\_let\_start=\_relax \_adef{ }{\n\ \n}\_edef\_tmpb{\_start^^J##2\_end}% \_replthis{^^J}{\n^^J\n}\_replthis{\n\_end}{\_end}% \_let\x=\_relax \_let\y=\_relax \_let\z=\_relax \_let\t=\_relax \_endlinechar=`\^^M \_lowercase{\_def\_tmpa{#1}}% \_ifcsname _hialias:\_tmpa\_endcsname \_edef\_tmpa{\_cs{_hialias:\_tmpa}}\_fi \_ifx\_tmpa\_empty \_else \_unless \_ifcsname _hisyntax\_tmpa\_endcsname \_isfile{hisyntax-\_tmpa.opm}\_iftrue \_opinput {hisyntax-\_tmpa.opm} \_fi\_fi \_ifcsname _hisyntax\_tmpa\_endcsname \_ifcsname hicolors\_tmpa\endcsname \_cs{_hicolors\_tmpa}=\_cs{hicolors\_tmpa}% \_else \_if^\_the\_hicolors^\_else \_ifcsname _hicolors\_tmpa\_endcsname \_global\_cs{_hicolors\_tmpa}=\_hicolors \_global\_hicolors={}% \_fi\_fi\_fi \_ea\_the \_csname _hisyntax\_tmpa\_endcsname % \_the\_hisyntax \_else\_opwarning{Syntax highlighting "\_tmpa" undeclared (no file hisyntax-\_tmpa.opm)} \_fi\_fi \_replthis{\_start\n^^J}{}\_replthis{^^J\_end}{^^J}% \_def\n{}% \_def\x####1####2{\_xscan{####1}####2^^J^}% \_def\y####1{\_ea \_noexpand \_csname ####1\_endcsname}% \_edef\_tmpb{\_tmpb}% \_def\z####1{\_cs{_z:####1}}% \_def\t{\_hskip \_dimexpr\_tabspaces em/2\_relax}% \_localcolor }} \_public \hisyntax \hicolor ; \_doc ----------------------------- Aliases for languages can be declared like this. When `\hisyntax{xml}` is used then this is the same as `\hisyntax{html}`. \_cod ----------------------------- \_sdef{_hialias:xml}{html} \_sdef{_hialias:json}{c} \_endcode % ------------------------------------------- The user can write \begtt \adef/{\_csstring\\} \begtt \hisytnax{C} ... /endtt \endtt and the code is colorized by C syntax. The user can write `\everytt={\hisyntax{C}}` and all verbatim listings are colorized. The \^`\hisyntax``{}` reads the file `hisyntax-.opm` where the colorization is declared. The parameter `` is case insensitive and the file name must include it in lowercase letters. For example the file `hisyntax-c.opm` looks like: \printdoc hisyntax-c.opm \OpTeX/ provides `hisyntax-{c,python,tex,html}.opm` files. You can take inspiration from these files and declare more languages. User can re-declare colors by `\hicolors={...}` This value has precedence before `\_hicolors` values declared in the `hicolors-.opm` file. What exactly to do: copy `\_hicolors={...}` from `hicolors-.opm` to your document, rename it as `\hicolors={...}` and do you own colors modifications. Another way to set non-default colors is to declare `\newtoks\hicolors` (without the `_` prefix) and set the colors palette here. It has precedence before `\_hicolors` (with the `_` prefix) declared in the `hicolors-.opm` file. This is useful when there are more hi-syntax languages used in one document. \medskip \noindent{\bf Notes for hi-syntax macro writers} The file `hisyntax-.opm` is read only once in the \TeX/ group. If there are definitions then they must be declared as global. The `hisyntax-.opm` file must (globally) declare `\_hisyntax` tokens string where the action over verbatim text is declared typically by `\replfromto` or `\replthis` macros. The verbatim text is preparared by {\em pre-processing phase}, then the `\_hisyntax` is applied and then {\em post-processing phase} does final corrections. Finally, the verbatim text is printed line by line. The pre-processing phase does: \begitems * Each space is replaced by {\visiblesp`\n\ \n`}, so `\n\n` should be a pattern to finding whole words (no subwords). The `\n` control sequence is removed in the post-processing phase. * Each end of line is repersented by `\n^^J\n`. * The `\_start` control sequence is added before the verbatim text and `\_end` control sequence is appended to the end of the verbatim text. These control sequences are removed in post-processing phase. \enditems There are special macros working only in a group when processing the verbatim text. \begitems * `\n` means noting but it should be used as a boundary of words as mentioned above. * `\t` means a tabelator. It is prepared as `\n\t\n` because it can be at the boundary of a word. * `\x {}` can be used as replacing text. Suppose the example \begtt \replfromto{/*}{*/}{\x C{/*#1*/}} \endtt This replaces all C comments `/*...*/` by `\x C{/*...*/}`. But the C comments may span more lines, i.e. the `^^J` should be inside it. The macro `\x {}` is replaced by one or more `\z {}` in post-processing phase where each parameter `` of `\z` keeps inside one line. Inside-line parameters are represented by `\x C{}` and they are replaced to `\z C{}` without any change. But: \begtt \catcode`\<=13 \x C{^^J^^J} is replaced by \z C{}^^J\z C{}^^J\z C{} \endtt The `\z {}` is expanded to `\_z:{}` and if `\hicolor ` is declared then `\_z:{}` expands to `{}`. So, required color is activated at all lines (separatelly) where C comment spans. * `\y {}` is replaced by `\` in the post processing phase. It should be used for macros without a parameter. You cannot use unprotected macros as replacement text before the post-processing phase, because the post-processing phase is based on expansion whole verbatim text. \enditems