summaryrefslogtreecommitdiff
path: root/macros/luatex/optex/base/hi-syntax.opm
diff options
context:
space:
mode:
Diffstat (limited to 'macros/luatex/optex/base/hi-syntax.opm')
-rw-r--r--macros/luatex/optex/base/hi-syntax.opm223
1 files changed, 223 insertions, 0 deletions
diff --git a/macros/luatex/optex/base/hi-syntax.opm b/macros/luatex/optex/base/hi-syntax.opm
new file mode 100644
index 0000000000..e0ac530c3d
--- /dev/null
+++ b/macros/luatex/optex/base/hi-syntax.opm
@@ -0,0 +1,223 @@
+%% This is part of OpTeX project, see http://petr.olsak.net/optex
+
+\_codedecl \hisyntax {Syntax highlithing of verbatim listings <2020-04-04>} % preloaded in format
+
+ \_doc -----------------------------
+ The following macros `\replfromto` and `\replthis` manipulate with the
+ verbatim text which has been read already and stored in the `\_tmpb` macro.
+
+ The \`\replfromto` `{<from>}{<to>}{<what>}` finds first `<from>` then the
+ first `<to>` following by `<from>` pattern and the
+ `<text>` between them is packed to `#1`.
+ Then `<from><text><to>` is replaced by `<what>`. The `<what>` parameter
+ can use `#1` which is replaced by the `<text>`.
+
+ The `\replfromto` continues by finding next `<from>`, then, next `<to>`
+ repeatedly over the whole verbatim text. If the verbatim text is ended by
+ opened `<from>` but not closing by `<to>` then `<to>` is appended to the
+ verbatim text automatically and the last part of verbatim text is replaced too.
+
+ First two parameters are expanded before usage of `\replfromto`. You can
+ use `\csstring\%` or something else here.
+ \_cod -----------------------------
+
+\_def\_replfromto #1#2{\_edef\_tmpa{{#1}{#2}}\_ea\_replfromtoE\_tmpa}
+\_def\_replfromtoE#1#2#3{% #1=from #2=to #3=what to replace
+ \_def\_replfrom##1#1##2{\_addto\_tmpb{##1}%
+ \_ifx\_end##2\_ea\_replstop \_else \_afterfi{\_replto##2}\_fi}%
+ \_def\_replto##1#2##2{%
+ \_ifx\_end##2\_afterfi{\_replfin##1}\_else
+ \_addto\_tmpb{#3}%
+ \_afterfi{\_replfrom##2}\_fi}%
+ \_def\_replfin##1#1\_end{\_addto\_tmpb{#3}\_replstop}%
+ \_edef\_tmpb{\_ea}\_ea\_replfrom\_tmpb#1\_end#2\_end\_end\_relax
+}
+\_def\_replstop#1\_end\_relax{}
+\_def\_finrepl{}
+
+ \_doc -----------------------------
+ The \`\replthis` `{<pattern>}{<what>}` replaces each `<pattern>` by `<what>`.
+ Both parameters of `\replthis` are expanded first.
+ \_cod -----------------------------
+
+\_def\_replthis#1#2{\_edef\_tmpa{{#1}{#2}}\_ea\_replstring\_ea\_tmpb \_tmpa}
+
+\_public \replfromto \replthis ;
+
+ \_doc -----------------------------
+ The patterns `<from>`, `<to>` and `<pattern>` are not found when they are
+ hidden in braces `{...}`. Example:
+ \begtt
+ \replfromto{/*}{*/}{\x C{/*#1/*}}
+ \endtt
+ replaces all C comments by `\x C{...}`. The patterns inside `{...}` are
+ not used by next usage of `\replfromto` or `\replthis` macros.
+
+ The \`\_xscan` macro does replacing `\x` by `\z` in the post-processing
+ phase. The `\x <letter>{<text>}` expands to `\_xscan {<letter>}<text>^^J^`.
+ If `#3` is `\_end` then it signals that something wrong happens, the
+ `<from>` was not terminated by legal `<to>` when `\replfromto` did work.
+ We must to fix it by the \`\_xscanR` macro.
+ \_cod -----------------------------
+
+\_def\_xscan#1#2^^J#3{\_ifx\_end#3 \_ea\_xscanR\_fi
+ \z{#1}{#2}%
+ \_ifx^#3\_else ^^J\_afterfi{\_xscan{#1}#3}\_fi}
+\_def\_xscanR#1\_fi#2^{^^J}
+
+ \_doc -----------------------------
+ The \`\hicolor` `<letter> <color>` defines `\_z<letter>{<text>}`
+ as `{<color><text>}`. It should be used in the context of
+ `\x <letter>{<text>}` macros.
+ \_cod -----------------------------
+
+\_def\_hicolor #1#2{\_sdef{_z:#1}##1{{#2##1}}}
+
+ \_doc -----------------------------
+ The \`\hisyntax``{<name>}` re-defines default \^`\_prepareverbdata``<macro><verbtext>`
+ in order to it does more things:
+ It saves `<verbtext>` to `\_tmpb`, appends `\n` around spaces and
+ `^^J` characters in pre-processing phase, it opens `hisyntax-<name>.opm`
+ file if `\_hisyntax<name>` is not defined. Then `\_the\_isyntax<name>`
+ is processed. Finally, the post-processing phase is realized by setting
+ appropriate values to `\x` and `\y` macros and doing
+ `\_edef\_tmpb{\_tmpb}`.
+ \_cod -----------------------------
+
+\_def\_hisyntax#1{\_def\_prepareverbdata##1##2{%
+ \_let\n=\_relax \_def\t{\n\_noexpand\t\n}\_let\_start=\_relax
+ \_adef{ }{\n\ \n}\_edef\_tmpb{\_start^^J##2\_end}%
+ \_replthis{^^J}{\n^^J\n}\_replthis{\n\_end}{\_end}%
+ \_let\x=\_relax \_let\y=\_relax \_let\z=\_relax \_let\t=\_relax
+ \_endlinechar=`\^^M
+ \_lowercase{\_def\_tmpa{#1}}%
+ \_ifcsname _hialias:\_tmpa\_endcsname \_edef\_tmpa{\_cs{_hialias:\_tmpa}}\_fi
+ \_ifx\_tmpa\_empty \_else
+ \_unless \_ifcsname _hisyntax\_tmpa\_endcsname
+ \_isfile{hisyntax-\_tmpa.opm}\_iftrue \_opinput {hisyntax-\_tmpa.opm} \_fi\_fi
+ \_ifcsname _hisyntax\_tmpa\_endcsname
+ \_ifcsname hicolors\_tmpa\_endcsname
+ \_cs{_hicolors\_tmpa}=\_cs{hicolors\_tmpa}%
+ \_else
+ \_if^\_the\_hicolors^\_else
+ \_ifcsname _hicolors\_tmpa\_endcsname
+ \_global\_cs{_hicolors\_tmpa}=\_hicolors \_global\_hicolors={}%
+ \_fi\_fi\_fi
+ \_ea\_the \_csname _hisyntax\_tmpa\_endcsname % \_the\_hisyntax<name>
+ \_else\_opwarning{Syntax highlighting "\_tmpa" undeclared (no file hisyntax-\_tmpa.opm)}
+ \_fi\_fi
+ \_replthis{\_start\n^^J}{}\_replthis{^^J\_end}{^^J}%
+ \_def\n{}%
+ \_def\x####1####2{\_xscan{####1}####2^^J^}%
+ \_def\y####1{\_ea \_noexpand \_csname ####1\_endcsname}%
+ \_edef\_tmpb{\_tmpb}%
+ \_def\z####1{\_cs{_z:####1}}%
+ \_def\t{\_hskip \_dimexpr\_tabspaces em/2\_relax}%
+ \_localcolor
+}}
+\_public \hisyntax \hicolor ;
+
+ \_doc -----------------------------
+ Aliases for languages can be declared like this.
+ When `\hisyntax{xml}` is used then this is the same as `\hisyntax{html}`.
+ \_cod -----------------------------
+
+\_sdef{_hialias:xml}{html}
+\_sdef{_hialias:json}{c}
+
+\_endcode % -------------------------------------------
+
+The user can write
+
+\begtt \adef/{\_csstring\\}
+\begtt \hisytnax{C}
+...
+/endtt
+\endtt
+and the code is colorized by C syntax.
+The user can write `\everytt={\hisyntax{C}}` and all verbatim listings are
+colorized.
+
+The \^`\hisyntax``{<name>}` reads the file `hisyntax-<name>.opm` where the
+colorization is declared. The parameter `<name>` is case insensitive and the
+file name must include it in lowercase letters. For example the file
+`hisyntax-c.opm` looks like:
+
+\printdoc hisyntax-c.opm
+
+\OpTeX/ provides `hisyntax-{c,python,tex,html}.opm` files.
+You can take inspiration from these files and declare more languages.
+
+User can re-declare colors by `\hicolors={...}` This value has precedence
+before `\_hicolors<name>` values declared in the `hicolors-<name>.opm` file.
+What exactly to do: copy `\_hicolors<name>={...}`
+from `hicolors-<name>.opm` to your document,
+rename it as `\hicolors={...}` and do you own colors modifications.
+
+Another way to set non-default colors is to declare
+`\newtoks\hicolors<name>` (without the `_` prefix) and set the colors palette here.
+It has precedence before `\_hicolors<name>` (with the `_` prefix) declared in
+the `hicolors-<name>.opm` file.
+This is useful when there are more hi-syntax languages used in one document.
+
+\medskip
+\noindent{\bf Notes for hi-syntax macro writers}
+
+The file `hisyntax-<name>.opm` is read only once in the \TeX/ group.
+If there are definitions then they must be declared as global.
+
+The `hisyntax-<name>.opm` file must (globally) declare `\_hisyntax<name>` tokens string
+where the action over verbatim text is declared typically by `\replfromto` or
+`\replthis` macros.
+
+The verbatim text is preparared by {\em pre-processing phase}, then the
+`\_hisyntax<name>` is applied and then {\em post-processing phase} does final
+corrections. Finally, the verbatim text is printed line by line.
+
+The pre-processing phase does:
+
+\begitems
+* Each space is replaced by {\visiblesp`\n\ \n`}, so `\n<word>\n` should be a pattern to
+ finding whole words (no subwords). The `\n` control sequence is removed in
+ the post-processing phase.
+* Each end of line is repersented by `\n^^J\n`.
+* The `\_start` control sequence is added before the verbatim text and `\_end` control
+ sequence is appended to the end of the verbatim text. These control
+ sequences are removed in post-processing phase.
+\enditems
+
+There are special macros working only in a group when processing the verbatim
+text.
+
+
+\begitems
+* `\n` means noting but it should be used as a boundary of words as mentioned above.
+* `\t` means a tabelator. It is prepared as `\n\t\n` because it can be at
+ the boundary of a word.
+* `\x <letter>{<text>}` can be used as replacing text. Suppose the example
+ \begtt
+ \replfromto{/*}{*/}{\x C{/*#1*/}}
+ \endtt
+ This replaces all C comments `/*...*/` by `\x C{/*...*/}`. But the
+ C comments may span more lines, i.e. the `^^J` should be inside it.
+
+ The macro `\x <letter>{<text>}` is replaced by one or more
+ `\z <letter>{<text>}` in post-processing phase where each parameter `<text>` of
+ `\z` keeps inside one line. Inside-line parameters are represented
+ by `\x C{<text>}` and they are replaced to `\z C{<text>}` without any change.
+ But:
+ \begtt \catcode`\<=13
+ \x C{<text1>^^J<text3>^^J<text3>}
+ is replaced by
+ \z C{<text1>}^^J\z C{<text2>}^^J\z C{<text3>}
+ \endtt
+ The `\z <letter>{<text>}` is expanded to `\_z:<letter>{<text>}` and if
+ `\hicolor <letter> <color>` is declared then
+ `\_z:<letter>{<text>}` expands to `{<color><text>}`. So, required color is
+ activated at all lines (separatelly) where C comment spans.
+* `\y {<text>}` is replaced by `\<text>` in the post processing phase. It should
+ be used for macros without a parameter. You cannot use unprotected macros
+ as replacement text before the post-processing phase, because the post-processing
+ phase is based on expansion whole verbatim text.
+\enditems
+