summaryrefslogtreecommitdiff
path: root/macros/luatex/optex/catcodes.opm
blob: d675e482bed416a9495b1ba758c06c35ff018bf0 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
%% This is part of OpTeX project, see http://petr.olsak.net/optex

\_codedecl \setcatcode {Initialization category codes <2020-01-23>}

\_def \_optexcatcodes {%
   \_setcatcode `\\=0 % backlslash starts control sequences
   \_setcatcode `\{=1 % left brace is begin-group character
   \_setcatcode `\}=2 % right brace is end-group character
   \_setcatcode `\$=3 % dollar sign is math shift
   \_setcatcode `\&=4 % ampersand is alignment tab
   \_setcatcode `\^^M=5 % ascii return is end-line
   \_setcatcode `\#=6 % hash mark is macro parameter character
   \_setcatcode `\^=7 
   \_setcatcode `\^^K=7 % circumflex and uparrow are for superscripts
   \_setcatcode `\^^A=8 % downarrow is for subscripts
   \_setcatcode `\ =10 % ascii space is blank space
   \_setcatcode `\^^I=10 % ascii tab is a blank space
   \_setcatcode `\_=11 % underline can be used in control sequences
   \_setcatcode `\~=13 % tilde is active
   \_setcatcode `\%=14 % percent sign is comment character
   \_setcatcode 127=12 % normal character
}
\_let \_setcatcode =\catcode   \_optexcatcodes   % catcode initialization

\_def \_normalasciicatcodes {%
   \_setcatcodes 33-34: 12
   \_setcatcodes 39-64: 12
   \_setcatcodes 65-90: 11
   \_setcatcode  `\[=12
   \_setcatcode  `\]=12
   \_setcatcode  `\`=12
   \_setcatcodes 97-122: 11
   \_setcatcode  `\|=12
   \_optexcatcodes
}
\_def \_setcatcodes #1-#2: #3 {%
   \_tmpnum=#1
   \_loop
      \_setcatcode \_the\_tmpnum =#3
      \_advance \_tmpnum by1
      \_unless \_ifnum \_tmpnum >#2 \_repeat
}
\_def \_setcatcode #1=#2 {%
   \_ifx \_setcatcode \_empty \_errmessage {Use \_noexpand\savecatcode before \_noexpand\setcatcode}\_fi
   \_unless \_ifnum \_catcode #1=#2
      \_ea \_setcatcodeA \_catcodelist \_end {#1}%
      \_catcode #1=#2 
   \_fi
}
\_def \_setcatcodeA #1#2\_end #3{\_edef \_catcodelist{{#1}\_catcode #3=\_the\_catcode #3 #2}}
\_def \_savecatcodes {\_edef\_catcodelist {{\_catcodelist}}}
\_def \_restorecatcodes {\_ea\_def \_ea\_catcodelist \_catcodelist}
\_def \_catcodelist{{}}

\_chardef\_active=13 \_catcode`\~=\_active % tilde is active
%\catcode`\^^L=\active \outer\def^^L{\par} % ascii form-feed is "\outer\par", obsolete

\public 
   \optexcatcodes \normalasciicatcodes \setcatcodes \setcatcode
   \savecatcodes \restorecatcodes \active ;

\_endcode % ----------------------------------------

Basic category codes inspired by plain \TeX/ are set here. The only
difference is underline: it can be used in control sequences in \SpTeX/, so
its category code is 11. But plain \TeX/ sets it to 8 for subscripts. We'll
define `_` subscript only in math mode, see `math-basics.spm`.

Second differnece from plain \TeX/ is `\catcode127`, which is set to invalid
character in plain \TeX/ but we set it as normal character.

`\normalasciicatodes` sets catcodes of all ascii visible characters to its
"normal" values used in \SpTeX/. 

We implement catcodes stack by three macros: `\savecatcodes`, `\setcatcode`
and `\restorecatcodes` which bahaves with catcode values like 
`\begingroup`, `\catcode` and `\endgroup` but it does not 
open/close any \TeX/ group. You can use it for eaxample

\begtt
\savecatcodes \normalasciicatcodes \input macrofile.tex \restorecatcodes
\endtt
%
and you are sure that 

\begitems
* `macrofile.tex` is read with normal catodes,
* previous catcode setting is restored after the `macrofile.tex` is read.
\enditems

The catcodes stack cannot be crossed by \TeX/ groups. But it is not
problem to re-implement it to global setting if you want.

You can use `\setcatcodes <from>-<to>: <value>` to set a range of characters.
This macro uses `\setcatcode` repeatedly.


\sec Implemetation details:

First, we define `\_plaintexcatcodes` and use it macro with
`\_let\_setcatcode=\catcode`. This code sets the "normal" catcodes.

The main idea of catcode stack implementation is that the macro
`\_setcatcode` does nothing if previus catcode value is the same as desired
value. Else it saves previous value into `\_catcodelist` and sets desired
value. Finally `\_restorecatcodes` restores values from the `\_catcodelist`.

We need to support stack behavior, it means that nested pairs
`\_savecatcodes` and `\_restorecatcodes` work well. This is a reason why the
current value of the `\_catcodelist` looks like this:

\begtt
{data actual before last \_savecatcodes} currently generated data
\endtt
%
Then `\_restorecatcodes` does 

\begtt
\_def \_catcodelist {data actual before savecatcodes} apply currently generated data
\endtt

We need to apply currently generated data in reverse order. This is done by
the macro `\setcatcodeA` (invoked by the `\setcatcode`) which inserts new
data HERE-NEW:

\begtt
{data actual before last savecatdodes} HERE-NEW currently generated data
\endtt

One record of data is in the form `\catode char-code=previous-value`, so we
need simply to "run" the currently generated data in order to restore
values.

Notice: the `\_codedecl` and `\_endcode` macros are redefined and explained in basic.opm