diff options
Diffstat (limited to 'Master/texmf-dist/doc/luatex/interpreter/interpreter-doc.txt')
| -rw-r--r-- | Master/texmf-dist/doc/luatex/interpreter/interpreter-doc.txt | 426 |
1 files changed, 410 insertions, 16 deletions
diff --git a/Master/texmf-dist/doc/luatex/interpreter/interpreter-doc.txt b/Master/texmf-dist/doc/luatex/interpreter/interpreter-doc.txt index 5bece3bac7a..0156c3e07d8 100644 --- a/Master/texmf-dist/doc/luatex/interpreter/interpreter-doc.txt +++ b/Master/texmf-dist/doc/luatex/interpreter/interpreter-doc.txt @@ -1,19 +1,23 @@ TABLE OF CONTENTS - (Make a search on the tag in the right - column to jump to the associated section. - Vim users can simply type * on the tag; - Emacs users do that with C-s C-w (I think); - Other editors: I don't know!) -======================================= - Introduction intro_tag - Input files input_tag - Paragraphs paragraphs_tag - Declaring patterns patterns_tag - Classes classes_tag - Protecting input protect_tag - Technical stuff technical_tag - An example: i-doc.lua example_tag -======================================= + (Make a search on the tag in the right column to jump + to the associated section. + Vim users can simply type * on the tag; + Emacs users do that with C-s C-w (I think); + Other editors: I don't know!) +=========================================================== + Introduction intro_tag + Input files input_tag + Paragraphs paragraphs_tag + Declaring patterns patterns_tag + Classes classes_tag + Protecting input protect_tag + Technical stuff technical_tag + An example: i-doc.lua example_tag + The Gates in Interpreter gates_tag + The interpreter table interpreter_tag + The interpreter.core.tools table interpreter_tools_tag + The interpreter.core.reader table interpreter_reader_tag +=========================================================== @@ -41,6 +45,14 @@ modified lines to TeX you can also translate the original lines into, say, HTML, and write them to an external file, thus creating both PDF and HTML output at once. +Interpreter has been rewritten with the Gates package (actually, only +the Lua side) in version 1.1. That hasn't changed anything to its default +behavior, but now it can also be customized quite deeply, since its +code is a collection of small chunks with names that can be externally +controlled and/or augmented. See the Gates documentation for further +information. The last sections of this documentation describe the gates +in Interpreter. + ================================== input_tag @@ -224,7 +236,7 @@ precisely, the process is as follows: Interpreter searches the entire paragraph for the first pattern in class~1, then for the second pattern in the same class, then for the third, etc., then when there is no pattern left in class~1 it does the same with class~2, up to class~_n_, where _n_ is the -hightest class number such that there exists a class _n - 1_ (in other words, +highest class number such that there exists a class _n - 1_ (in other words, classes should be numbered consecutively). Finally, the same goes for the patterns in class~0 (which always exists, even if it contains no pattern). @@ -381,3 +393,385 @@ does its job; actually you won't learn much anyway. ================================== \interpretfile{doc}{i-doc.lua} + + + +================================== gates_tag +=== The Gates in Interpreter ===== +================================== + +Interpreter is written with the Gates package (only the Lua side, +actually). It means that it can be hacked down to the core. Here I'll +simply list the gates involved; you should read the Gates +documentation to learn how to use them. + +There are three gates families: "interpreter", associated with the main +"interpreter" table, contains the user interface; "interpreter_tools", +associated with "interpreter.core.tools" table, contains internal +functions; finally "interpreter_reader", associated with the +"interpreter.core.reader" table, contains the main functions used to +read the file. + +Whenever I mention a conditional or a loop, I mean the local conditionals +and loops, relative to the l-gate where the gate appears. Also, the +syntax indicates the arguments a gate uses, not all the arguments that +are passed to it (which are simply what the previous gate has returned). + +As an example of customizing Interpreter with Gates, you could very +well add a bit of code which does something to all lines. Inserting a +small gate, say "everyline", after "check_direct" in "aggregate_lines" +below would do the trick, e.g.: + + + function interpreter.core.reader.everyline (file, line) + line = dosomething(line) + return file, line + end + interpreter.core.reader.add( + "everyline", "aggregate_lines", "after check_direct") + interpreter.core.reader.conditional( + "everyline", "aggregate_lines", function (f, l) return l end) + +(Note that it is important to check that the line really exist, +because one might have hit the end of the file; hence the conditional, +as with others gates in "aggregate_lines"). + + + + +================================== interpreter_tag +=== The "interpreter" table ======== +================================== + +All the user functions in "interpreter" are simple m-gates, so they can +be treated as ordinary functions, except "interpreter.add_pattern", +which is an l-gate containing, built as: + + add_pattern + . ensure_class + . apply_nomagic + . insert-pattern + . . do_insert + . . sort_class + +> ensure_class (<pattern>) [m-gate] + Creates the class of <pattern> if necessary, and set it as the metatable + for <pattern>. Classes themselves are kept in the "interpreter.core.classes" + table. The gate return <pattern> and the class number. + +> apply_nomagic (<pattern>, <class>) [m-gate] + Transforms the "pattern" entry in <pattern> with "intepreter.nomagic"; + tied to a conditional that returns <pattern>'s "nomagic" entry (so + the gate is executed only if "nomagic" is true); autoreturns both + arguments. + +> insert_pattern (<pattern>, <class>) [l-gate] + An autoreturning l-gate containing the following two gates. + +> do_insert (<pattern>, <class>) [m-gate] + Adds <pattern> to <class> (i.e. "interpreter.core.classes[<class>]"). + +> sort_class (<pattern>, <class>) [m-gate] + Sorts <class> with function "interpreter.core.tools.sort". This gate + can be skipped to apply the patterns in the order in which they were + declared. + + + +============================================= interpreter_tools_tag +=== The interpreter.core.tools table ======== +============================================= + +All the functions in the "interpreter.core.tools" all are simple m-gates. + +> sort (<patt1>, <patt2>) [m-gate] + Returns true if the pattern in "patt1" is longer than the one in + "patt2", or if they have the same length and the first ranks before + the second with respect to alphabetical order. The gate is used in + the "interpreter.sort_class" m-gate. + +> xsub (<string>, <index>, <pattern>, <replacement>) [m-gate] + Returns <string> with <pattern> replaced with <replacement>, but only + once, and only after <index>. + +> protector (<string>, <index>) [m-gate] + Checks whether <index> in <string> isn't between characters declared + with "interpreter.protector". If that is the case, the function returns + "nil" and the index of the second protector. Otherwise, it returns + <index>. + +> get_index (<string>, <pattern>, <index>) [m-gate] + Checks whether <pattern> occurs in <string>, starting at <index>. If + it does, but if <index>-1 is "interpreter.escape", calls itself with + <index>+1. Otherwise, calls "interpreter.core.tools.protector" to + check whether <index> is in a protected part of the string. If so, + calls itself with <right>+1 instead of <index>, where <right> is the + second return value of of "interpreter.core.tools.protector", i.e. + it searches again after the right protector. If <index> is found, + end of story, returns <index>, otherwise returns nothing. + +The "interpreter.core.tools" table also contains +"magic_characters", a table with an entry for each magic character in Lua except +`"."' and `"%"'; the values to those entries are the same characters +prefixed with `"%"'. The table is used by "interpreter.nomagic" to spot +and replace magic characters, with the dot and the percent sign dealt +with independantly. + + + +============================================= interpreter_reader_tag +=== The interpreter.core.reader table ======= +============================================= + +Interpreter works by hooking in the "open_read_file" callback; the +function registered there is the "interpreter.core.reader.input" +l-gate, built as follows: + + input + . unregister + . . set_unregister + . . use_unregister + . open_file + . set_reader + +> unregister (<filename>) [l-gate] + Contains the following two m-gates; <filename> is received from + "input", which itself receives it from the callback, i.e. that's the + file that's being input (the second argument to "\interpretfile"). + It is also automatically returned. + +> set_unregister () [m-gate] + Sets the function to remove "input" from the callback, namely + "interpreter.unregister"; the gate is called only if gate + "interpreter.unregister" doesn't already exits. If "luatexbase" is + detected, the functions there are used; otherwise, "callback.register" + is used with "nil" as the second argument. + +> use_unregister () [m-gate] + Calls "interpreter.unregister()". (You don't want the next input file + to be processed with Interpreter by default, that's why you remove + the callback function; not that the current one is nonetheless + processed with the current file, of course.) + +> open_file (<filename>) [m-gate] + Returns "io.open(<filename>)". + +> set_reader (<file>) [m-gate] + Returns a table with a "reader" entry containing a function whose + definition is + + function () + return interpreter.core.reader.read_file(f) + end + + That's the convention for the "open_read_file" callback: it should + return such a table, and the function will be called each time a + line is required from the input file. + +So most of the work is done by "interpreter.core.reader.read_file", +which is why it is so heavy; it receives a file handle: + + read_file + . make_paragraph + . . aggregate_lines + . . . read_line + . . . check_direct + . . . insert_line + . . apply_classes + . . . initialize + . . . pass_class + . . . . pass_pattern + . . . . . pass_line + . . . . . . check_index + . . . . . . . call + . . . . . . . replace + . . . . . . . protect + . . . . . . increment_line + . . . . . increment_pattern + . . . . increment_class + . . . apply_class0 + . . . . initialize0 + . . . . pass_pattern + . . . . . pass_line + . . . . . . check_index + . . . . . . . call + . . . . . . . replace + . . . . . . . protect + . . . . . . increment_line + . . . . . increment_pattern + . . . unprotect + . . . . undo_protected + . . . . unprotect_lines + . . . remove_escape + . return_line + +> make_paragraph (<file>) [l-gate] + The big l-gate that contains everything that follows, barring + "return_line". It is called if and only if the "interpreter.core.lines" + table is empty; that table is where lines of a paragraph are stored, + and it is emptied by "return_line". + +> aggregate_lines (<file>, <line>) [l-gate] + The main l-gate that reads line from <file> and stores them in + "interpreter.core.lines". It loops until <line> is "nil" or equivalent + to "interpreter.paragraph". (Of course, <line> is "nil" on the first + iteration, but the "loopuntil" conditional is evaluated after that + first iteration, during which the last subgate "insert_line" will + probably returns a line.) + +> read_line (<file>) [m-gate] + Reads the next line from <file>, and returns <file> and that line. + +> check_direct (<file>, <line>) [m-gate] + If <line> begins with "interpreter.direct", removes it and use + "loadstring" on the resulting string. Returns <file> and <line>, the + latter set to an empty string is the previous operation applied. The + gate depends on a conditional: <line> should be non-"nil" (of + course), and "interpreter.direct" should be defined. + +> insert_line (<file>, <line>) [m-gate] + Adds <line> to "interpreter.core.lines". Automatically returns the + two arguments (and if <line> isn't "nil" or equivalent to + "interpreter.paragraph", it will be executed again). + +> apply_classes () [l-gate] + The l-gate that applies transformations to the lines, once the paragraph + has been gathered, with the gates that follow. For each class, it + will apply each pattern on each line. It depends on a conditional: + "interpreter.core.lines" shouldn't be empty, and "interpreter.active" + should be true. + +> initialize () [m-gate] + Returns "1, 1, 1, 1"; it simply starts the following loop. + +> pass_class (<class>) [l-gate] + An l-gate called as long as "interpreter.core.reader.protected" isn't + a boolean value (see "unprotect" below) and <class> isn't larger than + "#interpreter.core.classes"; see "increment_class" below, which returns + "<class>+1". In other words, "pass_class" will be repeated for each + class. + +> pass_pattern (<class>, <pattern>) [l-gate] + Same as "pass_class", except it iterates on the patterns in "class": + it is executed as long as "interpreter.core.reader.protected" isn't + a boolean and "<pattern>" isn't larger than "#interpreter.core.classes[<class>]". + See "increment_pattern" below. In other words, it will be repeated + for each pattern in <class>. + +> pass_line (<class>, <pattern>, <line>), [l-gate] + Here we go again, except we check, besides protection, that <line> + isn't larger than "#interpreter.core.lines". See "increment_line" + below. + +> check_index (<class>, <pattern>, <line>, <index>) [l-gate] + This time we check that "interpreter.core.classes[<class>][<pattern>].pattern" + occurs in "interpreter.core.lines[<line>]", starting at <index>. + Protection is checked too, as is the type of "interpreter.core.lines[<line>]"; + indeed, individual lines are protected by turning them into tables + (with a single entry, the line itself), see "protect" below. The + occurrence of the pattern in the line is checked with + "interpreter.core.tools.get_index". + +> call (<class>, <pattern>, <line>, <index>) [m-gate] + This is executed if and only if the pattern table (i.e. + "interpreter.core.classes[<class>][<pattern>]") has a "call" entry. + The index where the pattern occurs is retrieved with + "interpreter.core.tools.get_index" and stored in <newindex>, and then + the "call" entry is executed as: + + call (interpreter.core.lines, + <line>, <newindex>, <pattern table>) + + where <pattern table> is defined as above; if call returns one value, + it is assigned to <index>; if it returns two, they are assigned to + <line> and <index>. The gate returns the four arguments. + +> replace (<class>, <pattern>, <line>, <index>) [m-gate] + This is executed only if the pattern table, as defined above, has a + "replace" field and no "call" field; it applies "interpreter.core.tools.xsub" + as: + + xsub (interpreter.core.lines[<line>], + <newindex>, <pattern table>.pattern, + <pattern table>.replace) + + where <newindex> is defined as in "call". The return value of "xsub" + is assigned to "interpreter.core.lines[<line>]". The gate returns + its four argument, with <index> incremented by the value of the + "offset" entry of the pattern, if any. + +> protect () [m-gate] + The "interpreter.protect()" function can either protect the whole + paragraph (when no argument is passed) or a single line (when a number + is passed). In the first case, "interpreter.core.tools.protected" + takes the value "true", which is checked in various gates above. In + the second case, "interpreter.core.tools.protected" is a table with + each index indicating a line to be protected. This gate implements + the protection in that case: for each entry in the table, it protects + the line with the same index in "interpreter.core.lines" by transforming + it into a table (with a single entry, the string representing the + original line); the type of the line is checked in "check_index" + above. This gate is executed if and only if "interpreter.core.tools.protected" + is a table. The four arguments returned by "call" or "replace" are + automatically returned, so that "check_index" is reexecuted if its + "loop" conditional evaluates to "true". + +> increment_line (<class>, <pattern>, <line>, <index>) [m-gate] + Returns <class>, <pattern>, <line> + 1, <index>, so "pass_line" is + reexecuted if its "loop" conditional evaluates to "true". + +> increment_pattern (<class>, <pattern>, <line>, <index>) [m-gate] + Returns <class>, <pattern> + 1, <line>, <index>, so "pass_pattern" + is reexecuted if its "loop" conditional evaluates to "true". + +> increment_class (<class>, <pattern>, <line>, <index>) [m-gate] + Returns <class> + 1, <pattern>, <line>, <index>, so "pass_class" is + reexecuted if its "loop" conditional evaluates to "true". + +> apply_class0 () [l-gate] + The l-gate that does roughly the same job as "apply_classes", but + with class 0. It is called if and only if "interpreter.core.reader.protected" + isn't a boolean. + +> initialize0 () [m-gate] + The same as "initialize", except it returns "0, 1, 1, 1", since + we're interested in class 0. + +> pass_pattern (<class>, <pattern>) [l-gate] + The same as above. (Note that this version of the gate doesn't occur + in the same l-gate as the previous one, so its local "loop" is also + different.) + +There is one important difference between "apply_class" and "apply_class0": +the latter has "pass_pattern" as its immediate subgate, without +"pass_class" as an intermediary l-gate. That makes sense, since we don't +want to loop on all the classes here (as that was done in "apply_class"), +but we're only interested in class 0. Apart from that, the "pass_pattern" +gate is really the same in both, so all the subgates of "pass_pattern" +are also reapplied here; I won't describe them again. + +> unprotect () [l-gate] + Now all the patterns in all the classes have been applied to the + entire paragraph, and protection must be removed. This l-gate + contains the following two gates. + +> undo_protected () [m-gate] + Simply sets "interpreter.core.tools.protected" to "nil" so it is + ready for the next paragraph. + +> unprotect_lines () [m-gate] + Restore all the lines in "interpreter.core.lines" as simple strings. + +> remove_escape () [m-gate] + If "interpreter.escape" is defined, removes all its occurrences in + each line of the paragraph. + +This is the end of the big "make_paragraph" l-gate. It won't be called +again until the paragraph has been fully passed to TeX, i.e. when +"interpreter.core.lines" is empty. + +> return_line () [m-gate] + Pops the first line from "interpreter.core.lines" and returns it. + Since this is the very last subgate of "read_file", the line is + passed to TeX. + |