updated reference manual for luatex 1.09.2

git-svn-id: svn://tug.org/texlive/trunk@49615 c570f23f-e606-0410-a88d-b1316a301751

24 files changed, 17821 insertions, 14540 deletions

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-backend.tex b/Master/texmf-dist/doc/luatex/base/luatex-backend.tex
index 46e9222e289..0ec9be9b0f7 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-backend.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-backend.tex
@@ -1,1043 +1,1084 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-backend
-
-\startchapter[reference=backend,title={The backend libraries}]
-
-\section{The \type {pdf} library}
-
-This library contains variables and functions that are related to the \PDF\
-backend. You can find more details about the expected values to setters in \in
-{section} [backendprimitives].
-
-\subsection{\type {mapfile}, \type {mapline}}
-
-\startfunctioncall
-pdf.mapfile(<string> map file)
-pdf.mapline(<string> map line)
-\stopfunctioncall
-
-These two functions can be used to replace primitives \type {\pdfmapfile} and
-\type {\pdfmapline} inherited from \PDFTEX. They expect a string as only
-parameter and have no return value. The first character in a map line can be
-\type {-}, \type {+} or \type {=} which means as much as remove, add or replace
-this line.
-
-\subsection {\type {[set|get][catalog|info|names|trailer]}}
-
-These functions complement the corresponding \PDF\ backend token lists dealing
-with metadata. The value types are strings and they are written out to the \PDF\
-file directly after the token registers.
-
-\subsection {\type {[set|get][pageattributes|pageresources|pagesattributes]}}
-
-These functions complement the corresponding \PDF\ backend token lists dealing
-with page resources. The variables have no interaction with the corresponding \PDF\
-backend token register. They are written out to the \PDF\ file directly after the
-token registers.
-
-\subsection{\type {[set|get][xformattributes|xformresources]}}
-
-These functions complement the corresponding \PDF\ backend token lists dealing
-with reuseable boxes and images. The variables have no interaction with the
-corresponding \PDF\ backend token register. They are written out to the \PDF\
-file directly after the token registers.
-
-\subsection{\type {getversion} and \type {[set|get]minorversion}}
-
-The version is frozen in the binary but you can set the minor version. What minor
-version you set depends on what \PDF\ features you use. This is out of control
-of \LUATEX.
-
-\subsection{\type {getcreationdate}}
-
-This function returns a string with the date in the format that ends up in the
-\PDF\ file, in this case it's: {\tttf \cldcontext{pdf.getcreationdate()}}.
-
-\subsection{\type {[set|get]inclusionerrorlevel}, \type {[set|get]ignoreunknownimages}}
-
-These variable control how error in included image are treated. They are modeled
-after the \PDFTEX\ equivalents.
-
-\subsection{\type {[set|get]suppressoptionalinfo}}
-
-This bitset determnines what kind of info gets flushes. By default we flush all.
-
-\subsection{\type {[set|get]trailerid}}
-
-You can set your own trailer id. This has to be a valid array string
-with checksums.
-
-\subsection{\type {[set|get]compresslevel}}
-
-These two functions set the level of compression. The minimum value is~0,
-the maximum is~9.
-
-\subsection{\type {[set|get]objcompresslevel}}
-
-These two functions set the level of compression. The minimum value is~0,
-the maximum is~9.
-
-\subsection{\type {[set|get]gentounicode}}
-
-This flag enables tounicode generation (like in \PDFTEX).
-
-\subsection{\type {[set|get]omitcidset}}
-
-This flag disables inclusion of a so called CIDSet which can be handy when aiming
-at some of the many \PDF\ substandards.
-
-\subsection{\type {[set|get]decimaldigits}}
-
-These two functions set the accuracy of floats written to the \PDF file. You can
-set any value but the backend will not go below~3 and above~6.
-
-\subsection{\type {[set|get]pkresolution}}
-
-These setter takes two arguments: the resolution and an optional zero or one that
-indicates if this is a fixed one. The getter returns these two values.
-
-\subsection{\type {getlast[obj|link|annot]} and \type {getretval}}
-
-These status variables are similar to the ones traditionally used in the backend
-interface at the \TEX\ end.
-
-\subsection{\type {maxobjnum} and \type {objtype}, \type {fontname}, \type {fontobjnum},
-\type {fontsize}, \type {xformname}}.
-
-These (and some other) introspective helpers were moved from the the \type {tex}
-namespace to the \type {pdf} namespace but kept their original names. They are
-mostly used when you construct \PDF\ objects yourself and need for instance
-information about a (to be) embedded font.
-
-\subsection{\type {[set|get]origin}}
-
-This one is used to set the horizonal and|/|or vertical offset, a traditional
-backend property.
-
-\starttyping
-pdf.setorigin() -- sets both to 0pt
-pdf.setorigin(tex.sp("1in")) -- sets both to 1in
-pdf.setorigin(tex.sp("1in"),tex.sp("1in"))
-\stoptyping
-
-The counterpart of this function returns two values.
-
-\subsection{\type {[set|get]imageresolution}}
-
-These two functions relate to the imageresolution that is used when the image
-itself doesn't provide a non|-|zero x or y resolution.
-
-\subsection {\type {[set|get][link|dest|thread|xform]margin}}
-
-These functions can be used to set and retrieve the margins that are added to the
-natural bounding boxes of the respective objects.
-
-\subsection {\type {get[pos|hpos|vpos]}}
-
-These function get current location on the output page, measured from its lower
-left corner. The values return scaled points as units.
-
-\starttyping
-local h, v = pdf.getpos()
-\stoptyping
-
-\subsection {\type {[has|get]matrix}}
-
-The current matrix transformation is available via the \type {getmatrix} command,
-which returns 6 values: \type {sx}, \type {rx}, \type {ry}, \type {sy}, \type
-{tx}, and \type {ty}. The \type {hasmatrix} function returns \type {true} when a
-matrix is applied.
-
-\starttyping
-if pdf.hasmatrix() then
-    local sx, rx, ry, sy, tx, ty = pdf.getmatrix()
-    -- do something useful or not
-end
-\stoptyping
-
-\subsection {\type {print}}
-
-You can print string to the \PDF\ document from within a within a \type
-{\latelua} call. This function is not to be used inside \type {\directlua} unless
-you know {\it exactly} what you are doing.
-
-\startfunctioncall
-pdf.print(<string> s)
-pdf.print(<string> type, <string> s)
-\stopfunctioncall
-
-The optional parameter can be used to mimic the behavior of \PDF\ literals: the
-\type {type} is \type {direct} or \type {page}.
-
-\subsection {\type {immediateobj}}
-
-This function creates a \PDF\ object and immediately writes it to the \PDF\ file.
-It is modelled after \PDFTEX's \type {\immediate} \type {\pdfobj} primitives. All
-function variants return the object number of the newly generated object.
-
-\startfunctioncall
-<number> n =
-    pdf.immediateobj(<string> objtext)
-<number> n =
-    pdf.immediateobj("file", <string> filename)
-<number> n =
-    pdf.immediateobj("stream", <string> streamtext, <string> attrtext)
-<number> n =
-    pdf.immediateobj("streamfile", <string> filename, <string> attrtext)
-\stopfunctioncall
-
-The first version puts the \type {objtext} raw into an object. Only the object
-wrapper is automatically generated, but any internal structure (like \type {<<
->>} dictionary markers) needs to provided by the user. The second version with
-keyword \type {file} as first argument puts the contents of the file with name
-\type {filename} raw into the object. The third version with keyword \type
-{stream} creates a stream object and puts the \type {streamtext} raw into the
-stream. The stream length is automatically calculated. The optional \type
-{attrtext} goes into the dictionary of that object. The fourth version with
-keyword \type {streamfile} does the same as the third one, it just reads the
-stream data raw from a file.
-
-An optional first argument can be given to make the function use a previously
-reserved \PDF\ object.
-
-\startfunctioncall
-<number> n =
-    pdf.immediateobj(<integer> n, <string> objtext)
-<number> n =
-    pdf.immediateobj(<integer> n, "file", <string> filename)
-<number> n =
-    pdf.immediateobj(<integer> n, "stream", <string> streamtext, <string> attrtext)
-<number> n =
-    pdf.immediateobj(<integer> n, "streamfile", <string> filename, <string> attrtext)
-\stopfunctioncall
-
-\subsection{\type{obj}}
-
-This function creates a \PDF\ object, which is written to the \PDF\ file only
-when referenced, e.g., by \type {refobj()}.
-
-All function variants return the object number of the newly generated object, and
-there are two separate calling modes. The first mode is modelled after \PDFTEX's
-\type {\pdfobj} primitive.
-
-\startfunctioncall
-<number> n =
-    pdf.obj(<string> objtext)
-<number> n =
-    pdf.obj("file", <string> filename)
-<number> n =
-    pdf.obj("stream", <string> streamtext, <string> attrtext)
-<number> n =
-    pdf.obj("streamfile", <string> filename, <string> attrtext)
-\stopfunctioncall
-
-An optional first argument can be given to make the function use a previously
-reserved \PDF\ object.
-
-\startfunctioncall
-<number> n =
-    pdf.obj(<integer> n, <string> objtext)
-<number> n =
-    pdf.obj(<integer> n, "file", <string> filename)
-<number> n =
-    pdf.obj(<integer> n, "stream", <string> streamtext, <string> attrtext)
-<number> n =
-    pdf.obj(<integer> n, "streamfile", <string> filename, <string> attrtext)
-\stopfunctioncall
-
-The second mode accepts a single argument table with key--value pairs.
-
-\startfunctioncall
-<number> n = pdf.obj {
-    type           = <string>,
-    immmediate     = <boolean>,
-    objnum         = <number>,
-    attr           = <string>,
-    compresslevel  = <number>,
-    objcompression = <boolean>,
-    file           = <string>,
-    string         = <string>
-}
-\stopfunctioncall
-
-The \type {type} field can have the values \type {raw} and \type {stream}, this
-field is required, the others are optional (within constraints).
-
-Note: this mode makes \type{obj} look more flexible than it actually is: the
-constraints from the separate parameter version still apply, so for example you
-can't have both \type {string} and \type {file} at the same time.
-
-\subsection {\type {refobj}}
-
-This function, the \LUA\ version of the \type {\pdfrefobj} primitive, references an
-object by its object number, so that the object will be written out.
-
-\startfunctioncall
-pdf.refobj(<integer> n)
-\stopfunctioncall
-
-This function works in both the \type {\directlua} and \type {\latelua} environment.
-Inside \type {\directlua} a new whatsit node \quote {pdf_refobj} is created, which
-will be marked for flushing during page output and the object is then written
-directly after the page, when also the resources objects are written out. Inside
-\type {\latelua} the object will be marked for flushing.
-
-This function has no return values.
-
-\subsection {\type {reserveobj}}
-
-This function creates an empty \PDF\ object and returns its number.
-
-\startfunctioncall
-<number> n = pdf.reserveobj()
-<number> n = pdf.reserveobj("annot")
-\stopfunctioncall
-
-\subsection {\type {registerannot}}
-
-This function adds an object number to the \type {/Annots} array for the current
-page without doing anything else. This function can only be used from within
-\type {\latelua}.
-
-\startfunctioncall
-pdf.registerannot (<number> objnum)
-\stopfunctioncall
-
-\subsection {\type {newcolorstack}}
-
-This function allocates a new color stack and returns it's id. The arguments
-are the same as for the similar backend extension primitive.
-
-\startfunctioncall
-pdf.newcolorstack("0 g","page",true) -- page|direct|origin
-\stopfunctioncall
-
-\subsection {\type {setfontattributes}}
-
-This function will force some additional code into the font resource. It can for
-instance be used to add a custom \type {ToUnicode} vector to a bitmap file.
-
-\startfunctioncall
-pdf.setfontattributes(<number> font id, <string> pdf code)
-\stopfunctioncall
-
-\section {The \type {pdfscanner} library}
-
-The \type {pdfscanner} library allows interpretation of \PDF\ content streams and
-\type {/ToUnicode} (cmap) streams. You can get those streams from the \type
-{epdf} library, as explained in an earlier section. There is only a single
-top|-|level function in this library:
-
-\startfunctioncall
-pdfscanner.scan (<Object> stream, <table> operatortable, <table> info)
-\stopfunctioncall
-
-The first argument, \type {stream}, should be either a \PDF\ stream object, or a
-\PDF\ array of \PDF\ stream objects (those options comprise the possible return
-values of \type {<Page>:getContents()} and \type {<Object>:getStream()} in the
-\type {epdf} library).
-
-The second argument, \type {operatortable}, should be a \LUA\ table where the
-keys are \PDF\ operator name strings and the values are \LUA\ functions (defined
-by you) that are used to process those operators. The functions are called
-whenever the scanner finds one of these \PDF\ operators in the content stream(s).
-The functions are called with two arguments: the \type {scanner} object itself,
-and the \type {info} table that was passed are the third argument to \type
-{pdfscanner.scan}.
-
-Internally, \type {pdfscanner.scan} loops over the \PDF\ operators in the
-stream(s), collecting operands on an internal stack until it finds a \PDF\
-operator. If that \PDF\ operator's name exists in \type {operatortable}, then the
-associated function is executed. After the function has run (or when there is no
-function to execute) the internal operand stack is cleared in preparation for the
-next operator, and processing continues.
-
-The \type {scanner} argument to the processing functions is needed because it
-offers various methods to get the actual operands from the internal operand
-stack.
-
-A simple example of processing a \PDF's document stream could look like this:
-
-\starttyping
-function Do (scanner, info)
-   local val       = scanner:pop()
-   local name      = val[2] -- val[1] == 'name'
-   local resources = info.resources
-   local xobject   = resources:lookup("XObject"):getDict():lookup(name)
-   print (info.space ..'Use XObject '.. name)
-   if xobject and xobject:isStream() then
-      local dict = xobject:getStream():getDict()
-      if dict then
-        local name = dict:lookup("Subtype")
-        if name:getName() == "Form" then
-          local newinfo =  {
-            space = info.space .. "  " ,
-            resources = dict:lookup("Resources"):getDict()
-          }
-          pdfscanner.scan(xobject, operatortable, newinfo)
-        end
-      end
-   end
-end
-
-operatortable = { Do = Do }
-
-doc      = epdf.open(arg[1])
-pagenum  = 1
-
-while pagenum <= doc:getNumPages() do
-   local page = doc:getCatalog():getPage(pagenum)
-   local info = {
-     space     = "  " ,
-     resources = page:getResourceDict()
-   }
-   print('Page ' .. pagenum)
-   pdfscanner.scan(page:getContents(), operatortable, info)
-   pagenum = pagenum + 1
-end
-\stoptyping
-
-This example iterates over all the actual content in the \PDF, and prints out the
-found \type {XObject} names. While the code demonstrates quite some of the \type
-{epdf} functions, let's focus on the type \type {pdfscanner} specific code
-instead.
-
-From the bottom up, the following line runs the scanner with the \PDF\ page's
-top-level content.
-
-\starttyping
-   pdfscanner.scan(page:getContents(), operatortable, info)
-\stoptyping
-
-The third argument, \type {info}, contains two entries: \type {space} is used to
-indent the printed output, and \type {resources} is needed so that embedded \type
-{XForms} can find their own content.
-
-The second argument, \type {operatortable} defines a processing function for a
-single \PDF\ operator, \type {Do}.
-
-The function \type {Do} prints the name of the current \type {XObject}, and then
-starts a new scanner for that object's content stream, under the condition that
-the \type {XObject} is in fact a \type {/Form}. That nested scanner is called
-with new \type {info} argument with an updated \type {space} value so that the
-indentation of the output nicely nests, and with an new \type {resources} field
-to help the next iteration down to properly process any other, embedded \type
-{XObject}s.
-
-Of course, this is not a very useful example in practise, but for the purpose of
-demonstrating \type {pdfscanner}, it is just long enough. It makes use of only
-one \type {scanner} method: \type {scanner:pop()}. That function pops the top
-operand of the internal stack, and returns a \LUA\ table where the object at index
-one is a string representing the type of the operand, and object two is its
-value.
-
-The list of possible operand types and associated \LUA\ value types is:
-
-\starttabulate[|l|c|]
-\NC \type{integer}  \NC <number>  \NC \NR
-\NC \type{real}     \NC <number>  \NC \NR
-\NC \type{boolean}  \NC <boolean> \NC \NR
-\NC \type{name}     \NC <string>  \NC \NR
-\NC \type{operator} \NC <string>  \NC \NR
-\NC \type{string}   \NC <string>  \NC \NR
-\NC \type{array}    \NC <table>   \NC \NR
-\NC \type{dict}     \NC <table>   \NC \NR
-\stoptabulate
-
-In case of \type {integer} or \type {real}, the value is always a \LUA\ (floating
-point) number.
-
-In case of \type {name}, the leading slash is always stripped.
-
-In case of \type {string}, please bear in mind that \PDF\ actually supports
-different types of strings (with different encodings) in different parts of the
-\PDF\ document, so may need to reencode some of the results; \type {pdfscanner}
-always outputs the byte stream without reencoding anything. \type {pdfscanner}
-does not differentiate between literal strings and hexadecimal strings (the
-hexadecimal values are decoded), and it treats the stream data for inline images
-as a string that is the single operand for \type {EI}.
-
-In case of \type {array}, the table content is a list of \type {pop} return
-values and in case of \type {dict}, the table keys are \PDF\ name strings and the
-values are \type {pop} return values.
-
-\blank
-
-There are few more methods defined that you can ask \type {scanner}:
-
-\starttabulate[|l|p|]
-\NC \type{pop}       \NC as explained above \NC \NR
-\NC \type{popNumber} \NC return only the value of a \type {real} or \type {integer} \NC \NR
-\NC \type{popName}   \NC return only the value of a \type {name} \NC \NR
-\NC \type{popString} \NC return only the value of a \type {string} \NC \NR
-\NC \type{popArray}  \NC return only the value of a \type {array} \NC \NR
-\NC \type{popDict}   \NC return only the value of a \type {dict} \NC \NR
-\NC \type{popBool}   \NC return only the value of a \type {boolean} \NC \NR
-\NC \type{done}      \NC abort further processing of this \type {scan()} call \NC \NR
-\stoptabulate
-
-The \type {popXXX} are convenience functions, and come in handy when you know the
-type of the operands beforehand (which you usually do, in \PDF). For example, the
-\type {Do} function could have used \type {local name = scanner:popName()}
-instead, because the single operand to the \type {Do} operator is always a \PDF\
-name object.
-
-The \type {done} function allows you to abort processing of a stream once you
-have learned everything you want to learn. This comes in handy while parsing
-\type {/ToUnicode}, because there usually is trailing garbage that you are not
-interested in. Without \type {done}, processing only end at the end of the
-stream, possibly wasting \CPU\ cycles.
-
-\section{The \type {epdf} library}
-
-The \type {epdf} library provides \LUA\ bindings to many \PDF\ access functions
-that are defined by the poppler \PDF\ viewer library (written in C$+{}+$ by
-Kristian H\o gsberg, based on xpdf by Derek Noonburg). Within \LUATEX\ xpdf
-functionality is being used since long time to embed \PDF\ files. The \type
-{epdf} library allows to scrutinize an external \PDF\ file. It gives access to
-its document structure: catalog, cross|-|reference table, individual pages,
-objects, annotations, info, and metadata. The \type {epdf} library only provides
-read|-|only access. At some point we might decide to limit the interface to a
-reasonable subset.
-
-\start \em
-
-The following functions will be removed in a next version. This is partly due to
-a change in the api of the used library and partly due to the fact that write
-access is not needed at all.
-
-\starttabulate[|T|T||]
-\NC Annot            \NC  getAppearance     \NC gone    \NC \NR
-\NC                  \NC  getBorder         \NC gone    \NC \NR
-\NC AnnotBorderStyle \NC  getWidth          \NC gone    \NC \NR
-\NC Array            \NC  incRef            \NC gone    \NC \NR
-\NC                  \NC  decRef            \NC gone    \NC \NR
-\NC                  \NC  add               \NC will go \NC \NR
-\NC Attribute        \NC  setRevision       \NC will go \NC \NR
-\NC                  \NC  setHidden         \NC will go \NC \NR
-\NC                  \NC  setFormattedValue \NC will go \NC \NR
-\NC EmbFile          \NC  save              \NC will go \NC \NR
-\NC Dict             \NC  incRef            \NC gone    \NC \NR
-\NC                  \NC  decRef            \NC gone    \NC \NR
-\NC                  \NC  add               \NC will go \NC \NR
-\NC                  \NC  set               \NC will go \NC \NR
-\NC                  \NC  remove            \NC will go \NC \NR
-\NC Object           \NC  initBool          \NC will go \NC \NR
-\NC                  \NC  initInt           \NC will go \NC \NR
-\NC                  \NC  initReal          \NC will go \NC \NR
-\NC                  \NC  initString        \NC will go \NC \NR
-\NC                  \NC  initName          \NC will go \NC \NR
-\NC                  \NC  initNull          \NC will go \NC \NR
-\NC                  \NC  initArray         \NC will go \NC \NR
-\NC                  \NC  initDict          \NC will go \NC \NR
-\NC                  \NC  initStream        \NC will go \NC \NR
-\NC                  \NC  initRef           \NC will go \NC \NR
-\NC                  \NC  initCmd           \NC will go \NC \NR
-\NC                  \NC  initError         \NC will go \NC \NR
-\NC                  \NC  initEOF           \NC will go \NC \NR
-\NC                  \NC  dictAdd           \NC will go \NC \NR
-\NC                  \NC  dictSet           \NC will go \NC \NR
-\NC                  \NC  streamReset       \NC will go \NC \NR
-\NC StructElement    \NC  setRevision       \NC will go \NC \NR
-\NC                  \NC  appendChild       \NC will go \NC \NR
-\NC                  \NC  appendAttribute   \NC will go \NC \NR
-\NC StructTreeRoot   \NC  appendChild       \NC will go \NC \NR
-\stoptabulate
-
-\stop
-
-For a start, a \PDF\ file is opened by \type {epdf.open()} with file name, e.g.:
-
-\starttyping
-doc = epdf.open("foo.pdf")
-\stoptyping
-
-This normally returns a \type {PDFDoc} userdata variable; but if the file could
-not be opened successfully, instead of a fatal error just the value \type {nil} is
-returned.
-
-All \LUA\ functions in the \type {epdf} library are named after the poppler
-functions listed in the poppler header files for the various classes, e.g., files
-\type {PDFDoc.h}, \type {Dict.h}, and \type {Array.h}. These files can be found
-in the poppler subdirectory within the \LUATEX\ sources. Which functions are
-already implemented in the \type {epdf} library can be found in the \LUATEX\
-source file \type {lepdflib.cc}. For using the \type {epdf} library, knowledge of
-the \PDF\ file architecture is indispensable.
-
-There are many different userdata types defined by the \type {epdf} library,
-currently these are \type {AnnotBorderStyle}, \type {AnnotBorder}, \type
-{Annots}, \type {Annot}, \type {Array}, \type {Attribute}, \type {Catalog}, \type
-{Dict}, \type {EmbFile}, \type {GString}, \type {LinkDest}, \type {Links}, \type
-{Link}, \type {ObjectStream}, \type {Object}, \type {PDFDoc}, \type
-{PDFRectangle}, \type {Page}, \type {Ref}, \type {Stream}, \type {StructElement},
-\type {StructTreeRoot} \type {TextSpan}, \type {XRefEntry} and \type {XRef}.
-
-All these userdata names and the \LUA\ access functions closely resemble the
-classes naming from the poppler header files, including the choice of mixed upper
-and lower case letters. The \LUA\ function calls use object|-|oriented syntax,
-e.g., the following calls return the \type {Page} object for page~1:
-
-\starttyping
-pageref = doc:getCatalog():getPageRef(1)
-pageobj = doc:getXRef():fetch(pageref.num, pageref.gen)
-\stoptyping
-
-But writing such chained calls is risky, as an intermediate function may return
-\type {nil} on error. Therefore between function calls there should be \LUA\ type
-checks (e.g., against \type {nil}) done. If a non|-|object item is requested (for
-instance a \type {Dict} item by calling \type {page:getPieceInfo()}, cf.~\type
-{Page.h}) but not available, the \LUA\ functions return \type {nil} (without
-error). If a function should return an \type {Object}, but it's not existing, a
-\type {Null} object is returned instead, also without error. This is in|-|line
-with poppler behavior.
-
-All library objects have a \type {__gc} metamethod for garbage collection. The
-\type {__tostring} metamethod gives the type name for each object.
-
-These are the object constructors:
-
-\startfunctioncall
-<PDFDoc>       = epdf.open(<string> PDF filename)
-<Annot>        = epdf.Annot(<XRef>, <Dict>, <Catalog>, <Ref>)
-<Annots>       = epdf.Annots(<XRef>, <Catalog>, <Object>)
-<Array>        = epdf.Array(<XRef>)
-<Attribute>    = epdf.Attribute(<Type>,<Object>)| epdf.Attribute(<string>, <int>, <Object>)
-<Dict>         = epdf.Dict(<XRef>)
-<Object>       = epdf.Object()|epdf.Object(<Type>)|epdf.Object(<Object_Type>,<string>)|
-<PDFRectangle> = epdf.PDFRectangle()
-\stopfunctioncall
-
-There is a new API for the \type {Object} constructors: the old \type {init<ObjType>} constructors
-are still present but they are deprecated. They will disappear after the next \TeX Live.
-The functions \type {StructElement_Type}, \type {Object_Type}, \type {Attribute_Type} and \type
-{AttributeOwner_Type} return a hash table \type {{<string>,<integer>}}.
-The hash table \type {Object_Type} is useful for the
-\type {epdf.Object(<Object_Type>,<string>)} \type {Object} constructor, where \type {<Object_Type>} is an
-integer.
-
-\type {Annot} methods:
-
-
-\startfunctioncall
-<boolean>     = <Annot>:isOK()
-<boolean>     = <Annot>:match(<Ref>)
-<Object>      = <Annot>:getAppearance() -- gone
-<AnnotBorder> = <Annot>:getBorder()     -- gone
-\stopfunctioncall
-
-\type {AnnotBorderStyle} methods (gone):
-
-\startfunctioncall
-<number> = <AnnotBorderStyle>:getWidth() -- gone
-\stopfunctioncall
-
-\type {Annots} methods:
-
-\startfunctioncall
-<integer> = <Annots>:getNumAnnots()
-<Annot>   = <Annots>:getAnnot(<integer>)
-\stopfunctioncall
-
-\type {Array} methods:
-
-\startfunctioncall
-            <Array>:incRef()      -- gone
-            <Array>:decRef()      -- gone
-            <Array>:add(<Object>) -- will go
-<integer> = <Array>:getLength()
-<Object>  = <Array>:get(<integer>)
-<Object>  = <Array>:getNF(<integer>)
-<string>  = <Array>:getString(<integer>)
-\stopfunctioncall
-
-\type {Attribute} methods:
-
-\startfunctioncall
-<boolean>  = <Attribute>:isOk()
-<integer>  = <Attribute>:getType()
-<integer>  = <Attribute>:getOwner()
-<string>   = <Attribute>:getTypeName()
-<string>   = <Attribute>:getOwnerName()
-<Object>   = <Attribute>:getValue()
-<Object>   = <Attribute>:getDefaultValue
-<string>   = <Attribute>:getName()
-<integer>  = <Attribute>:getRevision()
-             <Attribute>:setRevision(<unsigned integer>) -- will go
-<boolean>  = <Attribute>:isHidden()
-             <Attribute>:setHidden(<boolean>)            -- will go
-<string>   = <Attribute>:getFormattedValue()
-<string>   = <Attribute>:setFormattedValue(<string>)     -- will go
-\stopfunctioncall
-
-\type {Catalog} methods:
-
-\startfunctioncall
-<boolean>  = <Catalog>:isOK()
-<integer>  = <Catalog>:getNumPages()
-<Page>     = <Catalog>:getPage(<integer>)
-<Ref>      = <Catalog>:getPageRef(<integer>)
-<string>   = <Catalog>:getBaseURI()
-<string>   = <Catalog>:readMetadata()
-<Object>   = <Catalog>:getStructTreeRoot()
-<integer>  = <Catalog>:findPage(<integer> object number, <integer> object generation)
-<LinkDest> = <Catalog>:findDest(<string> name)
-<Object>   = <Catalog>:getDests()
-<integer>  = <Catalog>:numEmbeddedFiles()
-<EmbFile>  = <Catalog>:embeddedFile(<integer>)
-<integer>  = <Catalog>:numJS()
-<string>   = <Catalog>:getJS(<integer>)
-<Object>   = <Catalog>:getOutline()
-<Object>   = <Catalog>:getAcroForm()
-\stopfunctioncall
-
-\type {EmbFile} methods:
-
-\startfunctioncall
-<integer>  = <EmbFile>:size()
-<string>   = <EmbFile>:modDate()
-<string>   = <EmbFile>:createDate()
-<string>   = <EmbFile>:checksum()
-<string>   = <EmbFile>:mimeType()
-<boolean>  = <EmbFile>:isOk()
-<string>   = <EmbFile>:name()         -- not (yet) implemented
-<string>   = <EmbFile>:description()  -- not (yet) implemented
-<Object>   = <EmbFile>:streamObject() -- not (yet) implemented
-<boolean>  = <EmbFile>:save()         -- will go
-\stopfunctioncall
-
-\type {FileSpec} methods:
-
-\startfunctioncall
-<boolean> = <FileSpec>:isOk()
-<string>  = <FileSpec>:getFileName()
-<string>  = <FileSpec>:getFileNameForPlatform()
-<string>  = <FileSpec>:getDescription()
-<EmbFile> = <FileSpec>:getEmbeddedFile()
-\stopfunctioncall
-
-\type {Dict} methods:
-
-\startfunctioncall
-            <Dict>:incRef()                -- will go
-            <Dict>:decRef()                -- will go
-            <Dict>:add(<string>, <Object>) -- will go
-            <Dict>:set(<string>, <Object>) -- will go
-            <Dict>:remove(<string>)        -- will go
-<integer> = <Dict>:getLength()
-<boolean> = <Dict>:is(<string>)
-<Object>  = <Dict>:lookup(<string>)
-<Object>  = <Dict>:lookupNF(<string>)
-<integer> = <Dict>:lookupInt(<string>, <string>)
-<string>  = <Dict>:getKey(<integer>)
-<Object>  = <Dict>:getVal(<integer>)
-<Object>  = <Dict>:getValNF(<integer>)
-<boolean> = <Dict>:hasKey(<string>)
-\stopfunctioncall
-
-% \type {Link} methods:
-
-% \startfunctioncall
-% <boolean>  = <Link>:isOK()
-% <boolean>  = <Link>:inRect(<number>, <number>)
-% \stopfunctioncall
-
-\type {LinkDest} methods:
-
-\startfunctioncall
-<boolean>  = <LinkDest>:isOK()
-<integer>  = <LinkDest>:getKind()
-<string>   = <LinkDest>:getKindName()
-<boolean>  = <LinkDest>:isPageRef()
-<integer>  = <LinkDest>:getPageNum()
-<Ref>      = <LinkDest>:getPageRef()
-<number>   = <LinkDest>:getLeft()
-<number>   = <LinkDest>:getBottom()
-<number>   = <LinkDest>:getRight()
-<number>   = <LinkDest>:getTop()
-<number>   = <LinkDest>:getZoom()
-<boolean>  = <LinkDest>:getChangeLeft()
-<boolean>  = <LinkDest>:getChangeTop()
-<boolean>  = <LinkDest>:getChangeZoom()
-\stopfunctioncall
-
-\type {Links} methods:
-
-% <Link>     = <Links>:getLink(<integer>)
-
-\startfunctioncall
-<integer>  = <Links>:getNumLinks()
-\stopfunctioncall
-
-\type {Object} methods:
-
-\startfunctioncall
-            <Object>:initBool(<boolean>)              -- will go
-            <Object>:initInt(<integer>)               -- will go
-            <Object>:initReal(<number>)               -- will go
-            <Object>:initString(<string>)             -- will go
-            <Object>:initName(<string>)               -- will go
-            <Object>:initNull()                       -- will go
-            <Object>:initArray(<XRef>)                -- will go
-            <Object>:initDict(<XRef>)                 -- will go
-            <Object>:initStream(<Stream>)             -- will go
-            <Object>:initRef(<integer> object number,
-                <integer> object generation)          -- will go
-            <Object>:initCmd(<string>)                -- will go
-            <Object>:initError()                      -- will go
-            <Object>:initEOF()                        -- will go
-<Object>  = <Object>:fetch(<XRef>)
-<integer> = <Object>:getType()
-<string>  = <Object>:getTypeName()
-<boolean> = <Object>:isBool()
-<boolean> = <Object>:isInt()
-<boolean> = <Object>:isReal()
-<boolean> = <Object>:isNum()
-<boolean> = <Object>:isString()
-<boolean> = <Object>:isName()
-<boolean> = <Object>:isNull()
-<boolean> = <Object>:isArray()
-<boolean> = <Object>:isDict()
-<boolean> = <Object>:isStream()
-<boolean> = <Object>:isRef()
-<boolean> = <Object>:isCmd()
-<boolean> = <Object>:isError()
-<boolean> = <Object>:isEOF()
-<boolean> = <Object>:isNone()
-<boolean> = <Object>:getBool()
-<integer> = <Object>:getInt()
-<number>  = <Object>:getReal()
-<number>  = <Object>:getNum()
-<string>  = <Object>:getString()
-<string>  = <Object>:getName()
-<Array>   = <Object>:getArray()
-<Dict>    = <Object>:getDict()
-<Stream>  = <Object>:getStream()
-<Ref>     = <Object>:getRef()
-<integer> = <Object>:getRefNum()
-<integer> = <Object>:getRefGen()
-<string>  = <Object>:getCmd()
-<integer> = <Object>:arrayGetLength()
-          = <Object>:arrayAdd(<Object>)               -- will go
-<Object>  = <Object>:arrayGet(<integer>)
-<Object>  = <Object>:arrayGetNF(<integer>)
-<integer> = <Object>:dictGetLength(<integer>)
-          = <Object>:dictAdd(<string>, <Object>)      -- will go
-          = <Object>:dictSet(<string>, <Object>)      -- will go
-<Object>  = <Object>:dictLookup(<string>)
-<Object>  = <Object>:dictLookupNF(<string>)
-<string>  = <Object>:dictgetKey(<integer>)
-<Object>  = <Object>:dictgetVal(<integer>)
-<Object>  = <Object>:dictgetValNF(<integer>)
-<boolean> = <Object>:streamIs(<string>)
-          = <Object>:streamReset()                    -- will go
-<integer> = <Object>:streamGetChar()
-<integer> = <Object>:streamLookChar()
-<integer> = <Object>:streamGetPos()
-          = <Object>:streamSetPos(<integer>)
-<Dict>    = <Object>:streamGetDict()
-\stopfunctioncall
-
-\type {Page} methods:
-
-\startfunctioncall
-<boolean>      = <Page>:isOk()
-<integer>      = <Page>:getNum()
-<PDFRectangle> = <Page>:getMediaBox()
-<PDFRectangle> = <Page>:getCropBox()
-<boolean>      = <Page>:isCropped()
-<number>       = <Page>:getMediaWidth()
-<number>       = <Page>:getMediaHeight()
-<number>       = <Page>:getCropWidth()
-<number>       = <Page>:getCropHeight()
-<PDFRectangle> = <Page>:getBleedBox()
-<PDFRectangle> = <Page>:getTrimBox()
-<PDFRectangle> = <Page>:getArtBox()
-<integer>      = <Page>:getRotate()
-<string>       = <Page>:getLastModified()
-<Dict>         = <Page>:getBoxColorInfo()
-<Dict>         = <Page>:getGroup()
-<Stream>       = <Page>:getMetadata()
-<Dict>         = <Page>:getPieceInfo()
-<Dict>         = <Page>:getSeparationInfo()
-<Dict>         = <Page>:getResourceDict()
-<Object>       = <Page>:getAnnots()
-<Links>        = <Page>:getLinks(<Catalog>)
-<Object>       = <Page>:getContents()
-\stopfunctioncall
-
-\type {PDFDoc} methods:
-
-\startfunctioncall
-<boolean>  = <PDFDoc>:isOk()
-<integer>  = <PDFDoc>:getErrorCode()
-<string>   = <PDFDoc>:getErrorCodeName()
-<string>   = <PDFDoc>:getFileName()
-<XRef>     = <PDFDoc>:getXRef()
-<Catalog>  = <PDFDoc>:getCatalog()
-<number>   = <PDFDoc>:getPageMediaWidth()
-<number>   = <PDFDoc>:getPageMediaHeight()
-<number>   = <PDFDoc>:getPageCropWidth()
-<number>   = <PDFDoc>:getPageCropHeight()
-<integer>  = <PDFDoc>:getNumPages()
-<string>   = <PDFDoc>:readMetadata()
-<Object>   = <PDFDoc>:getStructTreeRoot()
-<integer>  = <PDFDoc>:findPage(<integer> object number,
-                <integer> object generation)
-<Links>    = <PDFDoc>:getLinks(<integer>)
-<LinkDest> = <PDFDoc>:findDest(<string>)
-<boolean>  = <PDFDoc>:isEncrypted()
-<boolean>  = <PDFDoc>:okToPrint()
-<boolean>  = <PDFDoc>:okToChange()
-<boolean>  = <PDFDoc>:okToCopy()
-<boolean>  = <PDFDoc>:okToAddNotes()
-<boolean>  = <PDFDoc>:isLinearized()
-<Object>   = <PDFDoc>:getDocInfo()
-<Object>   = <PDFDoc>:getDocInfoNF()
-<integer>  = <PDFDoc>:getPDFMajorVersion()
-<integer>  = <PDFDoc>:getPDFMinorVersion()
-\stopfunctioncall
-
-\type {PDFRectangle} methods:
-
-\startfunctioncall
-<boolean>  = <PDFRectangle>:isValid() -- setindex/newindex will go
-\stopfunctioncall
-
-%\type {Ref} methods:
-%
-%\startfunctioncall
-%\stopfunctioncall
-
-\type {Stream} methods:
-
-\startfunctioncall
-<integer>  = <Stream>:getKind()
-<string>   = <Stream>:getKindName()
-           = <Stream>:reset()
-           = <Stream>:close()
-<integer>  = <Stream>:getChar()
-<integer>  = <Stream>:lookChar()
-<integer>  = <Stream>:getRawChar()
-<integer>  = <Stream>:getUnfilteredChar()
-           = <Stream>:unfilteredReset()
-<integer>  = <Stream>:getPos()
-<boolean>  = <Stream>:isBinary()
-<Stream>   = <Stream>:getUndecodedStream()
-<Dict>     = <Stream>:getDict()
-\stopfunctioncall
-
-\type {StructElement} methods:
-
-\startfunctioncall
-<string>         = <StructElement>:getTypeName()
-<integer>        = <StructElement>:getType()
-<boolean>        = <StructElement>:isOk()
-<boolean>        = <StructElement>:isBlock()
-<boolean>        = <StructElement>:isInline()
-<boolean>        = <StructElement>:isGrouping()
-<boolean>        = <StructElement>:isContent()
-<boolean>        = <StructElement>:isObjectRef()
-<integer>        = <StructElement>:getMCID()
-<Ref>            = <StructElement>:getObjectRef()
-<Ref>            = <StructElement>:getParentRef()
-<boolean>        = <StructElement>:hasPageRef()
-<Ref>            = <StructElement>:getPageRef()
-<StructTreeRoot> = <StructElement>:getStructTreeRoot()
-<string>         = <StructElement>:getID()
-<string>         = <StructElement>:getLanguage()
-<integer>        = <StructElement>:getRevision()
-                   <StructElement>:setRevision(<unsigned integer>)   -- will go
-<string>         = <StructElement>:getTitle()
-<string>         = <StructElement>:getExpandedAbbr()
-<integer>        = <StructElement>:getNumChildren()
-<StructElement>  = <StructElement>:getChild()
-                 = <StructElement>:appendChild<StructElement>)       -- will go
-<integer>        = <StructElement>:getNumAttributes()
-<Attribute>      = <StructElement>:getAttribute(<integer>)
-<string>         = <StructElement>:appendAttribute(<Attribute>)      -- will go
-<Attribute>      = <StructElement>:findAttribute(<Attribute::Type>,
-                        boolean,Attribute::Owner)
-<string>         = <StructElement>:getAltText()
-<string>         = <StructElement>:getActualText()
-<string>         = <StructElement>:getText(<boolean>)
-<table>          = <StructElement>:getTextSpans()
-\stopfunctioncall
-
-\type {StructTreeRoot} methods:
-
-\startfunctioncall
-<StructElement> = <StructTreeRoot>:findParentElement
-<PDFDoc>        = <StructTreeRoot>:getDoc
-<Dict>          = <StructTreeRoot>:getRoleMap
-<Dict>          = <StructTreeRoot>:getClassMap
-<integer>       = <StructTreeRoot>:getNumChildren
-<StructElement> = <StructTreeRoot>:getChild
-                  <StructTreeRoot>:appendChild       -- will go
-<StructElement> = <StructTreeRoot>:findParentElement
-\stopfunctioncall
-
-\type {TextSpan} han only one method:
-
-\startfunctioncall
-<string> = <TestSpan>:getText()
-\stopfunctioncall
-
-\type {XRef} methods:
-
-\startfunctioncall
-<boolean>  = <XRef>:isOk()
-<integer>  = <XRef>:getErrorCode()
-<boolean>  = <XRef>:isEncrypted()
-<boolean>  = <XRef>:okToPrint()
-<boolean>  = <XRef>:okToPrintHighRes()
-<boolean>  = <XRef>:okToChange()
-<boolean>  = <XRef>:okToCopy()
-<boolean>  = <XRef>:okToAddNotes()
-<boolean>  = <XRef>:okToFillForm()
-<boolean>  = <XRef>:okToAccessibility()
-<boolean>  = <XRef>:okToAssemble()
-<Object>   = <XRef>:getCatalog()
-<Object>   = <XRef>:fetch(<integer> object number,
-                <integer> object generation)
-<Object>   = <XRef>:getDocInfo()
-<Object>   = <XRef>:getDocInfoNF()
-<integer>  = <XRef>:getNumObjects()
-<integer>  = <XRef>:getRootNum()
-<integer>  = <XRef>:getRootGen()
-<integer>  = <XRef>:getSize()
-<Object>   = <XRef>:getTrailerDict()
-\stopfunctioncall
-
-There is an experimental function \type {epdf.openMemStream} that takes three
-arguments:
-
-\starttabulate
-\NC \type {stream}  \NC this is a (in low level \LUA\ speak) light userdata
-                        object, i.e.\ a pointer to a sequence of bytes \NC \NR
-\NC \type {length}  \NC this is the length of the stream in bytes \NC \NR
-\NC \type {name}    \NC this is a unique identifier that is used for hashing the
-                        stream, so that multiple doesn't use more memory \NC \NR
-\stoptabulate
-
-Instead of a light userdata stream you can also pass a \LUA\ string, in which
-case the given length is (at most) the string length.
-
-The function returns a \type{epdf} object and a string. The string can be used in
-the \type {img} library instead of a filename. You need to prevent garbage
-collection of the object when you use it as image (for instance by storing it
-somewhere).
-
-Both the memory stream and it's use in the image library is experimental and can
-change. In case you wonder where this can be used: when you use the swiglib
-library for \type {graphicmagick}, it can return such a userdata object. This
-permits conversion in memory and passing the result directly to the backend. This
-might save some runtime in one|-|pass workflows. This feature is currently not
-meant for production and we might come up with a better implementation.
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-backend

+

+\startchapter[reference=backend,title={The backend libraries}]

+

+\startsection[title={The \type {pdf} library}][library=pdf]

+

+\topicindex{backend}

+\topicindex{\PDF}

+

+This library contains variables and functions that are related to the \PDF\

+backend. You can find more details about the expected values to setters in \in

+{section} [backendprimitives].

+

+\startsubsection[title={\type {mapfile}, \type {mapline}}]

+

+\topicindex{map files}

+\topicindex{files+map}

+

+\startfunctioncall

+pdf.mapfile(<string> map file)

+pdf.mapline(<string> map line)

+\stopfunctioncall

+

+These two functions can be used to replace primitives \orm {pdfmapfile} and

+\orm {pdfmapline} inherited from \PDFTEX. They expect a string as only

+parameter and have no return value. The first character in a map line can be

+\type {-}, \type {+} or \type {=} which means as much as remove, add or replace

+this line. They are not state setters but act immediately.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get][catalog|info|names|trailer]}}]

+

+\topicindex{\PDF+trailer}

+\topicindex{\PDF+catalog}

+\topicindex{\PDF+info}

+

+\libindex{setcatalog} \libindex{getcatalog}

+\libindex{setinfo}    \libindex{getinfo}

+\libindex{setnames}   \libindex{getnames}

+\libindex{settrailer} \libindex{gettrailer}

+

+These functions complement the corresponding \PDF\ backend token lists dealing

+with metadata. The value types are strings and they are written to the \PDF\

+file directly after the token registers set at the \TEX\ end are written.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get][pageattributes|pageresources|pagesattributes]}}]

+

+\libindex{setpageattributes}  \libindex{getpageattributes}

+\libindex{setpageresources}   \libindex{getpageresources}

+\libindex{setpagesattributes} \libindex{getpagesattributes}

+

+\topicindex{\PDF+page attributes}

+\topicindex{\PDF+page resources}

+

+These functions complement the corresponding \PDF\ backend token lists dealing

+with page resources. The variables have no interaction with the corresponding \PDF\

+backend token register. They are written to the \PDF\ file directly after the

+token registers set at the \TEX\ end are written.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get][xformattributes|xformresources]}}]

+

+\libindex{setxformattributes} \libindex{getxformattributes}

+\libindex{setxformresources}  \libindex{getxformresources}

+

+\topicindex{\PDF+xform attributes}

+\topicindex{\PDF+xform resources}

+

+These functions complement the corresponding \PDF\ backend token lists dealing

+with reuseable boxes and images. The variables have no interaction with the

+corresponding \PDF\ backend token register. They are written to the \PDF\

+file directly after the token registers set at the \TEX\ end are written.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get][major|minor]version}}]

+

+\topicindex{\PDF+version}

+

+\libindex{getmajorversion} \libindex{setmajorversion}

+\libindex{getminorversion} \libindex{setminorversion}

+

+You can set both the major and minor version of the output. The major version is

+normally~1 but when set to~2 some data will not be written to the file in order

+to comply with the standard. What minor version you set depends on what \PDF\

+features you use. This is out of control of \LUATEX.

+

+\stopsubsection

+

+\startsubsection[title={\type {getcreationdate}}]

+

+\topicindex{\PDF+date}

+

+\libindex{getcreationdate}

+

+This function returns a string with the date in the format that ends up in the

+\PDF\ file, in this case it's: {\tttf \cldcontext{pdf.getcreationdate()}}.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get]inclusionerrorlevel} and \type {[set|get]ignoreunknownimages}}]

+

+\topicindex{\PDF+options}

+

+\libindex{getinclusionerrorlevel} \libindex{setinclusionerrorlevel}

+\libindex{getignoreunknownimages} \libindex{setignoreunknownimages}

+

+These variable control how error in included image are treated. They are modeled

+after the \PDFTEX\ equivalents.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get]suppressoptionalinfo}, \type {[set|get]trailerid}

+and \type {[set|get]omitcidset}}]

+

+\topicindex{\PDF+options}

+\topicindex{\PDF+trailer}

+

+\libindex{getsuppressoptionalinfo} \libindex{setsuppressoptionalinfo}

+\libindex{gettrailerid}            \libindex{settrailerid}

+\libindex{getomitcidset}           \libindex{setomitcidset}

+\libindex{getomitcharset}          \libindex{setomitcharset}

+

+The optional info bitset (a number) determines what kind of info gets flushed.

+By default we flush all. See \in {section} [sec:pdfextensions] for more details.

+

+You can set your own trailer id. This has to be string containing valid \PDF\

+array content with checksums.

+

+The cidset and charset flags (numbers) disables inclusion of a so called \type

+{CIDSet} and \type {CharSet} entries, which can be handy when aiming at some of

+the many \PDF\ substandards.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get][obj|]compresslevel} and \type {[set|get]recompress}}]

+

+\topicindex{\PDF+compression}

+

+\libindex{getcompresslevel}    \libindex{setcompresslevel}

+\libindex{getobjcompresslevel} \libindex{setobjcompresslevel}

+\libindex{getrecompress}       \libindex{setrecompress}

+

+These functions set the level stream compression. When object compression is

+enabled multiple objects will be packed in a compressed stream which saves space.

+The minimum values are~0, the maxima are~9.

+

+When recompression is to~1 compressed objects will be decompressed and when

+compresslevel is larger than zero they will then be recompressed. This is mostly

+a debugging feature and should not be relied upon.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get]gentounicode}}]

+

+\topicindex{\PDF+unicode}

+

+\libindex{getgentounicode} \libindex{setgentounicode}

+

+This flag enables tounicode generation (like in \PDFTEX). Normally the values are

+provided by the font loader.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get]decimaldigits}}]

+

+\topicindex{\PDF+precision}

+

+\libindex{getdecimaldigits} \libindex{setdecimaldigits}

+

+These two functions set the accuracy of floats written to the \PDF file. You can

+set any value but the backend will not go below~3 and above~6.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get]pkresolution}}]

+

+\topicindex{\PDF+resolution}

+

+\libindex{getpkresolution} \libindex{setpkresolution}

+

+These setter takes two arguments: the resolution and an optional zero or one that

+indicates if this is a fixed one. The getter returns these two values.

+

+\stopsubsection

+

+\startsubsection[title={\type {getlast[obj|link|annot]} and \type {getretval}}]

+

+\topicindex{\PDF+objects}

+\topicindex{\PDF+annotations}

+

+\libindex{getlastobj}   \libindex{setlastobj}

+\libindex{getlastlink}  \libindex{setlastlink}

+\libindex{getlastannot} \libindex{setlastannot}

+\libindex{getretval}

+

+These status variables are similar to the ones traditionally used in the backend

+interface at the \TEX\ end.

+

+\stopsubsection

+

+\startsubsection[title={\type {getmaxobjnum} and \type {getobjtype}, \type {getfontname},

+\type {getfontobjnum}, \type {getfontsize}, \type {getxformname}}]

+

+\libindex{getmaxobjnum}

+\libindex{getobjtype}

+\libindex{getfontname}

+\libindex{getfontobjnum}

+\libindex{getfontsize}

+\libindex{getxformname}

+

+These introspective helpers are mostly used when you construct \PDF\ objects

+yourself and need for instance information about a (to be) embedded font.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get]origin}}]

+

+\topicindex{\PDF+positioning}

+

+\libindex{setorigin} \libindex{getorigin}

+

+This one is used to set the horizonal and|/|or vertical offset, a traditional

+backend property.

+

+\starttyping

+pdf.setorigin() -- sets both to 0pt

+pdf.setorigin(tex.sp("1in")) -- sets both to 1in

+pdf.setorigin(tex.sp("1in"),tex.sp("1in"))

+\stoptyping

+

+The counterpart of this function returns two values.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get]imageresolution}}]

+

+\topicindex{\PDF+resolution}

+

+\libindex{setimageresolution} \libindex{getimageresolution}

+

+These two functions relate to the imageresolution that is used when the image

+itself doesn't provide a non|-|zero x or y resolution.

+

+\stopsubsection

+

+\startsubsection[title={\type {[set|get][link|dest|thread|xform]margin}}]

+

+\topicindex{\PDF+margins}

+

+\libindex{getlinkmargin}   \libindex{setlinkmargin}

+\libindex{getdestmargin}   \libindex{setdestmargin}

+\libindex{getthreadmargin} \libindex{setthreadmargin}

+\libindex{getxformmargin}  \libindex{setxformmargin}

+\libindex{getmarginmargin} \libindex{setmarginmargin}

+

+These functions can be used to set and retrieve the margins that are added to the

+natural bounding boxes of the respective objects.

+

+\stopsubsection

+

+\startsubsection[title={\type {get[pos|hpos|vpos]}}]

+

+\topicindex{\PDF+positions}

+

+\libindex{getpos}

+\libindex{gethpos}

+\libindex{getvpos}

+

+These functions get current location on the output page, measured from its lower

+left corner. The values return scaled points as units.

+

+\starttyping

+local h, v = pdf.getpos()

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {[has|get]matrix}}]

+

+\topicindex{\PDF+matrix}

+

+\libindex{getmatrix} \libindex{hasmatrix}

+

+The current matrix transformation is available via the \type {getmatrix} command,

+which returns 6 values: \type {sx}, \type {rx}, \type {ry}, \type {sy}, \type

+{tx}, and \type {ty}. The \type {hasmatrix} function returns \type {true} when a

+matrix is applied.

+

+\starttyping

+if pdf.hasmatrix() then

+    local sx, rx, ry, sy, tx, ty = pdf.getmatrix()

+    -- do something useful or not

+end

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {print}}]

+

+\topicindex{\PDF+print to}

+

+\libindex{print}

+

+You can print a string to the \PDF\ document from within a \lpr {latelua} call.

+This function is not to be used inside \prm {directlua} unless you know {\it

+exactly} what you are doing.

+

+\startfunctioncall

+pdf.print(<string> s)

+pdf.print(<string> type, <string> s)

+\stopfunctioncall

+

+The optional parameter can be used to mimic the behavior of \PDF\ literals: the

+\type {type} is \type {direct} or \type {page}.

+

+\stopsubsection

+

+\startsubsection[title={\type {immediateobj}}]

+

+\topicindex{\PDF+objects}

+

+\libindex{immediateobj}

+

+This function creates a \PDF\ object and immediately writes it to the \PDF\ file.

+It is modelled after \PDFTEX's \prm {immediate} \orm {pdfobj} primitives. All

+function variants return the object number of the newly generated object.

+

+\startfunctioncall

+<number> n =

+    pdf.immediateobj(<string> objtext)

+<number> n =

+    pdf.immediateobj("file", <string> filename)

+<number> n =

+    pdf.immediateobj("stream", <string> streamtext, <string> attrtext)

+<number> n =

+    pdf.immediateobj("streamfile", <string> filename, <string> attrtext)

+\stopfunctioncall

+

+The first version puts the \type {objtext} raw into an object. Only the object

+wrapper is automatically generated, but any internal structure (like \type {<<

+>>} dictionary markers) needs to be provided by the user. The second version with

+keyword \type {file} as first argument puts the contents of the file with name

+\type {filename} raw into the object. The third version with keyword \type

+{stream} creates a stream object and puts the \type {streamtext} raw into the

+stream. The stream length is automatically calculated. The optional \type

+{attrtext} goes into the dictionary of that object. The fourth version with

+keyword \type {streamfile} does the same as the third one, it just reads the

+stream data raw from a file.

+

+An optional first argument can be given to make the function use a previously

+reserved \PDF\ object.

+

+\startfunctioncall

+<number> n =

+    pdf.immediateobj(<integer> n, <string> objtext)

+<number> n =

+    pdf.immediateobj(<integer> n, "file", <string> filename)

+<number> n =

+    pdf.immediateobj(<integer> n, "stream", <string> streamtext, <string> attrtext)

+<number> n =

+    pdf.immediateobj(<integer> n, "streamfile", <string> filename, <string> attrtext)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type{obj}}]

+

+\topicindex{\PDF+objects}

+

+\libindex{obj}

+

+This function creates a \PDF\ object, which is written to the \PDF\ file only

+when referenced, e.g., by \type {refobj()}.

+

+All function variants return the object number of the newly generated object, and

+there are two separate calling modes. The first mode is modelled after \PDFTEX's

+\orm {pdfobj} primitive.

+

+\startfunctioncall

+<number> n =

+    pdf.obj(<string> objtext)

+<number> n =

+    pdf.obj("file", <string> filename)

+<number> n =

+    pdf.obj("stream", <string> streamtext, <string> attrtext)

+<number> n =

+    pdf.obj("streamfile", <string> filename, <string> attrtext)

+\stopfunctioncall

+

+An optional first argument can be given to make the function use a previously

+reserved \PDF\ object.

+

+\startfunctioncall

+<number> n =

+    pdf.obj(<integer> n, <string> objtext)

+<number> n =

+    pdf.obj(<integer> n, "file", <string> filename)

+<number> n =

+    pdf.obj(<integer> n, "stream", <string> streamtext, <string> attrtext)

+<number> n =

+    pdf.obj(<integer> n, "streamfile", <string> filename, <string> attrtext)

+\stopfunctioncall

+

+The second mode accepts a single argument table with key--value pairs.

+

+\startfunctioncall

+<number> n = pdf.obj {

+    type           = <string>,

+    immediate      = <boolean>,

+    objnum         = <number>,

+    attr           = <string>,

+    compresslevel  = <number>,

+    objcompression = <boolean>,

+    file           = <string>,

+    string         = <string>,

+    nolength       = <boolean>,

+}

+\stopfunctioncall

+

+The \type {type} field can have the values \type {raw} and \type {stream}, this

+field is required, the others are optional (within constraints). When \type

+{nolength} is set, there will be no \type {/Length} entry added to the

+dictionary.

+

+Note: this mode makes \type{obj} look more flexible than it actually is: the

+constraints from the separate parameter version still apply, so for example you

+can't have both \type {string} and \type {file} at the same time.

+

+\stopsubsection

+

+\startsubsection[title={\type {refobj}}]

+

+\topicindex{\PDF+objects}

+

+\libindex{refobj}

+

+This function, the \LUA\ version of the \orm {pdfrefobj} primitive, references an

+object by its object number, so that the object will be written to the \PDF\ file.

+

+\startfunctioncall

+pdf.refobj(<integer> n)

+\stopfunctioncall

+

+This function works in both the \prm {directlua} and \lpr {latelua} environment.

+Inside \prm {directlua} a new whatsit node \quote {pdf_refobj} is created, which

+will be marked for flushing during page output and the object is then written

+directly after the page, when also the resources objects are written to the \PDF\

+file. Inside \lpr {latelua} the object will be marked for flushing.

+

+This function has no return values.

+

+\stopsubsection

+

+\startsubsection[title={\type {reserveobj}}]

+

+\topicindex{\PDF+objects}

+

+\libindex{reserveobj}

+

+This function creates an empty \PDF\ object and returns its number.

+

+\startfunctioncall

+<number> n = pdf.reserveobj()

+<number> n = pdf.reserveobj("annot")

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {getpageref}}]

+

+\topicindex{\PDF+pages}

+

+\libindex{getpageref}

+

+The object number of a page can be fetched with this function. This can be a

+forward reference so when you ask for a future page, you do get a number back.

+

+\startfunctioncall

+<number> n = pdf.getpageref(123)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {registerannot}}]

+

+\topicindex{\PDF+annotations}

+

+\libindex{registerannot}

+

+This function adds an object number to the \type {/Annots} array for the current

+page without doing anything else. This function can only be used from within

+\lpr {latelua}.

+

+\startfunctioncall

+pdf.registerannot (<number> objnum)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {newcolorstack}}]

+

+\topicindex{\PDF+color stack}

+

+\libindex{newcolorstack}

+

+This function allocates a new color stack and returns it's id. The arguments

+are the same as for the similar backend extension primitive.

+

+\startfunctioncall

+pdf.newcolorstack("0 g","page",true) -- page|direct|origin

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {setfontattributes}}]

+

+\topicindex{\PDF+fonts}

+

+\libindex{setfontattributes}

+

+This function will force some additional code into the font resource. It can for

+instance be used to add a custom \type {ToUnicode} vector to a bitmap file.

+

+\startfunctioncall

+pdf.setfontattributes(<number> font id, <string> pdf code)

+\stopfunctioncall

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={The \type {pdfe} library}][library=pdfe]

+

+\startsubsection[title={Introduction}]

+

+\topicindex{\PDF+objects}

+

+\topicindex{\PDF+analyze}

+\topicindex{\PDF+\type{pdfe}}

+

+The \type {pdfe} library replaces the \type {epdf} library and provides an

+interface to \PDF\ files. It uses the same code as is used for \PDF\ image

+inclusion. The \type {pplib} library by Paweł Jackowski replaces the \type

+{poppler} (derived from \type {xpdf}) library.

+

+A \PDF\ file is basically a tree of objects and one descends into the tree via

+dictionaries (key/value) and arrays (index/value). There are a few topmost

+dictionaries that start at root that are accessed more directly.

+

+Although everything in \PDF\ is basically an object we only wrap a few in so

+called userdata \LUA\ objects.

+

+\starttabulate

+\BC \PDF          \BC \LUA \NC \NR

+\NC null          \NC nil \NC \NR

+\NC boolean       \NC boolean \NC \NR

+\NC integer       \NC integer \NC \NR

+\NC float         \NC number \NC \NR

+\NC name          \NC string \NC \NR

+\NC string        \NC string \NC \NR

+\NC array         \NC array userdatum \NC \NR

+\NC dictionary    \NC dictionary userdatum \NC \NR

+\NC stream        \NC stream userdatum (with related dictionary) \NC \NR

+\NC reference     \NC reference userdatum \NC \NR

+\stoptabulate

+

+The regular getters return these \LUA\ data types but one can also get more

+detailed information.

+

+\stopsubsection

+

+\startsubsection[title={\type {open}, \type {new}, \type {status}, \type {close}, \type {unencrypt}}]

+

+\libindex {open}

+\libindex {new}

+\libindex {new}

+\libindex {status}

+\libindex {close}

+\libindex {unencrypt}

+

+A document is loaded from a file or string

+

+\starttyping

+<pdfe document> = pdfe.open(filename)

+<pdfe document> = pdfe.new(somestring,somelength)

+\stoptyping

+

+Such a document is closed with:

+

+\starttyping

+pdfe.close(<pdfe document>)

+\stoptyping

+

+You can check if a document opened well by:

+

+\starttyping

+pdfe.status(<pdfe document>)

+\stoptyping

+

+The returned codes are:

+

+\starttabulate[|c|l|]

+\DB value       \BC explanation \NC \NR

+\TB

+\NC \type {-2}  \NC the document failed to open \NC \NR

+\NC \type {-1}  \NC the document is (still) protected \NC \NR

+\NC \type {0}   \NC the document is not encrypted \NC \NR

+\NC \type {2}   \NC the document has been unencrypted \NC \NR

+\LL

+\stoptabulate

+

+An encrypted document can be unencrypted by the next command where instead of

+either password you can give \type {nil}:

+

+\starttyping

+pdfe.unencrypt(<pdfe document>,userpassword,ownerpassword)

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {size}, \type {version}, \type {getnofobjects}, \type {getnofpages}}]

+

+\libindex {size}

+\libindex {version}

+\libindex {getnofobjects}

+\libindex {getnofpages}

+

+A successfully opened document can provide some information:

+

+\starttyping

+bytes = size(<pdfe document>)

+major, minor = version(<pdfe document>)

+n = getnofobjects(<pdfe document>)

+n = getnofpages(<pdfe document>)

+bytes, waste = getnofpages(<pdfe document>)

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {get[catalog|trailer|info]}}]

+

+\libindex {getcatalog}

+\libindex {gettrailer}

+\libindex {getinfo}

+

+For accessing the document structure you start with the so called catalog, a

+dictionary:

+

+\starttyping

+<pdfe dictionary> = pdfe.getcatalog(<pdfe document>)

+\stoptyping

+

+The other two root dictionaries are accessed with:

+

+\starttyping

+<pdfe dictionary> = pdfe.gettrailer(<pdfe document>)

+<pdfe dictionary> = pdfe.getinfo(<pdfe document>)

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {getpage}, \type {getbox}}]

+

+\libindex {getpage}

+\libindex {getbox}

+

+A specific page can conveniently be reached with the next command, which

+returns a dictionary. The first argument is to be a page dictionary.

+

+\starttyping

+<pdfe dictionary> = pdfe.getpage(<pdfe dictionary>,pagenumber)

+\stoptyping

+

+Another convenience command gives you the (bounding) box of a (normally page)

+which can be inheritted from the document itself. An example of a valid box name

+is \type {MediaBox}.

+

+\starttyping

+pages = pdfe.getbox(<pdfe document>,boxname)

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {get[string|integer|number|boolean|name]}}]

+

+\libindex {getstring}

+\libindex {getinteger}

+\libindex {getnumber}

+\libindex {getboolean}

+\libindex {getname}

+

+Common values in dictionaries and arrays are strings, integers, floats, booleans

+and names (which are also strings) and these are also normal \LUA\ objects:

+

+\starttyping

+s = getstring (<pdfe array|dictionary>,index|key)

+i = getinteger(<pdfe array|dictionary>,index|key)

+n = getnumber (<pdfe array|dictionary>,index|key)

+b = getboolean(<pdfe array|dictionary>,index|key)

+n = getname   (<pdfe array|dictionary>,index|key)

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {get[from][dictionary|array|stream]}}]

+

+\libindex {getdictionary} \libindex {getfromdictionary}

+\libindex {getarray}      \libindex {getfromarray}

+\libindex {getstream}     \libindex {getfromstream}

+

+Normally you will use an index in an array and key in a dictionary but dictionaries

+also accept an index. The size of an array or dictionary is available with the

+usual \type {#} operator.

+

+\starttyping

+<pdfe dictionary>   = getdictionary(<pdfe array|dictionary>,index|key)

+<pdfe array>        = getarray     (<pdfe array|dictionary>,index|key)

+<pdfe stream>,

+<pdfe dictionary>   = getstream    (<pdfe array|dictionary>,index|key)

+\stoptyping

+

+These commands return dictionaries, arrays and streams, which are dictionaries

+with a blob of data attached.

+

+Before we come to an alternative access mode, we mention that the objects provide

+access in a different way too, for instance this is valid:

+

+\starttyping

+print(pdfe.open("foo.pdf").Catalog.Type)

+\stoptyping

+

+At the topmost level there are \type {Catalog}, \type {Info}, \type {Trailer}

+and \type {Pages}, so this is also okay:

+

+\starttyping

+print(pdfe.open("foo.pdf").Pages[1])

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {[open|close|readfrom][whole|]stream}}]

+

+\libindex {openstream}

+\libindex {closestream}

+\libindex {readfromstream}

+\libindex {readfromwholestream}

+

+Streams are sort of special. When your index or key hits a stream you get back a

+stream object and dictionary object. The dictionary you can access in the usual

+way and for the stream there are the following methods:

+

+\starttyping

+okay   = openstream(<pdfe stream>,[decode])

+         closestream(<pdfe stream>)

+str, n = readfromstream(<pdfe stream>)

+str, n = readwholestream(<pdfe stream>,[decode])

+\stoptyping

+

+You either read in chunks, or you ask for the whole. When reading in chunks, you

+need to open and close the stream yourself. The \type {n} value indicates the

+length read. The \type {decode} parameter controls if the stream data gets

+uncompressed.

+

+As with dictionaries, you can access fields in a stream dictionary in the usual

+\LUA\ way too. You get the content when you \quote {call} the stream. You can

+pass a boolean that indicates if the stream has to be decompressed.

+

+% pdfe.objectcodes      = objectcodes

+% pdfe.stringcodes      = stringcodes

+% pdfe.encryptioncodes  = encryptioncodes

+

+\stopsubsection

+

+\startsubsection[title={\type {getfrom[dictionary|array]}}]

+

+\libindex {getfromdictionary}

+\libindex {getfromarray}

+

+In addition to the interface described before, there is also a bit lower level

+interface available.

+

+\starttyping

+key, type, value, detail = getfromdictionary(<pdfe dictionary>,index)

+type, value, detail = getfromarray(<pdfe array>,index)

+\stoptyping

+

+\starttabulate[|c|l|l|l|]

+\DB type       \BC meaning    \BC value            \BC detail \NC \NR

+\NC \type {0}  \NC none       \NC nil              \NC \NC \NR

+\NC \type {1}  \NC null       \NC nil              \NC \NC \NR

+\NC \type {2}  \NC boolean    \NC boolean          \NC \NC \NR

+\NC \type {3}  \NC boolean    \NC integer          \NC \NC \NR

+\NC \type {4}  \NC number     \NC float            \NC \NC \NR

+\NC \type {5}  \NC name       \NC string           \NC \NC \NR

+\NC \type {6}  \NC string     \NC string           \NC hex \NC \NR

+\NC \type {7}  \NC array      \NC arrayobject      \NC size \NC \NR

+\NC \type {8}  \NC dictionary \NC dictionaryobject \NC size \NC \NR

+\NC \type {9}  \NC stream     \NC streamobject     \NC dictionary size \NC \NR

+\NC \type {10} \NC reference  \NC integer          \NC \NC \NR

+\LL

+\stoptabulate

+

+A \type {hex} string is (in the \PDF\ file) surrounded by \type {<>} while plain

+strings are bounded by \type {<>}.

+

+\stopsubsection

+

+\startsubsection[title={\type {[dictionary|array]totable}}]

+

+\libindex {dictionarytotable}

+\libindex {arraytotable}

+

+All entries in a dictionary or table can be fetched with the following commands

+where the return values are a hashed or indexed table.

+

+\starttyping

+hash = dictionarytotable(<pdfe dictionary>)

+list = arraytotable(<pdfe array>)

+\stoptyping

+

+You can get a list of pages with:

+

+\starttyping

+{ { <pdfe dictionary>, size, objnum }, ... } = pagestotable(<pdfe document>)

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\type {getfromreference}}]

+

+\libindex {getfromreference}

+

+Because you can have unresolved references, a reference object can be resolved

+with:

+

+\starttyping

+<pdfe dictionary|array|stream> = getfromreference(<pdfe reference>)

+\stoptyping

+

+So, you get back a new \type {pdfe} userdata object that you can query.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Memory streams}][library=pdfe]

+

+\topicindex{\PDF+memory streams}

+

+\libindex {new}

+

+The \type {pdfe.new} that takes three arguments:

+

+\starttabulate

+\DB value           \BC explanation      \NC \NR

+\TB

+\NC \type {stream}  \NC this is a (in low level \LUA\ speak) light userdata

+                        object, i.e.\ a pointer to a sequence of bytes \NC \NR

+\NC \type {length}  \NC this is the length of the stream in bytes (the stream can

+                        have embedded zeros) \NC \NR

+\NC \type {name}    \NC optional, this is a unique identifier that is used for

+                        hashing the stream, so that multiple doesn't use more

+                        memory \NC \NR

+\LL

+\stoptabulate

+

+The third argument is optional. When it is not given the function will return an

+\type {pdfe} document object as with a regular file, otherwise it will return a

+filename that can be used elsewhere (e.g.\ in the image library) to reference the

+stream as pseudo file.

+

+Instead of a light userdata stream (which is actually fragile but handy when you

+come from a library) you can also pass a \LUA\ string, in which case the given

+length is (at most) the string length.

+

+The function returns an \type {pdfe} object and a string. The string can be used in

+the \type {img} library instead of a filename. You need to prevent garbage

+collection of the object when you use it as image (for instance by storing it

+somewhere).

+

+Both the memory stream and it's use in the image library is experimental and can

+change. In case you wonder where this can be used: when you use the swiglib

+library for \type {graphicmagick}, it can return such a userdata object. This

+permits conversion in memory and passing the result directly to the backend. This

+might save some runtime in one|-|pass workflows. This feature is currently not

+meant for production and we might come up with a better implementation.

+

+\stopsection

+

+\startsection[title={The \type {pdfscanner} library}][library=pdfscanner]

+

+\topicindex{\PDF+scanner}

+

+\libindex {scan}

+

+The \type {pdfscanner} library allows interpretation of \PDF\ content streams and

+\type {/ToUnicode} (cmap) streams. You can get those streams from the \type

+{pdfe} library, as explained in an earlier section. There is only a single

+top|-|level function in this library:

+

+\startfunctioncall

+pdfscanner.scan (<pdfe stream>, <table> operatortable, <table> info)

+pdfscanner.scan (<pdfe array>, <table> operatortable, <table> info)

+pdfscanner.scan (<string>, <table> operatortable, <table> info)

+\stopfunctioncall

+

+The first argument should be a \LUA\ string or a stream or array onject coming

+from the \type {pdfe} library. The second argument, \type {operatortable}, should

+be a \LUA\ table where the keys are \PDF\ operator name strings and the values

+are \LUA\ functions (defined by you) that are used to process those operators.

+The functions are called whenever the scanner finds one of these \PDF\ operators

+in the content stream(s). The functions are called with two arguments: the \type

+{scanner} object itself, and the \type {info} table that was passed are the third

+argument to \type {pdfscanner.scan}.

+

+Internally, \type {pdfscanner.scan} loops over the \PDF\ operators in the

+stream(s), collecting operands on an internal stack until it finds a \PDF\

+operator. If that \PDF\ operator's name exists in \type {operatortable}, then the

+associated function is executed. After the function has run (or when there is no

+function to execute) the internal operand stack is cleared in preparation for the

+next operator, and processing continues.

+

+The \type {scanner} argument to the processing functions is needed because it

+offers various methods to get the actual operands from the internal operand

+stack.

+

+A simple example of processing a \PDF's document stream could look like this:

+

+\starttyping

+local operatortable = { }

+

+operatortable.Do = function(scanner,info)

+    local resources = info.resources

+    if resources then

+        local val     = scanner:pop()

+        local name    = val[2]

+        local xobject = resources.XObject

+        print(info.space .. "Uses XObject " .. name)

+        local resources = xobject.Resources

+        if resources then

+            local newinfo =  {

+                space     = info.space .. "  ",

+                resources = resources,

+            }

+            pdfscanner.scan(entry, operatortable, newinfo)

+        end

+    end

+end

+

+local function Analyze(filename)

+    local doc = pdfe.open(filename)

+    if doc then

+        local pages = doc.Pages

+        for i=1,#pages do

+            local page = pages[i]

+            local info = {

+              space     = "  " ,

+              resources = page.Resources,

+            }

+            print("Page " .. i)

+         -- pdfscanner.scan(page.Contents,operatortable,info)

+            pdfscanner.scan(page.Contents(),operatortable,info)

+        end

+    end

+end

+

+Analyze("foo.pdf")

+\stoptyping

+

+This example iterates over all the actual content in the \PDF, and prints out the

+found \type {XObject} names. While the code demonstrates quite some of the \type

+{pdfe} functions, let's focus on the type \type {pdfscanner} specific code

+instead.

+

+From the bottom up, the following line runs the scanner with the \PDF\ page's

+top|-|level content given in the first argument.

+

+The third argument, \type {info}, contains two entries: \type {space} is used to

+indent the printed output, and \type {resources} is needed so that embedded \type

+{XForms} can find their own content.

+

+The second argument, \type {operatortable} defines a processing function for a

+single \PDF\ operator, \type {Do}.

+

+The function \type {Do} prints the name of the current \type {XObject}, and then

+starts a new scanner for that object's content stream, under the condition that

+the \type {XObject} is in fact a \type {/Form}. That nested scanner is called

+with new \type {info} argument with an updated \type {space} value so that the

+indentation of the output nicely nests, and with a new \type {resources} field

+to help the next iteration down to properly process any other, embedded \type

+{XObject}s.

+

+Of course, this is not a very useful example in practice, but for the purpose of

+demonstrating \type {pdfscanner}, it is just long enough. It makes use of only

+one \type {scanner} method: \type {scanner:pop()}. That function pops the top

+operand of the internal stack, and returns a \LUA\ table where the object at index

+one is a string representing the type of the operand, and object two is its

+value.

+

+The list of possible operand types and associated \LUA\ value types is:

+

+\starttabulate[|l|l|]

+\DB types           \BC type      \NC \NR

+\TB

+\NC \type{integer}  \NC <number>  \NC \NR

+\NC \type{real}     \NC <number>  \NC \NR

+\NC \type{boolean}  \NC <boolean> \NC \NR

+\NC \type{name}     \NC <string>  \NC \NR

+\NC \type{operator} \NC <string>  \NC \NR

+\NC \type{string}   \NC <string>  \NC \NR

+\NC \type{array}    \NC <table>   \NC \NR

+\NC \type{dict}     \NC <table>   \NC \NR

+\LL

+\stoptabulate

+

+In case of \type {integer} or \type {real}, the value is always a \LUA\ (floating

+point) number. In case of \type {name}, the leading slash is always stripped.

+

+In case of \type {string}, please bear in mind that \PDF\ actually supports

+different types of strings (with different encodings) in different parts of the

+\PDF\ document, so you may need to reencode some of the results; \type {pdfscanner}

+always outputs the byte stream without reencoding anything. \type {pdfscanner}

+does not differentiate between literal strings and hexadecimal strings (the

+hexadecimal values are decoded), and it treats the stream data for inline images

+as a string that is the single operand for \type {EI}.

+

+In case of \type {array}, the table content is a list of \type {pop} return

+values and in case of \type {dict}, the table keys are \PDF\ name strings and the

+values are \type {pop} return values.

+

+\libindex{pop}

+\libindex{popnumber}

+\libindex{popname}

+\libindex{popstring}

+\libindex{poparray}

+\libindex{popdictionary}

+\libindex{popboolean}

+\libindex{done}

+

+There are a few more methods defined that you can ask \type {scanner}:

+

+\starttabulate[|l|p|]

+\DB method               \BC explanation \NC \NR

+\TB

+\NC \type{pop}           \NC see above \NC \NR

+\NC \type{popnumber}     \NC return only the value of a \type {real} or \type {integer} \NC \NR

+\NC \type{popname}       \NC return only the value of a \type {name} \NC \NR

+\NC \type{popstring}     \NC return only the value of a \type {string} \NC \NR

+\NC \type{poparray}      \NC return only the value of a \type {array} \NC \NR

+\NC \type{popdictionary} \NC return only the value of a \type {dict} \NC \NR

+\NC \type{popboolean}    \NC return only the value of a \type {boolean} \NC \NR

+\NC \type{done}          \NC abort further processing of this \type {scan()} call \NC \NR

+\LL

+\stoptabulate

+

+The \type {pop*} are convenience functions, and come in handy when you know the

+type of the operands beforehand (which you usually do, in \PDF). For example, the

+\type {Do} function could have used \type {local name = scanner:popname()}

+instead, because the single operand to the \type {Do} operator is always a \PDF\

+name object.

+

+The \type {done} function allows you to abort processing of a stream once you

+have learned everything you want to learn. This comes in handy while parsing

+\type {/ToUnicode}, because there usually is trailing garbage that you are not

+interested in. Without \type {done}, processing only ends at the end of the

+stream, possibly wasting \CPU\ cycles.

+

+{\em We keep the older names \type {popNumber}, \type {popName}, \type

+{popString}, \type {popArray}, \type {popDict} and \type {popBool} around.}

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-callbacks.tex b/Master/texmf-dist/doc/luatex/base/luatex-callbacks.tex
index cad7ad7c6f8..b2b706ea913 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-callbacks.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-callbacks.tex
@@ -1,958 +1,1149 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-callbacks
-
-\startchapter[reference=callbacks,title={\LUATEX\ \LUA\ callbacks}]
-
-\section {Registering callbacks}
-
-This library has functions that register, find and list callbacks. Callbacks are
-\LUA\ functions that are called in well defined places. There are two kind of
-callbacks: those that mix with existing functionality, and those that (when
-enabled) replace functionality. In mosty cases the second category is expected to
-behave similar to the built in functionality because in a next step specific
-data is expected. For instance, you can replace the hyphenation routine. The
-function gets a list that can be hyphenated (or not). The final list should be
-valid and is (normally) used for constructing a paragraph. Another function can
-replace the ligature builder and|/|or kerner. Doing something else is possible
-but in the end might not give the user the expected outcome.
-
-The first thing you need to do is registering a callback:
-
-\startfunctioncall
-id, error =
-    callback.register (<string> callback_name, <function> func)
-id, error =
-    callback.register (<string> callback_name, nil)
-id, error =
-    callback.register (<string> callback_name, false)
-\stopfunctioncall
-
-Here the \syntax {callback_name} is a predefined callback name, see below. The
-function returns the internal \type {id} of the callback or \type {nil}, if the
-callback could not be registered. In the latter case, \type {error} contains an
-error message, otherwise it is \type {nil}.
-
-\LUATEX\ internalizes the callback function in such a way that it does not matter
-if you redefine a function accidentally.
-
-Callback assignments are always global. You can use the special value \type {nil}
-instead of a function for clearing the callback.
-
-For some minor speed gain, you can assign the boolean \type {false} to the
-non|-|file related callbacks, doing so will prevent \LUATEX\ from executing
-whatever it would execute by default (when no callback function is registered at
-all). Be warned: this may cause all sorts of grief unless you know \notabene
-{exactly} what you are doing!
-
-\startfunctioncall
-<table> info =
-    callback.list()
-\stopfunctioncall
-
-The keys in the table are the known callback names, the value is a boolean where
-\type {true} means that the callback is currently set (active).
-
-\startfunctioncall
-<function> f = callback.find (callback_name)
-\stopfunctioncall
-
-If the callback is not set, \type {callback.find} returns \type {nil}.
-
-\section{File discovery callbacks}
-
-The behaviour documented in this subsection is considered stable in the sense that
-there will not be backward|-|incompatible changes any more.
-
-\subsection{\type {find_read_file} and \type {find_write_file}}
-
-Your callback function should have the following conventions:
-
-\startfunctioncall
-<string> actual_name =
-    function (<number> id_number, <string> asked_name)
-\stopfunctioncall
-
-Arguments:
-
-\startitemize
-
-\sym{id_number}
-
-This number is zero for the log or \type {\input} files. For \TEX's \type {\read}
-or \type {\write} the number is incremented by one, so \type {\read0} becomes~1.
-
-\sym{asked_name}
-
-This is the user|-|supplied filename, as found by \type {\input}, \type {\openin}
-or \type {\openout}.
-
-\stopitemize
-
-Return value:
-
-\startitemize
-
-\sym{actual_name}
-
-This is the filename used. For the very first file that is read in by \TEX, you
-have to make sure you return an \type {actual_name} that has an extension and
-that is suitable for use as \type {jobname}. If you don't, you will have to
-manually fix the name of the log file and output file after \LUATEX\ is finished,
-and an eventual format filename will become mangled. That is because these file
-names depend on the jobname.
-
-You have to return \type {nil} if the file cannot be found.
-
-\stopitemize
-
-\subsection{\type {find_font_file}}
-
-Your callback function should have the following conventions:
-
-\startfunctioncall
-<string> actual_name =
-    function (<string> asked_name)
-\stopfunctioncall
-
-The \type {asked_name} is an \OTF\ or \TFM\ font metrics file.
-
-Return \type {nil} if the file cannot be found.
-
-\subsection{\type {find_output_file}}
-
-Your callback function should have the following conventions:
-
-\startfunctioncall
-<string> actual_name =
-    function (<string> asked_name)
-\stopfunctioncall
-
-The \type {asked_name} is the \PDF\ or \DVI\ file for writing.
-
-\subsection{\type {find_format_file}}
-
-Your callback function should have the following conventions:
-
-\startfunctioncall
-<string> actual_name =
-    function (<string> asked_name)
-\stopfunctioncall
-
-The \type {asked_name} is a format file for reading (the format file for writing
-is always opened in the current directory).
-
-\subsection{\type {find_vf_file}}
-
-Like \type {find_font_file}, but for virtual fonts. This applies to both \ALEPH's
-\OVF\ files and traditional Knuthian \VF\ files.
-
-\subsection{\type {find_map_file}}
-
-Like \type {find_font_file}, but for map files.
-
-\subsection{\type {find_enc_file}}
-
-Like \type {find_font_file}, but for enc files.
-
-\subsection{\type {find_sfd_file}}
-
-Like \type {find_font_file}, but for subfont definition files.
-
-\subsection{\type {find_pk_file}}
-
-Like \type {find_font_file}, but for pk bitmap files. This callback takes two
-arguments: \type {name} and \type {dpi}. In your callback you can decide to
-look for:
-
-\starttyping
-<base res>dpi/<fontname>.<actual res>pk
-\stoptyping
-
-but other strategies are possible. It is up to you to find a \quote {reasonable}
-bitmap file to go with that specification.
-
-\subsection{\type {find_data_file}}
-
-Like \type {find_font_file}, but for embedded files (\type {\pdfobj file '...'}).
-
-\subsection{\type {find_opentype_file}}
-
-Like \type {find_font_file}, but for \OPENTYPE\ font files.
-
-\subsection{\type {find_truetype_file} and \type {find_type1_file}}
-
-Your callback function should have the following conventions:
-
-\startfunctioncall
-<string> actual_name =
-    function (<string> asked_name)
-\stopfunctioncall
-
-The \type {asked_name} is a font file. This callback is called while \LUATEX\ is
-building its internal list of needed font files, so the actual timing may
-surprise you. Your return value is later fed back into the matching \type
-{read_file} callback.
-
-Strangely enough, \type {find_type1_file} is also used for \OPENTYPE\ (\OTF)
-fonts.
-
-\subsection{\type {find_image_file}}
-
-Your callback function should have the following conventions:
-
-\startfunctioncall
-<string> actual_name =
-    function (<string> asked_name)
-\stopfunctioncall
-
-The \type {asked_name} is an image file. Your return value is used to open a file
-from the hard disk, so make sure you return something that is considered the name
-of a valid file by your operating system.
-
-\subsection[iocallback]{File reading callbacks}
-
-The behavior documented in this subsection is considered stable in the sense that
-there will not be backward-incompatible changes any more.
-
-\subsection{\type {open_read_file}}
-
-Your callback function should have the following conventions:
-
-\startfunctioncall
-<table> env =
-    function (<string> file_name)
-\stopfunctioncall
-
-Argument:
-
-\startitemize
-
-\sym{file_name}
-
-The filename returned by a previous \type {find_read_file} or the return value of
-\type {kpse.find_file()} if there was no such callback defined.
-
-\stopitemize
-
-Return value:
-
-\startitemize
-
-\sym{env}
-
-This is a table containing at least one required and one optional callback
-function for this file. The required field is \type {reader} and the associated
-function will be called once for each new line to be read, the optional one is
-\type {close} that will be called once when \LUATEX\ is done with the file.
-
-\LUATEX\ never looks at the rest of the table, so you can use it to store your
-private per|-|file data. Both the callback functions will receive the table as
-their only argument.
-
-\stopitemize
-
-\subsubsection{\type {reader}}
-
-\LUATEX\ will run this function whenever it needs a new input line from the file.
-
-\startfunctioncall
-function(<table> env)
-    return <string> line
-end
-\stopfunctioncall
-
-Your function should return either a string or \type {nil}. The value \type {nil}
-signals that the end of file has occurred, and will make \TEX\ call the optional
-\type {close} function next.
-
-\subsubsection{\type {close}}
-
-\LUATEX\ will run this optional function when it decides to close the file.
-
-\startfunctioncall
-function(<table> env)
-end
-\stopfunctioncall
-
-Your function should not return any value.
-
-\subsection{General file readers}
-
-There is a set of callbacks for the loading of binary data files. These all use
-the same interface:
-
-\startfunctioncall
-function(<string> name)
-    return <boolean> success, <string> data, <number> data_size
-end
-\stopfunctioncall
-
-The \type {name} will normally be a full path name as it is returned by either
-one of the file discovery callbacks or the internal version of \type
-{kpse.find_file()}.
-
-\startitemize
-
-\sym{success}
-
-Return \type {false} when a fatal error occurred (e.g.\ when the file cannot be
-found, after all).
-
-\sym{data}
-
-The bytes comprising the file.
-
-\sym{data_size}
-
-The length of the \type {data}, in bytes.
-
-\stopitemize
-
-Return an empty string and zero if the file was found but there was a
-reading problem.
-
-The list of functions is as follows:
-
-\starttabulate[|l|p|]
-\NC \type{read_font_file}     \NC ofm or tfm files \NC \NR
-\NC \type{read_vf_file}       \NC virtual fonts \NC \NR
-\NC \type{read_map_file}      \NC map files \NC \NR
-\NC \type{read_enc_file}      \NC encoding files \NC \NR
-\NC \type{read_sfd_file}      \NC subfont definition files \NC \NR
-\NC \type{read_pk_file}       \NC pk bitmap files \NC \NR
-\NC \type{read_data_file}     \NC embedded files (as is possible with \PDF\ objects) \NC \NR
-\NC \type{read_truetype_file} \NC \TRUETYPE\ font files \NC \NR
-\NC \type{read_type1_file}    \NC \TYPEONE\ font files \NC \NR
-\NC \type{read_opentype_file} \NC \OPENTYPE\ font files \NC \NR
-\stoptabulate
-
-\section{Data processing callbacks}
-
-\subsection{\type {process_input_buffer}}
-
-This callback allows you to change the contents of the line input buffer just
-before \LUATEX\ actually starts looking at it.
-
-\startfunctioncall
-function(<string> buffer)
-    return <string> adjusted_buffer
-end
-\stopfunctioncall
-
-If you return \type {nil}, \LUATEX\ will pretend like your callback never
-happened. You can gain a small amount of processing time from that. This callback
-does not replace any internal code.
-
-\subsection{\type {process_output_buffer}}
-
-This callback allows you to change the contents of the line output buffer just
-before \LUATEX\ actually starts writing it to a file as the result of a \type
-{\write} command. It is only called for output to an actual file (that is,
-excluding the log, the terminal, and \type {\write18} calls).
-
-\startfunctioncall
-function(<string> buffer)
-    return <string> adjusted_buffer
-end
-\stopfunctioncall
-
-If you return \type {nil}, \LUATEX\ will pretend like your callback never
-happened. You can gain a small amount of processing time from that. This callback
-does not replace any internal code.
-
-\subsection{\type {process_jobname}}
-
-This callback allows you to change the jobname given by \type {\jobname} in \TEX\
-and \type {tex.jobname} in Lua. It does not affect the internal job name or the
-name of the output or log files.
-
-\startfunctioncall
-function(<string> jobname)
-    return <string> adjusted_jobname
-end
-\stopfunctioncall
-
-The only argument is the actual job name; you should not use \type {tex.jobname}
-inside this function or infinite recursion may occur. If you return \type {nil},
-\LUATEX\ will pretend your callback never happened. This callback does not
-replace any internal code.
-
-\section{Node list processing callbacks}
-
-The description of nodes and node lists is in~\in{chapter}[nodes].
-
-\subsection{\type {contribute_filter}}
-
-This callback is called when \LUATEX\ adds contents to list:
-
-\startfunctioncall
-function(<string> extrainfo)
-end
-\stopfunctioncall
-
-The string reports the group code. From this you can deduce from
-what list you can give a treat.
-
-\starttabulate[|l|p|]
-\BC value             \BC explanation                                  \NC \NR
-\NC \type{pre_box}    \NC interline material is being added            \NC \NR
-\NC \type{pre_adjust} \NC \type {\vadjust} material is being added     \NC \NR
-\NC \type{box}        \NC a typeset box is being added (always called) \NC \NR
-\NC \type{adjust}     \NC \type {\vadjust} material is being added     \NC \NR
-\stoptabulate
-
-\subsection{\type {buildpage_filter}}
-
-This callback is called whenever \LUATEX\ is ready to move stuff to the main
-vertical list. You can use this callback to do specialized manipulation of the
-page building stage like imposition or column balancing.
-
-\startfunctioncall
-function(<string> extrainfo)
-end
-\stopfunctioncall
-
-The string \type {extrainfo} gives some additional information about what \TEX's
-state is with respect to the \quote {current page}. The possible values for the
-\type {buildpage_filter} callback are:
-
-\starttabulate[|l|p|]
-\BC value                  \BC explanation                               \NC \NR
-\NC \type{alignment}       \NC a (partial) alignment is being added      \NC \NR
-\NC \type{after_output}    \NC an output routine has just finished       \NC \NR
-\NC \type{new_graf}        \NC the beginning of a new paragraph          \NC \NR
-\NC \type{vmode_par}       \NC \type {\par} was found in vertical mode   \NC \NR
-\NC \type{hmode_par}       \NC \type {\par} was found in horizontal mode \NC \NR
-\NC \type{insert}          \NC an insert is added                        \NC \NR
-\NC \type{penalty}         \NC a penalty (in vertical mode)              \NC \NR
-\NC \type{before_display}  \NC immediately before a display starts       \NC \NR
-\NC \type{after_display}   \NC a display is finished                     \NC \NR
-\NC \type{end}             \NC \LUATEX\ is terminating (it's all over)   \NC \NR
-\stoptabulate
-
-\subsection{\type {build_page_insert}}
-
-This callback is called when the pagebuilder adds an insert. There is not much
-control over this mechanism but this callback permits some last minute
-manipulations of the spacing before an insert, something that might be handy when
-for instance multiple inserts (types) are appended in a row.
-
-\startfunctioncall
-function(<number> n, <number> i)
-    return <number> register
-end
-\stopfunctioncall
-
-with
-
-\starttabulate[|l|p|]
-\BC value    \BC explanation             \NC \NR
-\NC \type{n} \NC the insert class        \NC \NR
-\NC \type{i} \NC the order of the insert \NC \NR
-\stoptabulate
-
-The return value is a number indicating the skip register to use for the
-prepended spacing. This permits for instance a different top space (when \type
-{i} equals one) and intermediate space (when \type {i} is larger than one). Of course
-you can mess with the insert box but you need to make sure that \LUATEX\ is happy
-afterwards.
-
-\subsection{\type {pre_linebreak_filter}}
-
-This callback is called just before \LUATEX\ starts converting a list of nodes
-into a stack of \type {\hbox}es, after the addition of \type {\parfillskip}.
-
-\startfunctioncall
-function(<node> head, <string> groupcode)
-    return true | false | <node> newhead
-end
-\stopfunctioncall
-
-The string called \type {groupcode} identifies the nodelist's context within
-\TEX's processing. The range of possibilities is given in the table below, but
-not all of those can actually appear in \type {pre_linebreak_filter}, some are
-for the \type {hpack_filter} and \type {vpack_filter} callbacks that will be
-explained in the next two paragraphs.
-
-\starttabulate[|l|p|]
-\BC value                \BC explanation                                     \NC \NR
-\NC \type{<empty>}       \NC main vertical list                              \NC \NR
-\NC \type{hbox}          \NC \type {\hbox} in horizontal mode                \NC \NR
-\NC \type{adjusted_hbox} \NC \type {\hbox} in vertical mode                  \NC \NR
-\NC \type{vbox}          \NC \type {\vbox}                                   \NC \NR
-\NC \type{vtop}          \NC \type {\vtop}                                   \NC \NR
-\NC \type{align}         \NC \type {\halign} or \type {\valign}              \NC \NR
-\NC \type{disc}          \NC discretionaries                                 \NC \NR
-\NC \type{insert}        \NC packaging an insert                             \NC \NR
-\NC \type{vcenter}       \NC \type {\vcenter}                                \NC \NR
-\NC \type{local_box}     \NC \type {\localleftbox} or \type {\localrightbox} \NC \NR
-\NC \type{split_off}     \NC top of a \type {\vsplit}                        \NC \NR
-\NC \type{split_keep}    \NC remainder of a \type {\vsplit}                  \NC \NR
-\NC \type{align_set}     \NC alignment cell                                  \NC \NR
-\NC \type{fin_row}       \NC alignment row                                   \NC \NR
-\stoptabulate
-
-As for all the callbacks that deal with nodes, the return value can be one of
-three things:
-
-\startitemize
-\startitem
-    boolean \type {true} signals successful processing
-\stopitem
-\startitem
-    \type {<node>} signals that the \quote {head} node should be replaced by the
-    returned node
-\stopitem
-\startitem
-    boolean \type {false} signals that the \quote {head} node list should be
-    ignored and flushed from memory
-\stopitem
-\stopitemize
-
-This callback does not replace any internal code.
-
-\subsection{\type {linebreak_filter}}
-
-This callback replaces \LUATEX's line breaking algorithm.
-
-\startfunctioncall
-function(<node> head, <boolean> is_display)
-    return <node> newhead
-end
-\stopfunctioncall
-
-The returned node is the head of the list that will be added to the main vertical
-list, the boolean argument is true if this paragraph is interrupted by a
-following math display.
-
-If you return something that is not a \type {<node>}, \LUATEX\ will apply the
-internal linebreak algorithm on the list that starts at \type {<head>}.
-Otherwise, the \type {<node>} you return is supposed to be the head of a list of
-nodes that are all allowed in vertical mode, and at least one of those has to
-represent a hbox. Failure to do so will result in a fatal error.
-
-Setting this callback to \type {false} is possible, but dangerous, because it is
-possible you will end up in an unfixable \quote {deadcycles loop}.
-
-\subsection{\type {append_to_vlist_filter}}
-
-This callback is called whenever \LUATEX\ adds a box to a vertical list:
-
-\startfunctioncall
-function(<node> box, <string> locationcode, <number prevdepth>,
-    <boolean> mirrored)
-    return list, prevdepth
-end
-\stopfunctioncall
-
-It is ok to return nothing in which case you also need to flush the box or deal
-with it yourself. The prevdepth is also optional. Locations are \type {box},
-\type {alignment}, \type {equation}, \type {equation_number} and \type
-{post_linebreak}.
-
-\subsection{\type {post_linebreak_filter}}
-
-This callback is called just after \LUATEX\ has converted a list of nodes into a
-stack of \type {\hbox}es.
-
-\startfunctioncall
-function(<node> head, <string> groupcode)
-    return true | false | <node> newhead
-end
-\stopfunctioncall
-
-This callback does not replace any internal code.
-
-\subsection{\type {hpack_filter}}
-
-This callback is called when \TEX\ is ready to start boxing some horizontal mode
-material. Math items and line boxes are ignored at the moment.
-
-\startfunctioncall
-function(<node> head, <string> groupcode, <number> size,
-         <string> packtype [, <string> direction] [, <node> attributelist])
-    return true | false | <node> newhead
-end
-\stopfunctioncall
-
-The \type {packtype} is either \type {additional} or \type {exactly}. If \type
-{additional}, then the \type {size} is a \type {\hbox spread ...} argument. If
-\type {exactly}, then the \type {size} is a \type {\hbox to ...}. In both cases,
-the number is in scaled points.
-
-The \type {direction} is either one of the three-letter direction specifier
-strings, or \type {nil}.
-
-This callback does not replace any internal code.
-
-\subsection{\type {vpack_filter}}
-
-This callback is called when \TEX\ is ready to start boxing some vertical mode
-material. Math displays are ignored at the moment.
-
-This function is very similar to the \type {hpack_filter}. Besides the fact
-that it is called at different moments, there is an extra variable that matches
-\TEX's \type {\maxdepth} setting.
-
-\startfunctioncall
-function(<node> head, <string> groupcode, <number> size, <string> packtype,
-        <number> maxdepth [, <string> direction] [, <node> attributelist]))
-    return true | false | <node> newhead
-end
-\stopfunctioncall
-
-This callback does not replace any internal code.
-
-\subsection{\type {hpack_quality}}
-
-This callback can be used to intercept the overfull messages that can result from
-packing a horizontal list (as happens in the par builder). The function takes a
-few arguments:
-
-\startfunctioncall
-function(<string> incident, <number> detail, <node> head, <number> first,
-         <number> last)
-    return <node> whatever
-end
-\stopfunctioncall
-
-The incident is one of \type {overfull}, \type {underfull}, \type {loose} or
-\type {tight}. The detail is either the amount of overflow in case of \type
-{overfull}, or the badness otherwise. The head is the list that is constructed
-(when protrusion or expansion is enabled, this is an intermediate list).
-Optionally you can return a node, for instance an overfull rule indicator. That
-node will be appended to the list (just like \TEX's own rule would).
-
-\subsection{\type {vpack_quality}}
-
-This callback can be used to intercept the overfull messages that can result from
-packing a vertical list (as happens in the page builder). The function takes a
-few arguments:
-
-\startfunctioncall
-function(<string> incident, <number> detail, <node> head, <number> first,
-         <number> last)
-end
-\stopfunctioncall
-
-The incident is one of \type {overfull}, \type {underfull}, \type {loose} or
-\type {tight}. The detail is either the amount of overflow in case of \type
-{overfull}, or the badness otherwise. The head is the list that is constructed.
-
-\subsection{\type {process_rule}}
-
-This is an experimental callback. It can be used with rules of subtype~4
-(user). The callback gets three arguments: the node, the width and the
-height. The callback can use \type {pdf.print} to write code to the \PDF\
-file but beware of not messing up the final result. No checking is done.
-
-\subsection{\type {pre_output_filter}}
-
-This callback is called when \TEX\ is ready to start boxing the box 255 for \type
-{\output}.
-
-\startfunctioncall
-function(<node> head, <string> groupcode, <number> size, <string> packtype,
-        <number> maxdepth [, <string> direction])
-    return true | false | <node> newhead
-end
-\stopfunctioncall
-
-This callback does not replace any internal code.
-
-\subsection{\type {hyphenate}}
-
-\startfunctioncall
-function(<node> head, <node> tail)
-end
-\stopfunctioncall
-
-No return values. This callback has to insert discretionary nodes in the node
-list it receives.
-
-Setting this callback to \type {false} will prevent the internal discretionary
-insertion pass.
-
-\subsection{\type {ligaturing}}
-
-\startfunctioncall
-function(<node> head, <node> tail)
-end
-\stopfunctioncall
-
-No return values. This callback has to apply ligaturing to the node list it
-receives.
-
-You don't have to worry about return values because the \type {head} node that is
-passed on to the callback is guaranteed not to be a glyph_node (if need be, a
-temporary node will be prepended), and therefore it cannot be affected by the
-mutations that take place. After the callback, the internal value of the \quote
-{tail of the list} will be recalculated.
-
-The \type {next} of \type {head} is guaranteed to be non-nil.
-
-The \type {next} of \type {tail} is guaranteed to be nil, and therefore the
-second callback argument can often be ignored. It is provided for orthogonality,
-and because it can sometimes be handy when special processing has to take place.
-
-Setting this callback to \type {false} will prevent the internal ligature
-creation pass.
-
-You must not ruin the node list. For instance, the head normally is a local par node,
-and the tail a glue. Messing too much can push \LUATEX\ into panic mode.
-
-\subsection{\type {kerning}}
-
-\startfunctioncall
-function(<node> head, <node> tail)
-end
-\stopfunctioncall
-
-No return values. This callback has to apply kerning between the nodes in the
-node list it receives. See \type {ligaturing} for calling conventions.
-
-Setting this callback to \type {false} will prevent the internal kern insertion
-pass.
-
-You must not ruin the node list. For instance, the head normally is a local par node,
-and the tail a glue. Messing too much can push \LUATEX\ into panic mode.
-
-\subsection{\type {insert_local_par}}
-
-Each paragraph starts with a local par node that keeps track of for instance
-the direction. You can hook a callback into the creator:
-
-\startfunctioncall
-function(<node> local_par, <string> location)
-end
-\stopfunctioncall
-
-There is no return value and you should make sure that the node stays valid
-as otherwise \TEX\ can get confused.
-
-\subsection{\type {mlist_to_hlist}}
-
-This callback replaces \LUATEX's math list to node list conversion algorithm.
-
-\startfunctioncall
-function(<node> head, <string> display_type, <boolean> need_penalties)
-    return <node> newhead
-end
-\stopfunctioncall
-
-The returned node is the head of the list that will be added to the vertical or
-horizontal list, the string argument is either \quote {text} or \quote {display}
-depending on the current math mode, the boolean argument is \type {true} if
-penalties have to be inserted in this list, \type {false} otherwise.
-
-Setting this callback to \type {false} is bad, it will almost certainly result in
-an endless loop.
-
-\section{Information reporting callbacks}
-
-\subsection{\type {pre_dump}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-This function is called just before dumping to a format file starts. It does not
-replace any code and there are neither arguments nor return values.
-
-\subsection{\type {start_run}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-This callback replaces the code that prints \LUATEX's banner. Note that for
-successful use, this callback has to be set in the \LUA\ initialization script,
-otherwise it will be seen only after the run has already started.
-
-\subsection{\type {stop_run}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-This callback replaces the code that prints \LUATEX's statistics and \quote
-{output written to} messages.
-
-\subsection{\type {start_page_number}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-Replaces the code that prints the \type {[} and the page number at the begin of
-\type {\shipout}. This callback will also override the printing of box information
-that normally takes place when \type {\tracingoutput} is positive.
-
-\subsection{\type {stop_page_number}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-Replaces the code that prints the \type {]} at the end of \type {\shipout}.
-
-\subsection{\type {show_error_hook}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-This callback is run from inside the \TEX\ error function, and the idea is to
-allow you to do some extra reporting on top of what \TEX\ already does (none of
-the normal actions are removed). You may find some of the values in the \type
-{status} table useful. This callback does not replace any internal code.
-
-\subsection{\type {show_error_message}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-This callback replaces the code that prints the error message. The usual
-interaction after the message is not affected.
-
-\subsection{\type {show_lua_error_hook}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-This callback replaces the code that prints the extra \LUA\ error message.
-
-\subsection{\type {start_file}}
-
-\startfunctioncall
-function(category,filename)
-end
-\stopfunctioncall
-
-This callback replaces the code that prints \LUATEX's when a file is opened like
-\type {(filename} for regular files. The category is a number:
-
-\starttabulate[|||]
-\NC 1 \NC a normal data file, like a \TEX\ source \NC \NR
-\NC 2 \NC a font map coupling font names to resources \NC \NR
-\NC 3 \NC an image file (\type {png}, \type {pdf}, etc) \NC \NR
-\NC 4 \NC an embedded font subset \NC \NR
-\NC 5 \NC a fully embedded font \NC \NR
-\stoptabulate
-
-\subsection{\type {stop_file}}
-
-\startfunctioncall
-function(category)
-end
-\stopfunctioncall
-
-This callback replaces the code that prints \LUATEX's when a file is closed like
-the \type {)} for regular files.
-
-\subsection{\type {call_edit}}
-
-\startfunctioncall
-function(filename,linenumber)
-end
-\stopfunctioncall
-
-This callback replaces the call to an external editor when \quote{E} is pressed
-in reply to an error message. Processing will end immediately after the callback
-returns control to the main program.
-
-\subsection{\type {finish_synctex_callback}}
-
-This callback can be used to wrap up alternative synctex methods. It kicks in
-after the normal synctex finalizer (that happens to remove the synctex files
-after a run when native synctex is not enabled).
-
-\section{PDF-related callbacks}
-
-\subsection{\type {finish_pdffile}}
-
-\startfunctioncall
-function()
-end
-\stopfunctioncall
-
-This callback is called when all document pages are already written to the \PDF\
-file and \LUATEX\ is about to finalize the output document structure. Its
-intended use is final update of \PDF\ dictionaries such as \type {/Catalog} or
-\type {/Info}. The callback does not replace any code. There are neither
-arguments nor return values.
-
-\subsection{\type {finish_pdfpage}}
-
-\startfunctioncall
-function(shippingout)
-end
-\stopfunctioncall
-
-This callback is called after the \PDF\ page stream has been assembled and before
-the page object gets finalized.
-
-\section{Font-related callbacks}
-
-\subsection{\type {define_font}}
-
-\startfunctioncall
-function(<string> name, <number> size, <number> id)
-    return <table> font | <number> id
-end
-\stopfunctioncall
-
-The string \type {name} is the filename part of the font specification, as given
-by the user.
-
-The number \type {size} is a bit special:
-
-\startitemize[packed]
-\startitem
-    If it is positive, it specifies an \quote{at size} in scaled points.
-\stopitem
-\startitem
-    If it is negative, its absolute value represents a \quote {scaled} setting
-    relative to the designsize of the font.
-\stopitem
-\stopitemize
-
-The \type {id} is the internal number assigned to the font.
-
-The internal structure of the \type {font} table that is to be returned is
-explained in \in {chapter} [fonts]. That table is saved internally, so you can
-put extra fields in the table for your later \LUA\ code to use. In alternative,
-\type {retval} can be a previously defined fontid. This is useful if a previous
-definition can be reused instead of creating a whole new font structure.
-
-Setting this callback to \type {false} is pointless as it will prevent font
-loading completely but will nevertheless generate errors.
-
-\subsection{\type {glyph_not_found}}
-
-This callback kicks in when the backend cannot insert a glyph. When no callback
-is defined a message is written to the log.
-
-\startfunctioncall
-function(<number> id, <number> char)
-    -- do something with font id and char code
-end
-\stopfunctioncall
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-callbacks

+

+\startchapter[reference=callbacks,title={\LUA\ callbacks}]

+

+\startsection[title={Registering callbacks}][library=callback]

+

+\topicindex{callbacks}

+

+\libindex{register}

+\libindex{list}

+\libindex{find}

+

+This library has functions that register, find and list callbacks. Callbacks are

+\LUA\ functions that are called in well defined places. There are two kind of

+callbacks: those that mix with existing functionality, and those that (when

+enabled) replace functionality. In mosty cases the second category is expected to

+behave similar to the built in functionality because in a next step specific

+data is expected. For instance, you can replace the hyphenation routine. The

+function gets a list that can be hyphenated (or not). The final list should be

+valid and is (normally) used for constructing a paragraph. Another function can

+replace the ligature builder and|/|or kerner. Doing something else is possible

+but in the end might not give the user the expected outcome.

+

+The first thing you need to do is registering a callback:

+

+\startfunctioncall

+id, error =

+    callback.register(<string> callback_name, <function> func)

+id, error =

+    callback.register(<string> callback_name, nil)

+id, error =

+    callback.register(<string> callback_name, false)

+\stopfunctioncall

+

+Here the \syntax {callback_name} is a predefined callback name, see below. The

+function returns the internal \type {id} of the callback or \type {nil}, if the

+callback could not be registered. In the latter case, \type {error} contains an

+error message, otherwise it is \type {nil}.

+

+\LUATEX\ internalizes the callback function in such a way that it does not matter

+if you redefine a function accidentally.

+

+Callback assignments are always global. You can use the special value \type {nil}

+instead of a function for clearing the callback.

+

+For some minor speed gain, you can assign the boolean \type {false} to the

+non|-|file related callbacks, doing so will prevent \LUATEX\ from executing

+whatever it would execute by default (when no callback function is registered at

+all). Be warned: this may cause all sorts of grief unless you know \notabene

+{exactly} what you are doing!

+

+\startfunctioncall

+<table> info =

+    callback.list()

+\stopfunctioncall

+

+The keys in the table are the known callback names, the value is a boolean where

+\type {true} means that the callback is currently set (active).

+

+\startfunctioncall

+<function> f = callback.find(callback_name)

+\stopfunctioncall

+

+If the callback is not set, \type {find} returns \type {nil}.

+

+\stopsection

+

+\startsection[title={File discovery callbacks}][library=callback]

+

+The behaviour documented in this subsection is considered stable in the sense that

+there will not be backward|-|incompatible changes any more.

+

+\subsection{\cbk {find_read_file} and \cbk {find_write_file}}

+

+Your callback function should have the following conventions:

+

+\startfunctioncall

+<string> actual_name =

+    function (<number> id_number, <string> asked_name)

+\stopfunctioncall

+

+Arguments:

+

+\startitemize

+

+\sym{id_number}

+

+This number is zero for the log or \prm {input} files. For \TEX's \prm {read}

+or \prm {write} the number is incremented by one, so \type {\read0} becomes~1.

+

+\sym{asked_name}

+

+This is the user|-|supplied filename, as found by \prm {input}, \prm {openin}

+or \prm {openout}.

+

+\stopitemize

+

+Return value:

+

+\startitemize

+

+\sym{actual_name}

+

+This is the filename used. For the very first file that is read in by \TEX, you

+have to make sure you return an \type {actual_name} that has an extension and

+that is suitable for use as \type {jobname}. If you don't, you will have to

+manually fix the name of the log file and output file after \LUATEX\ is finished,

+and an eventual format filename will become mangled. That is because these file

+names depend on the jobname.

+

+You have to return \type {nil} if the file cannot be found.

+

+\stopitemize

+

+\subsection{\cbk {find_font_file}}

+

+\topicindex{callbacks+font files}

+

+Your callback function should have the following conventions:

+

+\startfunctioncall

+<string> actual_name =

+    function (<string> asked_name)

+\stopfunctioncall

+

+The \type {asked_name} is an \OTF\ or \TFM\ font metrics file.

+

+Return \type {nil} if the file cannot be found.

+

+\subsection{\cbk {find_output_file}}

+

+\topicindex{callbacks+output file}

+

+Your callback function should have the following conventions:

+

+\startfunctioncall

+<string> actual_name =

+    function (<string> asked_name)

+\stopfunctioncall

+

+The \type {asked_name} is the \PDF\ or \DVI\ file for writing.

+

+\subsection{\cbk {find_format_file}}

+

+\topicindex{callbacks+format file}

+

+Your callback function should have the following conventions:

+

+\startfunctioncall

+<string> actual_name =

+    function (<string> asked_name)

+\stopfunctioncall

+

+The \type {asked_name} is a format file for reading (the format file for writing

+is always opened in the current directory).

+

+\subsection{\cbk {find_vf_file}}

+

+\topicindex{callbacks+font files}

+

+Like \cbk {find_font_file}, but for virtual fonts. This applies to both \ALEPH's

+\OVF\ files and traditional Knuthian \VF\ files.

+

+\subsection{\cbk {find_map_file}}

+

+\topicindex{callbacks+font files}

+

+Like \cbk {find_font_file}, but for map files.

+

+\subsection{\cbk {find_enc_file}}

+

+\topicindex{callbacks+font files}

+

+Like \cbk {find_font_file}, but for enc files.

+

+\subsection{\cbk {find_pk_file}}

+

+\topicindex{callbacks+font files}

+

+Like \cbk {find_font_file}, but for pk bitmap files. This callback takes two

+arguments: \type {name} and \type {dpi}. In your callback you can decide to

+look for:

+

+\starttyping

+<base res>dpi/<fontname>.<actual res>pk

+\stoptyping

+

+but other strategies are possible. It is up to you to find a \quote {reasonable}

+bitmap file to go with that specification.

+

+\subsection{\cbk {find_data_file}}

+

+\topicindex{callbacks+data files}

+

+Like \cbk {find_font_file}, but for embedded files (\type {\pdfobj file '...'}).

+

+\subsection{\cbk {find_opentype_file}}

+

+\topicindex{callbacks+font files}

+

+Like \cbk {find_font_file}, but for \OPENTYPE\ font files.

+

+\subsection{\cbk {find_truetype_file} and \cbk {find_type1_file}}

+

+\topicindex{callbacks+font files}

+

+Your callback function should have the following conventions:

+

+\startfunctioncall

+<string> actual_name =

+    function (<string> asked_name)

+\stopfunctioncall

+

+The \type {asked_name} is a font file. This callback is called while \LUATEX\ is

+building its internal list of needed font files, so the actual timing may

+surprise you. Your return value is later fed back into the matching \type

+{read_file} callback.

+

+Strangely enough, \cbk {find_type1_file} is also used for \OPENTYPE\ (\OTF)

+fonts.

+

+\subsection{\cbk {find_image_file}}

+

+\topicindex{callbacks+image files}

+

+Your callback function should have the following conventions:

+

+\startfunctioncall

+<string> actual_name =

+    function (<string> asked_name)

+\stopfunctioncall

+

+The \type {asked_name} is an image file. Your return value is used to open a file

+from the hard disk, so make sure you return something that is considered the name

+of a valid file by your operating system.

+

+\subsection[iocallback]{File reading callbacks}

+

+The behavior documented in this subsection is considered stable in the sense that

+there will not be backward-incompatible changes any more.

+

+\subsection{\cbk {open_read_file}}

+

+\topicindex{callbacks+opening files}

+

+Your callback function should have the following conventions:

+

+\startfunctioncall

+<table> env =

+    function (<string> file_name)

+\stopfunctioncall

+

+Argument:

+

+\startitemize

+

+\sym{file_name}

+

+The filename returned by a previous \cbk {find_read_file} or the return value of

+\type {kpse.find_file()} if there was no such callback defined.

+

+\stopitemize

+

+Return value:

+

+\startitemize

+

+\sym{env}

+

+This is a table containing at least one required and one optional callback

+function for this file. The required field is \type {reader} and the associated

+function will be called once for each new line to be read, the optional one is

+\type {close} that will be called once when \LUATEX\ is done with the file.

+

+\LUATEX\ never looks at the rest of the table, so you can use it to store your

+private per|-|file data. Both the callback functions will receive the table as

+their only argument.

+

+\stopitemize

+

+\subsubsection{\type {reader}}

+

+\topicindex{callbacks+reader}

+

+\LUATEX\ will run this function whenever it needs a new input line from the file.

+

+\startfunctioncall

+function(<table> env)

+    return <string> line

+end

+\stopfunctioncall

+

+Your function should return either a string or \type {nil}. The value \type {nil}

+signals that the end of file has occurred, and will make \TEX\ call the optional

+\type {close} function next.

+

+\subsubsection{\type {close}}

+

+\topicindex{callbacks+closing files}

+

+\LUATEX\ will run this optional function when it decides to close the file.

+

+\startfunctioncall

+function(<table> env)

+end

+\stopfunctioncall

+

+Your function should not return any value.

+

+\subsection{General file readers}

+

+\topicindex{callbacks+readers}

+

+There is a set of callbacks for the loading of binary data files. These all use

+the same interface:

+

+\startfunctioncall

+function(<string> name)

+    return <boolean> success, <string> data, <number> data_size

+end

+\stopfunctioncall

+

+The \type {name} will normally be a full path name as it is returned by either

+one of the file discovery callbacks or the internal version of \type

+{kpse.find_file()}.

+

+\startitemize

+

+\sym{success}

+

+Return \type {false} when a fatal error occurred (e.g.\ when the file cannot be

+found, after all).

+

+\sym{data}

+

+The bytes comprising the file.

+

+\sym{data_size}

+

+The length of the \type {data}, in bytes.

+

+\stopitemize

+

+Return an empty string and zero if the file was found but there was a

+reading problem.

+

+The list of functions is:

+

+\starttabulate[|l|p|]

+\DB function  \BC usage \NC \NR

+\TB

+\NC \type{read_font_file}     \NC ofm or tfm files \NC \NR

+\NC \type{read_vf_file}       \NC virtual fonts \NC \NR

+\NC \type{read_map_file}      \NC map files \NC \NR

+\NC \type{read_enc_file}      \NC encoding files \NC \NR

+\NC \type{read_pk_file}       \NC pk bitmap files \NC \NR

+\NC \type{read_data_file}     \NC embedded files (as is possible with \PDF\ objects) \NC \NR

+\NC \type{read_truetype_file} \NC \TRUETYPE\ font files \NC \NR

+\NC \type{read_type1_file}    \NC \TYPEONE\ font files \NC \NR

+\NC \type{read_opentype_file} \NC \OPENTYPE\ font files \NC \NR

+\LL

+\stoptabulate

+

+\stopsection

+

+\startsection[title={Data processing callbacks}][library=callback]

+

+\subsection{\cbk {process_input_buffer}}

+

+\topicindex{callbacks+input buffer}

+

+This callback allows you to change the contents of the line input buffer just

+before \LUATEX\ actually starts looking at it.

+

+\startfunctioncall

+function(<string> buffer)

+    return <string> adjusted_buffer

+end

+\stopfunctioncall

+

+If you return \type {nil}, \LUATEX\ will pretend like your callback never

+happened. You can gain a small amount of processing time from that. This callback

+does not replace any internal code.

+

+\subsection{\cbk {process_output_buffer}}

+

+\topicindex{callbacks+output buffer}

+

+This callback allows you to change the contents of the line output buffer just

+before \LUATEX\ actually starts writing it to a file as the result of a \prm

+{write} command. It is only called for output to an actual file (that is,

+excluding the log, the terminal, and so called \prm {write} 18 calls).

+

+\startfunctioncall

+function(<string> buffer)

+    return <string> adjusted_buffer

+end

+\stopfunctioncall

+

+If you return \type {nil}, \LUATEX\ will pretend like your callback never

+happened. You can gain a small amount of processing time from that. This callback

+does not replace any internal code.

+

+\subsection{\cbk {process_jobname}}

+

+\topicindex{callbacks+jobname}

+

+This callback allows you to change the jobname given by \prm {jobname} in \TEX\

+and \type {tex.jobname} in Lua. It does not affect the internal job name or the

+name of the output or log files.

+

+\startfunctioncall

+function(<string> jobname)

+    return <string> adjusted_jobname

+end

+\stopfunctioncall

+

+The only argument is the actual job name; you should not use \type {tex.jobname}

+inside this function or infinite recursion may occur. If you return \type {nil},

+\LUATEX\ will pretend your callback never happened. This callback does not

+replace any internal code.

+

+\stopsection

+

+\startsection[title={Node list processing callbacks}][library=callback]

+

+The description of nodes and node lists is in~\in{chapter}[nodes].

+

+\subsection{\cbk {contribute_filter}}

+

+\topicindex{callbacks+contributions}

+

+This callback is called when \LUATEX\ adds contents to list:

+

+\startfunctioncall

+function(<string> extrainfo)

+end

+\stopfunctioncall

+

+The string reports the group code. From this you can deduce from

+what list you can give a treat.

+

+\starttabulate[|l|p|]

+\DB value             \BC explanation                                  \NC \NR

+\TB

+\NC \type{pre_box}    \NC interline material is being added            \NC \NR

+\NC \type{pre_adjust} \NC \prm {vadjust} material is being added       \NC \NR

+\NC \type{box}        \NC a typeset box is being added (always called) \NC \NR

+\NC \type{adjust}     \NC \prm {vadjust} material is being added       \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\cbk {buildpage_filter}}

+

+\topicindex{callbacks+building pages}

+

+This callback is called whenever \LUATEX\ is ready to move stuff to the main

+vertical list. You can use this callback to do specialized manipulation of the

+page building stage like imposition or column balancing.

+

+\startfunctioncall

+function(<string> extrainfo)

+end

+\stopfunctioncall

+

+The string \type {extrainfo} gives some additional information about what \TEX's

+state is with respect to the \quote {current page}. The possible values for the

+\cbk {buildpage_filter} callback are:

+

+\starttabulate[|l|p|]

+\DB value                  \BC explanation                             \NC \NR

+\TB

+\NC \type{alignment}       \NC a (partial) alignment is being added    \NC \NR

+\NC \type{after_output}    \NC an output routine has just finished     \NC \NR

+\NC \type{new_graf}        \NC the beginning of a new paragraph        \NC \NR

+\NC \type{vmode_par}       \NC \prm {par} was found in vertical mode   \NC \NR

+\NC \type{hmode_par}       \NC \prm {par} was found in horizontal mode \NC \NR

+\NC \type{insert}          \NC an insert is added                      \NC \NR

+\NC \type{penalty}         \NC a penalty (in vertical mode)            \NC \NR

+\NC \type{before_display}  \NC immediately before a display starts     \NC \NR

+\NC \type{after_display}   \NC a display is finished                   \NC \NR

+\NC \type{end}             \NC \LUATEX\ is terminating (it's all over) \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\cbk {build_page_insert}}

+

+\topicindex{callbacks+inserts}

+

+This callback is called when the pagebuilder adds an insert. There is not much

+control over this mechanism but this callback permits some last minute

+manipulations of the spacing before an insert, something that might be handy when

+for instance multiple inserts (types) are appended in a row.

+

+\startfunctioncall

+function(<number> n, <number> i)

+    return <number> register

+end

+\stopfunctioncall

+

+with

+

+\starttabulate[|l|p|]

+\DB value    \BC explanation             \NC \NR

+\TB

+\NC \type{n} \NC the insert class        \NC \NR

+\NC \type{i} \NC the order of the insert \NC \NR

+\LL

+\stoptabulate

+

+The return value is a number indicating the skip register to use for the

+prepended spacing. This permits for instance a different top space (when \type

+{i} equals one) and intermediate space (when \type {i} is larger than one). Of

+course you can mess with the insert box but you need to make sure that \LUATEX\

+is happy afterwards.

+

+\subsection{\cbk {pre_linebreak_filter}}

+

+\topicindex{callbacks+linebreaks}

+

+This callback is called just before \LUATEX\ starts converting a list of nodes

+into a stack of \prm {hbox}es, after the addition of \prm {parfillskip}.

+

+\startfunctioncall

+function(<node> head, <string> groupcode)

+    return true | false | <node> newhead

+end

+\stopfunctioncall

+

+The string called \type {groupcode} identifies the nodelist's context within

+\TEX's processing. The range of possibilities is given in the table below, but

+not all of those can actually appear in \cbk {pre_linebreak_filter}, some are

+for the \cbk {hpack_filter} and \cbk {vpack_filter} callbacks that will be

+explained in the next two paragraphs.

+

+\starttabulate[|l|p|]

+\DB value                \BC explanation                                 \NC \NR

+\TB

+\NC \type{<empty>}       \NC main vertical list                          \NC \NR

+\NC \type{hbox}          \NC \prm {hbox} in horizontal mode              \NC \NR

+\NC \type{adjusted_hbox} \NC \prm {hbox} in vertical mode                \NC \NR

+\NC \type{vbox}          \NC \prm {vbox}                                 \NC \NR

+\NC \type{vtop}          \NC \prm {vtop}                                 \NC \NR

+\NC \type{align}         \NC \prm {halign} or \prm {valign}              \NC \NR

+\NC \type{disc}          \NC discretionaries                             \NC \NR

+\NC \type{insert}        \NC packaging an insert                         \NC \NR

+\NC \type{vcenter}       \NC \prm {vcenter}                              \NC \NR

+\NC \type{local_box}     \NC \lpr {localleftbox} or \lpr {localrightbox} \NC \NR

+\NC \type{split_off}     \NC top of a \prm {vsplit}                      \NC \NR

+\NC \type{split_keep}    \NC remainder of a \prm {vsplit}                \NC \NR

+\NC \type{align_set}     \NC alignment cell                              \NC \NR

+\NC \type{fin_row}       \NC alignment row                               \NC \NR

+\LL

+\stoptabulate

+

+As for all the callbacks that deal with nodes, the return value can be one of

+three things:

+

+\startitemize

+\startitem

+    boolean \type {true} signals successful processing

+\stopitem

+\startitem

+    \type {<node>} signals that the \quote {head} node should be replaced by the

+    returned node

+\stopitem

+\startitem

+    boolean \type {false} signals that the \quote {head} node list should be

+    ignored and flushed from memory

+\stopitem

+\stopitemize

+

+This callback does not replace any internal code.

+

+\subsection{\cbk {linebreak_filter}}

+

+\topicindex{callbacks+linebreaks}

+

+This callback replaces \LUATEX's line breaking algorithm.

+

+\startfunctioncall

+function(<node> head, <boolean> is_display)

+    return <node> newhead

+end

+\stopfunctioncall

+

+The returned node is the head of the list that will be added to the main vertical

+list, the boolean argument is true if this paragraph is interrupted by a

+following math display.

+

+If you return something that is not a \type {<node>}, \LUATEX\ will apply the

+internal linebreak algorithm on the list that starts at \type {<head>}.

+Otherwise, the \type {<node>} you return is supposed to be the head of a list of

+nodes that are all allowed in vertical mode, and at least one of those has to

+represent a hbox. Failure to do so will result in a fatal error.

+

+Setting this callback to \type {false} is possible, but dangerous, because it is

+possible you will end up in an unfixable \quote {deadcycles loop}.

+

+\subsection{\type {append_to_vlist_filter}}

+

+\topicindex{callbacks+contributions}

+

+This callback is called whenever \LUATEX\ adds a box to a vertical list:

+

+\startfunctioncall

+function(<node> box, <string> locationcode, <number prevdepth>,

+    <boolean> mirrored)

+    return list, prevdepth

+end

+\stopfunctioncall

+

+It is ok to return nothing in which case you also need to flush the box or deal

+with it yourself. The prevdepth is also optional. Locations are \type {box},

+\type {alignment}, \type {equation}, \type {equation_number} and \type

+{post_linebreak}.

+

+\subsection{\cbk {post_linebreak_filter}}

+

+\topicindex{callbacks+linebreaks}

+

+This callback is called just after \LUATEX\ has converted a list of nodes into a

+stack of \prm {hbox}es.

+

+\startfunctioncall

+function(<node> head, <string> groupcode)

+    return true | false | <node> newhead

+end

+\stopfunctioncall

+

+This callback does not replace any internal code.

+

+\subsection{\cbk {hpack_filter}}

+

+\topicindex{callbacks+packing}

+

+This callback is called when \TEX\ is ready to start boxing some horizontal mode

+material. Math items and line boxes are ignored at the moment.

+

+\startfunctioncall

+function(<node> head, <string> groupcode, <number> size,

+         <string> packtype [, <string> direction] [, <node> attributelist])

+    return true | false | <node> newhead

+end

+\stopfunctioncall

+

+The \type {packtype} is either \type {additional} or \type {exactly}. If \type

+{additional}, then the \type {size} is a \type {\hbox spread ...} argument. If

+\type {exactly}, then the \type {size} is a \type {\hbox to ...}. In both cases,

+the number is in scaled points.

+

+The \type {direction} is either one of the three-letter direction specifier

+strings, or \type {nil}.

+

+This callback does not replace any internal code.

+

+\subsection{\cbk {vpack_filter}}

+

+\topicindex{callbacks+packing}

+

+This callback is called when \TEX\ is ready to start boxing some vertical mode

+material. Math displays are ignored at the moment.

+

+This function is very similar to the \cbk {hpack_filter}. Besides the fact

+that it is called at different moments, there is an extra variable that matches

+\TEX's \prm {maxdepth} setting.

+

+\startfunctioncall

+function(<node> head, <string> groupcode, <number> size, <string> packtype,

+        <number> maxdepth [, <string> direction] [, <node> attributelist]))

+    return true | false | <node> newhead

+end

+\stopfunctioncall

+

+This callback does not replace any internal code.

+

+\subsection{\type {hpack_quality}}

+

+\topicindex{callbacks+packing}

+

+This callback can be used to intercept the overfull messages that can result from

+packing a horizontal list (as happens in the par builder). The function takes a

+few arguments:

+

+\startfunctioncall

+function(<string> incident, <number> detail, <node> head, <number> first,

+         <number> last)

+    return <node> whatever

+end

+\stopfunctioncall

+

+The incident is one of \type {overfull}, \type {underfull}, \type {loose} or

+\type {tight}. The detail is either the amount of overflow in case of \type

+{overfull}, or the badness otherwise. The head is the list that is constructed

+(when protrusion or expansion is enabled, this is an intermediate list).

+Optionally you can return a node, for instance an overfull rule indicator. That

+node will be appended to the list (just like \TEX's own rule would).

+

+\subsection{\type {vpack_quality}}

+

+\topicindex{callbacks+packing}

+

+This callback can be used to intercept the overfull messages that can result from

+packing a vertical list (as happens in the page builder). The function takes a

+few arguments:

+

+\startfunctioncall

+function(<string> incident, <number> detail, <node> head, <number> first,

+         <number> last)

+end

+\stopfunctioncall

+

+The incident is one of \type {overfull}, \type {underfull}, \type {loose} or

+\type {tight}. The detail is either the amount of overflow in case of \type

+{overfull}, or the badness otherwise. The head is the list that is constructed.

+

+\subsection{\cbk {process_rule}}

+

+\topicindex{callbacks+rules}

+

+This is an experimental callback. It can be used with rules of subtype~4

+(user). The callback gets three arguments: the node, the width and the

+height. The callback can use \type {pdf.print} to write code to the \PDF\

+file but beware of not messing up the final result. No checking is done.

+

+\subsection{\type {pre_output_filter}}

+

+\topicindex{callbacks+output}

+

+This callback is called when \TEX\ is ready to start boxing the box 255 for \prm

+{output}.

+

+\startfunctioncall

+function(<node> head, <string> groupcode, <number> size, <string> packtype,

+        <number> maxdepth [, <string> direction])

+    return true | false | <node> newhead

+end

+\stopfunctioncall

+

+This callback does not replace any internal code.

+

+\subsection{\cbk {hyphenate}}

+

+\topicindex{callbacks+hyphenation}

+

+\startfunctioncall

+function(<node> head, <node> tail)

+end

+\stopfunctioncall

+

+No return values. This callback has to insert discretionary nodes in the node

+list it receives.

+

+Setting this callback to \type {false} will prevent the internal discretionary

+insertion pass.

+

+\subsection{\cbk {ligaturing}}

+

+\topicindex{callbacks+ligature building}

+

+\startfunctioncall

+function(<node> head, <node> tail)

+end

+\stopfunctioncall

+

+No return values. This callback has to apply ligaturing to the node list it

+receives.

+

+You don't have to worry about return values because the \type {head} node that is

+passed on to the callback is guaranteed not to be a glyph_node (if need be, a

+temporary node will be prepended), and therefore it cannot be affected by the

+mutations that take place. After the callback, the internal value of the \quote

+{tail of the list} will be recalculated.

+

+The \type {next} of \type {head} is guaranteed to be non-nil.

+

+The \type {next} of \type {tail} is guaranteed to be nil, and therefore the

+second callback argument can often be ignored. It is provided for orthogonality,

+and because it can sometimes be handy when special processing has to take place.

+

+Setting this callback to \type {false} will prevent the internal ligature

+creation pass.

+

+You must not ruin the node list. For instance, the head normally is a local par node,

+and the tail a glue. Messing too much can push \LUATEX\ into panic mode.

+

+\subsection{\cbk {kerning}}

+

+\topicindex{callbacks+kerning}

+

+\startfunctioncall

+function(<node> head, <node> tail)

+end

+\stopfunctioncall

+

+No return values. This callback has to apply kerning between the nodes in the

+node list it receives. See \cbk {ligaturing} for calling conventions.

+

+Setting this callback to \type {false} will prevent the internal kern insertion

+pass.

+

+You must not ruin the node list. For instance, the head normally is a local par node,

+and the tail a glue. Messing too much can push \LUATEX\ into panic mode.

+

+\subsection{\type {insert_local_par}}

+

+Each paragraph starts with a local par node that keeps track of for instance

+the direction. You can hook a callback into the creator:

+

+\startfunctioncall

+function(<node> local_par, <string> location)

+end

+\stopfunctioncall

+

+There is no return value and you should make sure that the node stays valid

+as otherwise \TEX\ can get confused.

+

+\subsection{\cbk {mlist_to_hlist}}

+

+\topicindex{callbacks+math}

+

+This callback replaces \LUATEX's math list to node list conversion algorithm.

+

+\startfunctioncall

+function(<node> head, <string> display_type, <boolean> need_penalties)

+    return <node> newhead

+end

+\stopfunctioncall

+

+The returned node is the head of the list that will be added to the vertical or

+horizontal list, the string argument is either \quote {text} or \quote {display}

+depending on the current math mode, the boolean argument is \type {true} if

+penalties have to be inserted in this list, \type {false} otherwise.

+

+Setting this callback to \type {false} is bad, it will almost certainly result in

+an endless loop.

+

+\stopsection

+

+\startsection[title={Information reporting callbacks}][library=callback]

+

+\subsection{\cbk {pre_dump}}

+

+\topicindex{callbacks+dump}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+This function is called just before dumping to a format file starts. It does not

+replace any code and there are neither arguments nor return values.

+

+\subsection{\cbk {start_run}}

+

+\topicindex{callbacks+job run}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+This callback replaces the code that prints \LUATEX's banner. Note that for

+successful use, this callback has to be set in the \LUA\ initialization script,

+otherwise it will be seen only after the run has already started.

+

+\subsection{\cbk {stop_run}}

+

+\topicindex{callbacks+job run}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+This callback replaces the code that prints \LUATEX's statistics and \quote

+{output written to} messages. The engine can still do housekeeping and therefore

+you should not rely on this hook for postprocessing the \PDF\ or log file.

+

+\subsection{\cbk {start_page_number}}

+

+\topicindex{callbacks+pages}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+Replaces the code that prints the \type {[} and the page number at the begin of

+\prm {shipout}. This callback will also override the printing of box information

+that normally takes place when \prm {tracingoutput} is positive.

+

+\subsection{\cbk {stop_page_number}}

+

+\topicindex{callbacks+pages}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+Replaces the code that prints the \type {]} at the end of \prm {shipout}.

+

+\subsection{\cbk {show_error_hook}}

+

+\topicindex{callbacks+errors}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+This callback is run from inside the \TEX\ error function, and the idea is to

+allow you to do some extra reporting on top of what \TEX\ already does (none of

+the normal actions are removed). You may find some of the values in the \type

+{status} table useful. This callback does not replace any internal code.

+

+\subsection{\cbk {show_error_message}}

+

+\topicindex{callbacks+errors}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+This callback replaces the code that prints the error message. The usual

+interaction after the message is not affected.

+

+\subsection{\cbk {show_lua_error_hook}}

+

+\topicindex{callbacks+errors}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+This callback replaces the code that prints the extra \LUA\ error message.

+

+\subsection{\cbk {start_file}}

+

+\topicindex{callbacks+files}

+

+\startfunctioncall

+function(category,filename)

+end

+\stopfunctioncall

+

+This callback replaces the code that prints \LUATEX's when a file is opened like

+\type {(filename} for regular files. The category is a number:

+

+\starttabulate[|c|l|]

+\DB value  \BC meaning \NC \NR

+\TB

+\NC 1 \NC a normal data file, like a \TEX\ source \NC \NR

+\NC 2 \NC a font map coupling font names to resources \NC \NR

+\NC 3 \NC an image file (\type {png}, \type {pdf}, etc) \NC \NR

+\NC 4 \NC an embedded font subset \NC \NR

+\NC 5 \NC a fully embedded font \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\cbk {stop_file}}

+

+\topicindex{callbacks+files}

+

+\startfunctioncall

+function(category)

+end

+\stopfunctioncall

+

+This callback replaces the code that prints \LUATEX's when a file is closed like

+the \type {)} for regular files.

+

+\subsection{\cbk {call_edit}}

+

+\topicindex{callbacks+editing}

+

+\startfunctioncall

+function(filename,linenumber)

+end

+\stopfunctioncall

+

+This callback replaces the call to an external editor when \quote{E} is pressed

+in reply to an error message. Processing will end immediately after the callback

+returns control to the main program.

+

+\subsection{\cbk {finish_synctex}}

+

+\topicindex{callbacks+synctex}

+

+This callback can be used to wrap up alternative synctex methods. It kicks in

+after the normal synctex finalizer (that happens to remove the synctex files

+after a run when native synctex is not enabled).

+

+\subsection{\cbk {wrapup_run}}

+

+\topicindex{callbacks+wrapping up}

+

+This callback is called after the \PDF\ and log files are closed. Use it at your own

+risk.

+

+\stopsection

+

+\startsection[title={\PDF\ related callbacks}][library=callback]

+

+\subsection{\cbk {finish_pdffile}}

+

+\topicindex{callbacks+\PDF\ file}

+

+\startfunctioncall

+function()

+end

+\stopfunctioncall

+

+This callback is called when all document pages are already written to the \PDF\

+file and \LUATEX\ is about to finalize the output document structure. Its

+intended use is final update of \PDF\ dictionaries such as \type {/Catalog} or

+\type {/Info}. The callback does not replace any code. There are neither

+arguments nor return values.

+

+\subsection{\cbk {finish_pdfpage}}

+

+\topicindex{callbacks+\PDF\ file}

+

+\startfunctioncall

+function(shippingout)

+end

+\stopfunctioncall

+

+This callback is called after the \PDF\ page stream has been assembled and before

+the page object gets finalized.

+

+\subsection{\cbk {page_objnum_provider}}

+

+\topicindex{callbacks+\PDF\ file}

+

+This is one that experts can use to juggle the page tree, a data structure

+that determines the order in a \PDF\ file:

+

+\startfunctioncall

+function(objnum)

+  return objnum

+end

+\stopfunctioncall

+

+We can for instance swap the first and last page:

+

+\starttyping

+local n = 0

+callback.register("page_objnum_provider",function(objnum)

+    n = n + 1

+    if n == 1 then

+        return pdf.getpageref(tex.count[0])

+    elseif n == tex.count[0] then

+        return pdf.getpageref(1)

+    else

+        return objnum

+    end

+end)

+\stoptyping

+

+When you mess things up \unknown\ don't complain.

+

+\subsection{\cbk {process_pdf_image_content}}

+

+\topicindex{callbacks+image content}

+

+When a page from a \PDF\ file is embedded its page stream as well as related

+objects are copied to the target file. However, it can be that the page stream

+has operators that assume additional resources, for instance marked text. You can

+decide to filter that for which \LUATEX\ provides a callback. Here is a simple

+demonstration of use:

+

+\starttyping

+pdf.setrecompress(1)

+

+callback.register("process_pdf_image_content",function(s)

+    print(s)

+    return s

+end)

+\stoptyping

+

+You need to explicitly enable recompression because otherwise the content stream

+gets just passed on in its original compressed form.

+

+\stopsection

+

+\startsection[title={Font-related callbacks}][library=callback]

+

+\subsection{\cbk {define_font}}

+

+\topicindex{callbacks+fonts}

+

+\startfunctioncall

+function(<string> name, <number> size, <number> id)

+    return <table> font | <number> id

+end

+\stopfunctioncall

+

+The string \type {name} is the filename part of the font specification, as given

+by the user.

+

+The number \type {size} is a bit special:

+

+\startitemize[packed]

+\startitem

+    If it is positive, it specifies an \quote{at size} in scaled points.

+\stopitem

+\startitem

+    If it is negative, its absolute value represents a \quote {scaled} setting

+    relative to the designsize of the font.

+\stopitem

+\stopitemize

+

+The \type {id} is the internal number assigned to the font.

+

+The internal structure of the \type {font} table that is to be returned is

+explained in \in {chapter} [fonts]. That table is saved internally, so you can

+put extra fields in the table for your later \LUA\ code to use. In alternative,

+\type {retval} can be a previously defined fontid. This is useful if a previous

+definition can be reused instead of creating a whole new font structure.

+

+Setting this callback to \type {false} is pointless as it will prevent font

+loading completely but will nevertheless generate errors.

+

+\subsection{\cbk {glyph_not_found}}

+

+\topicindex{callbacks+fonts}

+

+This callback kicks in when the backend cannot insert a glyph. When no callback

+is defined a message is written to the log.

+

+\startfunctioncall

+function(<number> id, <number> char)

+    -- do something with font id and char code

+end

+\stopfunctioncall

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-contents.tex b/Master/texmf-dist/doc/luatex/base/luatex-contents.tex
index 6d06b3ef082..815ca4bbe78 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-contents.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-contents.tex
@@ -1,20 +1,19 @@
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-contents
-
-\starttitle[title=Contents]
-
-\start
-
-    \definecolor[maincolor][black]
-
-    \placelist
-      [chapter,section,subsection]
-      [criterium=text]
-
-\stop
-
-\stoptitle
-
-\stopcomponent
+\environment luatex-style

+

+\startcomponent luatex-contents

+

+\starttitle[title=Contents]

+

+\start

+

+    \definecolor[maincolor][black]

+

+    \placelist

+      [chapter,section,subsection]

+      [criterium=text]

+

+\stop

+

+\stoptitle

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex b/Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex
index d1da33878e0..b04a6e29cfe 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex
@@ -1,916 +1,1367 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-enhancements
-
-\startchapter[reference=enhancements,title={Basic \TEX\ enhancements}]
-
-\section{Introduction}
-
-From day one, \LUATEX\ has offered extra features compared to the superset of
-\PDFTEX\ and \ALEPH. This has not been limited to the possibility to execute
-\LUA\ code via \type {\directlua}, but \LUATEX\ also adds functionality via new
-\TEX|-|side primitives or extensions to existing ones.
-
-When \LUATEX\ starts up in \quote {iniluatex} mode (\type {luatex -ini}), it
-defines only the primitive commands known by \TEX82 and the one extra command
-\type {\directlua}. As is fitting, a \LUA\ function has to be called to add the
-extra primitives to the user environment. The simplest method to get access to
-all of the new primitive commands is by adding this line to the format generation
-file:
-
-\starttyping
-\directlua { tex.enableprimitives('',tex.extraprimitives()) }
-\stoptyping
-
-But be aware that the curly braces may not have the proper \type {\catcode}
-assigned to them at this early time (giving a \quote {Missing number} error), so
-it may be needed to put these assignments before the above line:
-
-\starttyping
-\catcode `\{=1
-\catcode `\}=2
-\stoptyping
-
-More fine|-|grained primitives control is possible and you can look up the
-details in \in {section} [luaprimitives]. For simplicity's sake, this manual
-assumes that you have executed the \type {\directlua} command as given above.
-
-The startup behaviour documented above is considered stable in the sense that
-there will not be backward|-|incompatible changes any more. We have promoted some
-rather generic \PDFTEX\ primitives to core \LUATEX\ ones, and the ones inherited
-frome \ALEPH\ (\OMEGA) are also promoted. Effectively this means that we now only
-have the \type {tex}, \type {etex} and \type {luatex} sets left.
-
-In \in {Chapter} [modifications] we discuss several primitives that are derived
-from \PDFTEX\ and \ALEPH\ (\OMEGA). Here we stick to real new ones. In the
-chapters on fonts and math we discuss a few more new ones.
-
-\section{Version information}
-
-\subsection {\type {\luatexbanner}, \type {\luatexversion} and \type {\luatexrevision}}
-
-There are three new primitives to test the version of \LUATEX:
-
-\starttabulate[|l|pl|pl|]
-\BC primitive               \BC explanation                               \BC value              \NC \NR
-\NC \type {\luatexbanner}   \NC the banner reported on the command line   \NC \luatexbanner      \NC \NR
-\NC \type {\luatexversion}  \NC a combination of major and minor number   \NC \the\luatexversion \NC \NR
-\NC \type {\luatexrevision} \NC the revision number, the current value is \NC \luatexrevision    \NC \NR
-\stoptabulate
-
-The official \LUATEX\ version is defined as follows:
-
-\startitemize
-\startitem
-    The major version is the integer result of \type {\luatexversion} divided by
-    100. The primitive is an \quote {internal variable}, so you may need to prefix
-    its use with \type {\the} depending on the context.
-\stopitem
-\startitem
-    The minor version is the two-digit result of \type {\luatexversion} modulo 100.
-\stopitem
-\startitem
-    The revision is the given by \type {\luatexrevision}. This primitive expands to
-    a positive integer.
-\stopitem
-\startitem
-    The full version number consists of the major version, minor version and
-    revision, separated by dots.
-\stopitem
-\stopitemize
-
-\subsection{\type {\formatname}}
-
-The \type {\formatname} syntax is identical to \type {\jobname}. In \INITEX, the
-expansion is empty. Otherwise, the expansion is the value that \type {\jobname} had
-during the \INITEX\ run that dumped the currently loaded format. You can use this
-token list to provide your own version info.
-
-\section{\UNICODE\ text support}
-
-\subsection {Extended ranges}
-
-Text input and output is now considered to be \UNICODE\ text, so input characters
-can use the full range of \UNICODE\ ($2^{20}+2^{16}-1 = \hbox{0x10FFFF}$). Later
-chapters will talk of characters and glyphs. Although these are not
-interchangeable, they are closely related. During typesetting, a character is
-always converted to a suitable graphic representation of that character in a
-specific font. However, while processing a list of to|-|be|-|typeset nodes, its
-contents may still be seen as a character. Inside \LUATEX\ there is no clear
-separation between the two concepts. Because the subtype of a glyph node can be
-changed in \LUA\ it is up to the user: subtypes larger than 255 indicate that
-font processing has happened.
-
-A few primitives are affected by this, all in a similar fashion: each of them has
-to accommodate for a larger range of acceptable numbers. For instance, \type
-{\char} now accepts values between~0 and $1{,}114{,}111$. This should not be a
-problem for well|-|behaved input files, but it could create incompatibilities for
-input that would have generated an error when processed by older \TEX|-|based
-engines. The affected commands with an altered initial (left of the equals sign)
-or secondary (right of the equals sign) value are: \type {\char}, \type
-{\lccode}, \type {\uccode}, \type {\catcode}, \type {\sfcode}, \type {\efcode},
-\type {\lpcode}, \type {\rpcode}, \type {\chardef}.
-
-As far as the core engine is concerned, all input and output to text files is
-\UTF-8 encoded. Input files can be pre|-|processed using the \type {reader}
-callback. This will be explained in a later chapter.
-
-Output in byte|-|sized chunks can be achieved by using characters just outside of
-the valid \UNICODE\ range, starting at the value $1{,}114{,}112$ (0x110000). When
-the time comes to print a character $c>=1{,}114{,}112$, \LUATEX\ will actually
-print the single byte corresponding to $c$ minus 1{,}114{,}112.
-
-Output to the terminal uses \type {^^} notation for the lower control range
-($c<32$), with the exception of \type {^^I}, \type {^^J} and \type {^^M}. These
-are considered \quote {safe} and therefore printed as|-|is. You can disable
-escaping with \type {texio.setescape(false)} in which case you get the normal
-characters on the console.
-
-Normalization of the \UNICODE\ input can be handled by a macro package during
-callback processing (this will be explained in \in {section} [iocallback]).
-
-\subsection{\type {\Uchar}}
-
-The expandable command \type {\Uchar} reads a number between~0 and $1{,}114{,}111$
-and expands to the associated \UNICODE\ character.
-
-\section{Extended tables}
-
-All traditional \TEX\ and \ETEX\ registers can be 16-bit numbers. The affected
-commands are:
-
-\startfourcolumns
-\starttyping
-\count
-\dimen
-\skip
-\muskip
-\marks
-\toks
-\countdef
-\dimendef
-\skipdef
-\muskipdef
-\toksdef
-\insert
-\box
-\unhbox
-\unvbox
-\copy
-\unhcopy
-\unvcopy
-\wd
-\ht
-\dp
-\setbox
-\vsplit
-\stoptyping
-\stopfourcolumns
-
-Because font memory management has been rewritten, character properties in fonts
-are no longer shared among fonts instances that originate from the same metric
-file.
-
-\section{Attributes}
-
-\subsection{Attribute registers}
-
-Attributes are a completely new concept in \LUATEX. Syntactically, they behave a
-lot like counters: attributes obey \TEX's nesting stack and can be used after
-\type {\the} etc.\ just like the normal \type {\count} registers.
-
-\startsyntax
-\attribute <16-bit number> <optional equals> <32-bit number>!crlf
-\attributedef <csname> <optional equals> <16-bit number>
-\stopsyntax
-
-Conceptually, an attribute is either \quote {set} or \quote {unset}. Unset
-attributes have a special negative value to indicate that they are unset, that
-value is the lowest legal value: \type {-"7FFFFFFF} in hexadecimal, a.k.a.
-$-2147483647$ in decimal. It follows that the value \type {-"7FFFFFFF} cannot be
-used as a legal attribute value, but you {\it can\/} assign \type {-"7FFFFFFF} to
-\quote {unset} an attribute. All attributes start out in this \quote {unset}
-state in \INITEX.
-
-Attributes can be used as extra counter values, but their usefulness comes mostly
-from the fact that the numbers and values of all \quote {set} attributes are
-attached to all nodes created in their scope. These can then be queried from any
-\LUA\ code that deals with node processing. Further information about how to use
-attributes for node list processing from \LUA\ is given in~\in {chapter}[nodes].
-
-Attributes are stored in a sorted (sparse) linked list that are shared when
-possible. This permits efficient testing and updating.
-
-\subsection{Box attributes}
-
-Nodes typically receive the list of attributes that is in effect when they are
-created. This moment can be quite asynchronous. For example: in paragraph
-building, the individual line boxes are created after the \type {\par} command has
-been processed, so they will receive the list of attributes that is in effect
-then, not the attributes that were in effect in, say, the first or third line of
-the paragraph.
-
-Similar situations happen in \LUATEX\ regularly. A few of the more obvious
-problematic cases are dealt with: the attributes for nodes that are created
-during hyphenation, kerning and ligaturing borrow their attributes from their
-surrounding glyphs, and it is possible to influence box attributes directly.
-
-When you assemble a box in a register, the attributes of the nodes contained in
-the box are unchanged when such a box is placed, unboxed, or copied. In this
-respect attributes act the same as characters that have been converted to
-references to glyphs in fonts. For instance, when you use attributes to implement
-color support, each node carries information about its eventual color. In that
-case, unless you implement mechanisms that deal with it, applying a color to
-already boxed material will have no effect. Keep in mind that this
-incompatibility is mostly due to the fact that separate specials and literals are
-a more unnatural approach to colors than attributes.
-
-It is possible to fine-tune the list of attributes that are applied to a \type
-{hbox}, \type {vbox} or \type {vtop} by the use of the keyword \type {attr}. An
-example:
-
-\starttyping
-\attribute2=5
-\setbox0=\hbox {Hello}
-\setbox2=\hbox attr1=12 attr2=-"7FFFFFFF{Hello}
-\stoptyping
-
-This will set the attribute list of box~2 to $1=12$, and the attributes of box~0
-will be $2=5$. As you can see, assigning the maximum negative value causes an
-attribute to be ignored.
-
-The \type {attr} keyword(s) should come before a \type {to} or \type {spread}, if
-that is also specified.
-
-\section{\LUA\ related primitives}
-
-\subsection{\type {\directlua}}
-
-In order to merge \LUA\ code with \TEX\ input, a few new primitives are needed.
-The primitive \type {\directlua} is used to execute \LUA\ code immediately. The
-syntax is
-
-\startsyntax
-\directlua <general text>!crlf
-\directlua <16-bit number> <general text>
-\stopsyntax
-
-The \syntax {<general text>} is expanded fully, and then fed into the \LUA\
-interpreter. After reading and expansion has been applied to the \syntax
-{<general text>}, the resulting token list is converted to a string as if it was
-displayed using \type {\the\toks}. On the \LUA\ side, each \type {\directlua}
-block is treated as a separate chunk. In such a chunk you can use the \type
-{local} directive to keep your variables from interfering with those used by the
-macro package.
-
-The conversion to and from a token list means that you normally can not use \LUA\
-line comments (starting with \type {--}) within the argument. As there typically
-will be only one \quote {line} the first line comment will run on until the end
-of the input. You will either need to use \TEX|-|style line comments (starting
-with \%), or change the \TEX\ category codes locally. Another possibility is to
-say:
-
-\starttyping
-\begingroup
-\endlinechar=10
-\directlua ...
-\endgroup
-\stoptyping
-
-Then \LUA\ line comments can be used, since \TEX\ does not replace line endings
-with spaces.
-
-Likewise, the \syntax {<16-bit number>} designates a name of a \LUA\ chunk and is
-taken from the \type {lua.name} array (see the documentation of the \type {lua}
-table further in this manual). When a chunk name starts with a \type {@} it will
-be displayed as a file name. This is a side effect of the way \LUA\ implements
-error handling.
-
-The \type {\directlua} command is expandable. Since it passes \LUA\ code to the
-\LUA\ interpreter its expansion from the \TEX\ viewpoint is usually empty.
-However, there are some \LUA\ functions that produce material to be read by \TEX,
-the so called print functions. The most simple use of these is \type
-{tex.print(<string> s)}. The characters of the string \type {s} will be placed on
-the \TEX\ input buffer, that is, \quote {before \TEX's eyes} to be read by \TEX\
-immediately. For example:
-
-\startbuffer
-\count10=20
-a\directlua{tex.print(tex.count[10]+5)}b
-\stopbuffer
-
-\typebuffer
-
-expands to
-
-\getbuffer
-
-Here is another example:
-
-\startbuffer
-$\pi = \directlua{tex.print(math.pi)}$
-\stopbuffer
-
-\typebuffer
-
-will result in
-
-\getbuffer
-
-Note that the expansion of \type {\directlua} is a sequence of characters, not of
-tokens, contrary to all \TEX\ commands. So formally speaking its expansion is
-null, but it places material on a pseudo-file to be immediately read by \TEX, as
-\ETEX's \type {\scantokens}. For a description of print functions look at \in
-{section} [sec:luaprint].
-
-Because the \syntax {<general text>} is a chunk, the normal \LUA\ error handling
-is triggered if there is a problem in the included code. The \LUA\ error messages
-should be clear enough, but the contextual information is still pretty bad.
-Often, you will only see the line number of the right brace at the end of the
-code.
-
-While on the subject of errors: some of the things you can do inside \LUA\ code
-can break up \LUATEX\ pretty bad. If you are not careful while working with the
-node list interface, you may even end up with assertion errors from within the
-\TEX\ portion of the executable.
-
-The behaviour documented in the above subsection is considered stable in the sense
-that there will not be backward-incompatible changes any more.
-
-\subsection{\type {\latelua}}
-
-Contrary to \type {\directlua}, \type {\latelua} stores \LUA\ code in a whatsit
-that will be processed at the time of shipping out. Its intended use is a cross
-between \PDF\ literals (often available as \type {\pdfliteral}) and the
-traditional \TEX\ extension \type {\write}. Within the \LUA\ code you can print
-\PDF\ statements directly to the \PDF\ file via \type {pdf.print}, or you can
-write to other output streams via \type {texio.write} or simply using \LUA\ \IO\
-routines.
-
-\startsyntax
-\latelua <general text>!crlf
-\latelua <16-bit number> <general text>
-\stopsyntax
-
-Expansion of macros in the final \type {<general text>} is delayed until just
-before the whatsit is executed (like in \type {\write}). With regard to \PDF\
-output stream \type {\latelua} behaves as \PDF\ page literals. The \syntax
-{name <general text>} and \syntax {<16-bit number>} behave in the same way as
-they do for \type {\directlua}
-
-\subsection{\type {\luaescapestring}}
-
-This primitive converts a \TEX\ token sequence so that it can be safely used as
-the contents of a \LUA\ string: embedded backslashes, double and single quotes,
-and newlines and carriage returns are escaped. This is done by prepending an
-extra token consisting of a backslash with category code~12, and for the line
-endings, converting them to \type {n} and \type {r} respectively. The token
-sequence is fully expanded.
-
-\startsyntax
-\luaescapestring <general text>
-\stopsyntax
-
-Most often, this command is not actually the best way to deal with the
-differences between the \TEX\ and \LUA. In very short bits of \LUA\
-code it is often not needed, and for longer stretches of \LUA\ code it
-is easier to keep the code in a separate file and load it using \LUA's
-\type {dofile}:
-
-\starttyping
-\directlua { dofile('mysetups.lua') }
-\stoptyping
-
-\subsection{\type {\luafunction}}
-
-The \type {\directlua} commands involves tokenization of its argument (after
-picking up an optional name or number specification). The tokenlist is then
-converted into a string and given to \LUA\ to turn into a function that is
-called. The overhead is rather small but when you use this primitive hundreds of
-thousands of times, it can become noticeable. For this reason there is a variant
-call available: \type {\luafunction}. This command is used as follows:
-
-\starttyping
-\directlua {
-    local t = lua.get_functions_table()
-    t[1] = function() tex.print("!") end
-    t[2] = function() tex.print("?") end
-}
-
-\luafunction1
-\luafunction2
-\stoptyping
-
-Of course the functions can also be defined in a separate file. There is no limit
-on the number of functions apart from normal \LUA\ limitations. Of course there
-is the limitation of no arguments but that would involve parsing and thereby give
-no gain. The function, when called in fact gets one argument, being the index, so
-in the following example the number \type {8} gets typeset.
-
-\starttyping
-\directlua {
-    local t = lua.get_functions_table()
-    t[8] = function(slot) tex.print(slot) end
-}
-\stoptyping
-
-\section {Alignments}
-
-\subsection{\tex {alignmark}}
-
-This primitive duplicates the functionality of \type {#} inside alignment
-preambles.
-
-\subsection{\tex {aligntab}}
-
-This primitive duplicates the functionality of \type {&} inside alignments and
-preambles.
-
-\section{Catcode tables}
-
-Catcode tables are a new feature that allows you to switch to a predefined
-catcode regime in a single statement. You can have a practically unlimited number
-of different tables. This subsystem is backward compatible: if you never use the
-following commands, your document will not notice any difference in behaviour
-compared to traditional \TEX. The contents of each catcode table is independent
-from any other catcode tables, and their contents is stored and retrieved from
-the format file.
-
-\subsection{\type {\catcodetable}}
-
-\startsyntax
-\catcodetable <15-bit number>
-\stopsyntax
-
-The primitive \type {\catcodetable} switches to a different catcode table. Such a
-table has to be previously created using one of the two primitives below, or it
-has to be zero. Table zero is initialized by \INITEX.
-
-\subsection{\type {\initcatcodetable}}
-
-\startsyntax
-\initcatcodetable <15-bit number>
-\stopsyntax
-
-The primitive \type {\initcatcodetable} creates a new table with catcodes identical
-to those defined by \INITEX:
-
-\starttabulate[|r|l|l|l|]
-\NC  0 \NC \tttf \letterbackslash       \NC         \NC \type {escape}       \NC\NR
-\NC  5 \NC \tttf \letterhat\letterhat M \NC return  \NC \type {car_ret}      \NC\NR
-\NC  9 \NC \tttf \letterhat\letterhat @ \NC null    \NC \type {ignore}       \NC\NR
-\NC 10 \NC \tttf <space>                \NC space   \NC \type {spacer}       \NC\NR
-\NC 11 \NC {\tttf a} \endash\ {\tttf z} \NC         \NC \type {letter}       \NC\NR
-\NC 11 \NC {\tttf A} \endash\ {\tttf Z} \NC         \NC \type {letter}       \NC\NR
-\NC 12 \NC everything else              \NC         \NC \type {other}        \NC\NR
-\NC 14 \NC \tttf \letterpercent         \NC         \NC \type {comment}      \NC\NR
-\NC 15 \NC \tttf \letterhat\letterhat ? \NC delete  \NC \type {invalid_char} \NC\NR
-\stoptabulate
-
-The new catcode table is allocated globally: it will not go away after the
-current group has ended. If the supplied number is identical to the currently
-active table, an error is raised.
-
-\subsection{\type {\savecatcodetable}}
-
-\startsyntax
-\savecatcodetable <15-bit number>
-\stopsyntax
-
-\type {\savecatcodetable} copies the current set of catcodes to a new table with
-the requested number. The definitions in this new table are all treated as if
-they were made in the outermost level.
-
-The new table is allocated globally: it will not go away after the current group
-has ended. If the supplied number is the currently active table, an error is
-raised.
-
-\section{Suppressing errors}
-
-\subsection{\type {\suppressfontnotfounderror}}
-
-\startsyntax
-\suppressfontnotfounderror = 1
-\stopsyntax
-
-If this integer parameter is non|-|zero, then \LUATEX\ will not complain about
-font metrics that are not found. Instead it will silently skip the font
-assignment, making the requested csname for the font \type {\ifx} equal to \type
-{\nullfont}, so that it can be tested against that without bothering the user.
-
-\subsection{\type {\suppresslongerror}}
-
-\startsyntax
-\suppresslongerror = 1
-\stopsyntax
-
-If this integer parameter is non|-|zero, then \LUATEX\ will not complain about
-\type {\par} commands encountered in contexts where that is normally prohibited
-(most prominently in the arguments of non-long macros).
-
-\subsection{\type {\suppressifcsnameerror}}
-
-\startsyntax
-\suppressifcsnameerror = 1
-\stopsyntax
-
-If this integer parameter is non|-|zero, then \LUATEX\ will not complain about
-non-expandable commands appearing in the middle of a \type {\ifcsname} expansion.
-Instead, it will keep getting expanded tokens from the input until it encounters
-an \type {\endcsname} command. If the input expansion is unbalanced with respect
-to \type {\csname} \ldots \type {\endcsname} pairs, the \LUATEX\ process may hang
-indefinitely.
-
-\subsection{\type {\suppressoutererror}}
-
-\startsyntax
-\suppressoutererror = 1
-\stopsyntax
-
-If this new integer parameter is non|-|zero, then \LUATEX\ will not complain
-about \type {\outer} commands encountered in contexts where that is normally
-prohibited.
-
-\subsection{\type {\suppressmathparerror}}
-
-The following setting will permit \type {\par} tokens in a math formula:
-
-\startsyntax
-\suppressmathparerror = 1
-\stopsyntax
-
-So, the next code is valid then:
-
-\starttyping
-$ x + 1 =
-
-a $
-\stoptyping
-
-\subsection{\type {\suppressprimitiveerror}}
-
-When set to a non|-|zero value the following command will not issue an error:
-
-\starttyping
-\suppressprimitiveerror = 1
-
-\primitive\notaprimitive
-\stoptyping
-
-\section {Math}
-
-\subsection{Extensions}
-
-We will cover math in its own chapter because not only the font subsystem and
-spacing model have been enhanced (thereby introducing many new primitives) but
-also because some more control has been added to existing functionality.
-
-\subsection{\type {\matheqnogapstep}}
-
-By default \TEX\ will add one quad between the equation and the number. This is
-hard coded. A new primitive can control this:
-
-\startsyntax
-\matheqnogapstep = 1000
-\stopsyntax
-
-Because a math quad from the math text font is used instead of a dimension, we
-use a step to control the size. A value of zero will suppress the gap. The step
-is divided by 1000 which is the usual way to mimmick floating point factors in
-\TEX.
-
-\section{Fonts}
-
-\subsection{Font syntax}
-
-\LUATEX\ will accept a braced argument as a font name:
-
-\starttyping
-\font\myfont = {cmr10}
-\stoptyping
-
-This allows for embedded spaces, without the need for double quotes. Macro
-expansion takes place inside the argument.
-
-\subsection{\type {\fontid}}
-
-\startsyntax
-\fontid\font
-\stopsyntax
-
-This primitive expands into a number. It is not a register so there is no need to
-prefix with \type {\number} (and using \type {\the} gives an error). The currently
-used font id is \fontid\font. Here are some more:
-
-\starttabulate[|l|c|]
-\NC \type {\bf} \NC \bf \fontid\font \NC \NR
-\NC \type {\it} \NC \it \fontid\font \NC \NR
-\NC \type {\bi} \NC \bi \fontid\font \NC \NR
-\stoptabulate
-
-These numbers depend on the macro package used because each one has its own way
-of dealing with fonts. They can also differ per run, as they can depend on the
-order of loading fonts. For instance, when in \CONTEXT\ virtual math \UNICODE\
-fonts are used, we can easily get over a hundred ids in use. Not all ids have to
-be bound to a real font, after all it's just a number.
-
-\subsection{\type {\setfontid}}
-
-The primitive \type {\setfontid} can be used to enable a font with the given id
-(which of course needs to be a valid one).
-
-\subsection{\type {\noligs} and \type {\nokerns}}
-
-These primitives prohibit ligature and kerning insertion at the time when the
-initial node list is built by \LUATEX's main control loop. You can enable these
-primitives when you want to do node list processing of \quote {characters}, where
-\TEX's normal processing would get in the way.
-
-\startsyntax
-\noligs <integer>!crlf
-\nokerns <integer>
-\stopsyntax
-
-These primitives can also be implemented by overloading the ligature building and
-kerning functions, i.e.\ by assigning dummy functions to their associated
-callbacks. Keep in mind that when you define a font (using \LUA) you can also
-omit the kern and ligature tables, which has the same effect as the above.
-
-\subsection{\type{\nospaces}}
-
-This new primitive can be used to overrule the usual \type {\spaceskip}
-related heuristics when a space character is seen in a text flow. The
-value~\type{1} triggers no injection while \type{2} results in injection of
-a zero skip. Below we see the results for four characters separated by a
-space.
-
-\startlinecorrection
-\startcombination[3*2]
-    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 10mm}}
-    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 10mm}}
-    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 10mm}}
-    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 1mm}}
-    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 1mm}}
-    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 1mm}}
-\stopcombination
-\stoplinecorrection
-
-\section{Tokens, commands and strings}
-
-\subsection{\type {\scantextokens}}
-
-The syntax of \type {\scantextokens} is identical to \type {\scantokens}. This
-primitive is a slightly adapted version of \ETEX's \type {\scantokens}. The
-differences are:
-
-\startitemize
-\startitem
-    The last (and usually only) line does not have a \type {\endlinechar}
-    appended.
-\stopitem
-\startitem
-    \type {\scantextokens} never raises an EOF error, and it does not execute
-    \type {\everyeof} tokens.
-\stopitem
-\startitem
-    There are no \quote {\unknown\ while end of file \unknown} error tests
-    executed. This allows the expansion to end on a different grouping level or
-    while a conditional is still incomplete.
-\stopitem
-\stopitemize
-
-\subsection{\type {\toksapp}, \type {\tokspre}, \type {\etoksapp} and \type {\etokspre}}
-
-Instead of:
-
-\starttyping
-\toks0\expandafter{\the\toks0 foo}
-\stoptyping
-
-you can use:
-
-\starttyping
-\etoksapp0{foo}
-\stoptyping
-
-The \type {pre} variants prepend instead of append, and the \type {e} variants
-expand the passed general text.
-
-\subsection{\type {\csstring}, \type {\begincsname} and \type {\lastnamedcs}}
-
-These are somewhat special. The \type {\csstring} primitive is like
-\type {\string} but it omits the leading escape character. This can be
-somewhat more efficient that stripping it of afterwards.
-
-The \type {\begincsname} primitive is like \type {\csname} but doesn't create
-a relaxed equivalent when there is no such name. It is equivalent to
-
-\starttyping
-\ifcsname foo\endcsname
-  \csname foo\endcsname
-\fi
-\stoptyping
-
-The advantage is that it saves a lookup (don't expect much speedup) but more
-important is that it avoids using the \type {\if}.
-
-The \type {\lastnamedcs} is one that should be used with care. The above
-example could be written as:
-
-\starttyping
-\ifcsname foo\endcsname
-  \lastnamedcs
-\fi
-\stoptyping
-
-This is slightly more efficient than constructing the string twice (deep down in
-\LUATEX\ this also involves some \UTF8 juggling), but probably more relevant is
-that it saves a few tokens and can make code a bit more more readable.
-
-\subsection{\type {\clearmarks}}
-
-This primitive complements the \ETEX\ mark primitives and clears a mark class
-completely, resetting all three connected mark texts to empty. It is an
-immediate command.
-
-\startsyntax
-\clearmarks <16-bit number>
-\stopsyntax
-
-\subsection{\type{\letcharcode}}
-
-This primitive is still experimental but can be used to assign a meaning to an active
-character, as in:
-
-\starttyping
-\def\foo{bar} \letcharcode123=\foo
-\stoptyping
-
-This can be a bit nicer that using the uppercase tricks (using the property of
-\type {\uppercase} that it treats active characters special).
-
-\section{Boxes, rules and leaders}
-
-\subsection{\type {\outputbox}}
-
-\startsyntax
-\outputbox = 65535
-\stopsyntax
-
-This new integer parameter allows you to alter the number of the box that will be
-used to store the page sent to the output routine. Its default value is 255, and
-the acceptable range is from 0 to 65535.
-
-\subsection{\type {\vpack}, \type {\hpack} and \type {\tpack}}
-
-These three primitives are like \type {\vbox}, \type {\hbox} and \type {\vtop}
-but don't apply the related callbacks.
-
-\subsection{\type {\vsplit}}
-
-The \type {\vsplit} primitive has to be followed by a specification of the
-required height. As alternative for the \type {to} keyword you can use \type
-{upto} to get a split of the given size but result has the natural dimensions
-then.
-
-\subsection{Images and Forms}
-
-These two concepts are now core concepts and no longer whatsits. They are in fact
-now implemented as rules with special properties. Normal rules have subtype~0,
-saved boxes have subtype~1 and images have subtype~2. This has the positive side
-effect that whenever we need to take content with dimensions into account, when we
-look at rule nodes, we automatically also deal with these two types.
-
-The syntax of the \type {\save...resource} is the same as in \PDFTEX\ but you
-should consider them to be backend specific. This means that a macro package
-should treat them as such and check for the current output mode if applicable.
-Here are the equivalents:
-
-\starttabulate[|l|l|]
-\NC \type {\saveboxresource}             \EQ \type {\pdfxform}           \NC \NR
-\NC \type {\saveimageresource}           \EQ \type {\pdfximage}          \NC \NR
-\NC \type {\useboxresource}              \EQ \type {\pdfrefxform}        \NC \NR
-\NC \type {\useimageresource}            \EQ \type {\pdfrefximage}       \NC \NR
-\NC \type {\lastsavedboxresourceindex}   \EQ \type {\pdflastxform}       \NC \NR
-\NC \type {\lastsavedimageresourceindex} \EQ \type {\pdflastximage}      \NC \NR
-\NC \type {\lastsavedimageresourcepages} \EQ \type {\pdflastximagepages} \NC \NR
-\stoptabulate
-
-\LUATEX\ accepts optional dimension parameters for \type {\use...resource} in the
-same format as for rules. With images, these dimensions are then used instead of
-the ones given to \type {\useimageresource} but the original dimensions are not
-overwritten, so that a \type {\useimageresource} without dimensions still
-provides the image with dimensions defined by \type {\saveimageresource}. These
-optional parameters are not implemented for \type {\saveboxresource}.
-
-\starttyping
-\useimageresource width 20mm height 10mm depth 5mm \lastsavedimageresourceindex
-\useboxresource   width 20mm height 10mm depth 5mm \lastsavedboxresourceindex
-\stoptyping
-
-The box resources are of course implemented in the backend and therefore we do
-support the \type {attr} and \type {resources} keys that accept a token list. New
-is the \type {type} key. When set to non|-|zero the \type {/Type} entry is
-omitted. A value of 1 or 3 still writes a \type {/BBox}, while 2 or 3 will write
-a \type {/Matrix}.
-
-\subsection{\type {\nohrule} and \type {\novrule}}
-
-Because introducing a new keyword can cause incompatibilities, two new primitives
-were introduced: \type {\nohrule} and \type {\novrule}. These can be used to
-reserve space. This is often more efficient than creating an empty box with fake
-dimensions).
-
-\subsection{\type {\gleaders}}
-
-This type of leaders is anchored to the origin of the box to be shipped out. So
-they are like normal \type {\leaders} in that they align nicely, except that the
-alignment is based on the {\it largest\/} enclosing box instead of the {\it
-smallest\/}. The \type {g} stresses this global nature.
-
-\section {Languages}
-
-\subsection{\type {\hyphenationmin}}
-
-This primitive can be used to set the minimal word length, so setting it to a value
-of~$5$ means that only words of 6 characters and more will be hyphenated, of course
-within the constraints of the \type {\lefthyphenmin} and \type {\righthyphenmin}
-values (as stored in the glyph node). This primitive accepts a number and stores
-the value with the language.
-
-\subsection{\type {\boundary}, \type {\noboundary}, \type {\protrusionboundary} and \type
-{\wordboundary}}
-
-The \type {\noboundary} commands used to inject a whatsit node but now injects a normal
-node with type \type {boundary} and subtype~0. In addition you can say:
-
-\starttyping
-x\boundary 123\relax y
-\stoptyping
-
-This has the same effect but the subtype is now~1 and the value~123 is stored.
-The traditional ligature builder still sees this as a cancel boundary directive
-but at the \LUA\ end you can implement different behaviour. The added benefit of
-passing this value is a side effect of the generalization. The subtypes~2 and~3
-are used to control protrusion and word boundaries in hyphenation.
-
-\section{Control and debugging}
-
-\subsection {Tracing}
-
-If \type {\tracingonline} is larger than~2, the node list display will also print
-the node number of the nodes.
-
-\subsection{\type {\outputmode} and \type {\draftmode}}
-
-The \type {\outputmode} variable tells \LUATEX\ what it has to produce:
-
-\starttabulate[|l|l|]
-\NC \type {0} \NC \DVI\ code \NC \NR
-\NC \type {1} \NC \PDF\ code \NC \NR
-\stoptabulate
-
-The value of the \type {\draftmode} counter signals the backend if it should
-output less. The \PDF\ backend accepts a value of~$1$, while the \DVI\ backend
-ignores the value.
-
-\section {Files}
-
-\subsection{File syntax}
-
-\LUATEX\ will accept a braced argument as a file name:
-
-\starttyping
-\input {plain}
-\openin 0 {plain}
-\stoptyping
-
-This allows for embedded spaces, without the need for double quotes. Macro
-expansion takes place inside the argument.
-
-The \type {\tracingfonts} primitive that has been inherited from \PDFTEX\ has
-been adapted to support variants in reporting the font. The reason for this
-extension is that a csname not always makes sense. The zero case is the default.
-
-\starttabulate[|T||]
-\NC 0 \EQ \type{\foo xyz} \NC \NR
-\NC 1 \EQ \type{\foo (bar)} \NC \NR
-\NC 2 \EQ \type{<bar> xyz} \NC \NR
-\NC 3 \EQ \type{<bar @ ..pt> xyz} \NC \NR
-\NC 4 \EQ \type{<id>} \NC \NR
-\NC 5 \EQ \type{<id: bar>} \NC \NR
-\NC 6 \EQ \type{<id: bar @ ..pt> xyz} \NC \NR
-\stoptabulate
-
-\subsection{Writing to file}
-
-You can now open upto 127 files with \type {\openout}. When no file is open
-writes will go to the console and log. As a consequence a system command is
-no longer possible but one can use \type {os.execute} to do the same.
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-enhancements

+

+\startchapter[reference=enhancements,title={Basic \TEX\ enhancements}]

+

+\startsection[title={Introduction}]

+

+\startsubsection[title={Primitive behaviour}]

+

+From day one, \LUATEX\ has offered extra features compared to the superset of

+\PDFTEX, which includes \ETEX, and \ALEPH. This has not been limited to the

+possibility to execute \LUA\ code via \prm {directlua}, but \LUATEX\ also adds

+functionality via new \TEX|-|side primitives or extensions to existing ones.

+

+When \LUATEX\ starts up in \quote {iniluatex} mode (\type {luatex -ini}), it

+defines only the primitive commands known by \TEX82 and the one extra command

+\prm {directlua}. As is fitting, a \LUA\ function has to be called to add the

+extra primitives to the user environment. The simplest method to get access to

+all of the new primitive commands is by adding this line to the format generation

+file:

+

+\starttyping

+\directlua { tex.enableprimitives('',tex.extraprimitives()) }

+\stoptyping

+

+But be aware that the curly braces may not have the proper \prm {catcode}

+assigned to them at this early time (giving a \quote {Missing number} error), so

+it may be needed to put these assignments before the above line:

+

+\starttyping

+\catcode `\{=1

+\catcode `\}=2

+\stoptyping

+

+More fine|-|grained primitives control is possible and you can look up the

+details in \in {section} [luaprimitives]. For simplicity's sake, this manual

+assumes that you have executed the \prm {directlua} command as given above.

+

+The startup behaviour documented above is considered stable in the sense that

+there will not be backward|-|incompatible changes any more. We have promoted some

+rather generic \PDFTEX\ primitives to core \LUATEX\ ones, and the few that we

+inherited from \ALEPH\ (\OMEGA) are also promoted. Effectively this means that we

+now only have the \type {tex}, \type {etex} and \type {luatex} sets left.

+

+In \in {Chapter} [modifications] we discuss several primitives that are derived

+from \PDFTEX\ and \ALEPH\ (\OMEGA). Here we stick to real new ones. In the

+chapters on fonts and math we discuss a few more new ones.

+

+\stopsubsection

+

+\startsubsection[title={Version information}]

+

+\startsubsubsection[title={\lpr {luatexbanner}, \lpr {luatexversion} and \lpr {luatexrevision}}]

+

+\topicindex{version}

+\topicindex{banner}

+

+There are three new primitives to test the version of \LUATEX:

+

+\unexpanded\def\VersionHack#1% otherwise different luatex and luajittex runs

+  {\cldcontext{(string.gsub(string.match("\luatexbanner","(.+)\letterpercent("),"jit",""))}}

+

+\starttabulate[|l|l|pl|]

+\DB primitive             \BC value

+                          \BC explanation \NC \NR

+\TB

+\NC \lpr {luatexbanner}   \NC \VersionHack{\luatexbanner}

+                          \NC the banner reported on the command line \NC \NR

+\NC \lpr {luatexversion}  \NC \the\luatexversion

+                          \NC a combination of major and minor number \NC \NR

+\NC \lpr {luatexrevision} \NC \luatexrevision

+                          \NC the revision number, the current value is \NC \NR

+\LL

+\stoptabulate

+

+The official \LUATEX\ version is defined as follows:

+

+\startitemize

+\startitem

+    The major version is the integer result of \lpr {luatexversion} divided by

+    100. The primitive is an \quote {internal variable}, so you may need to prefix

+    its use with \prm {the} depending on the context.

+\stopitem

+\startitem

+    The minor version is the two|-|digit result of \lpr {luatexversion} modulo 100.

+\stopitem

+\startitem

+    The revision is reported by \lpr {luatexrevision}. This primitive expands to

+    a positive integer.

+\stopitem

+\startitem

+    The full version number consists of the major version, minor version and

+    revision, separated by dots.

+\stopitem

+\stopitemize

+

+\stopsubsubsection

+

+\startsubsubsection[title={\lpr {formatname}}]

+

+\topicindex{format}

+

+The \lpr {formatname} syntax is identical to \prm {jobname}. In \INITEX, the

+expansion is empty. Otherwise, the expansion is the value that \prm {jobname} had

+during the \INITEX\ run that dumped the currently loaded format. You can use this

+token list to provide your own version info.

+

+\stopsubsubsection

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={\UNICODE\ text support}]

+

+\startsubsection[title={Extended ranges}]

+

+\topicindex{\UNICODE}

+

+Text input and output is now considered to be \UNICODE\ text, so input characters

+can use the full range of \UNICODE\ ($2^{20}+2^{16}-1 = \hbox{0x10FFFF}$). Later

+chapters will talk of characters and glyphs. Although these are not

+interchangeable, they are closely related. During typesetting, a character is

+always converted to a suitable graphic representation of that character in a

+specific font. However, while processing a list of to|-|be|-|typeset nodes, its

+contents may still be seen as a character. Inside \LUATEX\ there is no clear

+separation between the two concepts. Because the subtype of a glyph node can be

+changed in \LUA\ it is up to the user. Subtypes larger than 255 indicate that

+font processing has happened.

+

+A few primitives are affected by this, all in a similar fashion: each of them has

+to accommodate for a larger range of acceptable numbers. For instance, \prm

+{char} now accepts values between~0 and $1{,}114{,}111$. This should not be a

+problem for well|-|behaved input files, but it could create incompatibilities for

+input that would have generated an error when processed by older \TEX|-|based

+engines. The affected commands with an altered initial (left of the equal sign)

+or secondary (right of the equal sign) value are: \prm {char}, \prm {lccode},

+\prm {uccode}, \lpr {hjcode}, \prm {catcode}, \prm {sfcode}, \lpr {efcode}, \lpr

+{lpcode}, \lpr {rpcode}, \prm {chardef}.

+

+As far as the core engine is concerned, all input and output to text files is

+\UTF-8 encoded. Input files can be pre|-|processed using the \type {reader}

+callback. This will be explained in \in {section} [iocallback]. Normalization of

+the \UNICODE\ input is on purpose not built|-|in and can be handled by a macro

+package during callback processing. We have made some practical choices and the

+user has to live with those.

+

+Output in byte|-|sized chunks can be achieved by using characters just outside of

+the valid \UNICODE\ range, starting at the value $1{,}114{,}112$ (0x110000). When

+the time comes to print a character $c>=1{,}114{,}112$, \LUATEX\ will actually

+print the single byte corresponding to $c$ minus 1{,}114{,}112.

+

+Output to the terminal uses \type {^^} notation for the lower control range

+($c<32$), with the exception of \type {^^I}, \type {^^J} and \type {^^M}. These

+are considered \quote {safe} and therefore printed as|-|is. You can disable

+escaping with \type {texio.setescape(false)} in which case you get the normal

+characters on the console.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {Uchar}}]

+

+\topicindex{\UNICODE}

+

+The expandable command \lpr {Uchar} reads a number between~0 and $1{,}114{,}111$

+and expands to the associated \UNICODE\ character.

+

+\stopsubsection

+

+\startsubsection[title={Extended tables}]

+

+All traditional \TEX\ and \ETEX\ registers can be 16-bit numbers. The affected

+commands are:

+

+\startfourcolumns

+\startlines

+\prm {count}

+\prm {dimen}

+\prm {skip}

+\prm {muskip}

+\prm {marks}

+\prm {toks}

+\prm {countdef}

+\prm {dimendef}

+\prm {skipdef}

+\prm {muskipdef}

+\prm {toksdef}

+\prm {insert}

+\prm {box}

+\prm {unhbox}

+\prm {unvbox}

+\prm {copy}

+\prm {unhcopy}

+\prm {unvcopy}

+\prm {wd}

+\prm {ht}

+\prm {dp}

+\prm {setbox}

+\prm {vsplit}

+\stoplines

+\stopfourcolumns

+

+Because font memory management has been rewritten, character properties in fonts

+are no longer shared among font instances that originate from the same metric

+file. Of course we share fonts in the backend when possible so that the resulting

+\PDF\ file is as efficient as possible, but for instance also expansion and

+protrusion no longer use copies as in \PDFTEX.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Attributes}]

+

+\startsubsection[title={Nodes}]

+

+\topicindex {nodes}

+

+When \TEX\ reads input it will interpret the stream according to the properties

+of the characters. Some signal a macro name and trigger expansion, others open

+and close groups, trigger math mode, etc. What's left over becomes the typeset

+text. Internally we get linked list of nodes. Characters become \nod {glyph}

+nodes that have for instance a \type {font} and \type {char} property and \typ

+{\kern 10pt} becomes a \nod {kern} node with a \type {width} property. Spaces are

+alien to \TEX\ as they are turned into \nod {glue} nodes. So, a simple paragraph

+is mostly a mix of sequences of \nod {glyph} nodes (words) and \nod {glue} nodes

+(spaces).

+

+The sequences of characters at some point are extended with \nod {disc} nodes

+that relate to hyphenation. After that font logic can be applied and we get a

+list where some characters can be replaced, for instance multiple characters can

+become one ligature, and font kerns can be injected. This is driven by the

+font properties.

+

+Boxes (like \prm {hbox} and \prm {vbox}) become \nod {hlist} or \nod {vlist}

+nodes with \type {width}, \type {height}, \type {depth} and \type {shift}

+properties and a pointer \type {list} to its actual content. Boxes can be

+constructed explicitly or can be the result of subprocesses. For instance, when

+lines are broken into paragraphs, the lines are a linked list of \nod {hlist}

+nodes.

+

+So, to summarize: all that you enter as content eventually becomes a node, often

+as part of a (nested) list structure. They have a relative small memory footprint

+and carry only the minimal amount of information needed. In traditional \TEX\ a

+character node only held the font and slot number, in \LUATEX\ we also store some

+language related information, the expansion factor, etc. Now that we have access

+to these nodes from \LUA\ it makes sense to be able to carry more information

+with an node and this is where attributes kick in.

+

+\stopsubsection

+

+\startsubsection[title={Attribute registers}]

+

+\topicindex {attributes}

+

+Attributes are a completely new concept in \LUATEX. Syntactically, they behave a

+lot like counters: attributes obey \TEX's nesting stack and can be used after

+\prm {the} etc.\ just like the normal \prm {count} registers.

+

+\startsyntax

+\attribute <16-bit number> <optional equals> <32-bit number>!crlf

+\attributedef <csname> <optional equals> <16-bit number>

+\stopsyntax

+

+Conceptually, an attribute is either \quote {set} or \quote {unset}. Unset

+attributes have a special negative value to indicate that they are unset, that

+value is the lowest legal value: \type {-"7FFFFFFF} in hexadecimal, a.k.a.

+$-2147483647$ in decimal. It follows that the value \type {-"7FFFFFFF} cannot be

+used as a legal attribute value, but you {\it can\/} assign \type {-"7FFFFFFF} to

+\quote {unset} an attribute. All attributes start out in this \quote {unset}

+state in \INITEX.

+

+Attributes can be used as extra counter values, but their usefulness comes mostly

+from the fact that the numbers and values of all \quote {set} attributes are

+attached to all nodes created in their scope. These can then be queried from any

+\LUA\ code that deals with node processing. Further information about how to use

+attributes for node list processing from \LUA\ is given in~\in {chapter}[nodes].

+

+Attributes are stored in a sorted (sparse) linked list that are shared when

+possible. This permits efficient testing and updating. You can define many

+thousands of attributes but normally such a large number makes no sense and is

+also not that efficient because each node carries a (possibly shared) link to a

+list of currently set attributes. But they are a convenient extension and one of

+the first extensions we implemented in \LUATEX.

+

+\stopsubsection

+

+\startsubsection[title={Box attributes}]

+

+\topicindex {attributes}

+\topicindex {boxes}

+

+Nodes typically receive the list of attributes that is in effect when they are

+created. This moment can be quite asynchronous. For example: in paragraph

+building, the individual line boxes are created after the \prm {par} command has

+been processed, so they will receive the list of attributes that is in effect

+then, not the attributes that were in effect in, say, the first or third line of

+the paragraph.

+

+Similar situations happen in \LUATEX\ regularly. A few of the more obvious

+problematic cases are dealt with: the attributes for nodes that are created

+during hyphenation, kerning and ligaturing borrow their attributes from their

+surrounding glyphs, and it is possible to influence box attributes directly.

+

+When you assemble a box in a register, the attributes of the nodes contained in

+the box are unchanged when such a box is placed, unboxed, or copied. In this

+respect attributes act the same as characters that have been converted to

+references to glyphs in fonts. For instance, when you use attributes to implement

+color support, each node carries information about its eventual color. In that

+case, unless you implement mechanisms that deal with it, applying a color to

+already boxed material will have no effect. Keep in mind that this

+incompatibility is mostly due to the fact that separate specials and literals are

+a more unnatural approach to colors than attributes.

+

+It is possible to fine-tune the list of attributes that are applied to a \type

+{hbox}, \type {vbox} or \type {vtop} by the use of the keyword \type {attr}. The

+\type {attr} keyword(s) should come before a \type {to} or \type {spread}, if

+that is also specified. An example is:

+

+\startbuffer[tex]

+\attribute997=123

+\attribute998=456

+\setbox0=\hbox {Hello}

+\setbox2=\hbox attr 999 = 789 attr 998 = -"7FFFFFFF{Hello}

+\stopbuffer

+

+\startbuffer[lua]

+  for b=0,2,2 do

+    for a=997, 999 do

+      tex.sprint("box ", b, " : attr ",a," : ",tostring(tex.box[b]     [a]))

+      tex.sprint("\\quad\\quad")

+      tex.sprint("list ",b, " : attr ",a," : ",tostring(tex.box[b].list[a]))

+      tex.sprint("\\par")

+    end

+  end

+\stopbuffer

+

+\typebuffer[tex]

+

+Box 0 now has attributes 997 and 998 set while box 2 has attributes 997 and 999

+set while the nodes inside that box will all have attributes 997 and 998 set.

+Assigning the maximum negative value causes an attribute to be ignored.

+

+To give you an idea of what this means at the \LUA\ end, take the following

+code:

+

+\typebuffer[lua]

+

+Later we will see that you can access properties of a node. The boxes here are so

+called \nod {hlist} nodes that have a field \type {list} that points to the

+content. Because the attributes are a list themselves you can access them by

+indexing the node (here we do that with \type {[a]}. Running this snippet gives:

+

+\start

+    \getbuffer[tex]

+    \startpacked \tt

+        \ctxluabuffer[lua]

+    \stoppacked

+\stop

+

+Because some values are not set we need to apply the \type {tostring} function

+here so that we get the word \type {nil}.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={\LUA\ related primitives}]

+

+\startsubsection[title={\prm {directlua}}]

+

+In order to merge \LUA\ code with \TEX\ input, a few new primitives are needed.

+The primitive \prm {directlua} is used to execute \LUA\ code immediately. The

+syntax is

+

+\startsyntax

+\directlua <general text>!crlf

+\directlua <16-bit number> <general text>

+\stopsyntax

+

+The \syntax {<general text>} is expanded fully, and then fed into the \LUA\

+interpreter. After reading and expansion has been applied to the \syntax

+{<general text>}, the resulting token list is converted to a string as if it was

+displayed using \type {\the\toks}. On the \LUA\ side, each \prm {directlua} block

+is treated as a separate chunk. In such a chunk you can use the \type {local}

+directive to keep your variables from interfering with those used by the macro

+package.

+

+The conversion to and from a token list means that you normally can not use \LUA\

+line comments (starting with \type {--}) within the argument. As there typically

+will be only one \quote {line} the first line comment will run on until the end

+of the input. You will either need to use \TEX|-|style line comments (starting

+with \%), or change the \TEX\ category codes locally. Another possibility is to

+say:

+

+\starttyping

+\begingroup

+\endlinechar=10

+\directlua ...

+\endgroup

+\stoptyping

+

+Then \LUA\ line comments can be used, since \TEX\ does not replace line endings

+with spaces. Of course such an approach depends on the macro package that you

+use.

+

+The \syntax {<16-bit number>} designates a name of a \LUA\ chunk and is

+taken from the \type {lua.name} array (see the documentation of the \type {lua}

+table further in this manual). When a chunk name starts with a \type {@} it will

+be displayed as a file name. This is a side effect of the way \LUA\ implements

+error handling.

+

+The \prm {directlua} command is expandable. Since it passes \LUA\ code to the

+\LUA\ interpreter its expansion from the \TEX\ viewpoint is usually empty.

+However, there are some \LUA\ functions that produce material to be read by \TEX,

+the so called print functions. The most simple use of these is \type

+{tex.print(<string> s)}. The characters of the string \type {s} will be placed on

+the \TEX\ input buffer, that is, \quote {before \TEX's eyes} to be read by \TEX\

+immediately. For example:

+

+\startbuffer

+\count10=20

+a\directlua{tex.print(tex.count[10]+5)}b

+\stopbuffer

+

+\typebuffer

+

+expands to

+

+\getbuffer

+

+Here is another example:

+

+\startbuffer

+$\pi = \directlua{tex.print(math.pi)}$

+\stopbuffer

+

+\typebuffer

+

+will result in

+

+\getbuffer

+

+Note that the expansion of \prm {directlua} is a sequence of characters, not of

+tokens, contrary to all \TEX\ commands. So formally speaking its expansion is

+null, but it places material on a pseudo-file to be immediately read by \TEX, as

+\ETEX's \prm {scantokens}. For a description of print functions look at \in

+{section} [sec:luaprint].

+

+Because the \syntax {<general text>} is a chunk, the normal \LUA\ error handling

+is triggered if there is a problem in the included code. The \LUA\ error messages

+should be clear enough, but the contextual information is still pretty bad.

+Often, you will only see the line number of the right brace at the end of the

+code.

+

+While on the subject of errors: some of the things you can do inside \LUA\ code

+can break up \LUATEX\ pretty bad. If you are not careful while working with the

+node list interface, you may even end up with assertion errors from within the

+\TEX\ portion of the executable.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {latelua} and \lpr {lateluafunction}}]

+

+Contrary to \prm {directlua}, \lpr {latelua} stores \LUA\ code in a whatsit

+that will be processed at the time of shipping out. Its intended use is a cross

+between \PDF\ literals (often available as \orm {pdfliteral}) and the

+traditional \TEX\ extension \prm {write}. Within the \LUA\ code you can print

+\PDF\ statements directly to the \PDF\ file via \type {pdf.print}, or you can

+write to other output streams via \type {texio.write} or simply using \LUA\ \IO\

+routines.

+

+\startsyntax

+\latelua <general text>!crlf

+\latelua <16-bit number> <general text>

+\stopsyntax

+

+Expansion of macros in the final \type {<general text>} is delayed until just

+before the whatsit is executed (like in \prm {write}). With regard to \PDF\

+output stream \lpr {latelua} behaves as \PDF\ page literals. The \syntax

+{name <general text>} and \syntax {<16-bit number>} behave in the same way as

+they do for \prm {directlua}.

+

+The \lpr {lateluafunction} primitive takes a number and is similar to \lpr

+{luafunction} but gets delated to shipout time. It's just there for completeness.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {luaescapestring}}]

+

+\topicindex {escaping}

+

+This primitive converts a \TEX\ token sequence so that it can be safely used as

+the contents of a \LUA\ string: embedded backslashes, double and single quotes,

+and newlines and carriage returns are escaped. This is done by prepending an

+extra token consisting of a backslash with category code~12, and for the line

+endings, converting them to \type {n} and \type {r} respectively. The token

+sequence is fully expanded.

+

+\startsyntax

+\luaescapestring <general text>

+\stopsyntax

+

+Most often, this command is not actually the best way to deal with the

+differences between \TEX\ and \LUA. In very short bits of \LUA\ code it is often

+not needed, and for longer stretches of \LUA\ code it is easier to keep the code

+in a separate file and load it using \LUA's \type {dofile}:

+

+\starttyping

+\directlua { dofile('mysetups.lua') }

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\lpr {luafunction}, \lpr {luafunctioncall} and \lpr {luadef}}]

+

+The \prm {directlua} commands involves tokenization of its argument (after

+picking up an optional name or number specification). The tokenlist is then

+converted into a string and given to \LUA\ to turn into a function that is

+called. The overhead is rather small but when you have millions of calls it can

+have some impact. For this reason there is a variant call available: \lpr

+{luafunction}. This command is used as follows:

+

+\starttyping

+\directlua {

+    local t = lua.get_functions_table()

+    t[1] = function() tex.print("!") end

+    t[2] = function() tex.print("?") end

+}

+

+\luafunction1

+\luafunction2

+\stoptyping

+

+Of course the functions can also be defined in a separate file. There is no limit

+on the number of functions apart from normal \LUA\ limitations. Of course there

+is the limitation of no arguments but that would involve parsing and thereby give

+no gain. The function, when called in fact gets one argument, being the index, so

+in the following example the number \type {8} gets typeset.

+

+\starttyping

+\directlua {

+    local t = lua.get_functions_table()

+    t[8] = function(slot) tex.print(slot) end

+}

+\stoptyping

+

+The \lpr {luafunctioncall} primitive does the same but is unexpandable, for

+instance in an \prm {edef}. In addition \LUATEX\ provides a definer:

+

+\starttyping

+                 \luadef\MyFunctionA 1

+          \global\luadef\MyFunctionB 2

+\protected\global\luadef\MyFunctionC 3

+\stoptyping

+

+You should really use these commands with care. Some references get stored in

+tokens and assume that the function is available when that token expands. On the

+other hand, as we have tested this functionality in relative complex situations

+normal usage should not give problems.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {luabytecode} and \lpr {luabytecodecall}}]

+

+Analogue to the function callers discussed in the previous section we have byte

+code callers. Again the call variant is unexpandable.

+

+\starttyping

+\directlua {

+    lua.bytecode[9998] = function(s)

+        tex.sprint(s*token.scan_int())

+    end

+    lua.bytecode[5555] = function(s)

+        tex.sprint(s*token.scan_dimen())

+    end

+}

+\stoptyping

+

+This works with:

+

+\starttyping

+\luabytecode    9998 5  \luabytecode    5555 5sp

+\luabytecodecall9998 5  \luabytecodecall5555 5sp

+\stoptyping

+

+The variable \type {s} in the code is the number of the byte code register that

+can be used for diagnostic purposes. The advantage of bytecode registers over

+function calls is that they are stored in the format (but without upvalues).

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Catcode tables}]

+

+\startsubsection[title={Catcodes}]

+

+\topicindex {catcodes}

+

+Catcode tables are a new feature that allows you to switch to a predefined

+catcode regime in a single statement. You can have a practically unlimited number

+of different tables. This subsystem is backward compatible: if you never use the

+following commands, your document will not notice any difference in behaviour

+compared to traditional \TEX. The contents of each catcode table is independent

+from any other catcode table, and its contents is stored and retrieved from the

+format file.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {catcodetable}}]

+

+\startsyntax

+\catcodetable <15-bit number>

+\stopsyntax

+

+The primitive \lpr {catcodetable} switches to a different catcode table. Such a

+table has to be previously created using one of the two primitives below, or it

+has to be zero. Table zero is initialized by \INITEX.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {initcatcodetable}}]

+

+\startsyntax

+\initcatcodetable <15-bit number>

+\stopsyntax

+

+The primitive \lpr {initcatcodetable} creates a new table with catcodes

+identical to those defined by \INITEX. The new catcode table is allocated

+globally: it will not go away after the current group has ended. If the supplied

+number is identical to the currently active table, an error is raised. The

+initial values are:

+

+\starttabulate[|c|c|l|l|]

+\DB catcode \BC character               \BC equivalent \BC category          \NC \NR

+\TB

+\NC  0 \NC \tttf \letterbackslash       \NC         \NC \type {escape}       \NC \NR

+\NC  5 \NC \tttf \letterhat\letterhat M \NC return  \NC \type {car_ret}      \NC \NR

+\NC  9 \NC \tttf \letterhat\letterhat @ \NC null    \NC \type {ignore}       \NC \NR

+\NC 10 \NC \tttf <space>                \NC space   \NC \type {spacer}       \NC \NR

+\NC 11 \NC {\tttf a} \endash\ {\tttf z} \NC         \NC \type {letter}       \NC \NR

+\NC 11 \NC {\tttf A} \endash\ {\tttf Z} \NC         \NC \type {letter}       \NC \NR

+\NC 12 \NC everything else              \NC         \NC \type {other}        \NC \NR

+\NC 14 \NC \tttf \letterpercent         \NC         \NC \type {comment}      \NC \NR

+\NC 15 \NC \tttf \letterhat\letterhat ? \NC delete  \NC \type {invalid_char} \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\lpr {savecatcodetable}}]

+

+\startsyntax

+\savecatcodetable <15-bit number>

+\stopsyntax

+

+\lpr {savecatcodetable} copies the current set of catcodes to a new table with

+the requested number. The definitions in this new table are all treated as if

+they were made in the outermost level.

+

+The new table is allocated globally: it will not go away after the current group

+has ended. If the supplied number is the currently active table, an error is

+raised.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Suppressing errors}]

+

+\startsubsection[title={\lpr {suppressfontnotfounderror}}]

+

+\topicindex {errors}

+

+If this integer parameter is non|-|zero, then \LUATEX\ will not complain about

+font metrics that are not found. Instead it will silently skip the font

+assignment, making the requested csname for the font \prm {ifx} equal to \prm

+{nullfont}, so that it can be tested against that without bothering the user.

+

+\startsyntax

+\suppressfontnotfounderror = 1

+\stopsyntax

+

+\stopsubsection

+

+\startsubsection[title={\lpr {suppresslongerror}}]

+

+\topicindex {errors}

+

+If this integer parameter is non|-|zero, then \LUATEX\ will not complain about

+\prm {par} commands encountered in contexts where that is normally prohibited

+(most prominently in the arguments of macros not defined as \prm {long}).

+

+\startsyntax

+\suppresslongerror = 1

+\stopsyntax

+

+\stopsubsection

+

+\startsubsection[title={\lpr {suppressifcsnameerror}}]

+

+\topicindex {errors}

+

+If this integer parameter is non|-|zero, then \LUATEX\ will not complain about

+non-expandable commands appearing in the middle of a \prm {ifcsname} expansion.

+Instead, it will keep getting expanded tokens from the input until it encounters

+an \prm {endcsname} command. If the input expansion is unbalanced with respect

+to \prm {csname} \ldots \prm {endcsname} pairs, the \LUATEX\ process may hang

+indefinitely.

+

+\startsyntax

+\suppressifcsnameerror = 1

+\stopsyntax

+

+\stopsubsection

+

+\startsubsection[title={\lpr {suppressoutererror}}]

+

+\topicindex {errors}

+

+If this new integer parameter is non|-|zero, then \LUATEX\ will not complain

+about \prm {outer} commands encountered in contexts where that is normally

+prohibited.

+

+\startsyntax

+\suppressoutererror = 1

+\stopsyntax

+

+\stopsubsection

+

+\startsubsection[title={\lpr {suppressmathparerror}}]

+

+\topicindex {errors}

+\topicindex {math}

+

+The following setting will permit \prm {par} tokens in a math formula:

+

+\startsyntax

+\suppressmathparerror = 1

+\stopsyntax

+

+So, the next code is valid then:

+

+\starttyping

+$ x + 1 =

+

+a $

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={\lpr {suppressprimitiveerror}}]

+

+\topicindex {errors}

+\topicindex {primitives}

+

+When set to a non|-|zero value the following command will not issue an error:

+

+\startsyntax

+\suppressprimitiveerror = 1

+

+\primitive\notaprimitive

+\stopsyntax

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Fonts}]

+

+\startsubsection[title={Font syntax}]

+

+\topicindex {fonts}

+

+\LUATEX\ will accept a braced argument as a font name:

+

+\starttyping

+\font\myfont = {cmr10}

+\stoptyping

+

+This allows for embedded spaces, without the need for double quotes. Macro

+expansion takes place inside the argument.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {fontid} and \lpr {setfontid}}]

+

+\startsyntax

+\fontid\font

+\stopsyntax

+

+This primitive expands into a number. It is not a register so there is no need to

+prefix with \prm {number} (and using \prm {the} gives an error). The currently

+used font id is \fontid\font. Here are some more:

+

+\starttabulate[|l|c|c|]

+\DB style \BC command \BC font id \NC \NR

+\TB

+\NC normal      \NC \type {\tf} \NC \bf \fontid\font \NC \NR

+\NC bold        \NC \type {\bf} \NC \bf \fontid\font \NC \NR

+\NC italic      \NC \type {\it} \NC \it \fontid\font \NC \NR

+\NC bold italic \NC \type {\bi} \NC \bi \fontid\font \NC \NR

+\LL

+\stoptabulate

+

+These numbers depend on the macro package used because each one has its own way

+of dealing with fonts. They can also differ per run, as they can depend on the

+order of loading fonts. For instance, when in \CONTEXT\ virtual math \UNICODE\

+fonts are used, we can easily get over a hundred ids in use. Not all ids have to

+be bound to a real font, after all it's just a number.

+

+The primitive \lpr {setfontid} can be used to enable a font with the given id,

+which of course needs to be a valid one.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {noligs} and \lpr {nokerns}}]

+

+\topicindex {ligatures+suppress}

+\topicindex {kerns+suppress}

+

+These primitives prohibit ligature and kerning insertion at the time when the

+initial node list is built by \LUATEX's main control loop. You can enable these

+primitives when you want to do node list processing of \quote {characters}, where

+\TEX's normal processing would get in the way.

+

+\startsyntax

+\noligs <integer>!crlf

+\nokerns <integer>

+\stopsyntax

+

+These primitives can also be implemented by overloading the ligature building and

+kerning functions, i.e.\ by assigning dummy functions to their associated

+callbacks. Keep in mind that when you define a font (using \LUA) you can also

+omit the kern and ligature tables, which has the same effect as the above.

+

+\stopsubsection

+

+\startsubsection[title={\type{\nospaces}}]

+

+\topicindex {spaces+suppress}

+

+This new primitive can be used to overrule the usual \prm {spaceskip} related

+heuristics when a space character is seen in a text flow. The value~\type{1}

+triggers no injection while \type{2} results in injection of a zero skip. In \in

+{figure} [fig:nospaces] we see the results for four characters separated by a

+space.

+

+\startplacefigure[reference=fig:nospaces,title={The \lpr {nospaces} options.}]

+\startcombination[3*2]

+    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 10mm}}

+    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 10mm}}

+    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 10mm}}

+    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 1mm}}

+    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 1mm}}

+    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 1mm}}

+\stopcombination

+\stopplacefigure

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Tokens, commands and strings}]

+

+\startsubsection[title={\lpr {scantextokens}}]

+

+\topicindex {tokens+scanning}

+

+The syntax of \lpr {scantextokens} is identical to \prm {scantokens}. This

+primitive is a slightly adapted version of \ETEX's \prm {scantokens}. The

+differences are:

+

+\startitemize

+\startitem

+    The last (and usually only) line does not have a \prm {endlinechar}

+    appended.

+\stopitem

+\startitem

+    \lpr {scantextokens} never raises an EOF error, and it does not execute

+    \prm {everyeof} tokens.

+\stopitem

+\startitem

+    There are no \quote {\unknown\ while end of file \unknown} error tests

+    executed. This allows the expansion to end on a different grouping level or

+    while a conditional is still incomplete.

+\stopitem

+\stopitemize

+

+\stopsubsection

+

+\startsubsection[title={\lpr {toksapp}, \lpr {tokspre}, \lpr {etoksapp}, \lpr {etokspre},

+\lpr {gtoksapp}, \lpr {gtokspre}, \lpr {xtoksapp},  \lpr {xtokspre}}]

+

+Instead of:

+

+\starttyping

+\toks0\expandafter{\the\toks0 foo}

+\stoptyping

+

+you can use:

+

+\starttyping

+\etoksapp0{foo}

+\stoptyping

+

+The \type {pre} variants prepend instead of append, and the \type {e} variants

+expand the passed general text. The \type {g} and \type {x} variants are global.

+

+\stopsubsection

+

+\startsubsection[title={\prm {csstring}, \lpr {begincsname} and \lpr {lastnamedcs}}]

+

+These are somewhat special. The \prm {csstring} primitive is like

+\prm {string} but it omits the leading escape character. This can be

+somewhat more efficient than stripping it afterwards.

+

+The \lpr {begincsname} primitive is like \prm {csname} but doesn't create

+a relaxed equivalent when there is no such name. It is equivalent to

+

+\starttyping

+\ifcsname foo\endcsname

+  \csname foo\endcsname

+\fi

+\stoptyping

+

+The advantage is that it saves a lookup (don't expect much speedup) but more

+important is that it avoids using the \prm {if} test. The \lpr {lastnamedcs}

+is one that should be used with care. The above example could be written as:

+

+\starttyping

+\ifcsname foo\endcsname

+  \lastnamedcs

+\fi

+\stoptyping

+

+This is slightly more efficient than constructing the string twice (deep down in

+\LUATEX\ this also involves some \UTF8 juggling), but probably more relevant is

+that it saves a few tokens and can make code a bit more readable.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {clearmarks}}]

+

+\topicindex {marks}

+

+This primitive complements the \ETEX\ mark primitives and clears a mark class

+completely, resetting all three connected mark texts to empty. It is an

+immediate command.

+

+\startsyntax

+\clearmarks <16-bit number>

+\stopsyntax

+

+\stopsubsection

+

+\startsubsection[title={\lpr {alignmark} and \lpr {aligntab}}]

+

+The primitive \lpr {alignmark} duplicates the functionality of \type {#} inside

+alignment preambles, while \lpr {aligntab} duplicates the functionality of \type

+{&}.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {letcharcode}}]

+

+This primitive can be used to assign a meaning to an active character, as in:

+

+\starttyping

+\def\foo{bar} \letcharcode123=\foo

+\stoptyping

+

+This can be a bit nicer than using the uppercase tricks (using the property of

+\prm {uppercase} that it treats active characters special).

+

+\stopsubsection

+

+\startsubsection[title={\lpr {glet}}]

+

+This primitive is similar to:

+

+\starttyping

+\protected\def\glet{\global\let}

+\stoptyping

+

+but faster (only measurable with millions of calls) and probably more convenient

+(after all we also have \type {\gdef}).

+

+\stopsubsection

+

+\startsubsection[title={\lpr {expanded}, \lpr {immediateassignment} and \lpr {immediateassigned}}]

+

+\topicindex {expansion}

+

+The \lpr {expanded} primitive takes a token list and expands it content which can

+come in handy: it avoids a tricky mix of \prm {expandafter} and \prm {noexpand}.

+You can compare it with what happens inside the body of an \prm {edef}. But this

+kind of expansion it still doesn't expand some primitive operations.

+

+\startbuffer

+\newcount\NumberOfCalls

+

+\def\TestMe{\advance\NumberOfCalls1 }

+

+\edef\Tested{\TestMe foo:\the\NumberOfCalls}

+\edef\Tested{\TestMe foo:\the\NumberOfCalls}

+\edef\Tested{\TestMe foo:\the\NumberOfCalls}

+

+\meaning\Tested

+\stopbuffer

+

+\typebuffer

+

+The result is a macro that has the not expanded code in its body:

+

+\getbuffer

+

+Instead we can define \tex {TestMe} in a way that expands the assignment

+immediately. You need of course to be aware of preventing look ahead interference

+by using a space or \tex {relax} (often an expression works better as it doesn't

+leave an \tex {relax}).

+

+\startbuffer

+\def\TestMe{\immediateassignment\advance\NumberOfCalls1 }

+

+\edef\Tested{\TestMe foo:\the\NumberOfCalls}

+\edef\Tested{\TestMe foo:\the\NumberOfCalls}

+\edef\Tested{\TestMe foo:\the\NumberOfCalls}

+

+\meaning\Tested

+\stopbuffer

+

+\typebuffer

+

+This time the counter gets updates and we don't see interference in the

+resulting \tex {Tested} macro:

+

+\getbuffer

+

+Here is a somewhat silly example of expanded comparison:

+

+\startbuffer

+\def\expandeddoifelse#1#2#3#4%

+  {\immediateassignment\edef\tempa{#1}%

+   \immediateassignment\edef\tempb{#2}%

+   \ifx\tempa\tempb

+     \immediateassignment\def\next{#3}%

+   \else

+     \immediateassignment\def\next{#4}%

+   \fi

+   \next}

+

+\edef\Tested

+  {(\expandeddoifelse{abc}{def}{yes}{nop}/%

+    \expandeddoifelse{abc}{abc}{yes}{nop})}

+

+\meaning\Tested

+\stopbuffer

+

+\typebuffer

+

+It gives:

+

+\getbuffer

+

+A variant is:

+

+\starttyping

+\def\expandeddoifelse#1#2#3#4%

+  {\immediateassigned{

+     \edef\tempa{#1}%

+     \edef\tempb{#2}%

+   }%

+   \ifx\tempa\tempb

+     \immediateassignment\def\next{#3}%

+   \else

+     \immediateassignment\def\next{#4}%

+   \fi

+   \next}

+\stoptyping

+

+The possible error messages are the same as using assignments in preambles of

+alignments and after the \prm {accent} command. The supported assignments are the

+so called prefixed commands (except box assignments).

+

+\stopsubsection

+

+\startsubsection[title={\lpr {ifcondition}}]

+

+\topicindex {conditions}

+

+This is a somewhat special one. When you write macros conditions need to be

+properly balanced in order to let \TEX's fast branch skipping work well. This new

+primitive is basically a no||op flagged as a condition so that the scanner can

+recognize it as an if|-|test. However, when a real test takes place the work is

+done by what follows, in the next example \tex {something}.

+

+\starttyping

+\unexpanded\def\something#1#2%

+  {\edef\tempa{#1}%

+   \edef\tempb{#2}

+   \ifx\tempa\tempb}

+

+\ifcondition\something{a}{b}%

+    \ifcondition\something{a}{a}%

+        true 1

+    \else

+        false 1

+    \fi

+\else

+    \ifcondition\something{a}{a}%

+        true 2

+    \else

+        false 2

+    \fi

+\fi

+\stoptyping

+

+If you are familiar with \METAPOST, this is a bit like \type {vardef} where the macro

+has a return value. Here the return value is a test.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Boxes, rules and leaders}]

+

+\startsubsection[title={\lpr {outputbox}}]

+

+\topicindex {output}

+

+This integer parameter allows you to alter the number of the box that will be

+used to store the page sent to the output routine. Its default value is 255, and

+the acceptable range is from 0 to 65535.

+

+\startsyntax

+\outputbox = 12345

+\stopsyntax

+

+\stopsubsection

+

+\startsubsection[title={\prm {vpack}, \prm {hpack} and \prm {tpack}}]

+

+These three primitives are like \prm {vbox}, \prm {hbox} and \prm {vtop}

+but don't apply the related callbacks.

+

+\stopsubsection

+

+\startsubsection[title={\prm {vsplit}}]

+

+\topicindex {splitting}

+

+The \prm {vsplit} primitive has to be followed by a specification of the required

+height. As alternative for the \type {to} keyword you can use \type {upto} to get

+a split of the given size but result has the natural dimensions then.

+

+\stopsubsection

+

+\startsubsection[title={Images and reused box objects},reference=sec:imagedandforms]

+

+These two concepts are now core concepts and no longer whatsits. They are in fact

+now implemented as rules with special properties. Normal rules have subtype~0,

+saved boxes have subtype~1 and images have subtype~2. This has the positive side

+effect that whenever we need to take content with dimensions into account, when

+we look at rule nodes, we automatically also deal with these two types.

+

+The syntax of the \type {\save...resource} is the same as in \PDFTEX\ but you

+should consider them to be backend specific. This means that a macro package

+should treat them as such and check for the current output mode if applicable.

+

+\starttabulate[|l|p|]

+\DB command \BC explanation \NC \NR

+\TB

+\NC \lpr {saveboxresource}             \NC save the box as an object to be included later \NC \NR

+\NC \lpr {saveimageresource}           \NC save the image as an object to be included later \NC \NR

+\NC \lpr {useboxresource}              \NC include the saved box object here (by index) \NC \NR

+\NC \lpr {useimageresource}            \NC include the saved image object here (by index) \NC \NR

+\NC \lpr {lastsavedboxresourceindex}   \NC the index of the last saved box object \NC \NR

+\NC \lpr {lastsavedimageresourceindex} \NC the index of the last saved image object \NC \NR

+\NC \lpr {lastsavedimageresourcepages} \NC the number of pages in the last saved image object \NC \NR

+\LL

+\stoptabulate

+

+\LUATEX\ accepts optional dimension parameters for \type {\use...resource} in the

+same format as for rules. With images, these dimensions are then used instead of

+the ones given to \lpr {useimageresource} but the original dimensions are not

+overwritten, so that a \lpr {useimageresource} without dimensions still

+provides the image with dimensions defined by \lpr {saveimageresource}. These

+optional parameters are not implemented for \lpr {saveboxresource}.

+

+\starttyping

+\useimageresource width 20mm height 10mm depth 5mm \lastsavedimageresourceindex

+\useboxresource   width 20mm height 10mm depth 5mm \lastsavedboxresourceindex

+\stoptyping

+

+The box resources are of course implemented in the backend and therefore we do

+support the \type {attr} and \type {resources} keys that accept a token list. New

+is the \type {type} key. When set to non|-|zero the \type {/Type} entry is

+omitted. A value of 1 or 3 still writes a \type {/BBox}, while 2 or 3 will write

+a \type {/Matrix}.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {nohrule} and \lpr {novrule}}]

+

+\topicindex {rules}

+

+Because introducing a new keyword can cause incompatibilities, two new primitives

+were introduced: \lpr {nohrule} and \lpr {novrule}. These can be used to

+reserve space. This is often more efficient than creating an empty box with fake

+dimensions.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {gleaders}}]

+

+\topicindex {leaders}

+

+This type of leaders is anchored to the origin of the box to be shipped out. So

+they are like normal \prm {leaders} in that they align nicely, except that the

+alignment is based on the {\it largest\/} enclosing box instead of the {\it

+smallest\/}. The \type {g} stresses this global nature.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Languages}]

+

+\startsubsection[title={\lpr {hyphenationmin}}]

+

+\topicindex {languages}

+\topicindex {hyphenation}

+

+This primitive can be used to set the minimal word length, so setting it to a value

+of~$5$ means that only words of 6 characters and more will be hyphenated, of course

+within the constraints of the \prm {lefthyphenmin} and \prm {righthyphenmin}

+values (as stored in the glyph node). This primitive accepts a number and stores

+the value with the language.

+

+\stopsubsection

+

+\startsubsection[title={\prm {boundary}, \prm {noboundary}, \prm {protrusionboundary} and \prm {wordboundary}}]

+

+The \prm {noboundary} command is used to inject a whatsit node but now injects a normal

+node with type \nod {boundary} and subtype~0. In addition you can say:

+

+\starttyping

+x\boundary 123\relax y

+\stoptyping

+

+This has the same effect but the subtype is now~1 and the value~123 is stored.

+The traditional ligature builder still sees this as a cancel boundary directive

+but at the \LUA\ end you can implement different behaviour. The added benefit of

+passing this value is a side effect of the generalization. The subtypes~2 and~3

+are used to control protrusion and word boundaries in hyphenation and have

+related primitives.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Control and debugging}]

+

+\startsubsection[title={Tracing}]

+

+\topicindex {tracing}

+

+If \prm {tracingonline} is larger than~2, the node list display will also print

+the node number of the nodes.

+

+\stopsubsection

+

+\startsubsection[title={\lpr {outputmode}}]

+

+\topicindex {output}

+\topicindex {backend}

+

+The \lpr {outputmode} variable tells \LUATEX\ what it has to produce:

+

+\starttabulate[|l|l|]

+\DB value \BC output \NC \NR

+\TB

+\NC \type {0} \NC \DVI\ code \NC \NR

+\NC \type {1} \NC \PDF\ code \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\lpr {draftmode}}]

+

+The value of the \lpr {draftmode} counter signals the backend if it should output

+less. The \PDF\ backend accepts a value of~1, while the \DVI\ backend ignores the

+value. This is no critical feature so we can remove it in future versions when it

+can make the backend cleaner.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Files}]

+

+\startsubsection[title={File syntax}]

+

+\topicindex {files+names}

+

+\LUATEX\ will accept a braced argument as a file name:

+

+\starttyping

+\input {plain}

+\openin 0 {plain}

+\stoptyping

+

+This allows for embedded spaces, without the need for double quotes. Macro

+expansion takes place inside the argument.

+

+The \lpr {tracingfonts} primitive that has been inherited from \PDFTEX\ has

+been adapted to support variants in reporting the font. The reason for this

+extension is that a csname not always makes sense. The zero case is the default.

+

+\starttabulate[|l|l|]

+\DB value \BC reported \NC \NR

+\TB

+\NC \type{0} \NC \type{\foo xyz} \NC \NR

+\NC \type{1} \NC \type{\foo (bar)} \NC \NR

+\NC \type{2} \NC \type{<bar> xyz} \NC \NR

+\NC \type{3} \NC \type{<bar @ ..pt> xyz} \NC \NR

+\NC \type{4} \NC \type{<id>} \NC \NR

+\NC \type{5} \NC \type{<id: bar>} \NC \NR

+\NC \type{6} \NC \type{<id: bar @ ..pt> xyz} \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={Writing to file}]

+

+\topicindex {files+writing}

+

+You can now open upto 127 files with \prm {openout}. When no file is open

+writes will go to the console and log. As a consequence a system command is

+no longer possible but one can use \type {os.execute} to do the same.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={Math}]

+

+\topicindex {math}

+

+We will cover math extensions in its own chapter because not only the font

+subsystem and spacing model have been enhanced (thereby introducing many new

+primitives) but also because some more control has been added to existing

+functionality. Much of this relates to the different approaches of traditional

+\TEX\ fonts and \OPENTYPE\ math.

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-export-titlepage.tex b/Master/texmf-dist/doc/luatex/base/luatex-export-titlepage.tex
index 54a24c2e7bd..db841a78518 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-export-titlepage.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-export-titlepage.tex
@@ -1,32 +1,31 @@
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-export-titlepage
-
-\setupexport
-  [cssfile=extra-styles.css]
-
-\settaggedmetadata
-  [    title={LuaTeX Reference Manual},
-   copyright={LuaTeX Development Team},
-     version={\documentvariable{version}},
-      status={\documentvariable{status}},
-    snapshot={\documentvariable{snapshot}},
-        date={\rawdate[weekday,day,month,year]},
-         url={www.luatex.org}]
-
-\setupbackgrounds
-  [leftpage]
-  [setups=pagenumber:left]
-
-\setupbackgrounds
-  [rightpage]
-  [setups=pagenumber:right]
-
-\startpreamble
-    This document is derived from the original manual. When examples are given,
-    they assume processing by \LUATEX\ and therefore will not show up as
-    intended. The \PDF\ version is the real reference.
-\stoppreamble
-
-\stopcomponent
+\environment luatex-style

+

+\startcomponent luatex-export-titlepage

+

+\setupexport

+  [cssfile=extra-styles.css]

+

+\settaggedmetadata

+  [    title={LuaTeX Reference Manual},

+   copyright={LuaTeX Development Team},

+     version={\documentvariable{version}},

+      status={\documentvariable{status}},

+    snapshot={\documentvariable{snapshot}},

+        date={\rawdate[weekday,day,month,year]},

+         url={www.luatex.org}]

+

+\setupbackgrounds

+  [leftpage]

+  [setups=pagenumber:left]

+

+\setupbackgrounds

+  [rightpage]

+  [setups=pagenumber:right]

+

+\startpreamble

+    This document is derived from the original manual. When examples are given,

+    they assume processing by \LUATEX\ and therefore will not show up as

+    intended. The \PDF\ version is the real reference.

+\stoppreamble

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-firstpage.tex b/Master/texmf-dist/doc/luatex/base/luatex-firstpage.tex
new file mode 100644
index 00000000000..e64cc615286
--- /dev/null
+++ b/Master/texmf-dist/doc/luatex/base/luatex-firstpage.tex
@@ -0,0 +1,32 @@
+\startcomponent luatex-firstpage

+

+\startstandardmakeup

+

+    \start

+        \raggedleft

+        \definedfont[Bold*default at 48pt]

+        \setupinterlinespace

+        \blue \documentvariable{manual} \endgraf Reference \endgraf Manual \endgraf

+    \stop

+

+    \vfill

+

+    \definedfont[Bold*default at 12pt]

+

+    \starttabulate[|l|l|]

+        \NC copyright \EQ Lua\TeX\ development team \NC \NR

+        \NC more info \EQ www.luatex.org \NC \NR

+        \NC version   \EQ \currentdate \doifsomething{\documentvariable{snapshot}}{(snapshot \documentvariable{snapshot})} \NC \NR

+    \stoptabulate

+

+\stopstandardmakeup

+

+\setupbackgrounds

+  [leftpage]

+  [setups=pagenumber:left]

+

+\setupbackgrounds

+  [rightpage]

+  [setups=pagenumber:right]

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-fontloader.tex b/Master/texmf-dist/doc/luatex/base/luatex-fontloader.tex
index c2d82cbba36..e99ea1d2ff2 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-fontloader.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-fontloader.tex
@@ -1,1068 +1,1164 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-fontloader
-
-\startchapter[reference=fontloader,title={The fontloader}]
-
-The fontloader library is sort of independent of the rest in the sense that it
-can load font into a \LUA\ table that then can be converted into a table suitable
-for \TEX. The library is an adapted subset of \FONTFORGE\ and as such gives a
-similar view on a font (which has advantages when you want to debug.)
-
-\section{Getting quick information on a font}
-
-When you want to locate font by name you need some basic information that is
-hidden in the font files. For that reason we provide an efficient helper that
-gets the basic information without loading all of the font. Normally this helper
-is used to create a font (name) database.
-
-\startfunctioncall
-<table> info =
-    fontloader.info(<string> filename)
-\stopfunctioncall
-
-This function returns either \type {nil}, or a \type {table}, or an array of
-small tables (in the case of a \TRUETYPE\ collection). The returned table(s) will
-contain some fairly interesting information items from the font(s) defined by the
-file:
-
-\starttabulate[|l|l|p|]
-\BC key                 \BC type     \BC explanation \NC \NR
-\NC \type{fontname}     \NC string   \NC the \POSTSCRIPT\ name of the font\NC \NR
-\NC \type{fullname}     \NC string   \NC the formal name of the font\NC \NR
-\NC \type{familyname}   \NC string   \NC the family name this font belongs to\NC \NR
-\NC \type{weight}       \NC string   \NC a string indicating the color value of the font\NC \NR
-\NC \type{version}      \NC string   \NC the internal font version\NC \NR
-\NC \type{italicangle}  \NC float    \NC the slant angle\NC \NR
-\NC \type{units_per_em} \NC number   \NC 1000 for \POSTSCRIPT-based fonts, usually 2048 for \TRUETYPE\NC \NR
-\NC \type{pfminfo}      \NC table    \NC (see \in{section}[fontloaderpfminfotable])\NC \NR
-\stoptabulate
-
-Getting information through this function is (sometimes much) more efficient than
-loading the font properly, and is therefore handy when you want to create a
-dictionary of available fonts based on a directory contents.
-
-\section{Loading an \OPENTYPE\ or \TRUETYPE\ file}
-
-If you want to use an \OPENTYPE\ font, you have to get the metric information
-from somewhere. Using the \type {fontloader} library, the simplest way to get
-that information is thus:
-
-\starttyping
-function load_font (filename)
-  local metrics = nil
-  local font = fontloader.open(filename)
-  if font then
-     metrics = fontloader.to_table(font)
-     fontloader.close(font)
-  end
-  return metrics
-end
-
-myfont = load_font('/opt/tex/texmf/fonts/data/arial.ttf')
-\stoptyping
-
-The main function call is
-
-\startfunctioncall
-<userdata> f, <table> w = fontloader.open(<string> filename)
-<userdata> f, <table> w = fontloader.open(<string> filename, <string> fontname)
-\stopfunctioncall
-
-The first return value is a userdata representation of the font. The second
-return value is a table containing any warnings and errors reported by fontloader
-while opening the font. In normal typesetting, you would probably ignore the
-second argument, but it can be useful for debugging purposes.
-
-For \TRUETYPE\ collections (when filename ends in 'ttc') and \DFONT\ collections,
-you have to use a second string argument to specify which font you want from the
-collection. Use the \type {fontname} strings that are returned by \type
-{fontloader.info} for that.
-
-To turn the font into a table, \type {fontloader.to_table} is used on the font
-returned by \type {fontloader.open}.
-
-\startfunctioncall
-<table> f = fontloader.to_table(<userdata> font)
-\stopfunctioncall
-
-This table cannot be used directly by \LUATEX\ and should be turned into another
-one as described in~\in {chapter} [fonts]. Do not forget to store the \type
-{fontname} value in the \type {psname} field of the metrics table to be returned
-to \LUATEX, otherwise the font inclusion backend will not be able to find the
-correct font in the collection.
-
-See \in {section} [fontloadertables] for details on the userdata object returned
-by \type {fontloader.open()} and the layout of the \type {metrics} table returned
-by \type {fontloader.to_table()}.
-
-The font file is parsed and partially interpreted by the font loading routines
-from \FONTFORGE. The file format can be \OPENTYPE, \TRUETYPE, \TRUETYPE\
-Collection, \CFF, or \TYPEONE.
-
-There are a few advantages to this approach compared to reading the actual font
-file ourselves:
-
-\startitemize
-
-\startitem
-    The font is automatically re|-|encoded, so that the \type {metrics} table for
-    \TRUETYPE\ and \OPENTYPE\ fonts is using \UNICODE\ for the character indices.
-\stopitem
-
-\startitem
-    Many features are pre|-|processed into a format that is easier to handle than
-    just the bare tables would be.
-\stopitem
-
-\startitem
-    \POSTSCRIPT|-|based \OPENTYPE\ fonts do not store the character height and
-    depth in the font file, so the character boundingbox has to be calculated in
-    some way.
-\stopitem
-
-\startitem
-    In the future, it may be interesting to allow \LUA\ scripts access to
-    the font program itself, perhaps even creating or changing the font.
-\stopitem
-
-\stopitemize
-
-A loaded font is discarded with:
-
-\startfunctioncall
-fontloader.close(<userdata> font)
-\stopfunctioncall
-
-\section{Applying a \quote{feature file}}
-
-You can apply a \quote{feature file} to a loaded font:
-
-\startfunctioncall
-<table> errors = fontloader.apply_featurefile(<userdata> font, <string> filename)
-\stopfunctioncall
-
-A \quote {feature file} is a textual representation of the features in an
-\OPENTYPE\ font. See
-
-\starttyping
-http://www.adobe.com/devnet/opentype/afdko/topic_feature_file_syntax.html
-\stoptyping
-
-and
-
-\starttyping
-http://fontforge.sourceforge.net/featurefile.html
-\stoptyping
-
-for a more detailed description of feature files.
-
-If the function fails, the return value is a table containing any errors reported
-by fontloader while applying the feature file. On success, \type {nil} is
-returned.
-
-\section{Applying an \quote{\AFM\ file}}
-
-You can apply an \quote {\AFM\ file} to a loaded font:
-
-\startfunctioncall
-<table> errors = fontloader.apply_afmfile(<userdata> font, <string> filename)
-\stopfunctioncall
-
-An \AFM\ file is a textual representation of (some of) the meta information
-in a \TYPEONE\ font. See
-
-\starttyping
-ftp://ftp.math.utah.edu/u/ma/hohn/linux/postscript/5004.AFM_Spec.pdf
-\stoptyping
-
-for more information about \AFM\ files.
-
-Note: If you \type {fontloader.open()} a \TYPEONE\ file named \type {font.pfb},
-the library will automatically search for and apply \type {font.afm} if it exists
-in the same directory as the file \type {font.pfb}. In that case, there is no
-need for an explicit call to \type {apply_afmfile()}.
-
-If the function fails, the return value is a table containing any errors reported
-by fontloader while applying the AFM file. On success, \type {nil} is returned.
-
-\section[fontloadertables]{Fontloader font tables}
-
-As mentioned earlier, the return value of \type {fontloader.open()} is a userdata
-object. One way to have access to the actual metrics is to call \type
-{fontloader.to_table()} on this object, returning the table structure that is
-explained in the following sections. In teh following sections we will not
-explain each field in detail. Most fields are self descriptive and for the more
-technical aspects you need to consult the relevant font references.
-
-It turns out that the result from \type {fontloader.to_table()} sometimes needs
-very large amounts of memory (depending on the font's complexity and size) so it
-is possible to access the userdata object directly.
-
-\startitemize
-\startitem
-    All top|-|level keys that would be returned by \type {to_table()}
-    can also be accessed directly.
-\stopitem
-\startitem
-    The top|-|level key \quote {glyphs} returns a {\it virtual\/} array that
-    allows indices from \type {f.glyphmin} to (\type {f.glyphmax}).
-\stopitem
-\startitem
-    The items in that virtual array (the actual glyphs) are themselves also
-    userdata objects, and each has accessors for all of the keys explained in the
-    section \quote {Glyph items} below.
-\stopitem
-\startitem
-    The top|-|level key \quote {subfonts} returns an {\it actual} array of userdata
-    objects, one for each of the subfonts (or nil, if there are no subfonts).
-\stopitem
-\stopitemize
-
-A short example may be helpful. This code generates a printout of all
-the glyph names in the font \type {PunkNova.kern.otf}:
-
-\starttyping
-local f = fontloader.open('PunkNova.kern.otf')
-print (f.fontname)
-local i = 0
-if f.glyphcnt > 0 then
-    for i=f.glyphmin,f.glyphmax do
-       local g = f.glyphs[i]
-       if g then
-          print(g.name)
-       end
-       i = i + 1
-    end
-end
-fontloader.close(f)
-\stoptyping
-
-In this case, the \LUATEX\ memory requirement stays below 100MB on the test
-computer, while the internal structure generated by \type {to_table()} needs more
-than 2GB of memory (the font itself is 6.9MB in disk size).
-
-Only the top|-|level font, the subfont table entries, and the glyphs are virtual
-objects, everything else still produces normal \LUA\ values and tables.
-
-If you want to know the valid fields in a font or glyph structure, call the \type
-{fields} function on an object of a particular type (either glyph or font):
-
-\startfunctioncall
-<table> fields = fontloader.fields(<userdata> font)
-<table> fields = fontloader.fields(<userdata> font_glyph)
-\stopfunctioncall
-
-For instance:
-
-\startfunctioncall
-local fields = fontloader.fields(f)
-local fields = fontloader.fields(f.glyphs[0])
-\stopfunctioncall
-
-\section{Table types}
-
-\subsection{Top-level}
-
-The top|-|level keys in the returned table are (the explanations in this part of
-the documentation are not yet finished):
-
-\starttabulate[|l|l|p|]
-\BC key                                 \BC type     \NC explanation \NC \NR
-\NC \type{table_version}                \NC number   \NC indicates the metrics version (currently~0.3)\NC \NR
-\NC \type{fontname}                     \NC string   \NC \POSTSCRIPT\ font name\NC \NR
-\NC \type{fullname}                     \NC string   \NC official (human-oriented) font name\NC \NR
-\NC \type{familyname}                   \NC string   \NC family name\NC \NR
-\NC \type{weight}                       \NC string   \NC weight indicator\NC \NR
-\NC \type{copyright}                    \NC string   \NC copyright information\NC \NR
-\NC \type{filename}                     \NC string   \NC the file name\NC \NR
-\NC \type{version}                      \NC string   \NC font version\NC \NR
-\NC \type{italicangle}                  \NC float    \NC slant angle\NC \NR
-\NC \type{units_per_em}                 \NC number   \NC 1000 for \POSTSCRIPT-based fonts, usually 2048 for \TRUETYPE\NC \NR
-\NC \type{ascent}                       \NC number   \NC height of ascender in \type {units_per_em}\NC \NR
-\NC \type{descent}                      \NC number   \NC depth of descender in \type {units_per_em}\NC \NR
-\NC \type{upos}                         \NC float    \NC \NC \NR
-\NC \type{uwidth}                       \NC float    \NC \NC \NR
-\NC \type{uniqueid}                     \NC number   \NC \NC \NR
-\NC \type{glyphs}                       \NC array    \NC \NC \NR
-\NC \type{glyphcnt}                     \NC number   \NC number of included glyphs\NC \NR
-\NC \type{glyphmax}                     \NC number   \NC maximum used index the glyphs array\NC \NR
-\NC \type{glyphmin}                     \NC number   \NC minimum used index the glyphs array\NC \NR
-\NC \type{notdef_loc}                   \NC number   \NC location of the \type {.notdef} glyph
-                                                         or \type {-1} when not present \NC \NR
-\NC \type{hasvmetrics}                  \NC number   \NC \NC \NR
-\NC \type{onlybitmaps}                  \NC number   \NC \NC \NR
-\NC \type{serifcheck}                   \NC number   \NC \NC \NR
-\NC \type{isserif}                      \NC number   \NC \NC \NR
-\NC \type{issans}                       \NC number   \NC \NC \NR
-\NC \type{encodingchanged}              \NC number   \NC \NC \NR
-\NC \type{strokedfont}                  \NC number   \NC \NC \NR
-\NC \type{use_typo_metrics}             \NC number   \NC \NC \NR
-\NC \type{weight_width_slope_only}      \NC number   \NC \NC \NR
-\NC \type{head_optimized_for_cleartype} \NC number   \NC \NC \NR
-\NC \type{uni_interp}                   \NC enum     \NC \type {unset}, \type {none}, \type {adobe},
-                                                         \type {greek}, \type {japanese}, \type {trad_chinese},
-                                                         \type {simp_chinese}, \type {korean}, \type {ams}\NC \NR
-\NC \type{origname}                     \NC string   \NC the file name, as supplied by the user\NC \NR
-\NC \type{map}                          \NC table    \NC \NC \NR
-\NC \type{private}                      \NC table    \NC \NC \NR
-\NC \type{xuid}                         \NC string   \NC \NC \NR
-\NC \type{pfminfo}                      \NC table    \NC \NC \NR
-\NC \type{names}                        \NC table    \NC \NC \NR
-\NC \type{cidinfo}                      \NC table    \NC \NC \NR
-\NC \type{subfonts}                     \NC array    \NC \NC \NR
-\NC \type{commments}                    \NC string   \NC \NC \NR
-\NC \type{fontlog}                      \NC string   \NC \NC \NR
-\NC \type{cvt_names}                    \NC string   \NC \NC \NR
-\NC \type{anchor_classes}               \NC table    \NC \NC \NR
-\NC \type{ttf_tables}                   \NC table    \NC \NC \NR
-\NC \type{ttf_tab_saved}                \NC table    \NC \NC \NR
-\NC \type{kerns}                        \NC table    \NC \NC \NR
-\NC \type{vkerns}                       \NC table    \NC \NC \NR
-\NC \type{texdata}                      \NC table    \NC \NC \NR
-\NC \type{lookups}                      \NC table    \NC \NC \NR
-\NC \type{gpos}                         \NC table    \NC \NC \NR
-\NC \type{gsub}                         \NC table    \NC \NC \NR
-\NC \type{mm}                           \NC table    \NC \NC \NR
-\NC \type{chosenname}                   \NC string   \NC \NC \NR
-\NC \type{macstyle}                     \NC number   \NC \NC \NR
-\NC \type{fondname}                     \NC string   \NC \NC \NR
-%NC \type{design_size}                  \NC number   \NC \NC \NR
-\NC \type{fontstyle_id}                 \NC number   \NC \NC \NR
-\NC \type{fontstyle_name}               \NC table    \NC \NC \NR
-%NC \type{design_range_bottom}          \NC number   \NC \NC \NR
-%NC \type{design_range_top}             \NC number   \NC \NC \NR
-\NC \type{strokewidth}                  \NC float    \NC \NC \NR
-\NC \type{mark_classes}                 \NC table    \NC \NC \NR
-\NC \type{creationtime}                 \NC number   \NC \NC \NR
-\NC \type{modificationtime}             \NC number   \NC \NC \NR
-\NC \type{os2_version}                  \NC number   \NC \NC \NR
-\NC \type{sfd_version}                  \NC number   \NC \NC \NR
-\NC \type{math}                         \NC table    \NC \NC \NR
-\NC \type{validation_state}             \NC table    \NC \NC \NR
-\NC \type{horiz_base}                   \NC table    \NC \NC \NR
-\NC \type{vert_base}                    \NC table    \NC \NC \NR
-\NC \type{extrema_bound}                \NC number   \NC \NC \NR
-\NC \type{truetype}                     \NC boolean  \NC signals a \TRUETYPE\ font \NC \NR
-\stoptabulate
-
-\subsection{Glyph items}
-
-The \type {glyphs} is an array containing the per|-|character
-information (quite a few of these are only present if nonzero).
-
-\starttabulate[|l|l|p|]
-\BC \type{key}               \BC type     \BC explanation \NC \NR
-\NC \type{name}              \NC string   \NC the glyph name \NC \NR
-\NC \type{unicode}           \NC number   \NC unicode code point, or -1 \NC \NR
-\NC \type{boundingbox}       \NC array    \NC array of four numbers, see note below \NC \NR
-\NC \type{width}             \NC number   \NC only for horizontal fonts \NC \NR
-\NC \type{vwidth}            \NC number   \NC only for vertical fonts \NC \NR
-\NC \type{tsidebearing}      \NC number   \NC only for vertical ttf/otf fonts, and only if nonzero \NC \NR
-\NC \type{lsidebearing}      \NC number   \NC only if nonzero and not equal to boundingbox[1] \NC \NR
-\NC \type{class}             \NC string   \NC one of "none", "base", "ligature", "mark", "component"
-                                              (if not present, the glyph class is \quote {automatic}) \NC \NR
-\NC \type{kerns}             \NC array    \NC only for horizontal fonts, if set \NC \NR
-\NC \type{vkerns}            \NC array    \NC only for vertical fonts, if set \NC \NR
-\NC \type{dependents}        \NC array    \NC linear array of glyph name strings, only if nonempty\NC \NR
-\NC \type{lookups}           \NC table    \NC only if nonempty \NC \NR
-\NC \type{ligatures}         \NC table    \NC only if nonempty \NC \NR
-\NC \type{anchors}           \NC table    \NC only if set \NC \NR
-\NC \type{comment}           \NC string   \NC only if set \NC \NR
-\NC \type{tex_height}        \NC number   \NC only if set \NC \NR
-\NC \type{tex_depth}         \NC number   \NC only if set \NC \NR
-\NC \type{italic_correction} \NC number   \NC only if set \NC \NR
-\NC \type{top_accent}        \NC number   \NC only if set \NC \NR
-\NC \type{is_extended_shape} \NC number   \NC only if this character is part of a math extension list \NC \NR
-\NC \type{altuni}            \NC table    \NC alternate \UNICODE\ items \NC \NR
-\NC \type{vert_variants}     \NC table    \NC \NC \NR
-\NC \type{horiz_variants}    \NC table    \NC \NC \NR
-\NC \type{mathkern}          \NC table    \NC \NC \NR
-\stoptabulate
-
-On \type {boundingbox}: The boundingbox information for \TRUETYPE\ fonts and
-\TRUETYPE-based \OTF\ fonts is read directly from the font file.
-\POSTSCRIPT-based fonts do not have this information, so the boundingbox of
-traditional \POSTSCRIPT\ fonts is generated by interpreting the actual bezier
-curves to find the exact boundingbox. This can be a slow process, so the
-boundingboxes of \POSTSCRIPT-based \OTF\ fonts (and raw \CFF\ fonts) are
-calculated using an approximation of the glyph shape based on the actual glyph
-points only, instead of taking the whole curve into account. This means that
-glyphs that have missing points at extrema will have a too|-|tight boundingbox,
-but the processing is so much faster that in our opinion the tradeoff is worth
-it.
-
-The \type {kerns} and \type {vkerns} are linear arrays of small hashes:
-
-\starttabulate[|l|l|p|]
-\BC key           \BC type   \BC explanation \NC \NR
-\NC \type{char}   \NC string \NC \NC \NR
-\NC \type{off}    \NC number \NC \NC \NR
-\NC \type{lookup} \NC string \NC \NC \NR
-\stoptabulate
-
-The \type {lookups} is a hash, based on lookup subtable names, with
-the value of each key inside that a linear array of small hashes:
-
-% TODO: fix this description
-
-\starttabulate[|l|l|p|]
-\BC key                  \BC type \BC explanation \NC \NR
-\NC \type{type}          \NC enum \NC \type {position}, \type {pair}, \type
-                                      {substitution}, \type {alternate}, \type
-                                      {multiple}, \type {ligature}, \type {lcaret},
-                                      \type {kerning}, \type {vkerning}, \type
-                                      {anchors}, \type {contextpos}, \type
-                                      {contextsub}, \type {chainpos}, \type
-                                      {chainsub}, \type {reversesub}, \type {max},
-                                      \type {kernback}, \type {vkernback} \NC \NR
-\NC \type{specification} \NC table \NC extra data \NC \NR
-\stoptabulate
-
-For the first seven values of \type {type}, there can be additional
-sub|-|information, stored in the sub-table \type {specification}:
-
-\starttabulate[|l|l|p|]
-\BC value               \BC type     \BC explanation \NC \NR
-\NC \type{position}     \NC table    \NC a table of the \type {offset_specs} type \NC \NR
-\NC \type{pair}         \NC table    \NC one string: \type {paired}, and an array of one
-                                         or two \type {offset_specs} tables: \type
-                                         {offsets} \NC \NR
-\NC \type{substitution} \NC table    \NC one string: \type {variant} \NC \NR
-\NC \type{alternate}    \NC table    \NC one string: \type {components} \NC \NR
-\NC \type{multiple}     \NC table    \NC one string: \type {components} \NC \NR
-\NC \type{ligature}     \NC table    \NC two strings: \type {components}, \type {char} \NC \NR
-\NC \type{lcaret}       \NC array    \NC linear array of numbers \NC \NR
-\stoptabulate
-
-Tables for \type {offset_specs} contain up to four number|-|valued fields: \type
-{x} (a horizontal offset), \type {y} (a vertical offset), \type {h} (an advance
-width correction) and \type {v} (an advance height correction).
-
-The \type {ligatures} is a linear array of small hashes:
-
-\starttabulate[|l|l|p|]
-\BC key               \BC type   \BC explanation \NC \NR
-\NC \type{lig}        \NC table  \NC uses the same substructure as a single item in
-                                     the \type {lookups} table explained above \NC \NR
-\NC \type{char}       \NC string \NC \NC \NR
-\NC \type{components} \NC array  \NC linear array of named components \NC \NR
-\NC \type{ccnt}       \NC number \NC \NC \NR
-\stoptabulate
-
-The \type {anchor} table is indexed by a string signifying the anchor type, which
-is one of
-
-\starttabulate[|l|l|p|]
-\BC key             \BC type  \BC explanation \NC \NR
-\NC \type{mark}     \NC table \NC placement mark \NC \NR
-\NC \type{basechar} \NC table \NC mark for attaching combining items to a base char \NC \NR
-\NC \type{baselig}  \NC table \NC mark for attaching combining items to a ligature \NC \NR
-\NC \type{basemark} \NC table \NC generic mark for attaching combining items to connect to \NC \NR
-\NC \type{centry}   \NC table \NC cursive entry point \NC \NR
-\NC \type{cexit}    \NC table \NC cursive exit point \NC \NR
-\stoptabulate
-
-The content of these is a short array of defined anchors, with the
-entry keys being the anchor names. For all except \type {baselig}, the
-value is a single table with this definition:
-
-\starttabulate[|l|l|p|]
-\BC key                 \BC type   \BC explanation \NC \NR
-\NC \type{x}            \NC number \NC x location \NC \NR
-\NC \type{y}            \NC number \NC y location \NC \NR
-\NC \type{ttf_pt_index} \NC number \NC truetype point index, only if given \NC \NR
-\stoptabulate
-
-For \type {baselig}, the value is a small array of such anchor sets sets, one for
-each constituent item of the ligature.
-
-For clarification, an anchor table could for example look like this :
-
-\starttyping
-['anchor'] = {
-    ['basemark'] = {
-        ['Anchor-7'] = { ['x']=170, ['y']=1080 }
-    },
-    ['mark'] ={
-        ['Anchor-1'] = { ['x']=160, ['y']=810 },
-        ['Anchor-4'] = { ['x']=160, ['y']=800 }
-    },
-    ['baselig'] = {
-        [1] = { ['Anchor-2'] = { ['x']=160, ['y']=650 } },
-        [2] = { ['Anchor-2'] = { ['x']=460, ['y']=640 } }
-        }
-    }
-\stoptyping
-
-Note: The \type {baselig} table can be sparse!
-
-\subsection{map table}
-
-The top|-|level map is a list of encoding mappings. Each of those is a table
-itself.
-
-\starttabulate[|l|l|p|]
-\BC key             \BC type   \BC explanation \NC \NR
-\NC \type{enccount} \NC number \NC \NC \NR
-\NC \type{encmax}   \NC number \NC \NC \NR
-\NC \type{backmax}  \NC number \NC \NC \NR
-\NC \type{remap}    \NC table  \NC \NC \NR
-\NC \type{map}      \NC array  \NC non|-|linear array of mappings\NC \NR
-\NC \type{backmap}  \NC array  \NC non|-|linear array of backward mappings\NC \NR
-\NC \type{enc}      \NC table  \NC \NC \NR
-\stoptabulate
-
-The \type {remap} table is very small:
-
-\starttabulate[|l|l|p|]
-\BC key             \BC type   \BC explanation \NC \NR
-\NC \type{firstenc} \NC number \NC \NC \NR
-\NC \type{lastenc}  \NC number \NC \NC \NR
-\NC \type{infont}   \NC number \NC \NC \NR
-\stoptabulate
-
-The \type {enc} table is a bit more verbose:
-
-\starttabulate[|l|l|p|]
-\BC key                     \BC type   \BC explanation \NC \NR
-\NC \type{enc_name}         \NC string \NC \NC \NR
-\NC \type{char_cnt}         \NC number \NC \NC \NR
-\NC \type{char_max}         \NC number \NC \NC \NR
-\NC \type{unicode}          \NC array  \NC of \UNICODE\ position numbers\NC \NR
-\NC \type{psnames}          \NC array  \NC of \POSTSCRIPT\ glyph names\NC \NR
-\NC \type{builtin}          \NC number \NC \NC \NR
-\NC \type{hidden}           \NC number \NC \NC \NR
-\NC \type{only_1byte}       \NC number \NC \NC \NR
-\NC \type{has_1byte}        \NC number \NC \NC \NR
-\NC \type{has_2byte}        \NC number \NC \NC \NR
-\NC \type{is_unicodebmp}    \NC number \NC only if nonzero\NC \NR
-\NC \type{is_unicodefull}   \NC number \NC only if nonzero\NC \NR
-\NC \type{is_custom}        \NC number \NC only if nonzero\NC \NR
-\NC \type{is_original}      \NC number \NC only if nonzero\NC \NR
-\NC \type{is_compact}       \NC number \NC only if nonzero\NC \NR
-\NC \type{is_japanese}      \NC number \NC only if nonzero\NC \NR
-\NC \type{is_korean}        \NC number \NC only if nonzero\NC \NR
-\NC \type{is_tradchinese}   \NC number \NC only if nonzero [name?]\NC \NR
-\NC \type{is_simplechinese} \NC number \NC only if nonzero\NC \NR
-\NC \type{low_page}         \NC number \NC \NC \NR
-\NC \type{high_page}        \NC number \NC \NC \NR
-\NC \type{iconv_name}       \NC string \NC \NC \NR
-\NC \type{iso_2022_escape}  \NC string \NC \NC \NR
-\stoptabulate
-
-\subsection{private table}
-
-This is the font's private \POSTSCRIPT\ dictionary, if any. Keys and values are
-both strings.
-
-\subsection{cidinfo table}
-
-\starttabulate[|l|l|p|]
-\BC key               \BC type   \BC explanation \NC \NR
-\NC \type{registry}   \NC string \NC \NC \NR
-\NC \type{ordering}   \NC string \NC \NC \NR
-\NC \type{supplement} \NC number \NC \NC \NR
-\NC \type{version}    \NC number \NC \NC \NR
-\stoptabulate
-
-\subsection[fontloaderpfminfotable]{pfminfo table}
-
-The \type {pfminfo} table contains most of the OS/2 information:
-
-\starttabulate[|l|l|p|]
-\BC key                     \BC type   \BC explanation \NC \NR
-\NC \type{pfmset}           \NC number \NC \NC \NR
-\NC \type{winascent_add}    \NC number \NC \NC \NR
-\NC \type{windescent_add}   \NC number \NC \NC \NR
-\NC \type{hheadascent_add}  \NC number \NC \NC \NR
-\NC \type{hheaddescent_add} \NC number \NC \NC \NR
-\NC \type{typoascent_add}   \NC number \NC \NC \NR
-\NC \type{typodescent_add}  \NC number \NC \NC \NR
-\NC \type{subsuper_set}     \NC number \NC \NC \NR
-\NC \type{panose_set}       \NC number \NC \NC \NR
-\NC \type{hheadset}         \NC number \NC \NC \NR
-\NC \type{vheadset}         \NC number \NC \NC \NR
-\NC \type{pfmfamily}        \NC number \NC \NC \NR
-\NC \type{weight}           \NC number \NC \NC \NR
-\NC \type{width}            \NC number \NC \NC \NR
-\NC \type{avgwidth}         \NC number \NC \NC \NR
-\NC \type{firstchar}        \NC number \NC \NC \NR
-\NC \type{lastchar}         \NC number \NC \NC \NR
-\NC \type{fstype}           \NC number \NC \NC \NR
-\NC \type{linegap}          \NC number \NC \NC \NR
-\NC \type{vlinegap}         \NC number \NC \NC \NR
-\NC \type{hhead_ascent}     \NC number \NC \NC \NR
-\NC \type{hhead_descent}    \NC number \NC \NC \NR
-\NC \type{os2_typoascent}   \NC number \NC \NC \NR
-\NC \type{os2_typodescent}  \NC number \NC \NC \NR
-\NC \type{os2_typolinegap}  \NC number \NC \NC \NR
-\NC \type{os2_winascent}    \NC number \NC \NC \NR
-\NC \type{os2_windescent}   \NC number \NC \NC \NR
-\NC \type{os2_subxsize}     \NC number \NC \NC \NR
-\NC \type{os2_subysize}     \NC number \NC \NC \NR
-\NC \type{os2_subxoff}      \NC number \NC \NC \NR
-\NC \type{os2_subyoff}      \NC number \NC \NC \NR
-\NC \type{os2_supxsize}     \NC number \NC \NC \NR
-\NC \type{os2_supysize}     \NC number \NC \NC \NR
-\NC \type{os2_supxoff}      \NC number \NC \NC \NR
-\NC \type{os2_supyoff}      \NC number \NC \NC \NR
-\NC \type{os2_strikeysize}  \NC number \NC \NC \NR
-\NC \type{os2_strikeypos}   \NC number \NC \NC \NR
-\NC \type{os2_family_class} \NC number \NC \NC \NR
-\NC \type{os2_xheight}      \NC number \NC \NC \NR
-\NC \type{os2_capheight}    \NC number \NC \NC \NR
-\NC \type{os2_defaultchar}  \NC number \NC \NC \NR
-\NC \type{os2_breakchar}    \NC number \NC \NC \NR
-\NC \type{os2_vendor}       \NC string \NC \NC \NR
-\NC \type{codepages}        \NC table  \NC A two-number array of encoded code pages \NC \NR
-\NC \type{unicoderages}     \NC table  \NC A four-number array of encoded unicode ranges \NC \NR
-\NC \type{panose}           \NC table  \NC \NC \NR
-\stoptabulate
-
-The \type {panose} subtable has exactly 10 string keys:
-
-\starttabulate[|l|l|p|]
-\BC key                    \BC type    \BC explanation \NC \NR
-\NC \type{familytype}      \NC string  \NC Values as in the \OPENTYPE\ font
-                                           specification: \type {Any}, \type {No Fit},
-                                           \type {Text and Display}, \type {Script},
-                                           \type {Decorative}, \type {Pictorial} \NC
-                                           \NR
-\NC \type{serifstyle}      \NC string  \NC See the \OPENTYPE\ font specification for
-                                           values \NC \NR
-\NC \type{weight}          \NC string  \NC idem \NC \NR
-\NC \type{proportion}      \NC string  \NC idem \NC \NR
-\NC \type{contrast}        \NC string  \NC idem \NC \NR
-\NC \type{strokevariation} \NC string  \NC idem \NC \NR
-\NC \type{armstyle}        \NC string  \NC idem \NC \NR
-\NC \type{letterform}      \NC string  \NC idem \NC \NR
-\NC \type{midline}         \NC string  \NC idem \NC \NR
-\NC \type{xheight}         \NC string  \NC idem \NC \NR
-\stoptabulate
-
-\subsection[fontloadernamestable]{names table}
-
-Each item has two top|-|level keys:
-
-\starttabulate[|l|l|p|]
-\BC key          \BC type   \BC explanation \NC \NR
-\NC \type{lang}  \NC string \NC language for this entry \NC \NR
-\NC \type{names} \NC table  \NC \NC \NR
-\stoptabulate
-
-The \type {names} keys are the actual \TRUETYPE\ name strings. The possible keys
-are:
-
-\starttabulate[|l|p|]
-\BC key                    \BC explanation \NC \NR
-\NC \type{copyright}       \NC \NC \NR
-\NC \type{family}          \NC \NC \NR
-\NC \type{subfamily}       \NC \NC \NR
-\NC \type{uniqueid}        \NC \NC \NR
-\NC \type{fullname}        \NC \NC \NR
-\NC \type{version}         \NC \NC \NR
-\NC \type{postscriptname}  \NC \NC \NR
-\NC \type{trademark}       \NC \NC \NR
-\NC \type{manufacturer}    \NC \NC \NR
-\NC \type{designer}        \NC \NC \NR
-\NC \type{descriptor}      \NC \NC \NR
-\NC \type{venderurl}       \NC \NC \NR
-\NC \type{designerurl}     \NC \NC \NR
-\NC \type{license}         \NC \NC \NR
-\NC \type{licenseurl}      \NC \NC \NR
-\NC \type{idontknow}       \NC \NC \NR
-\NC \type{preffamilyname}  \NC \NC \NR
-\NC \type{prefmodifiers}   \NC \NC \NR
-\NC \type{compatfull}      \NC \NC \NR
-\NC \type{sampletext}      \NC \NC \NR
-\NC \type{cidfindfontname} \NC \NC \NR
-\NC \type{wwsfamily}       \NC \NC \NR
-\NC \type{wwssubfamily}    \NC \NC \NR
-\stoptabulate
-
-\subsection{anchor_classes table}
-
-The anchor_classes classes:
-
-\starttabulate[|l|l|p|]
-\BC key           \BC type   \BC explanation \NC \NR
-\NC \type{name}   \NC string \NC a descriptive id of this anchor class\NC \NR
-\NC \type{lookup} \NC string \NC \NC \NR
-\NC \type{type}   \NC string \NC one of \type {mark}, \type {mkmk}, \type {curs}, \type {mklg} \NC \NR
-\stoptabulate
-
-% type is actually a lookup subtype, not a feature name. Officially, these
-% strings should be gpos_mark2mark etc.
-
-\subsection{gpos table}
-
-The \type {gpos} table has one array entry for each lookup. (The \type {gpos_}
-prefix is somewhat redundant.)
-
-\starttabulate[|l|l|p|]
-\BC key              \BC type   \BC explanation \NC \NR
-\NC \type{type}      \NC string \NC one of \type {gpos_single}, \type {gpos_pair},
-                                    \type {gpos_cursive}, \type {gpos_mark2base},\crlf
-                                    \type {gpos_mark2ligature}, \type
-                                    {gpos_mark2mark}, \type {gpos_context},\crlf \type
-                                    {gpos_contextchain} \NC \NR
-\NC \type{flags}     \NC table  \NC \NC \NR
-\NC \type{name}      \NC string \NC \NC \NR
-\NC \type{features}  \NC array  \NC \NC \NR
-\NC \type{subtables} \NC array  \NC \NC \NR
-\stoptabulate
-
-The flags table has a true value for each of the lookup flags that is actually
-set:
-
-\starttabulate[|l|l|p|]
-\BC key                         \BC type    \BC explanation \NC \NR
-\NC \type{r2l}                  \NC boolean \NC \NC \NR
-\NC \type{ignorebaseglyphs}     \NC boolean \NC \NC \NR
-\NC \type{ignoreligatures}      \NC boolean \NC \NC \NR
-\NC \type{ignorecombiningmarks} \NC boolean \NC \NC \NR
-\NC \type{mark_class}           \NC string  \NC \NC \NR
-\stoptabulate
-
-The features subtable items of gpos have:
-
-\starttabulate[|l|l|p|]
-\BC key            \BC type   \BC explanation \NC \NR
-\NC \type{tag}     \NC string \NC \NC \NR
-\NC \type{scripts} \NC table  \NC \NC \NR
-\stoptabulate
-
-The scripts table within features has:
-
-\starttabulate[|l|l|p|]
-\BC key           \BC type             \BC explanation \NC \NR
-\NC \type{script} \NC string           \NC \NC \NR
-\NC \type{langs}  \NC array of strings \NC \NC \NR
-\stoptabulate
-
-The subtables table has:
-
-\starttabulate[|l|l|p|]
-\BC key                     \BC type   \BC explanation \NC \NR
-\NC \type{name}             \NC string \NC \NC \NR
-\NC \type{suffix}           \NC string \NC (only if used)\NC \NR % used by gpos_single to get a default
-\NC \type{anchor_classes}   \NC number \NC (only if used)\NC \NR
-\NC \type{vertical_kerning} \NC number \NC (only if used)\NC \NR
-\NC \type{kernclass}        \NC table  \NC (only if used)\NC \NR
-\stoptabulate
-
-The kernclass with subtables table has:
-
-\starttabulate[|l|l|p|]
-\BC key            \BC type             \BC explanation \NC \NR
-\NC \type{firsts}  \NC array of strings \NC \NC \NR
-\NC \type{seconds} \NC array of strings \NC \NC \NR
-\NC \type{lookup}  \NC string or array  \NC associated lookup(s) \NC \NR
-\NC \type{offsets} \NC array of numbers \NC \NC \NR
-\stoptabulate
-
-Note: the kernclass (as far as we can see) always has one entry so it could be one level
-deep instead. Also the seconds start at \type {[2]} which is close to the fontforge
-internals so we keep that too.
-
-\subsection{gsub table}
-
-This has identical layout to the \type {gpos} table, except for the
-type:
-
-\starttabulate[|l|l|p|]
-\BC key         \BC type   \BC explanation \NC \NR
-\NC \type{type} \NC string \NC one of \type {gsub_single}, \type {gsub_multiple},
-                               \type {gsub_alternate}, \type
-                               {gsub_ligature},\crlf \type {gsub_context}, \type
-                               {gsub_contextchain}, \type
-                               {gsub_reversecontextchain} \NC \NR
-\stoptabulate
-
-\subsection{ttf_tables and ttf_tab_saved tables}
-
-\starttabulate[|l|l|p|]
-\BC key           \BC type   \BC explanation \NC \NR
-\NC \type{tag}    \NC string \NC \NC \NR
-\NC \type{len}    \NC number \NC \NC \NR
-\NC \type{maxlen} \NC number \NC \NC \NR
-\NC \type{data}   \NC number \NC \NC \NR
-\stoptabulate
-
-\subsection{mm table}
-
-\starttabulate[|l|l|p|]
-\BC key                   \BC type   \BC explanation \NC \NR
-\NC \type{axes}           \NC table  \NC array of axis names \NC \NR
-\NC \type{instance_count} \NC number \NC \NC \NR
-\NC \type{positions}      \NC table  \NC array of instance positions
-                                         (\#axes * instances )\NC \NR
-\NC \type{defweights}     \NC table  \NC array of default weights for instances \NC \NR
-\NC \type{cdv}            \NC string \NC \NC \NR
-\NC \type{ndv}            \NC string \NC \NC \NR
-\NC \type{axismaps}       \NC table  \NC \NC \NR
-\stoptabulate
-
-The \type {axismaps}:
-
-\starttabulate[|l|l|p|]
-\BC key            \BC type   \BC explanation \NC \NR
-\NC \type{blends}  \NC table  \NC an array of blend points \NC \NR
-\NC \type{designs} \NC table  \NC an array of design values \NC \NR
-\NC \type{min}     \NC number \NC \NC \NR
-\NC \type{def}     \NC number \NC \NC \NR
-\NC \type{max}     \NC number \NC \NC \NR
-\stoptabulate
-
-\subsection{mark_classes table}
-
-The keys in this table are mark class names, and the values are a
-space|-|separated string of glyph names in this class.
-
-\subsection{math table}
-
-\starttabulate[|l|p|]
-\NC \type{ScriptPercentScaleDown}                   \NC \NC \NR
-\NC \type{ScriptScriptPercentScaleDown}             \NC \NC \NR
-\NC \type{DelimitedSubFormulaMinHeight}             \NC \NC \NR
-\NC \type{DisplayOperatorMinHeight}                 \NC \NC \NR
-\NC \type{MathLeading}                              \NC \NC \NR
-\NC \type{AxisHeight}                               \NC \NC \NR
-\NC \type{AccentBaseHeight}                         \NC \NC \NR
-\NC \type{FlattenedAccentBaseHeight}                \NC \NC \NR
-\NC \type{SubscriptShiftDown}                       \NC \NC \NR
-\NC \type{SubscriptTopMax}                          \NC \NC \NR
-\NC \type{SubscriptBaselineDropMin}                 \NC \NC \NR
-\NC \type{SuperscriptShiftUp}                       \NC \NC \NR
-\NC \type{SuperscriptShiftUpCramped}                \NC \NC \NR
-\NC \type{SuperscriptBottomMin}                     \NC \NC \NR
-\NC \type{SuperscriptBaselineDropMax}               \NC \NC \NR
-\NC \type{SubSuperscriptGapMin}                     \NC \NC \NR
-\NC \type{SuperscriptBottomMaxWithSubscript}        \NC \NC \NR
-\NC \type{SpaceAfterScript}                         \NC \NC \NR
-\NC \type{UpperLimitGapMin}                         \NC \NC \NR
-\NC \type{UpperLimitBaselineRiseMin}                \NC \NC \NR
-\NC \type{LowerLimitGapMin}                         \NC \NC \NR
-\NC \type{LowerLimitBaselineDropMin}                \NC \NC \NR
-\NC \type{StackTopShiftUp}                          \NC \NC \NR
-\NC \type{StackTopDisplayStyleShiftUp}              \NC \NC \NR
-\NC \type{StackBottomShiftDown}                     \NC \NC \NR
-\NC \type{StackBottomDisplayStyleShiftDown}         \NC \NC \NR
-\NC \type{StackGapMin}                              \NC \NC \NR
-\NC \type{StackDisplayStyleGapMin}                  \NC \NC \NR
-\NC \type{StretchStackTopShiftUp}                   \NC \NC \NR
-\NC \type{StretchStackBottomShiftDown}              \NC \NC \NR
-\NC \type{StretchStackGapAboveMin}                  \NC \NC \NR
-\NC \type{StretchStackGapBelowMin}                  \NC \NC \NR
-\NC \type{FractionNumeratorShiftUp}                 \NC \NC \NR
-\NC \type{FractionNumeratorDisplayStyleShiftUp}     \NC \NC \NR
-\NC \type{FractionDenominatorShiftDown}             \NC \NC \NR
-\NC \type{FractionDenominatorDisplayStyleShiftDown} \NC \NC \NR
-\NC \type{FractionNumeratorGapMin}                  \NC \NC \NR
-\NC \type{FractionNumeratorDisplayStyleGapMin}      \NC \NC \NR
-\NC \type{FractionRuleThickness}                    \NC \NC \NR
-\NC \type{FractionDenominatorGapMin}                \NC \NC \NR
-\NC \type{FractionDenominatorDisplayStyleGapMin}    \NC \NC \NR
-\NC \type{SkewedFractionHorizontalGap}              \NC \NC \NR
-\NC \type{SkewedFractionVerticalGap}                \NC \NC \NR
-\NC \type{OverbarVerticalGap}                       \NC \NC \NR
-\NC \type{OverbarRuleThickness}                     \NC \NC \NR
-\NC \type{OverbarExtraAscender}                     \NC \NC \NR
-\NC \type{UnderbarVerticalGap}                      \NC \NC \NR
-\NC \type{UnderbarRuleThickness}                    \NC \NC \NR
-\NC \type{UnderbarExtraDescender}                   \NC \NC \NR
-\NC \type{RadicalVerticalGap}                       \NC \NC \NR
-\NC \type{RadicalDisplayStyleVerticalGap}           \NC \NC \NR
-\NC \type{RadicalRuleThickness}                     \NC \NC \NR
-\NC \type{RadicalExtraAscender}                     \NC \NC \NR
-\NC \type{RadicalKernBeforeDegree}                  \NC \NC \NR
-\NC \type{RadicalKernAfterDegree}                   \NC \NC \NR
-\NC \type{RadicalDegreeBottomRaisePercent}          \NC \NC \NR
-\NC \type{MinConnectorOverlap}                      \NC \NC \NR
-\NC \type{FractionDelimiterSize}                    \NC \NC \NR
-\NC \type{FractionDelimiterDisplayStyleSize}        \NC \NC \NR
-\stoptabulate
-
-\subsection{validation_state table}
-
-\starttabulate[|l|p|]
-\BC key                      \BC explanation \NC \NR
-\NC \type{bad_ps_fontname}   \NC \NC \NR
-\NC \type{bad_glyph_table}   \NC \NC \NR
-\NC \type{bad_cff_table}     \NC \NC \NR
-\NC \type{bad_metrics_table} \NC \NC \NR
-\NC \type{bad_cmap_table}    \NC \NC \NR
-\NC \type{bad_bitmaps_table} \NC \NC \NR
-\NC \type{bad_gx_table}      \NC \NC \NR
-\NC \type{bad_ot_table}      \NC \NC \NR
-\NC \type{bad_os2_version}   \NC \NC \NR
-\NC \type{bad_sfnt_header}   \NC \NC \NR
-\stoptabulate
-
-\subsection{horiz_base and vert_base table}
-
-\starttabulate[|l|l|p|]
-\BC key            \BC type  \BC explanation \NC \NR
-\NC \type{tags}    \NC table \NC an array of script list tags\NC \NR
-\NC \type{scripts} \NC table \NC \NC \NR
-\stoptabulate
-
-The \type {scripts} subtable:
-
-\starttabulate[|l|l|p|]
-\BC key                     \BC type    \BC explanation \NC \NR
-\NC \type{baseline}         \NC table   \NC \NC \NR
-\NC \type{default_baseline} \NC number  \NC \NC \NR
-\NC \type{lang}             \NC table   \NC \NC \NR
-\stoptabulate
-
-
-The \type {lang} subtable:
-
-\starttabulate[|l|l|p|]
-\BC key             \BC type   \BC explanation \NC \NR
-\NC \type{tag}      \NC string \NC a script tag \NC \NR
-\NC \type{ascent}   \NC number \NC \NC \NR
-\NC \type{descent}  \NC number \NC \NC \NR
-\NC \type{features} \NC table  \NC \NC \NR
-\stoptabulate
-
-The \type {features} points to an array of tables with the same layout except
-that in those nested tables, the tag represents a language.
-
-\subsection{altuni table}
-
-An array of alternate \UNICODE\ values. Inside that array are hashes with:
-
-\starttabulate[|l|l|p|]
-\BC key            \BC type   \BC explanation \NC \NR
-\NC \type{unicode} \NC number \NC this glyph is also used for this unicode \NC \NR
-\NC \type{variant} \NC number \NC the alternative is driven by this unicode selector \NC \NR
-\stoptabulate
-
-\subsection{vert_variants and horiz_variants table}
-
-\starttabulate[|l|l|p|]
-\BC key                      \BC type   \BC explanation \NC \NR
-\NC \type{variants}          \NC string \NC \NC \NR
-\NC \type{italic_correction} \NC number \NC \NC \NR
-\NC \type{parts}             \NC table  \NC \NC \NR
-\stoptabulate
-
-The \type {parts} table is an array of smaller tables:
-
-\starttabulate[|l|l|p|]
-\BC key              \BC type   \BC explanation \NC \NR
-\NC \type{component} \NC string \NC \NC \NR
-\NC \type{extender}  \NC number \NC \NC \NR
-\NC \type{start}     \NC number \NC \NC \NR
-\NC \type{end}       \NC number \NC \NC \NR
-\NC \type{advance}   \NC number \NC \NC \NR
-\stoptabulate
-
-\subsection{mathkern table}
-
-\starttabulate[|l|l|p|]
-\BC key                 \BC type  \BC explanation \NC \NR
-\NC \type{top_right}    \NC table \NC \NC \NR
-\NC \type{bottom_right} \NC table \NC \NC \NR
-\NC \type{top_left}     \NC table \NC \NC \NR
-\NC \type{bottom_left}  \NC table \NC \NC \NR
-\stoptabulate
-
-Each of the subtables is an array of small hashes with two keys:
-
-\starttabulate[|l|l|p|]
-\BC key           \BC type   \BC explanation \NC \NR
-\NC \type{height} \NC number \NC \NC \NR
-\NC \type{kern}   \NC number \NC \NC \NR
-\stoptabulate
-
-\subsection{kerns table}
-
-Substructure is identical to the per|-|glyph subtable.
-
-\subsection{vkerns table}
-
-Substructure is identical to the per|-|glyph subtable.
-
-\subsection{texdata table}
-
-\starttabulate[|l|l|p|]
-\BC key           \BC type   \BC explanation \NC \NR
-\NC \type{type}   \NC string \NC \type {unset}, \type {text}, \type {math}, \type {mathext} \NC \NR
-\NC \type{params} \NC array  \NC 22 font numeric parameters \NC \NR
-\stoptabulate
-
-\subsection{lookups table}
-
-Top|-|level \type {lookups} is quite different from the ones at character level.
-The keys in this hash are strings, the values the actual lookups, represented as
-dictionary tables.
-
-\starttabulate[|l|l|p|]
-\BC key                  \BC type   \BC explanation \NC \NR
-\NC \type{type}          \NC string \NC \NC \NR
-\NC \type{format}        \NC enum   \NC one of \type {glyphs}, \type {class}, \type {coverage}, \type {reversecoverage} \NC \NR
-\NC \type{tag}           \NC string \NC \NC \NR
-\NC \type{current_class} \NC array  \NC \NC \NR
-\NC \type{before_class}  \NC array  \NC \NC \NR
-\NC \type{after_class}   \NC array  \NC \NC \NR
-\NC \type{rules}         \NC array  \NC an array of rule items\NC \NR
-\stoptabulate
-
-Rule items have one common item and one specialized item:
-
-\starttabulate[|l|l|p|]
-\BC key                    \BC type  \BC explanation \NC \NR
-\NC \type{lookups}         \NC array \NC a linear array of lookup names\NC \NR
-\NC \type{glyphs}          \NC array \NC only if the parent's format is \type {glyphs}\NC \NR
-\NC \type{class}           \NC array \NC only if the parent's format is \type {class}\NC \NR
-\NC \type{coverage}        \NC array \NC only if the parent's format is \type {coverage}\NC \NR
-\NC \type{reversecoverage} \NC array \NC only if the parent's format is \type {reversecoverage}\NC \NR
-\stoptabulate
-
-A glyph table is:
-
-\starttabulate[|l|l|p|]
-\BC key          \BC type   \BC explanation \NC \NR
-\NC \type{names} \NC string \NC \NC \NR
-\NC \type{back}  \NC string \NC \NC \NR
-\NC \type{fore}  \NC string \NC \NC \NR
-\stoptabulate
-
-A class table is:
-
-\starttabulate[|l|l|p|]
-\BC key            \BC type  \BC explanation \NC \NR
-\NC \type{current} \NC array \NC of numbers \NC \NR
-\NC \type{before}  \NC array \NC of numbers  \NC \NR
-\NC \type{after}   \NC array \NC of numbers  \NC \NR
-\stoptabulate
-
-coverage:
-
-\starttabulate[|l|l|p|]
-\BC key            \BC type  \BC explanation \NC \NR
-\NC \type{current} \NC array \NC of strings \NC \NR
-\NC \type{before}  \NC array \NC of strings\NC \NR
-\NC \type{after}   \NC array \NC of strings \NC \NR
-\stoptabulate
-
-reversecoverage:
-
-\starttabulate[|l|l|p|]
-\BC key                 \BC type   \BC explanation \NC \NR
-\NC \type{current}      \NC array  \NC of strings \NC \NR
-\NC \type{before}       \NC array  \NC of strings\NC \NR
-\NC \type{after}        \NC array  \NC of strings \NC \NR
-\NC \type{replacements} \NC string \NC \NC \NR
-\stoptabulate
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-fontloader

+

+\startchapter[reference=fontloader,title={The fontloader}]

+

+\topicindex {fonts+loading}

+

+The fontloader library is sort of independent of the rest in the sense that it

+can load font into a \LUA\ table that then can be converted into a table suitable

+for \TEX. The library is an adapted subset of \FONTFORGE\ and as such gives a

+similar view on a font (which has advantages when you want to debug). We will not

+discuss \OPENTYPE\ in detail here as the \MICROSOFT\ website offers enough

+information about it. The tables returned by the loader are not that far from the

+standard. We have no plans to extend the loader (it may even become an external

+module at some time).

+

+\startsection[title={Getting quick information on a font}][library=fontloader]

+

+\topicindex {fonts+information}

+

+\libindex{info}

+

+When you want to locate font by name you need some basic information that is

+hidden in the font files. For that reason we provide an efficient helper that

+gets the basic information without loading all of the font. Normally this helper

+is used to create a font (name) database.

+

+\startfunctioncall

+<table> info =

+    fontloader.info(<string> filename)

+\stopfunctioncall

+

+This function returns either \type {nil}, or a \type {table}, or an array of

+small tables (in the case of a \TRUETYPE\ collection). The returned table(s) will

+contain some fairly interesting information items from the font(s) defined by the

+file:

+

+\starttabulate[|l|l|p|]

+\DB key                 \BC type     \BC explanation \NC \NR

+\TB

+\NC \type{fontname}     \NC string   \NC the \POSTSCRIPT\ name of the font\NC \NR

+\NC \type{fullname}     \NC string   \NC the formal name of the font\NC \NR

+\NC \type{familyname}   \NC string   \NC the family name this font belongs to\NC \NR

+\NC \type{weight}       \NC string   \NC a string indicating the color value of the font\NC \NR

+\NC \type{version}      \NC string   \NC the internal font version\NC \NR

+\NC \type{italicangle}  \NC float    \NC the slant angle\NC \NR

+\NC \type{units_per_em} \NC number   \NC 1000 for \POSTSCRIPT-based fonts, usually 2048 for \TRUETYPE\NC \NR

+\NC \type{pfminfo}      \NC table    \NC (see \in{section}[fontloaderpfminfotable])\NC \NR

+\LL

+\stoptabulate

+

+Getting information through this function is (sometimes much) more efficient than

+loading the font properly, and is therefore handy when you want to create a

+dictionary of available fonts based on a directory contents.

+

+\stopsection

+

+\startsection[title={Loading an \OPENTYPE\ or \TRUETYPE\ file}][library=fontloader]

+

+\topicindex {\OPENTYPE}

+\topicindex {\TRUETYPE}

+

+\libindex{open}

+\libindex{close}

+\libindex{to_table}

+

+If you want to use an \OPENTYPE\ font, you have to get the metric information

+from somewhere. Using the \type {fontloader} library, the simplest way to get

+that information is thus:

+

+\starttyping

+function load_font (filename)

+  local metrics = nil

+  local font = fontloader.open(filename)

+  if font then

+     metrics = fontloader.to_table(font)

+     fontloader.close(font)

+  end

+  return metrics

+end

+

+myfont = load_font('/opt/tex/texmf/fonts/data/arial.ttf')

+\stoptyping

+

+The main function call is

+

+\startfunctioncall

+<userdata> f, <table> w = fontloader.open(<string> filename)

+<userdata> f, <table> w = fontloader.open(<string> filename, <string> fontname)

+\stopfunctioncall

+

+The first return value is a userdata representation of the font. The second

+return value is a table containing any warnings and errors reported by fontloader

+while opening the font. In normal typesetting, you would probably ignore the

+second argument, but it can be useful for debugging purposes.

+

+For \TRUETYPE\ collections (when filename ends in 'ttc') and \DFONT\ collections,

+you have to use a second string argument to specify which font you want from the

+collection. Use the \type {fontname} strings that are returned by \type

+{fontloader.info} for that.

+

+To turn the font into a table, \type {fontloader.to_table} is used on the font

+returned by \type {fontloader.open}.

+

+\startfunctioncall

+<table> f = fontloader.to_table(<userdata> font)

+\stopfunctioncall

+

+This table cannot be used directly by \LUATEX\ and should be turned into another

+one as described in~\in {chapter} [fonts]. Do not forget to store the \type

+{fontname} value in the \type {psname} field of the metrics table to be returned

+to \LUATEX, otherwise the font inclusion backend will not be able to find the

+correct font in the collection.

+

+See \in {section} [fontloadertables] for details on the userdata object returned

+by \type {fontloader.open} and the layout of the \type {metrics} table returned

+by \type {fontloader.to_table}.

+

+The font file is parsed and partially interpreted by the font loading routines

+from \FONTFORGE. The file format can be \OPENTYPE, \TRUETYPE, \TRUETYPE\

+Collection, \CFF, or \TYPEONE.

+

+There are a few advantages to this approach compared to reading the actual font

+file ourselves:

+

+\startitemize

+

+\startitem

+    The font is automatically re|-|encoded, so that the \type {metrics} table for

+    \TRUETYPE\ and \OPENTYPE\ fonts is using \UNICODE\ for the character indices.

+\stopitem

+

+\startitem

+    Many features are pre|-|processed into a format that is easier to handle than

+    just the bare tables would be.

+\stopitem

+

+\startitem

+    \POSTSCRIPT|-|based \OPENTYPE\ fonts do not store the character height and

+    depth in the font file, so the character boundingbox has to be calculated in

+    some way.

+\stopitem

+

+\stopitemize

+

+A loaded font is discarded with:

+

+\startfunctioncall

+fontloader.close(<userdata> font)

+\stopfunctioncall

+

+\stopsection

+

+\startsection[title={Applying a \quote{feature file}}][library=fontloader]

+

+\libindex{apply_featurefile}

+

+You can apply a \quote{feature file} to a loaded font:

+

+\startfunctioncall

+<table> errors = fontloader.apply_featurefile(<userdata> font, <string> filename)

+\stopfunctioncall

+

+A \quote {feature file} is a textual representation of the features in an

+\OPENTYPE\ font. See

+

+\starttyping

+http://www.adobe.com/devnet/opentype/afdko/topic_feature_file_syntax.html

+\stoptyping

+

+and

+

+\starttyping

+http://fontforge.sourceforge.net/featurefile.html

+\stoptyping

+

+for a more detailed description of feature files.

+

+If the function fails, the return value is a table containing any errors reported

+by fontloader while applying the feature file. On success, \type {nil} is

+returned.

+

+\stopsection

+

+\startsection[title={Applying an \quote{\AFM\ file}}][library=fontloader]

+

+\topicindex {\TYPEONE}

+

+\libindex{apply_afmfile}

+

+You can apply an \quote {\AFM\ file} to a loaded font:

+

+\startfunctioncall

+<table> errors = fontloader.apply_afmfile(<userdata> font, <string> filename)

+\stopfunctioncall

+

+An \AFM\ file is a textual representation of (some of) the meta information

+in a \TYPEONE\ font. See

+

+\starttyping

+ftp://ftp.math.utah.edu/u/ma/hohn/linux/postscript/5004.AFM_Spec.pdf

+\stoptyping

+

+for more information about \AFM\ files.

+

+Note: If you \type {fontloader.open} a \TYPEONE\ file named \type {font.pfb},

+the library will automatically search for and apply \type {font.afm} if it exists

+in the same directory as the file \type {font.pfb}. In that case, there is no

+need for an explicit call to \type {apply_afmfile()}.

+

+If the function fails, the return value is a table containing any errors reported

+by fontloader while applying the AFM file. On success, \type {nil} is returned.

+

+\stopsection

+

+\startsection[title={Fontloader font tables},reference=fontloadertables][library=fontloader]

+

+\topicindex {fontloader+tables}

+

+\libindex{fields}

+

+As mentioned earlier, the return value of \type {fontloader.open} is a userdata

+object. One way to have access to the actual metrics is to call \type

+{fontloader.to_table} on this object, returning the table structure that is

+explained in the following sections. In teh following sections we will not

+explain each field in detail. Most fields are self descriptive and for the more

+technical aspects you need to consult the relevant font references.

+

+It turns out that the result from \type {fontloader.to_table} sometimes needs

+very large amounts of memory (depending on the font's complexity and size) so it

+is possible to access the userdata object directly.

+

+\startitemize

+\startitem

+    All top|-|level keys that would be returned by \type {to_table()}

+    can also be accessed directly.

+\stopitem

+\startitem

+    The top|-|level key \quote {glyphs} returns a {\it virtual\/} array that

+    allows indices from \type {f.glyphmin} to (\type {f.glyphmax}).

+\stopitem

+\startitem

+    The items in that virtual array (the actual glyphs) are themselves also

+    userdata objects, and each has accessors for all of the keys explained in the

+    section \quote {Glyph items} below.

+\stopitem

+\startitem

+    The top|-|level key \quote {subfonts} returns an {\it actual} array of userdata

+    objects, one for each of the subfonts (or nil, if there are no subfonts).

+\stopitem

+\stopitemize

+

+A short example may be helpful. This code generates a printout of all

+the glyph names in the font \type {PunkNova.kern.otf}:

+

+\starttyping

+local f = fontloader.open('PunkNova.kern.otf')

+print (f.fontname)

+local i = 0

+if f.glyphcnt > 0 then

+    for i=f.glyphmin,f.glyphmax do

+       local g = f.glyphs[i]

+       if g then

+          print(g.name)

+       end

+       i = i + 1

+    end

+end

+fontloader.close(f)

+\stoptyping

+

+In this case, the \LUATEX\ memory requirement stays below 100MB on the test

+computer, while the internal structure generated by \type {to_table()} needs more

+than 2GB of memory (the font itself is 6.9MB in disk size).

+

+Only the top|-|level font, the subfont table entries, and the glyphs are virtual

+objects, everything else still produces normal \LUA\ values and tables.

+

+If you want to know the valid fields in a font or glyph structure, call the \type

+{fields} function on an object of a particular type (either glyph or font):

+

+\startfunctioncall

+<table> fields = fontloader.fields(<userdata> font)

+<table> fields = fontloader.fields(<userdata> font_glyph)

+\stopfunctioncall

+

+For instance:

+

+\startfunctioncall

+local fields = fontloader.fields(f)

+local fields = fontloader.fields(f.glyphs[0])

+\stopfunctioncall

+

+\stopsection

+

+\startsection[title={Table types}][library=fontloader]

+

+\startsubsection[title={The main table}]

+

+The top|-|level keys in the returned table are (the explanations in this part of

+the documentation are not yet finished):

+

+\starttabulate[|l|l|p|]

+\DB key                                 \BC type     \NC explanation \NC \NR

+\TB

+\NC \type{table_version}                \NC number   \NC indicates the metrics version (currently~0.3)\NC \NR

+\NC \type{fontname}                     \NC string   \NC \POSTSCRIPT\ font name\NC \NR

+\NC \type{fullname}                     \NC string   \NC official (human-oriented) font name\NC \NR

+\NC \type{familyname}                   \NC string   \NC family name\NC \NR

+\NC \type{weight}                       \NC string   \NC weight indicator\NC \NR

+\NC \type{copyright}                    \NC string   \NC copyright information\NC \NR

+\NC \type{filename}                     \NC string   \NC the file name\NC \NR

+\NC \type{version}                      \NC string   \NC font version\NC \NR

+\NC \type{italicangle}                  \NC float    \NC slant angle\NC \NR

+\NC \type{units_per_em}                 \NC number   \NC 1000 for \POSTSCRIPT-based fonts, usually 2048 for \TRUETYPE\NC \NR

+\NC \type{ascent}                       \NC number   \NC height of ascender in \type {units_per_em}\NC \NR

+\NC \type{descent}                      \NC number   \NC depth of descender in \type {units_per_em}\NC \NR

+\NC \type{upos}                         \NC float    \NC \NC \NR

+\NC \type{uwidth}                       \NC float    \NC \NC \NR

+\NC \type{uniqueid}                     \NC number   \NC \NC \NR

+\NC \type{glyphs}                       \NC array    \NC \NC \NR

+\NC \type{glyphcnt}                     \NC number   \NC number of included glyphs\NC \NR

+\NC \type{glyphmax}                     \NC number   \NC maximum used index the glyphs array\NC \NR

+\NC \type{glyphmin}                     \NC number   \NC minimum used index the glyphs array\NC \NR

+\NC \type{notdef_loc}                   \NC number   \NC location of the \type {.notdef} glyph

+                                                         or \type {-1} when not present \NC \NR

+\NC \type{hasvmetrics}                  \NC number   \NC \NC \NR

+\NC \type{onlybitmaps}                  \NC number   \NC \NC \NR

+\NC \type{serifcheck}                   \NC number   \NC \NC \NR

+\NC \type{isserif}                      \NC number   \NC \NC \NR

+\NC \type{issans}                       \NC number   \NC \NC \NR

+\NC \type{encodingchanged}              \NC number   \NC \NC \NR

+\NC \type{strokedfont}                  \NC number   \NC \NC \NR

+\NC \type{use_typo_metrics}             \NC number   \NC \NC \NR

+\NC \type{weight_width_slope_only}      \NC number   \NC \NC \NR

+\NC \type{head_optimized_for_cleartype} \NC number   \NC \NC \NR

+\NC \type{uni_interp}                   \NC enum     \NC \nod {unset}, \type {none}, \type {adobe},

+                                                         \type {greek}, \type {japanese}, \type {trad_chinese},

+                                                         \type {simp_chinese}, \type {korean}, \type {ams}\NC \NR

+\NC \type{origname}                     \NC string   \NC the file name, as supplied by the user\NC \NR

+\NC \type{map}                          \NC table    \NC \NC \NR

+\NC \type{private}                      \NC table    \NC \NC \NR

+\NC \type{xuid}                         \NC string   \NC \NC \NR

+\NC \type{pfminfo}                      \NC table    \NC \NC \NR

+\NC \type{names}                        \NC table    \NC \NC \NR

+\NC \type{cidinfo}                      \NC table    \NC \NC \NR

+\NC \type{subfonts}                     \NC array    \NC \NC \NR

+\NC \type{commments}                    \NC string   \NC \NC \NR

+\NC \type{fontlog}                      \NC string   \NC \NC \NR

+\NC \type{cvt_names}                    \NC string   \NC \NC \NR

+\NC \type{anchor_classes}               \NC table    \NC \NC \NR

+\NC \type{ttf_tables}                   \NC table    \NC \NC \NR

+\NC \type{ttf_tab_saved}                \NC table    \NC \NC \NR

+\NC \type{kerns}                        \NC table    \NC \NC \NR

+\NC \type{vkerns}                       \NC table    \NC \NC \NR

+\NC \type{texdata}                      \NC table    \NC \NC \NR

+\NC \type{lookups}                      \NC table    \NC \NC \NR

+\NC \type{gpos}                         \NC table    \NC \NC \NR

+\NC \type{gsub}                         \NC table    \NC \NC \NR

+\NC \type{mm}                           \NC table    \NC \NC \NR

+\NC \type{chosenname}                   \NC string   \NC \NC \NR

+\NC \type{macstyle}                     \NC number   \NC \NC \NR

+\NC \type{fondname}                     \NC string   \NC \NC \NR

+%NC \type{design_size}                  \NC number   \NC \NC \NR

+\NC \type{fontstyle_id}                 \NC number   \NC \NC \NR

+\NC \type{fontstyle_name}               \NC table    \NC \NC \NR

+%NC \type{design_range_bottom}          \NC number   \NC \NC \NR

+%NC \type{design_range_top}             \NC number   \NC \NC \NR

+\NC \type{strokewidth}                  \NC float    \NC \NC \NR

+\NC \type{mark_classes}                 \NC table    \NC \NC \NR

+\NC \type{creationtime}                 \NC number   \NC \NC \NR

+\NC \type{modificationtime}             \NC number   \NC \NC \NR

+\NC \type{os2_version}                  \NC number   \NC \NC \NR

+\NC \type{math}                         \NC table    \NC \NC \NR

+\NC \type{validation_state}             \NC table    \NC \NC \NR

+\NC \type{horiz_base}                   \NC table    \NC \NC \NR

+\NC \type{vert_base}                    \NC table    \NC \NC \NR

+\NC \type{extrema_bound}                \NC number   \NC \NC \NR

+\NC \type{truetype}                     \NC boolean  \NC signals a \TRUETYPE\ font \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {glyphs}}]

+

+The \type {glyphs} is an array containing the per|-|character

+information (quite a few of these are only present if non|-|zero).

+

+\starttabulate[|l|l|p|]

+\DB key                      \BC type     \BC explanation \NC \NR

+\TB

+\NC \type{name}              \NC string   \NC the glyph name \NC \NR

+\NC \type{unicode}           \NC number   \NC unicode code point, or -1 \NC \NR

+\NC \type{boundingbox}       \NC array    \NC array of four numbers, see note below \NC \NR

+\NC \type{width}             \NC number   \NC only for horizontal fonts \NC \NR

+\NC \type{vwidth}            \NC number   \NC only for vertical fonts \NC \NR

+\NC \type{tsidebearing}      \NC number   \NC only for vertical ttf/otf fonts, and only if non|-|zero \NC \NR

+\NC \type{lsidebearing}      \NC number   \NC only if non|-|zero and not equal to boundingbox[1] \NC \NR

+\NC \type{class}             \NC string   \NC one of "none", "base", "ligature", "mark", "component"

+                                              (if not present, the glyph class is \quote {automatic}) \NC \NR

+\NC \type{kerns}             \NC array    \NC only for horizontal fonts, if set \NC \NR

+\NC \type{vkerns}            \NC array    \NC only for vertical fonts, if set \NC \NR

+\NC \type{dependents}        \NC array    \NC linear array of glyph name strings, only if nonempty\NC \NR

+\NC \type{lookups}           \NC table    \NC only if nonempty \NC \NR

+\NC \type{ligatures}         \NC table    \NC only if nonempty \NC \NR

+\NC \type{anchors}           \NC table    \NC only if set \NC \NR

+\NC \type{comment}           \NC string   \NC only if set \NC \NR

+\NC \type{tex_height}        \NC number   \NC only if set \NC \NR

+\NC \type{tex_depth}         \NC number   \NC only if set \NC \NR

+\NC \type{italic_correction} \NC number   \NC only if set \NC \NR

+\NC \type{top_accent}        \NC number   \NC only if set \NC \NR

+\NC \type{is_extended_shape} \NC number   \NC only if this character is part of a math extension list \NC \NR

+\NC \type{altuni}            \NC table    \NC alternate \UNICODE\ items \NC \NR

+\NC \type{vert_variants}     \NC table    \NC \NC \NR

+\NC \type{horiz_variants}    \NC table    \NC \NC \NR

+\NC \type{mathkern}          \NC table    \NC \NC \NR

+\LL

+\stoptabulate

+

+On \type {boundingbox}: The boundingbox information for \TRUETYPE\ fonts and

+\TRUETYPE-based \OTF\ fonts is read directly from the font file.

+\POSTSCRIPT-based fonts do not have this information, so the boundingbox of

+traditional \POSTSCRIPT\ fonts is generated by interpreting the actual bezier

+curves to find the exact boundingbox. This can be a slow process, so the

+boundingboxes of \POSTSCRIPT-based \OTF\ fonts (and raw \CFF\ fonts) are

+calculated using an approximation of the glyph shape based on the actual glyph

+points only, instead of taking the whole curve into account. This means that

+glyphs that have missing points at extrema will have a too|-|tight boundingbox,

+but the processing is so much faster that in our opinion the tradeoff is worth

+it.

+

+The \type {kerns} and \type {vkerns} are linear arrays of small hashes:

+

+\starttabulate[|l|l|p|]

+\DB key           \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{char}   \NC string \NC \NC \NR

+\NC \type{off}    \NC number \NC \NC \NR

+\NC \type{lookup} \NC string \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {lookups} is a hash, based on lookup subtable names, with

+the value of each key inside that a linear array of small hashes:

+

+% TODO: fix this description

+

+\starttabulate[|l|l|p|]

+\DB key                  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{type}          \NC enum \NC \type {position}, \type {pair}, \type

+                                      {substitution}, \type {alternate}, \type

+                                      {multiple}, \type {ligature}, \type

+                                      {lcaret}, \cbk {kerning}, \type {vkerning},

+                                      \type {anchors}, \type {contextpos}, \type

+                                      {contextsub}, \type {chainpos}, \type

+                                      {chainsub}, \type {reversesub}, \type

+                                      {max}, \type {kernback}, \type {vkernback}

+                                      \NC \NR

+\NC \type{specification} \NC table \NC extra data \NC \NR

+\LL

+\stoptabulate

+

+For the first seven values of \type {type}, there can be additional

+sub|-|information, stored in the sub-table \type {specification}:

+

+\starttabulate[|l|l|p|]

+\DB value               \BC type     \BC explanation \NC \NR

+\TB

+\NC \type{position}     \NC table    \NC a table of the \type {offset_specs} type \NC \NR

+\NC \type{pair}         \NC table    \NC one string: \type {paired}, and an array of one

+                                         or two \type {offset_specs} tables: \type {offsets} \NC \NR

+\NC \type{substitution} \NC table    \NC one string: \type {variant} \NC \NR

+\NC \type{alternate}    \NC table    \NC one string: \type {components} \NC \NR

+\NC \type{multiple}     \NC table    \NC one string: \type {components} \NC \NR

+\NC \type{ligature}     \NC table    \NC two strings: \type {components}, \type {char} \NC \NR

+\NC \type{lcaret}       \NC array    \NC linear array of numbers \NC \NR

+\LL

+\stoptabulate

+

+Tables for \type {offset_specs} contain up to four number|-|valued fields: \type

+{x} (a horizontal offset), \type {y} (a vertical offset), \type {h} (an advance

+width correction) and \type {v} (an advance height correction).

+

+The \type {ligatures} is a linear array of small hashes:

+

+\starttabulate[|l|l|p|]

+\DB key               \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{lig}        \NC table  \NC uses the same substructure as a single item in

+                                     the \type {lookups} table explained above \NC \NR

+\NC \type{char}       \NC string \NC \NC \NR

+\NC \type{components} \NC array  \NC linear array of named components \NC \NR

+\NC \type{ccnt}       \NC number \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {anchor} table is indexed by a string signifying the anchor type, which

+is one of:

+

+\starttabulate[|l|l|p|]

+\DB key             \BC type  \BC explanation \NC \NR

+\TB

+\NC \type{mark}     \NC table \NC placement mark \NC \NR

+\NC \type{basechar} \NC table \NC mark for attaching combining items to a base char \NC \NR

+\NC \type{baselig}  \NC table \NC mark for attaching combining items to a ligature \NC \NR

+\NC \type{basemark} \NC table \NC generic mark for attaching combining items to connect to \NC \NR

+\NC \type{centry}   \NC table \NC cursive entry point \NC \NR

+\NC \type{cexit}    \NC table \NC cursive exit point \NC \NR

+\LL

+\stoptabulate

+

+The content of these is a short array of defined anchors, with the

+entry keys being the anchor names. For all except \type {baselig}, the

+value is a single table with this definition:

+

+\starttabulate[|l|l|p|]

+\DB key                 \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{x}            \NC number \NC x location \NC \NR

+\NC \type{y}            \NC number \NC y location \NC \NR

+\NC \type{ttf_pt_index} \NC number \NC truetype point index, only if given \NC \NR

+\LL

+\stoptabulate

+

+For \type {baselig}, the value is a small array of such anchor sets sets, one for

+each constituent item of the ligature.

+

+For clarification, an anchor table could for example look like this :

+

+\starttyping

+['anchor'] = {

+    ['basemark'] = {

+        ['Anchor-7'] = { ['x']=170, ['y']=1080 }

+    },

+    ['mark'] ={

+        ['Anchor-1'] = { ['x']=160, ['y']=810 },

+        ['Anchor-4'] = { ['x']=160, ['y']=800 }

+    },

+    ['baselig'] = {

+        [1] = { ['Anchor-2'] = { ['x']=160, ['y']=650 } },

+        [2] = { ['Anchor-2'] = { ['x']=460, ['y']=640 } }

+        }

+    }

+\stoptyping

+

+Note: The \type {baselig} table can be sparse!

+

+\stopsubsection

+

+\startsubsection[title={\type {map}}]

+

+The top|-|level map is a list of encoding mappings. Each of those is a table

+itself.

+

+\starttabulate[|l|l|p|]

+\DB key             \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{enccount} \NC number \NC \NC \NR

+\NC \type{encmax}   \NC number \NC \NC \NR

+\NC \type{backmax}  \NC number \NC \NC \NR

+\NC \type{remap}    \NC table  \NC \NC \NR

+\NC \type{map}      \NC array  \NC non|-|linear array of mappings\NC \NR

+\NC \type{backmap}  \NC array  \NC non|-|linear array of backward mappings\NC \NR

+\NC \type{enc}      \NC table  \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {remap} table is very small:

+

+\starttabulate[|l|l|p|]

+\DB key             \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{firstenc} \NC number \NC \NC \NR

+\NC \type{lastenc}  \NC number \NC \NC \NR

+\NC \type{infont}   \NC number \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {enc} table is a bit more verbose:

+

+\starttabulate[|l|l|p|]

+\DB key                     \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{enc_name}         \NC string \NC \NC \NR

+\NC \type{char_cnt}         \NC number \NC \NC \NR

+\NC \type{char_max}         \NC number \NC \NC \NR

+\NC \type{unicode}          \NC array  \NC of \UNICODE\ position numbers\NC \NR

+\NC \type{psnames}          \NC array  \NC of \POSTSCRIPT\ glyph names\NC \NR

+\NC \type{builtin}          \NC number \NC \NC \NR

+\NC \type{hidden}           \NC number \NC \NC \NR

+\NC \type{only_1byte}       \NC number \NC \NC \NR

+\NC \type{has_1byte}        \NC number \NC \NC \NR

+\NC \type{has_2byte}        \NC number \NC \NC \NR

+\NC \type{is_unicodebmp}    \NC number \NC only if non|-|zero\NC \NR

+\NC \type{is_unicodefull}   \NC number \NC only if non|-|zero\NC \NR

+\NC \type{is_custom}        \NC number \NC only if non|-|zero\NC \NR

+\NC \type{is_original}      \NC number \NC only if non|-|zero\NC \NR

+\NC \type{is_compact}       \NC number \NC only if non|-|zero\NC \NR

+\NC \type{is_japanese}      \NC number \NC only if non|-|zero\NC \NR

+\NC \type{is_korean}        \NC number \NC only if non|-|zero\NC \NR

+\NC \type{is_tradchinese}   \NC number \NC only if non|-|zero [name?]\NC \NR

+\NC \type{is_simplechinese} \NC number \NC only if non|-|zero\NC \NR

+\NC \type{low_page}         \NC number \NC \NC \NR

+\NC \type{high_page}        \NC number \NC \NC \NR

+\NC \type{iconv_name}       \NC string \NC \NC \NR

+\NC \type{iso_2022_escape}  \NC string \NC \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {private}}]

+

+This is the font's private \POSTSCRIPT\ dictionary, if any. Keys and values are

+both strings.

+

+\stopsubsection

+

+\startsubsection[title={\type {cidinfo}}]

+

+\starttabulate[|l|l|p|]

+\DB key               \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{registry}   \NC string \NC \NC \NR

+\NC \type{ordering}   \NC string \NC \NC \NR

+\NC \type{supplement} \NC number \NC \NC \NR

+\NC \type{version}    \NC number \NC \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[reference=fontloaderpfminfotable,title={\type {pfminfo}}]

+

+The \type {pfminfo} table contains most of the OS/2 information:

+

+\starttabulate[|l|l|p|]

+\DB key                     \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{pfmset}           \NC number \NC \NC \NR

+\NC \type{winascent_add}    \NC number \NC \NC \NR

+\NC \type{windescent_add}   \NC number \NC \NC \NR

+\NC \type{hheadascent_add}  \NC number \NC \NC \NR

+\NC \type{hheaddescent_add} \NC number \NC \NC \NR

+\NC \type{typoascent_add}   \NC number \NC \NC \NR

+\NC \type{typodescent_add}  \NC number \NC \NC \NR

+\NC \type{subsuper_set}     \NC number \NC \NC \NR

+\NC \type{panose_set}       \NC number \NC \NC \NR

+\NC \type{hheadset}         \NC number \NC \NC \NR

+\NC \type{vheadset}         \NC number \NC \NC \NR

+\NC \type{pfmfamily}        \NC number \NC \NC \NR

+\NC \type{weight}           \NC number \NC \NC \NR

+\NC \type{width}            \NC number \NC \NC \NR

+\NC \type{avgwidth}         \NC number \NC \NC \NR

+\NC \type{firstchar}        \NC number \NC \NC \NR

+\NC \type{lastchar}         \NC number \NC \NC \NR

+\NC \type{fstype}           \NC number \NC \NC \NR

+\NC \type{linegap}          \NC number \NC \NC \NR

+\NC \type{vlinegap}         \NC number \NC \NC \NR

+\NC \type{hhead_ascent}     \NC number \NC \NC \NR

+\NC \type{hhead_descent}    \NC number \NC \NC \NR

+\NC \type{os2_typoascent}   \NC number \NC \NC \NR

+\NC \type{os2_typodescent}  \NC number \NC \NC \NR

+\NC \type{os2_typolinegap}  \NC number \NC \NC \NR

+\NC \type{os2_winascent}    \NC number \NC \NC \NR

+\NC \type{os2_windescent}   \NC number \NC \NC \NR

+\NC \type{os2_subxsize}     \NC number \NC \NC \NR

+\NC \type{os2_subysize}     \NC number \NC \NC \NR

+\NC \type{os2_subxoff}      \NC number \NC \NC \NR

+\NC \type{os2_subyoff}      \NC number \NC \NC \NR

+\NC \type{os2_supxsize}     \NC number \NC \NC \NR

+\NC \type{os2_supysize}     \NC number \NC \NC \NR

+\NC \type{os2_supxoff}      \NC number \NC \NC \NR

+\NC \type{os2_supyoff}      \NC number \NC \NC \NR

+\NC \type{os2_strikeysize}  \NC number \NC \NC \NR

+\NC \type{os2_strikeypos}   \NC number \NC \NC \NR

+\NC \type{os2_family_class} \NC number \NC \NC \NR

+\NC \type{os2_xheight}      \NC number \NC \NC \NR

+\NC \type{os2_capheight}    \NC number \NC \NC \NR

+\NC \type{os2_defaultchar}  \NC number \NC \NC \NR

+\NC \type{os2_breakchar}    \NC number \NC \NC \NR

+\NC \type{os2_vendor}       \NC string \NC \NC \NR

+\NC \type{codepages}        \NC table  \NC A two-number array of encoded code pages \NC \NR

+\NC \type{unicoderages}     \NC table  \NC A four-number array of encoded unicode ranges \NC \NR

+\NC \type{panose}           \NC table  \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {panose} subtable has exactly 10 string keys:

+

+\starttabulate[|l|l|p|]

+\DB key                    \BC type    \BC explanation \NC \NR

+\TB

+\NC \type{familytype}      \NC string  \NC Values as in the \OPENTYPE\ font

+                                           specification: \type {Any}, \type {No Fit},

+                                           \type {Text and Display}, \type {Script},

+                                           \type {Decorative}, \type {Pictorial} \NC

+                                           \NR

+\NC \type{serifstyle}      \NC string  \NC See the \OPENTYPE\ font specification for

+                                           values \NC \NR

+\NC \type{weight}          \NC string  \NC idem \NC \NR

+\NC \type{proportion}      \NC string  \NC idem \NC \NR

+\NC \type{contrast}        \NC string  \NC idem \NC \NR

+\NC \type{strokevariation} \NC string  \NC idem \NC \NR

+\NC \type{armstyle}        \NC string  \NC idem \NC \NR

+\NC \type{letterform}      \NC string  \NC idem \NC \NR

+\NC \type{midline}         \NC string  \NC idem \NC \NR

+\NC \type{xheight}         \NC string  \NC idem \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[reference=fontloadernamestable,title={\type {names}}]

+

+Each item has two top|-|level keys:

+

+\starttabulate[|l|l|p|]

+\DB key          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{lang}  \NC string \NC language for this entry \NC \NR

+\NC \type{names} \NC table  \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {names} keys are the actual \TRUETYPE\ name strings. The possible keys

+are: \type {copyright}, \type {family}, \type {subfamily}, \type {uniqueid},

+\type {fullname}, \type {version}, \type {postscriptname}, \type {trademark},

+\type {manufacturer}, \type {designer}, \type {descriptor}, \type {venderurl},

+\type {designerurl}, \type {license}, \type {licenseurl}, \type {idontknow},

+\type {preffamilyname}, \type {prefmodifiers}, \type {compatfull}, \type

+{sampletext}, \type {cidfindfontname}, \type {wwsfamily} and \type

+{wwssubfamily}.

+

+\stopsubsection

+

+\startsubsection[title={\type {anchor_classes}}]

+

+The anchor_classes classes:

+

+\starttabulate[|l|l|p|]

+\DB key           \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{name}   \NC string \NC a descriptive id of this anchor class\NC \NR

+\NC \type{lookup} \NC string \NC \NC \NR

+\NC \type{type}   \NC string \NC one of \nod {mark}, \type {mkmk}, \type {curs}, \type {mklg} \NC \NR

+\LL

+\stoptabulate

+

+% type is actually a lookup subtype, not a feature name. Officially, these

+% strings should be gpos_mark2mark etc.

+

+\stopsubsection

+

+\startsubsection[title={\type {gpos}}]

+

+The \type {gpos} table has one array entry for each lookup. (The \type {gpos_}

+prefix is somewhat redundant.)

+

+\starttabulate[|l|l|p|]

+\DB key              \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{type}      \NC string \NC one of \type {gpos_single}, \type {gpos_pair},

+                                    \type {gpos_cursive}, \type {gpos_mark2base},\crlf

+                                    \type {gpos_mark2ligature}, \type {gpos_mark2mark}, \type {gpos_context},\crlf \type {gpos_contextchain} \NC \NR

+\NC \type{flags}     \NC table  \NC \NC \NR

+\NC \type{name}      \NC string \NC \NC \NR

+\NC \type{features}  \NC array  \NC \NC \NR

+\NC \type{subtables} \NC array  \NC \NC \NR

+\LL

+\stoptabulate

+

+The flags table has a true value for each of the lookup flags that is actually

+set:

+

+\starttabulate[|l|l|p|]

+\DB key                         \BC type    \BC explanation \NC \NR

+\TB

+\NC \type{r2l}                  \NC boolean \NC \NC \NR

+\NC \type{ignorebaseglyphs}     \NC boolean \NC \NC \NR

+\NC \type{ignoreligatures}      \NC boolean \NC \NC \NR

+\NC \type{ignorecombiningmarks} \NC boolean \NC \NC \NR

+\NC \type{mark_class}           \NC string  \NC \NC \NR

+\LL

+\stoptabulate

+

+The features subtable items of gpos have:

+

+\starttabulate[|l|l|p|]

+\DB key            \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{tag}     \NC string \NC \NC \NR

+\NC \type{scripts} \NC table  \NC \NC \NR

+\LL

+\stoptabulate

+

+The scripts table within features has:

+

+\starttabulate[|l|l|p|]

+\DB key           \BC type             \BC explanation \NC \NR

+\TB

+\NC \type{script} \NC string           \NC \NC \NR

+\NC \type{langs}  \NC array of strings \NC \NC \NR

+\LL

+\stoptabulate

+

+The subtables table has:

+

+\starttabulate[|l|l|p|]

+\DB key                     \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{name}             \NC string \NC \NC \NR

+\NC \type{suffix}           \NC string \NC (only if used)\NC \NR % used by gpos_single to get a default

+\NC \type{anchor_classes}   \NC number \NC (only if used)\NC \NR

+\NC \type{vertical_kerning} \NC number \NC (only if used)\NC \NR

+\NC \type{kernclass}        \NC table  \NC (only if used)\NC \NR

+\LL

+\stoptabulate

+

+The kernclass with subtables table has:

+

+\starttabulate[|l|l|p|]

+\DB key            \BC type             \BC explanation \NC \NR

+\TB

+\NC \type{firsts}  \NC array of strings \NC \NC \NR

+\NC \type{seconds} \NC array of strings \NC \NC \NR

+\NC \type{lookup}  \NC string or array  \NC associated lookup(s) \NC \NR

+\NC \type{offsets} \NC array of numbers \NC \NC \NR

+\LL

+\stoptabulate

+

+Note: the kernclass (as far as we can see) always has one entry so it could be one level

+deep instead. Also the seconds start at \type {[2]} which is close to the fontforge

+internals so we keep that too.

+

+\stopsubsection

+

+\startsubsection[title={\type {gsub}}]

+

+This has identical layout to the \type {gpos} table, except for the

+type:

+

+\starttabulate[|l|l|p|]

+\DB key         \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{type} \NC string \NC one of \type {gsub_single}, \type {gsub_multiple},

+                               \type {gsub_alternate}, \type {gsub_ligature},\crlf

+                               \type {gsub_context}, \type {gsub_contextchain},

+                               \type {gsub_reversecontextchain} \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {ttf_tables} and \type {ttf_tab_saved}}]

+

+\starttabulate[|l|l|p|]

+\DB key           \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{tag}    \NC string \NC \NC \NR

+\NC \type{len}    \NC number \NC \NC \NR

+\NC \type{maxlen} \NC number \NC \NC \NR

+\NC \type{data}   \NC number \NC \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {mm}}]

+

+\starttabulate[|l|l|p|]

+\DB key                   \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{axes}           \NC table  \NC array of axis names \NC \NR

+\NC \type{instance_count} \NC number \NC \NC \NR

+\NC \type{positions}      \NC table  \NC array of instance positions

+                                         (\#axes * instances )\NC \NR

+\NC \type{defweights}     \NC table  \NC array of default weights for instances \NC \NR

+\NC \type{cdv}            \NC string \NC \NC \NR

+\NC \type{ndv}            \NC string \NC \NC \NR

+\NC \type{axismaps}       \NC table  \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {axismaps}:

+

+\starttabulate[|l|l|p|]

+\DB key            \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{blends}  \NC table  \NC an array of blend points \NC \NR

+\NC \type{designs} \NC table  \NC an array of design values \NC \NR

+\NC \type{min}     \NC number \NC \NC \NR

+\NC \type{def}     \NC number \NC \NC \NR

+\NC \type{max}     \NC number \NC \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {mark_classes}}]

+

+The keys in this table are mark class names, and the values are a

+space|-|separated string of glyph names in this class.

+

+\stopsubsection

+

+\startsubsection[title={\type {math}}]

+

+The math table has the variables that are also discussed in the chapter aboout

+math: \type {ScriptPercentScaleDown}, \type {ScriptScriptPercentScaleDown}, \type

+{DelimitedSubFormulaMinHeight}, \type {DisplayOperatorMinHeight}, \type

+{MathLeading}, \type {AxisHeight}, \type {AccentBaseHeight}, \type

+{FlattenedAccentBaseHeight}, \type {SubscriptShiftDown}, \type {SubscriptTopMax},

+\type {SubscriptBaselineDropMin}, \type {SuperscriptShiftUp}, \type

+{SuperscriptShiftUpCramped}, \type {SuperscriptBottomMin}, \type

+{SuperscriptBaselineDropMax}, \type {SubSuperscriptGapMin}, \type

+{SuperscriptBottomMaxWithSubscript}, \type {SpaceAfterScript}, \type

+{UpperLimitGapMin}, \type {UpperLimitBaselineRiseMin}, \type {LowerLimitGapMin},

+\type {LowerLimitBaselineDropMin}, \type {StackTopShiftUp}, \type

+{StackTopDisplayStyleShiftUp}, \type {StackBottomShiftDown}, \type

+{StackBottomDisplayStyleShiftDown}, \type {StackGapMin}, \type

+{StackDisplayStyleGapMin}, \type {StretchStackTopShiftUp}, \type

+{StretchStackBottomShiftDown}, \type {StretchStackGapAboveMin}, \type

+{StretchStackGapBelowMin}, \type {FractionNumeratorShiftUp}, \type

+{FractionNumeratorDisplayStyleShiftUp}, \type {FractionDenominatorShiftDown},

+\type {FractionDenominatorDisplayStyleShiftDown}, \type

+{FractionNumeratorGapMin}, \type {FractionNumeratorDisplayStyleGapMin} \type

+{FractionRuleThickness}, \type {FractionDenominatorGapMin}, \type

+{FractionDenominatorDisplayStyleGapMin}, \type {SkewedFractionHorizontalGap},

+\type {SkewedFractionVerticalGap}, \type {OverbarVerticalGap}, \type

+{OverbarRuleThickness}, \type {OverbarExtraAscender}, \type

+{UnderbarVerticalGap}, \type {UnderbarRuleThickness}, \type

+{UnderbarExtraDescender}, \type {RadicalVerticalGap}, \type

+{RadicalDisplayStyleVerticalGap}, \type {RadicalRuleThickness}, \type

+{RadicalExtraAscender}, \type {RadicalKernBeforeDegree}, \type

+{RadicalKernAfterDegree}, \type {RadicalDegreeBottomRaisePercent}, \type

+{MinConnectorOverlap}, \type {FractionDelimiterSize} and \type

+{FractionDelimiterDisplayStyleSize}.

+

+\stopsubsection

+

+\startsubsection[title={\type {validation_state}}]

+

+This is just a bonus table with keys: \type {bad_ps_fontname}, \type {bad_glyph_table}, \type {bad_cff_table}, \type {bad_metrics_table}, \type {bad_cmap_table}, \type {bad_bitmaps_table}, \type {bad_gx_table}, \type {bad_ot_table}, \type {bad_os2_version} and \type {bad_sfnt_header}.

+

+\stopsubsection

+

+\startsubsection[title={\type {horiz_base} and \type {vert_base}}]

+

+\starttabulate[|l|l|p|]

+\DB key            \BC type  \BC explanation \NC \NR

+\TB

+\NC \type{tags}    \NC table \NC an array of script list tags\NC \NR

+\NC \type{scripts} \NC table \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {scripts} subtable:

+

+\starttabulate[|l|l|p|]

+\DB key                     \BC type    \BC explanation \NC \NR

+\TB

+\NC \type{baseline}         \NC table   \NC \NC \NR

+\NC \type{default_baseline} \NC number  \NC \NC \NR

+\NC \type{lang}             \NC table   \NC \NC \NR

+\LL

+\stoptabulate

+

+

+The \type {lang} subtable:

+

+\starttabulate[|l|l|p|]

+\DB key             \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{tag}      \NC string \NC a script tag \NC \NR

+\NC \type{ascent}   \NC number \NC \NC \NR

+\NC \type{descent}  \NC number \NC \NC \NR

+\NC \type{features} \NC table  \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {features} points to an array of tables with the same layout except

+that in those nested tables, the tag represents a language.

+

+\stopsubsection

+

+\startsubsection[title={\type {altuni}}]

+

+An array of alternate \UNICODE\ values. Inside that array are hashes with:

+

+\starttabulate[|l|l|p|]

+\DB key            \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{unicode} \NC number \NC this glyph is also used for this unicode \NC \NR

+\NC \type{variant} \NC number \NC the alternative is driven by this unicode selector \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {vert_variants} and \type {horiz_variants}}]

+

+\starttabulate[|l|l|p|]

+\DB key                      \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{variants}          \NC string \NC \NC \NR

+\NC \type{italic_correction} \NC number \NC \NC \NR

+\NC \type{parts}             \NC table  \NC \NC \NR

+\LL

+\stoptabulate

+

+The \type {parts} table is an array of smaller tables:

+

+\starttabulate[|l|l|p|]

+\DB key              \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{component} \NC string \NC \NC \NR

+\NC \type{extender}  \NC number \NC \NC \NR

+\NC \type{start}     \NC number \NC \NC \NR

+\NC \type{end}       \NC number \NC \NC \NR

+\NC \type{advance}   \NC number \NC \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {mathkern}}]

+

+\starttabulate[|l|l|p|]

+\DB key                 \BC type  \BC explanation \NC \NR

+\TB

+\NC \type{top_right}    \NC table \NC \NC \NR

+\NC \type{bottom_right} \NC table \NC \NC \NR

+\NC \type{top_left}     \NC table \NC \NC \NR

+\NC \type{bottom_left}  \NC table \NC \NC \NR

+\LL

+\stoptabulate

+

+Each of the subtables is an array of small hashes with two keys:

+

+\starttabulate[|l|l|p|]

+\DB key           \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{height} \NC number \NC \NC \NR

+\NC \type{kern}   \NC number \NC \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {kerns}}]

+

+Substructure is identical to the per|-|glyph subtable.

+

+\stopsubsection

+

+\startsubsection[title={\type {vkerns}}]

+

+Substructure is identical to the per|-|glyph subtable.

+

+\stopsubsection

+

+\startsubsection[title={\type {texdata}}]

+

+\starttabulate[|l|l|p|]

+\DB key           \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{type}   \NC string \NC \nod {unset}, \type {text}, \nod {math}, \type {mathext} \NC \NR

+\NC \type{params} \NC array  \NC 22 font numeric parameters \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {lookups}}]

+

+Top|-|level \type {lookups} is quite different from the ones at character level.

+The keys in this hash are strings, the values the actual lookups, represented as

+dictionary tables.

+

+\starttabulate[|l|l|p|]

+\DB key                  \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{type}          \NC string \NC \NC \NR

+\NC \type{format}        \NC enum   \NC one of \type {glyphs}, \type {class}, \type {coverage},

+                                        \type {reversecoverage} \NC \NR

+\NC \type{tag}           \NC string \NC \NC \NR

+\NC \type{current_class} \NC array  \NC \NC \NR

+\NC \type{before_class}  \NC array  \NC \NC \NR

+\NC \type{after_class}   \NC array  \NC \NC \NR

+\NC \type{rules}         \NC array  \NC an array of rule items\NC \NR

+\LL

+\stoptabulate

+

+Rule items have one common item and one specialized item:

+

+\starttabulate[|l|l|p|]

+\DB key                    \BC type  \BC explanation \NC \NR

+\TB

+\NC \type{lookups}         \NC array \NC a linear array of lookup names                         \NC \NR

+\NC \type{glyphs}          \NC array \NC only if the parent's format is \type {glyphs}          \NC \NR

+\NC \type{class}           \NC array \NC only if the parent's format is \type {class}           \NC \NR

+\NC \type{coverage}        \NC array \NC only if the parent's format is \type {coverage}        \NC \NR

+\NC \type{reversecoverage} \NC array \NC only if the parent's format is \type {reversecoverage} \NC \NR

+\LL

+\stoptabulate

+

+A glyph table is:

+

+\starttabulate[|l|l|p|]

+\DB key          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{names} \NC string \NC \NC \NR

+\NC \type{back}  \NC string \NC \NC \NR

+\NC \type{fore}  \NC string \NC \NC \NR

+\LL

+\stoptabulate

+

+A class table is:

+

+\starttabulate[|l|l|p|]

+\DB key            \BC type  \BC explanation \NC \NR

+\TB

+\NC \type{current} \NC array \NC of numbers  \NC \NR

+\NC \type{before}  \NC array \NC of numbers  \NC \NR

+\NC \type{after}   \NC array \NC of numbers  \NC \NR

+\LL

+\stoptabulate

+

+for coverage:

+

+\starttabulate[|l|l|p|]

+\DB key            \BC type  \BC explanation \NC \NR

+\TB

+\NC \type{current} \NC array \NC of strings \NC \NR

+\NC \type{before}  \NC array \NC of strings \NC \NR

+\NC \type{after}   \NC array \NC of strings \NC \NR

+\LL

+\stoptabulate

+

+and for reverse coverage:

+

+\starttabulate[|l|l|p|]

+\DB key                 \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{current}      \NC array  \NC of strings  \NC \NR

+\NC \type{before}       \NC array  \NC of strings  \NC \NR

+\NC \type{after}        \NC array  \NC of strings  \NC \NR

+\NC \type{replacements} \NC string \NC             \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-fonts.tex b/Master/texmf-dist/doc/luatex/base/luatex-fonts.tex
index 6d72364839a..0e1cc8cc5a2 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-fonts.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-fonts.tex
@@ -1,776 +1,858 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-fonts
-
-\startchapter[reference=fonts,title={Font structure}]
-
-\section {The font tables}
-
-All \TEX\ fonts are represented to \LUA\ code as tables, and internally as
-\CCODE~structures. All keys in the table below are saved in the internal font
-structure if they are present in the table returned by the \type {define_font}
-callback, or if they result from the normal \TFM|/|\VF\ reading routines if there
-is no \type {define_font} callback defined.
-
-The column \quote {\VF} means that this key will be created by the \type
-{font.read_vf()} routine, \quote {\TFM} means that the key will be created by the
-\type {font.read_tfm()} routine, and \quote{used} means whether or not the
-\LUATEX\ engine itself will do something with the key.
-
-The top|-|level keys in the table are as follows:
-
-\starttabulate[|l|c|c|c|l|p|]
-\BC key                     \BC vf  \BC tfm \BC used \BC value type \BC description \NC \NR
-\NC \type{name}             \NC yes \NC yes \NC yes  \NC string     \NC metric (file) name \NC \NR
-\NC \type{area}             \NC no  \NC yes \NC yes  \NC string     \NC (directory) location, typically empty \NC \NR
-\NC \type{used}             \NC no  \NC yes \NC yes  \NC boolean    \NC indicates usage (initial: false) \NC \NR
-\NC \type{characters}       \NC yes \NC yes \NC yes  \NC table      \NC the defined glyphs of this font \NC \NR
-\NC \type{checksum}         \NC yes \NC yes \NC no   \NC number     \NC default: 0 \NC \NR
-\NC \type{designsize}       \NC no  \NC yes \NC yes  \NC number     \NC expected size (default: 655360 == 10pt) \NC \NR
-\NC \type{direction}        \NC no  \NC yes \NC yes  \NC number     \NC default: 0 \NC \NR
-\NC \type{encodingbytes}    \NC no  \NC no  \NC yes  \NC number     \NC default: depends on \type {format} \NC \NR
-\NC \type{encodingname}     \NC no  \NC no  \NC yes  \NC string     \NC encoding name \NC \NR
-\NC \type{fonts}            \NC yes \NC no  \NC yes  \NC table      \NC locally used fonts \NC \NR
-\NC \type{psname}           \NC no  \NC no  \NC yes  \NC string     \NC This is the \POSTSCRIPT\ fontname in the incoming font
-                                                                        source, and it's used as fontname identifier in the \PDF\
-                                                                        output. This has to be a valid string, e.g.\ no spaces
-                                                                        and such, as the backend will not do a cleanup. This gives
-                                                                        complete control to the loader. \NC \NR
-\NC \type{fullname}         \NC no  \NC no  \NC yes  \NC string     \NC output font name, used as a fallback in the \PDF\ output
-                                                                        if the \type {psname} is not set \NC \NR
-\NC \type{header}           \NC yes \NC no  \NC no   \NC string     \NC header comments, if any \NC \NR
-\NC \type{hyphenchar}       \NC no  \NC no  \NC yes  \NC number     \NC default: \TEX's \type {\hyphenchar} \NC \NR
-\NC \type{parameters}       \NC no  \NC yes \NC yes  \NC hash       \NC default: 7 parameters, all zero \NC \NR
-\NC \type{size}             \NC no  \NC yes \NC yes  \NC number     \NC loaded (at) size. (default: same as designsize) \NC \NR
-\NC \type{skewchar}         \NC no  \NC no  \NC yes  \NC number     \NC default: \TEX's \type {\skewchar} \NC \NR
-\NC \type{type}             \NC yes \NC no  \NC yes  \NC string     \NC basic type of this font \NC \NR
-\NC \type{format}           \NC no  \NC no  \NC yes  \NC string     \NC disk format type \NC \NR
-\NC \type{embedding}        \NC no  \NC no  \NC yes  \NC string     \NC \PDF\ inclusion  \NC \NR
-\NC \type{filename}         \NC no  \NC no  \NC yes  \NC string     \NC the name of the font on disk \NC \NR
-\NC \type{tounicode}        \NC no  \NC yes \NC yes  \NC number     \NC When this is set to~1 \LUATEX\ assumes per|-|glyph
-                                                                        tounicode entries are present in the font. \NC \NR
-\NC \type{stretch}          \NC no  \NC no  \NC yes  \NC number     \NC the \quote {stretch} value from \type
-                                                                        {\expandglyphsinfont} \NC \NR
-\NC \type{shrink}           \NC no  \NC no  \NC yes  \NC number     \NC the \quote {shrink} value from \type
-                                                                        {\expandglyphsinfont} \NC \NR
-\NC \type{step}             \NC no  \NC no  \NC yes  \NC number     \NC the \quote {step} value from \type
-                                                                        {\expandglyphsinfont} \NC \NR
-\NC \type{expansion_factor} \NC no  \NC no  \NC no   \NC number     \NC the actual expansion factor of an expanded font \NC \NR
-\NC \type{attributes}       \NC no  \NC no  \NC yes  \NC string     \NC the \type {\pdffontattr} \NC \NR
-\NC \type{cache}            \NC no  \NC no  \NC yes  \NC string     \NC This key controls caching of the \LUA\ table on the
-                                                                        \TEX\ end where \type {yes} means: use a reference to
-                                                                        the table that is passed to \LUATEX\ (this is the
-                                                                        default), and no \type {no} means: don't store the
-                                                                        table reference, don't cache any \LUA\ data for this
-                                                                        font while \type {renew} means: don't store the table
-                                                                        reference, but save a reference to the table that is
-                                                                        created at the first access to one of its fields in font.
-                                                                        Note: the saved reference is thread|-|local, so be
-                                                                        careful when you are using coroutines: an error will be
-                                                                        thrown if the table has been cached in one thread, but
-                                                                        you reference it from another thread. \NC \NR
-\NC \type{nomath}           \NC no  \NC no  \NC yes  \NC boolean    \NC This key allows a minor speedup for text fonts. If it
-                                                                        is present and true, then \LUATEX\ will not check the
-                                                                        character entries for math|-|specific keys. \NC \NR
-\NC \type{oldmath}          \NC no  \NC no  \NC yes  \NC boolean    \NC This key flags a font as representing an old school \TEX\
-                                                                        math font and disables the \OPENTYPE\ code path. \NC \NR
-\NC \type{slant}            \NC no  \NC no  \NC yes  \NC number     \NC This has the same semantics as the \type {SlantFont}
-                                                                        operator in font map files. \NC \NR
-\NC \type{extent}           \NC no  \NC no  \NC yes  \NC number     \NC This has the same semantics as the \type {ExtendFont}
-                                                                        operator in font map files. \NC \NR
-\stoptabulate
-
-The key \type {name} is always required. The keys \type {stretch}, \type
-{shrink}, \type {step} only have meaning when used together: they can be used to
-replace a post|-|loading \type {\expandglyphsinfont} command. The \type
-{auto_expand} option is not supported in \LUATEX. In fact, the primitives
-that create expanded or protruding copies are probably only useful when used with
-traditional fonts because all these extra \OPENTYPE\ properties are kept out of
-the picture. The \type {expansion_factor} is value that can be present inside a
-font in \type {font.fonts}. It is the actual expansion factor (a value between
-\type {-shrink} and \type {stretch}, with step \type {step}) of a font that was
-automatically generated by the font expansion algorithm.
-
-Because we store the actual state of expansion with each glyph and don't have
-special font instances, we can change some font related parameters before lines
-are constructed, like:
-
-\starttyping
-font.setexpansion(font.current(),100,100,20)
-\stoptyping
-
-This is mostly meant for experiments (or an optimizing routing written in \LUA)
-so there is no primitive.
-
-The key \type {attributes} can be used to set font attributes in the \PDF\ file.
-The key \type {used} is set by the engine when a font is actively in use, this
-makes sure that the font's definition is written to the output file (\DVI\ or
-\PDF). The \TFM\ reader sets it to false. The \type {direction} is a number
-signalling the \quote {normal} direction for this font. There are sixteen
-possibilities:
-
-\starttabulate[|c|c|c|c|]
-\BC number   \BC meaning   \BC number    \BC meaning   \NC \NR
-\NC \type{0} \NC \type{LT} \NC \type {8} \NC \type{TT} \NC \NR
-\NC \type{1} \NC \type{LL} \NC \type {9} \NC \type{TL} \NC \NR
-\NC \type{2} \NC \type{LB} \NC \type{10} \NC \type{TB} \NC \NR
-\NC \type{3} \NC \type{LR} \NC \type{11} \NC \type{TR} \NC \NR
-\NC \type{4} \NC \type{RT} \NC \type{12} \NC \type{BT} \NC \NR
-\NC \type{5} \NC \type{RL} \NC \type{13} \NC \type{BL} \NC \NR
-\NC \type{6} \NC \type{RB} \NC \type{14} \NC \type{BB} \NC \NR
-\NC \type{7} \NC \type{RR} \NC \type{15} \NC \type{BR} \NC \NR
-\stoptabulate
-
-These are \OMEGA|-|style direction abbreviations: the first character indicates
-the \quote {first} edge of the character glyphs (the edge that is seen first in
-the writing direction), the second the \quote {top} side. Keep in mind that
-\LUATEX\ has a bit different directional model so these values are not used for
-anything.
-
-The \type {parameters} is a hash with mixed key types. There are seven possible
-string keys, as well as a number of integer indices (these start from 8 up). The
-seven strings are actually used instead of the bottom seven indices, because that
-gives a nicer user interface.
-
-The names and their internal remapping are:
-
-\starttabulate[|l|c|]
-\BC name                  \BC remapping \NC \NR
-\NC \type {slant}         \NC 1 \NC \NR
-\NC \type {space}         \NC 2 \NC \NR
-\NC \type {space_stretch} \NC 3 \NC \NR
-\NC \type {space_shrink}  \NC 4 \NC \NR
-\NC \type {x_height}      \NC 5 \NC \NR
-\NC \type {quad}          \NC 6 \NC \NR
-\NC \type {extra_space}   \NC 7 \NC \NR
-\stoptabulate
-
-The keys \type {type}, \type {format}, \type {embedding}, \type {fullname} and
-\type {filename} are used to embed \OPENTYPE\ fonts in the result \PDF.
-
-The \type {characters} table is a list of character hashes indexed by an integer
-number. The number is the \quote {internal code} \TEX\ knows this character by.
-
-Two very special string indexes can be used also: \type {left_boundary} is a
-virtual character whose ligatures and kerns are used to handle word boundary
-processing. \type {right_boundary} is similar but not actually used for anything
-(yet).
-
-Other index keys are ignored.
-
-Each character hash itself is a hash. For example, here is the character \quote
-{f} (decimal 102) in the font \type {cmr10 at 10pt}:
-
-\starttyping
-[102] = {
-    ['width'] = 200250,
-    ['height'] = 455111,
-    ['depth'] = 0,
-    ['italic'] = 50973,
-    ['kerns'] = {
-        [63] = 50973,
-        [93] = 50973,
-        [39] = 50973,
-        [33] = 50973,
-        [41] = 50973
-    },
-    ['ligatures'] = {
-        [102] = {
-            ['char'] = 11,
-            ['type'] = 0
-        },
-        [108] = {
-            ['char'] = 13,
-            ['type'] = 0
-        },
-        [105] = {
-            ['char'] = 12,
-            ['type'] = 0
-        }
-    }
-}
-\stoptyping
-
-The following top|-|level keys can be present inside a character hash:
-
-\starttabulate[|l|c|c|c|l|p|]
-\BC key              \BC vf  \BC tfm \BC used  \BC type    \BC description \NC\NR
-\NC \type{width}            \NC yes \NC yes \NC yes   \NC number  \NC character's width, in sp (default 0) \NC\NR
-\NC \type{height}           \NC no  \NC yes \NC yes   \NC number  \NC character's height, in sp (default 0) \NC\NR
-\NC \type{depth}            \NC no  \NC yes \NC yes   \NC number  \NC character's depth, in sp (default 0) \NC\NR
-\NC \type{italic}           \NC no  \NC yes \NC yes   \NC number  \NC character's italic correction, in sp (default zero) \NC\NR
-\NC \type{top_accent}       \NC no  \NC no  \NC maybe \NC number  \NC character's top accent alignment place, in sp (default zero) \NC\NR
-\NC \type{bot_accent}       \NC no  \NC no  \NC maybe \NC number  \NC character's bottom accent alignment place, in sp (default zero) \NC\NR
-\NC \type{left_protruding}  \NC no  \NC no  \NC maybe \NC number  \NC character's \type {\lpcode} \NC\NR
-\NC \type{right_protruding} \NC no  \NC no  \NC maybe \NC number  \NC character's \type {\rpcode} \NC\NR
-\NC \type{expansion_factor} \NC no  \NC no  \NC maybe \NC number  \NC character's \type {\efcode} \NC\NR
-\NC \type{tounicode}        \NC no  \NC no  \NC maybe \NC string  \NC character's \UNICODE\ equivalent(s), in \UTF|-|16BE hexadecimal format \NC\NR
-\NC \type{next}             \NC no  \NC yes \NC yes   \NC number  \NC the \quote {next larger} character index \NC\NR
-\NC \type{extensible}       \NC no  \NC yes \NC yes   \NC table   \NC the constituent parts of an extensible recipe \NC\NR
-\NC \type{vert_variants}    \NC no  \NC no  \NC yes   \NC table   \NC constituent parts of a vertical variant set \NC \NR
-\NC \type{horiz_variants}   \NC no  \NC no  \NC yes   \NC table   \NC constituent parts of a horizontal variant set \NC \NR
-\NC \type{kerns}            \NC no  \NC yes \NC yes   \NC table   \NC kerning information \NC\NR
-\NC \type{ligatures}        \NC no  \NC yes \NC yes   \NC table   \NC ligaturing information \NC\NR
-\NC \type{commands}         \NC yes \NC no  \NC yes   \NC array   \NC virtual font commands \NC\NR
-\NC \type{name}             \NC no  \NC no  \NC no    \NC string  \NC the character (\POSTSCRIPT) name \NC\NR
-\NC \type{index}            \NC no  \NC no  \NC yes   \NC number  \NC the (\OPENTYPE\ or \TRUETYPE) font glyph index \NC\NR
-\NC \type{used}             \NC no  \NC yes \NC yes   \NC boolean \NC typeset already (default: false)? \NC\NR
-\NC \type{mathkern}         \NC no  \NC no  \NC yes   \NC table   \NC math cut-in specifications \NC\NR
-\stoptabulate
-
-The values of \type {top_accent}, \type {bot_accent} and \type {mathkern} are
-used only for math accent and superscript placement, see the \at {math chapter}
-[math] in this manual for details.
-
-The values of \type {left_protruding} and \type {right_protruding} are used only
-when \type {\protrudechars} is non-zero.
-
-Whether or not \type {expansion_factor} is used depends on the font's global
-expansion settings, as well as on the value of \type {\adjustspacing}.
-
-The usage of \type {tounicode} is this: if this font specifies a \type
-{tounicode=1} at the top level, then \LUATEX\ will construct a \type {/ToUnicode}
-entry for the \PDF\ font (or font subset) based on the character|-|level \type
-{tounicode} strings, where they are available. If a character does not have a
-sensible \UNICODE\ equivalent, do not provide a string either (no empty strings).
-
-If the font level \type {tounicode} is not set, then \LUATEX\ will build up \type
-{/ToUnicode} based on the \TEX\ code points you used, and any character-level
-\type {tounicodes} will be ignored. The string format is exactly the format that
-is expected by Adobe \CMAP\ files (\UTF-16BE in hexadecimal encoding), minus the
-enclosing angle brackets. For instance the \type {tounicode} for a \type {fi}
-ligature would be \type {00660069}. When you pass a number the conversion will be
-done for you.
-
-The presence of \type {extensible} will overrule \type {next}, if that is also
-present. It in in turn can be overruled by \type {vert_variants}.
-
-The \type {extensible} table is very simple:
-
-\starttabulate[|l|l|p|]
-\BC key        \BC type   \BC description                \NC\NR
-\NC \type{top} \NC number \NC top character index        \NC\NR
-\NC \type{mid} \NC number \NC middle character index     \NC\NR
-\NC \type{bot} \NC number \NC bottom character index     \NC\NR
-\NC \type{rep} \NC number \NC repeatable character index \NC\NR
-\stoptabulate
-
-The \type {horiz_variants} and \type {vert_variants} are arrays of components.
-Each of those components is itself a hash of up to five keys:
-
-\starttabulate[|l|l|p|]
-\BC key             \BC type   \BC explanation \NC \NR
-\NC \type{glyph}    \NC number \NC The character index. Note that this is an encoding number, not a name. \NC \NR
-\NC \type{extender} \NC number \NC One (1) if this part is repeatable, zero (0) otherwise. \NC \NR
-\NC \type{start}    \NC number \NC The maximum overlap at the starting side (in scaled points). \NC \NR
-\NC \type{end}      \NC number \NC The maximum overlap at the ending side (in scaled points). \NC \NR
-\NC \type{advance}  \NC number \NC The total advance width of this item. It can be zero or missing,
-                                   then the natural size of the glyph for character \type {component}
-                                   is used. \NC \NR
-\stoptabulate
-
-The \type {kerns} table is a hash indexed by character index (and \quote
-{character index} is defined as either a non|-|negative integer or the string
-value \type {right_boundary}), with the values the kerning to be applied, in
-scaled points.
-
-The \type {ligatures} table is a hash indexed by character index (and \quote
-{character index} is defined as either a non|-|negative integer or the string
-value \type {right_boundary}), with the values being yet another small hash, with
-two fields:
-
-\starttabulate[|l|l|p|]
-\BC key         \BC type   \BC description \NC \NR
-\NC \type{type} \NC number \NC the type of this ligature command, default 0 \NC \NR
-\NC \type{char} \NC number \NC the character index of the resultant ligature \NC \NR
-\stoptabulate
-
-The \type {char} field in a ligature is required.
-
-The \type {type} field inside a ligature is the numerical or string value of one
-of the eight possible ligature types supported by \TEX. When \TEX\ inserts a new
-ligature, it puts the new glyph in the middle of the left and right glyphs. The
-original left and right glyphs can optionally be retained, and when at least one
-of them is kept, it is also possible to move the new \quote {insertion point}
-forward one or two places. The glyph that ends up to the right of the insertion
-point will become the next \quote {left}.
-
-\starttabulate[|l|c|l|l|]
-\BC textual (Knuth)       \BC number \BC string        \BC result      \NC\NR
-\NC \type{l + r =: n}     \NC 0      \NC \type{=:}     \NC \type{|n}   \NC\NR
-\NC \type{l + r =:| n}    \NC 1      \NC \type{=:|}    \NC \type{|nr}  \NC\NR
-\NC \type{l + r |=: n}    \NC 2      \NC \type{|=:}    \NC \type{|ln}  \NC\NR
-\NC \type{l + r |=:| n}   \NC 3      \NC \type{|=:|}   \NC \type{|lnr} \NC\NR
-\NC \type{l + r  =:|> n}  \NC 5      \NC \type{=:|>}   \NC \type{n|r}  \NC\NR
-\NC \type{l + r |=:> n}   \NC 6      \NC \type{|=:>}   \NC \type{l|n}  \NC\NR
-\NC \type{l + r |=:|> n}  \NC 7      \NC \type{|=:|>}  \NC \type{l|nr} \NC\NR
-\NC \type{l + r |=:|>> n} \NC 11     \NC \type{|=:|>>} \NC \type{ln|r} \NC\NR
-\stoptabulate
-
-The default value is~0, and can be left out. That signifies a \quote {normal}
-ligature where the ligature replaces both original glyphs. In this table the~\type {|}
-indicates the final insertion point.
-
-The \type {commands} array is explained below.
-
-\section {Real fonts}
-
-Whether or not a \TEX\ font is a \quote {real} font that should be written to the
-\PDF\ document is decided by the \type {type} value in the top|-|level font
-structure. If the value is \type {real}, then this is a proper font, and the
-inclusion mechanism will attempt to add the needed font object definitions to the
-\PDF. Values for \type {type} are:
-
-\starttabulate[|l|p|]
-\BC value          \BC description            \NC\NR
-\NC \type{real}    \NC this is a base font    \NC\NR
-\NC \type{virtual} \NC this is a virtual font \NC\NR
-\stoptabulate
-
-The actions to be taken depend on a number of different variables:
-
-\startitemize[packed]
-\startitem
-    Whether the used font fits in an 8-bit encoding scheme or not.
-\stopitem
-\startitem
-    The type of the disk font file.
-\stopitem
-\startitem
-    The level of embedding requested.
-\stopitem
-\stopitemize
-
-A font that uses anything other than an 8-bit encoding vector has to be written
-to the \PDF\ in a different way.
-
-The rule is: if the font table has \type {encodingbytes} set to~2, then this is a
-wide font, in all other cases it isn't. The value~2 is the default for \OPENTYPE\
-and \TRUETYPE\ fonts loaded via \LUA. For \TYPEONE\ fonts, you have to set \type
-{encodingbytes} to~2 explicitly. For \PK\ bitmap fonts, wide font encoding is not
-supported at all.
-
-If no special care is needed, \LUATEX\ currently falls back to the
-mapfile|-|based solution used by \PDFTEX\ and \DVIPS. This behaviour might
-silently be removed in the future, in which case the related primitives and \LUA\
-functions will become no|-|ops.
-
-If a \quote {wide} font is used, the new subsystem kicks in, and some
-extra fields have to be present in the font structure. In this case, \LUATEX\
-does not use a map file at all.
-
-The extra fields are: \type {format}, \type {embedding}, \type {fullname}, \type
-{cidinfo} (as explained above), \type {filename}, and the \type {index} key in
-the separate characters.
-
-Values for \type {format} are:
-
-\starttabulate[|l|p|]
-\BC value           \BC description                                               \NC \NR
-\NC \type{type1}    \NC this is a \POSTSCRIPT\ \TYPEONE\ font                     \NC \NR
-\NC \type{type3}    \NC this is a bitmapped (\PK) font                            \NC \NR
-\NC \type{truetype} \NC this is a \TRUETYPE\ or \TRUETYPE|-|based \OPENTYPE\ font \NC \NR
-\NC \type{opentype} \NC this is a \POSTSCRIPT|-|based \OPENTYPE\ font             \NC \NR
-\stoptabulate
-
-\type {type3} fonts are provided for backward compatibility only, and do not
-support the new wide encoding options.
-
-Values for \type {embedding} are:
-
-\starttabulate[|l|p|]
-\BC value         \BC description                             \NC \NR
-\NC \type{no}     \NC don't embed the font at all             \NC \NR
-\NC \type{subset} \NC include and atttempt to subset the font \NC \NR
-\NC \type{full}   \NC include this font in its entirety       \NC \NR
-\stoptabulate
-
-The other fields are used as follows: The \type {fullname} will be the
-\POSTSCRIPT|/|\PDF\ font name. The \type {cidinfo} will be used as the character
-set (the CID \type {/Ordering} and \type {/Registry} keys). The \type {filename}
-points to the actual font file. If you include the full path in the \type
-{filename} or if the file is in the local directory, \LUATEX\ will run a little
-bit more efficient because it will not have to re|-|run the \type {find_xxx_file}
-callback in that case.
-
-Be careful: when mixing old and new fonts in one document, it is possible to
-create \POSTSCRIPT\ name clashes that can result in printing errors. When this
-happens, you have to change the \type {fullname} of the font.
-
-Typeset strings are written out in a wide format using 2~bytes per glyph, using
-the \type {index} key in the character information as value. The overall effect
-is like having an encoding based on numbers instead of traditional (\POSTSCRIPT)
-name|-|based reencoding. The way to get the correct \type {index} numbers for
-\TYPEONE\ fonts is by loading the font via \type {fontloader.open} and use the table
-indices as \type {index} fields.
-
-In order to make sure that cut and paste of the final document works okay you can
-best make sure that there is a \type {tounicode} vector enforced.
-
-\section[virtualfonts]{Virtual fonts}
-
-\subsection{The structure}
-
-You have to take the following steps if you want \LUATEX\ to treat the returned
-table from \type {define_font} as a virtual font:
-
-\startitemize[packed]
-\startitem
-    Set the top|-|level key \type {type} to \type {virtual}.
-\stopitem
-\startitem
-    Make sure there is at least one valid entry in \type {fonts} (see below).
-\stopitem
-\startitem
-    Give a \type {commands} array to every character (see below).
-\stopitem
-\stopitemize
-
-The presence of the toplevel \type {type} key with the specific value \type
-{virtual} will trigger handling of the rest of the special virtual font fields in
-the table, but the mere existence of 'type' is enough to prevent \LUATEX\ from
-looking for a virtual font on its own.
-
-Therefore, this also works \quote {in reverse}: if you are absolutely certain
-that a font is not a virtual font, assigning the value \type {base} or \type
-{real} to \type {type} will inhibit \LUATEX\ from looking for a virtual font
-file, thereby saving you a disk search.
-
-The \type {fonts} is another \LUA\ array. The values are one- or two|-|key
-hashes themselves, each entry indicating one of the base fonts in a virtual font.
-In case your font is referring to itself, you can use the \type {font.nextid()}
-function which returns the index of the next to be defined font which is probably
-the currently defined one.
-
-An example makes this easy to understand
-
-\starttyping
-fonts = {
-    { name = 'ptmr8a', size = 655360 },
-    { name = 'psyr', size = 600000 },
-    { id = 38 }
-}
-\stoptyping
-
-says that the first referenced font (index 1) in this virtual font is \type
-{ptrmr8a} loaded at 10pt, and the second is \type {psyr} loaded at a little over
-9pt. The third one is previously defined font that is known to \LUATEX\ as font id
-\quote {38}.
-
-The array index numbers are used by the character command definitions that are
-part of each character.
-
-The \type {commands} array is a hash where each item is another small array,
-with the first entry representing a command and the extra items being the
-parameters to that command. The allowed commands and their arguments are:
-
-\starttabulate[|l|l|l|p|]
-\BC command name   \BC arguments \BC type      \BC description \NC \NR
-\NC \type{font}    \NC 1         \NC number    \NC select a new font from the local \type {fonts} table \NC \NR
-\NC \type{char}    \NC 1         \NC number    \NC typeset this character number from the current font,
-                                                   and move right by the character's width \NC \NR
-\NC \type{node}    \NC 1         \NC node      \NC output this node (list), and move right
-                                                   by the width of this list\NC \NR
-\NC \type{slot}    \NC 2         \NC 2 numbers \NC a shortcut for the combination of a font and char command\NC \NR
-\NC \type{push}    \NC 0         \NC           \NC save current position\NC \NR
-\NC \type{nop}     \NC 0         \NC           \NC do nothing \NC \NR
-\NC \type{pop}     \NC 0         \NC           \NC pop position \NC \NR
-\NC \type{rule}    \NC 2         \NC 2 numbers \NC output a rule $ht*wd$, and move right. \NC \NR
-\NC \type{down}    \NC 1         \NC number    \NC move down on the page \NC \NR
-\NC \type{right}   \NC 1         \NC number    \NC move right on the page \NC \NR
-\NC \type{special} \NC 1         \NC string    \NC output a \type {\special} command \NC \NR
-\NC \type{pdf}     \NC 2         \NC 2 strings \NC output a \PDF\ literal, the first string is one of \type {origin},
-                                                   \type {page}, \type {direct} or \type {raw}; if you have one string
-                                                   only \type {origin} is assumed \NC \NR
-\NC \type{lua}     \NC 1         \NC string    \NC execute a \LUA\ script (at \type {\latelua} time) \NC \NR
-\NC \type{image}   \NC 1         \NC image     \NC output an image (the argument can be either an \type
-                                                   {<image>} variable or an \type {image_spec} table) \NC \NR
-\NC \type{comment} \NC any       \NC any       \NC the arguments of this command are ignored \NC \NR
-\stoptabulate
-
-When a font id is set to~0 then it will be replaced by the currently assigned
-font id. This prevents the need for hackery with future id's (normally one could
-use \type {font.nextid} but when more complex fonts are built in the meantime
-other instances could have been loaded.
-
-Here is a rather elaborate glyph commands example:
-
-\starttyping
-...
-commands = {
-    { "push" },                     -- remember where we are
-    { "right", 5000 },              -- move right about 0.08pt
-    { "font", 3 },                  -- select the fonts[3] entry
-    { "char", 97 },                 -- place character 97 (ASCII 'a')
-    { "pop" },                      -- go all the way back
-    { "down", -200000 },            -- move upwards by about 3pt
-    { "special", "pdf: 1 0 0 rg" }  -- switch to red color
- -- { "pdf", "origin", "1 0 0 rg" } -- switch to red color (alternative)
-    { "rule", 500000, 20000 }       -- draw a bar
-    { "special", "pdf: 0 g" }       -- back to black
- -- { "pdf", "origin", "0 g" }      -- back to black (alternative)
-}
-...
-\stoptyping
-
-The default value for \type {font} is always~1 at the start of the
-\type {commands} array. Therefore, if the virtual font is essentially only a
-re|-|encoding, then you do usually not have create an explicit \quote {font}
-command in the array.
-
-Rules inside of \type {commands} arrays are built up using only two dimensions:
-they do not have depth. For correct vertical placement, an extra \type {down}
-command may be needed.
-
-Regardless of the amount of movement you create within the \type {commands}, the
-output pointer will always move by exactly the width that was given in the \type
-{width} key of the character hash. Any movements that take place inside the \type
-{commands} array are ignored on the upper level.
-
-The special can have a \type {pdf:}, \type {pdf:origin:},  \type {pdf:page:},
-\type {pdf:direct:} or  \type {pdf:raw:} prefix. When you have to concatenate
-strings using the \type {pdf} command might be more efficient.
-
-\subsection{Artificial fonts}
-
-Even in a \quote {real} font, there can be virtual characters. When \LUATEX\
-encounters a \type {commands} field inside a character when it becomes time to
-typeset the character, it will interpret the commands, just like for a true
-virtual character. In this case, if you have created no \quote {fonts} array,
-then the default (and only) \quote {base} font is taken to be the current font
-itself. In practice, this means that you can create virtual duplicates of
-existing characters which is useful if you want to create composite characters.
-
-Note: this feature does {\it not\/} work the other way around. There can not be
-\quote {real} characters in a virtual font! You cannot use this technique for
-font re-encoding either; you need a truly virtual font for that (because
-characters that are already present cannot be altered).
-
-\subsection{Example virtual font}
-
-Finally, here is a plain \TEX\ input file with a virtual font demonstration:
-
-\startbuffer
-\directlua {
-  callback.register('define_font',
-    function (name,size)
-      if name == 'cmr10-red' then
-        f = font.read_tfm('cmr10',size)
-        f.name = 'cmr10-red'
-        f.type = 'virtual'
-        f.fonts = {{ name = 'cmr10', size = size }}
-        for i,v in pairs(f.characters) do
-          if (string.char(i)):find('[tacohanshartmut]') then
-             v.commands = {
-               {'special','pdf: 1 0 0 rg'},
-               {'char',i},
-               {'special','pdf: 0 g'},
-              }
-          else
-             v.commands = {{'char',i}}
-          end
-        end
-      else
-        f = font.read_tfm(name,size)
-      end
-      return f
-    end
-  )
-}
-
-\font\myfont = cmr10-red at 10pt \myfont  This is a line of text \par
-\font\myfontx= cmr10     at 10pt \myfontx Here is another line of text \par
-\stopbuffer
-
-\typebuffer
-
-\section{The \type {font} library}
-
-The font library provides the interface into the internals of the font system,
-and also it contains helper functions to load traditional \TEX\ font metrics
-formats. Other font loading functionality is provided by the \type {fontloader}
-library that will be discussed in the next section.
-
-\subsection{Loading a \TFM\ file}
-
-The behavior documented in this subsection is considered stable in the sense that
-there will not be backward-incompatible changes any more.
-
-\startfunctioncall
-<table> fnt =
-    font.read_tfm(<string> name, <number> s)
-\stopfunctioncall
-
-The number is a bit special:
-
-\startitemize
-\startitem
-    If it is positive, it specifies an \quote {at size} in scaled points.
-\stopitem
-\startitem
-    If it is negative, its absolute value represents a \quote {scaled}
-    setting relative to the designsize of the font.
-\stopitem
-\stopitemize
-
-The internal structure of the metrics font table that is returned is explained in
-\in {chapter} [fonts].
-
-\subsection{Loading a \VF\ file}
-
-The behavior documented in this subsection is considered stable in the sense that
-there will not be backward-incompatible changes any more.
-
-\startfunctioncall
-<table> vf_fnt =
-    font.read_vf(<string> name, <number> s)
-\stopfunctioncall
-
-The meaning of the number \type {s} and the format of the returned table are
-similar to the ones in the \type {read_tfm()} function.
-
-\subsection{The fonts array}
-
-The whole table of \TEX\ fonts is accessible from \LUA\ using a virtual array.
-
-\starttyping
-font.fonts[n] = { ... }
-<table> f = font.fonts[n]
-\stoptyping
-
-See \in {chapter} [fonts] for the structure of the tables. Because this is a
-virtual array, you cannot call \type {pairs} on it, but see below for the \type
-{font.each} iterator.
-
-The two metatable functions implementing the virtual array are:
-
-\startfunctioncall
-<table> f = font.getfont(<number> n)
-font.setfont(<number> n, <table> f)
-\stopfunctioncall
-
-Note that at the moment, each access to the \type {font.fonts} or call to \type
-{font.getfont} creates a \LUA\ table for the whole font. This process can be quite
-slow. In a later version of \LUATEX, this interface will change (it will start
-using userdata objects instead of actual tables).
-
-Also note the following: assignments can only be made to fonts that have already
-been defined in \TEX, but have not been accessed {\it at all\/} since that
-definition. This limits the usability of the write access to \type {font.fonts}
-quite a lot, a less stringent ruleset will likely be implemented later.
-
-\subsection{Checking a font's status}
-
-You can test for the status of a font by calling this function:
-
-\startfunctioncall
-<boolean> f =
-    font.frozen(<number> n)
-\stopfunctioncall
-
-The return value is one of \type {true} (unassignable), \type {false} (can be
-changed) or \type {nil} (not a valid font at all).
-
-\subsection{Defining a font directly}
-
-You can define your own font into \type {font.fonts} by calling this function:
-
-\startfunctioncall
-<number> i =
-    font.define(<table> f)
-\stopfunctioncall
-
-The return value is the internal id number of the defined font (the index into
-\type {font.fonts}). If the font creation fails, an error is raised. The table
-is a font structure, as explained in \in {chapter} [fonts]. An alternative call
-is:
-
-\startfunctioncall
-<number> i =
-    font.define(<number> n, <table> f)
-\stopfunctioncall
-
-Where the first argument is a reserved font id (see below).
-
-\subsection{Extending a font}
-
-Within reasonable bounds you can extend a font after it has been defined. Because
-some properties are best left unchanged this is limited to adding characters.
-
-\startfunctioncall
-font.addcharacters(<number n>, <table> f)
-\stopfunctioncall
-
-The table passed can have the fields \type {characters} which is a (sub)table
-like the one used in define, and for virtual fonts a \type {fonts} table can be
-added. The characters defined in the \type {characters} table are added (when not
-yet present) or replace an existing entry. Keep in mind that replacing can have
-side effects because a character already can have been used. Instead of posing
-restrictions we expect the user to be careful. (The \type {setfont} helper is
-a more drastic replacer.)
-
-\subsection{Projected next font id}
-
-\startfunctioncall
-<number> i =
-    font.nextid()
-\stopfunctioncall
-
-This returns the font id number that would be returned by a \type {font.define}
-call if it was executed at this spot in the code flow. This is useful for virtual
-fonts that need to reference themselves. If you pass \type {true} as argument,
-the id gets reserved and you can pass to \type {font.define} as first argument.
-This can be handy when you create complex virtual fonts.
-
-\startfunctioncall
-<number> i =
-    font.nextid(true)
-\stopfunctioncall
-
-\subsection{Font id}
-
-\startfunctioncall
-<number> i =
-    font.id(<string> csname)
-\stopfunctioncall
-
-This returns the font id associated with \type {csname} string, or $-1$ if \type
-{csname} is not defined.
-
-\subsection{Currently active font}
-
-\startfunctioncall
-<number> i = font.current()
-font.current(<number> i)
-\stopfunctioncall
-
-This gets or sets the currently used font number.
-
-\subsection{Maximum font id}
-
-\startfunctioncall
-<number> i =
-    font.max()
-\stopfunctioncall
-
-This is the largest used index in \type {font.fonts}.
-
-\subsection{Iterating over all fonts}
-
-\startfunctioncall
-for i,v in font.each() do
-  ...
-end
-\stopfunctioncall
-
-This is an iterator over each of the defined \TEX\ fonts. The first returned
-value is the index in \type {font.fonts}, the second the font itself, as a \LUA\
-table. The indices are listed incrementally, but they do not always form an array
-of consecutive numbers: in some cases there can be holes in the sequence.
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-fonts

+

+\startchapter[reference=fonts,title={Font structure}]

+

+\startsection[title={The font tables}]

+

+\topicindex {fonts}

+\topicindex {fonts+tables}

+

+All \TEX\ fonts are represented to \LUA\ code as tables, and internally as

+\CCODE~structures. All keys in the table below are saved in the internal font

+structure if they are present in the table returned by the \cbk {define_font}

+callback, or if they result from the normal \TFM|/|\VF\ reading routines if there

+is no \cbk {define_font} callback defined.

+

+The column \quote {\VF} means that this key will be created by the \type

+{font.read_vf()} routine, \quote {\TFM} means that the key will be created by the

+\type {font.read_tfm()} routine, and \quote {used} means whether or not the

+\LUATEX\ engine itself will do something with the key. The top|-|level keys in

+the table are as follows:

+

+\starttabulate[|l|c|c|c|l|pl|]

+\DB key                     \BC vf  \BC tfm \BC used \BC value type \BC description \NC \NR

+\TB

+\NC \type{name}             \NC yes \NC yes \NC yes  \NC string     \NC metric (file) name \NC \NR

+\NC \type{area}             \NC no  \NC yes \NC yes  \NC string     \NC (directory) location, typically empty \NC \NR

+\NC \type{used}             \NC no  \NC yes \NC yes  \NC boolean    \NC indicates usage (initial: false) \NC \NR

+\NC \type{characters}       \NC yes \NC yes \NC yes  \NC table      \NC the defined glyphs of this font \NC \NR

+\NC \type{checksum}         \NC yes \NC yes \NC no   \NC number     \NC default: 0 \NC \NR

+\NC \type{designsize}       \NC no  \NC yes \NC yes  \NC number     \NC expected size (default: 655360 == 10pt) \NC \NR

+\NC \type{direction}        \NC no  \NC yes \NC yes  \NC number     \NC default: 0 \NC \NR

+\NC \type{encodingbytes}    \NC no  \NC no  \NC yes  \NC number     \NC default: depends on \type {format} \NC \NR

+\NC \type{encodingname}     \NC no  \NC no  \NC yes  \NC string     \NC encoding name \NC \NR

+\NC \type{fonts}            \NC yes \NC no  \NC yes  \NC table      \NC locally used fonts \NC \NR

+\NC \type{psname}           \NC no  \NC no  \NC yes  \NC string     \NC This is the \POSTSCRIPT\ fontname in the incoming font

+                                                                        source, and it's used as fontname identifier in the \PDF\

+                                                                        output. This has to be a valid string, e.g.\ no spaces

+                                                                        and such, as the backend will not do a cleanup. This gives

+                                                                        complete control to the loader. \NC \NR

+\NC \type{fullname}         \NC no  \NC no  \NC yes  \NC string     \NC output font name, used as a fallback in the \PDF\ output

+                                                                        if the \type {psname} is not set \NC \NR

+\NC \type{header}           \NC yes \NC no  \NC no   \NC string     \NC header comments, if any \NC \NR

+\NC \type{hyphenchar}       \NC no  \NC no  \NC yes  \NC number     \NC default: \TEX's \prm {hyphenchar} \NC \NR

+\NC \type{parameters}       \NC no  \NC yes \NC yes  \NC hash       \NC default: 7 parameters, all zero \NC \NR

+\NC \type{size}             \NC no  \NC yes \NC yes  \NC number     \NC the required scaling (by default the same as designsize) \NC \NR

+\NC \type{skewchar}         \NC no  \NC no  \NC yes  \NC number     \NC default: \TEX's \prm {skewchar} \NC \NR

+\NC \type{type}             \NC yes \NC no  \NC yes  \NC string     \NC basic type of this font \NC \NR

+\NC \type{format}           \NC no  \NC no  \NC yes  \NC string     \NC disk format type \NC \NR

+\NC \type{embedding}        \NC no  \NC no  \NC yes  \NC string     \NC \PDF\ inclusion  \NC \NR

+\NC \type{filename}         \NC no  \NC no  \NC yes  \NC string     \NC the name of the font on disk \NC \NR

+\NC \type{tounicode}        \NC no  \NC yes \NC yes  \NC number     \NC When this is set to~1 \LUATEX\ assumes per|-|glyph

+                                                                        tounicode entries are present in the font. \NC \NR

+\NC \type{stretch}          \NC no  \NC no  \NC yes  \NC number     \NC the \quote {stretch} value from \lpr {expandglyphsinfont} \NC \NR

+\NC \type{shrink}           \NC no  \NC no  \NC yes  \NC number     \NC the \quote {shrink} value from \lpr {expandglyphsinfont} \NC \NR

+\NC \type{step}             \NC no  \NC no  \NC yes  \NC number     \NC the \quote {step} value from \lpr {expandglyphsinfont} \NC \NR

+\NC \type{expansion_factor} \NC no  \NC no  \NC no   \NC number     \NC the actual expansion factor of an expanded font \NC \NR

+\NC \type{attributes}       \NC no  \NC no  \NC yes  \NC string     \NC the \orm {pdffontattr} \NC \NR

+\NC \type{cache}            \NC no  \NC no  \NC yes  \NC string     \NC This key controls caching of the \LUA\ table on the

+                                                                        \TEX\ end where \type {yes} means: use a reference to

+                                                                        the table that is passed to \LUATEX\ (this is the

+                                                                        default), and \type {no} means: don't store the

+                                                                        table reference, don't cache any \LUA\ data for this

+                                                                        font while \type {renew} means: don't store the table

+                                                                        reference, but save a reference to the table that is

+                                                                        created at the first access to one of its fields in the

+                                                                        font. \NC \NR

+\NC \type{nomath}           \NC no  \NC no  \NC yes  \NC boolean    \NC This key allows a minor speedup for text fonts. If it

+                                                                        is present and true, then \LUATEX\ will not check the

+                                                                        character entries for math|-|specific keys. \NC \NR

+\NC \type{oldmath}          \NC no  \NC no  \NC yes  \NC boolean    \NC This key flags a font as representing an old school \TEX\

+                                                                        math font and disables the \OPENTYPE\ code path. \NC \NR

+\NC \type{slant}            \NC no  \NC no  \NC yes  \NC number     \NC This parameter will tilt the font and

+                                                                        does the same as \type {SlantFont} in the map file for

+                                                                        \TYPEONE\ fonts. \NC \NR

+\NC \type{extend}           \NC no  \NC no  \NC yes  \NC number     \NC This parameter will scale the font horizontally and

+                                                                        does the same as \type {ExtendFont} in the map file for

+                                                                        \TYPEONE\ fonts. \NC \NR

+\NC \type{squeeze}          \NC no  \NC no  \NC yes  \NC number     \NC This parameter will scale the font vertically and has

+                                                                        no equivalent in the map file. \NC \NR

+\NC \type{width}            \NC no  \NC no  \NC yes  \NC number     \NC The backend will inject \PDF\ operators that set the

+                                                                        penwidth. The value is (as usual in \TEX) divided by 1000.

+                                                                        It works with the \type {mode} file. \NC \NR

+\NC \type{mode}             \NC no  \NC no  \NC yes  \NC number     \NC The backend will inject \PDF\ operators that relate to the

+                                                                        drawing mode with 0~being a fill, 1~being an outline,

+                                                                        2~both draw and fill and 3~no painting at all. \NC \NR

+\LL

+\stoptabulate

+

+The saved reference in the \type {cache} option is thread|-|local, so be careful

+when you are using coroutines: an error will be thrown if the table has been

+cached in one thread, but you reference it from another thread.

+

+The key \type {name} is always required. The keys \type {stretch}, \type

+{shrink}, \type {step} only have meaning when used together: they can be used to

+replace a post|-|loading \lpr {expandglyphsinfont} command. The \type

+{auto_expand} option is not supported in \LUATEX. In fact, the primitives that

+create expanded or protruding copies are probably only useful when used with

+traditional fonts because all these extra \OPENTYPE\ properties are kept out of

+the picture. The \type {expansion_factor} is value that can be present inside a

+font in \type {font.fonts}. It is the actual expansion factor (a value between

+\type {-shrink} and \type {stretch}, with step \type {step}) of a font that was

+automatically generated by the font expansion algorithm.

+

+Because we store the actual state of expansion with each glyph and don't have

+special font instances, we can change some font related parameters before lines

+are constructed, like:

+

+\starttyping

+font.setexpansion(font.current(),100,100,20)

+\stoptyping

+

+This is mostly meant for experiments (or an optimizing routing written in \LUA)

+so there is no primitive.

+

+The key \type {attributes} can be used to set font attributes in the \PDF\ file.

+The key \type {used} is set by the engine when a font is actively in use, this

+makes sure that the font's definition is written to the output file (\DVI\ or

+\PDF). The \TFM\ reader sets it to false. The \type {direction} is a number

+signalling the \quote {normal} direction for this font. There are sixteen

+possibilities:

+

+\starttabulate[|Tc|c|Tc|c|Tc|c|Tc|c|]

+\DB \# \BC dir \BC \# \BC dir \BC \# \BC dir \BC \# \BC dir \NC \NR

+\TB

+\NC  0 \NC LT  \NC  4 \NC RT  \NC  8 \NC TT  \NC 12 \NC BT  \NC \NR

+\NC  1 \NC LL  \NC  5 \NC RL  \NC  9 \NC TL  \NC 13 \NC BL  \NC \NR

+\NC  2 \NC LB  \NC  6 \NC RB  \NC 10 \NC TB  \NC 14 \NC BB  \NC \NR

+\NC  3 \NC LR  \NC  7 \NC RR  \NC 11 \NC TR  \NC 15 \NC BR  \NC \NR

+\LL

+\stoptabulate

+

+These are \OMEGA|-|style direction abbreviations: the first character indicates

+the \quote {first} edge of the character glyphs (the edge that is seen first in

+the writing direction), the second the \quote {top} side. Keep in mind that

+\LUATEX\ has a bit different directional model so these values are not used for

+anything.

+

+The \type {parameters} is a hash with mixed key types. There are seven possible

+string keys, as well as a number of integer indices (these start from 8 up). The

+seven strings are actually used instead of the bottom seven indices, because that

+gives a nicer user interface.

+

+The names and their internal remapping are:

+

+\starttabulate[|l|c|]

+\DB name                  \BC remapping \NC \NR

+\TB

+\NC \type {slant}         \NC 1 \NC \NR

+\NC \type {space}         \NC 2 \NC \NR

+\NC \type {space_stretch} \NC 3 \NC \NR

+\NC \type {space_shrink}  \NC 4 \NC \NR

+\NC \type {x_height}      \NC 5 \NC \NR

+\NC \type {quad}          \NC 6 \NC \NR

+\NC \type {extra_space}   \NC 7 \NC \NR

+\LL

+\stoptabulate

+

+The keys \type {type}, \type {format}, \type {embedding}, \type {fullname} and

+\type {filename} are used to embed \OPENTYPE\ fonts in the result \PDF.

+

+The \type {characters} table is a list of character hashes indexed by an integer

+number. The number is the \quote {internal code} \TEX\ knows this character by.

+

+Two very special string indexes can be used also: \type {left_boundary} is a

+virtual character whose ligatures and kerns are used to handle word boundary

+processing. \type {right_boundary} is similar but not actually used for anything

+(yet).

+

+Each character hash itself is a hash. For example, here is the character \quote

+{f} (decimal 102) in the font \type {cmr10 at 10pt}. The numbers that represent

+dimensions are in scaled points.

+

+\starttyping

+[102] = {

+    ["width"]  = 200250,

+    ["height"] = 455111,

+    ["depth"]  = 0,

+    ["italic"] = 50973,

+    ["kerns"]  = {

+        [63] = 50973,

+        [93] = 50973,

+        [39] = 50973,

+        [33] = 50973,

+        [41] = 50973

+    },

+    ["ligatures"] = {

+        [102] = { ["char"] = 11, ["type"] = 0 },

+        [108] = { ["char"] = 13, ["type"] = 0 },

+        [105] = { ["char"] = 12, ["type"] = 0 }

+    }

+}

+\stoptyping

+

+The following top|-|level keys can be present inside a character hash:

+

+\starttabulate[|l|c|c|c|l|p|]

+\DB key                     \BC vf  \BC tfm \BC used  \BC type    \BC description \NC\NR

+\TB

+\NC \type{width}            \NC yes \NC yes \NC yes   \NC number  \NC character's width, in sp (default 0) \NC\NR

+\NC \type{height}           \NC no  \NC yes \NC yes   \NC number  \NC character's height, in sp (default 0) \NC\NR

+\NC \type{depth}            \NC no  \NC yes \NC yes   \NC number  \NC character's depth, in sp (default 0) \NC\NR

+\NC \type{italic}           \NC no  \NC yes \NC yes   \NC number  \NC character's italic correction, in sp (default zero) \NC\NR

+\NC \type{top_accent}       \NC no  \NC no  \NC maybe \NC number  \NC character's top accent alignment place, in sp (default zero) \NC\NR

+\NC \type{bot_accent}       \NC no  \NC no  \NC maybe \NC number  \NC character's bottom accent alignment place, in sp (default zero) \NC\NR

+\NC \type{left_protruding}  \NC no  \NC no  \NC maybe \NC number  \NC character's \lpr {lpcode} \NC\NR

+\NC \type{right_protruding} \NC no  \NC no  \NC maybe \NC number  \NC character's \lpr {rpcode} \NC\NR

+\NC \type{expansion_factor} \NC no  \NC no  \NC maybe \NC number  \NC character's \lpr {efcode} \NC\NR

+\NC \type{tounicode}        \NC no  \NC no  \NC maybe \NC string  \NC character's \UNICODE\ equivalent(s), in \UTF|-|16BE hexadecimal format \NC\NR

+\NC \type{next}             \NC no  \NC yes \NC yes   \NC number  \NC the \quote {next larger} character index \NC\NR

+\NC \type{extensible}       \NC no  \NC yes \NC yes   \NC table   \NC the constituent parts of an extensible recipe \NC\NR

+\NC \type{vert_variants}    \NC no  \NC no  \NC yes   \NC table   \NC constituent parts of a vertical variant set \NC \NR

+\NC \type{horiz_variants}   \NC no  \NC no  \NC yes   \NC table   \NC constituent parts of a horizontal variant set \NC \NR

+\NC \type{kerns}            \NC no  \NC yes \NC yes   \NC table   \NC kerning information \NC\NR

+\NC \type{ligatures}        \NC no  \NC yes \NC yes   \NC table   \NC ligaturing information \NC\NR

+\NC \type{commands}         \NC yes \NC no  \NC yes   \NC array   \NC virtual font commands \NC\NR

+\NC \type{name}             \NC no  \NC no  \NC no    \NC string  \NC the character (\POSTSCRIPT) name \NC\NR

+\NC \type{index}            \NC no  \NC no  \NC yes   \NC number  \NC the (\OPENTYPE\ or \TRUETYPE) font glyph index \NC\NR

+\NC \type{used}             \NC no  \NC yes \NC yes   \NC boolean \NC typeset already (default: false) \NC\NR

+\NC \type{mathkern}         \NC no  \NC no  \NC yes   \NC table   \NC math cut-in specifications \NC\NR

+\LL

+\stoptabulate

+

+The values of \type {top_accent}, \type {bot_accent} and \type {mathkern} are

+used only for math accent and superscript placement, see \at {page} [math] in

+this manual for details. The values of \type {left_protruding} and \type

+{right_protruding} are used only when \lpr {protrudechars} is non-zero. Whether

+or not \type {expansion_factor} is used depends on the font's global expansion

+settings, as well as on the value of \lpr {adjustspacing}.

+

+The usage of \type {tounicode} is this: if this font specifies a \type

+{tounicode=1} at the top level, then \LUATEX\ will construct a \type {/ToUnicode}

+entry for the \PDF\ font (or font subset) based on the character|-|level \type

+{tounicode} strings, where they are available. If a character does not have a

+sensible \UNICODE\ equivalent, do not provide a string either (no empty strings).

+

+If the font level \type {tounicode} is not set, then \LUATEX\ will build up \type

+{/ToUnicode} based on the \TEX\ code points you used, and any character-level

+\type {tounicodes} will be ignored. The string format is exactly the format that

+is expected by Adobe \CMAP\ files (\UTF-16BE in hexadecimal encoding), minus the

+enclosing angle brackets. For instance the \type {tounicode} for a \type {fi}

+ligature would be \type {00660069}. When you pass a number the conversion will be

+done for you.

+

+A math character can have a \type {next} field that points to a next larger

+shape. However, the presence of \type {extensible} will overrule \type {next}, if

+that is also present. The \type {extensible} field in turn can be overruled by

+\type {vert_variants}, the \OPENTYPE\ version. The \type {extensible} table is

+very simple:

+

+\starttabulate[|l|l|p|]

+\DB key        \BC type   \BC description                \NC\NR

+\TB

+\NC \type{top} \NC number \NC top character index        \NC\NR

+\NC \type{mid} \NC number \NC middle character index     \NC\NR

+\NC \type{bot} \NC number \NC bottom character index     \NC\NR

+\NC \type{rep} \NC number \NC repeatable character index \NC\NR

+\LL

+\stoptabulate

+

+The \type {horiz_variants} and \type {vert_variants} are arrays of components.

+Each of those components is itself a hash of up to five keys:

+

+\starttabulate[|l|l|p|]

+\DB key             \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{glyph}    \NC number \NC The character index. Note that this is an encoding number, not a name. \NC \NR

+\NC \type{extender} \NC number \NC One (1) if this part is repeatable, zero (0) otherwise. \NC \NR

+\NC \type{start}    \NC number \NC The maximum overlap at the starting side (in scaled points). \NC \NR

+\NC \type{end}      \NC number \NC The maximum overlap at the ending side (in scaled points). \NC \NR

+\NC \type{advance}  \NC number \NC The total advance width of this item. It can be zero or missing,

+                                   then the natural size of the glyph for character \type {component}

+                                   is used. \NC \NR

+\LL

+\stoptabulate

+

+The \type {kerns} table is a hash indexed by character index (and \quote

+{character index} is defined as either a non|-|negative integer or the string

+value \type {right_boundary}), with the values of the kerning to be applied, in

+scaled points.

+

+The \type {ligatures} table is a hash indexed by character index (and \quote

+{character index} is defined as either a non|-|negative integer or the string

+value \type {right_boundary}), with the values being yet another small hash, with

+two fields:

+

+\starttabulate[|l|l|p|]

+\DB key         \BC type   \BC description \NC \NR

+\TB

+\NC \type{type} \NC number \NC the type of this ligature command, default 0 \NC \NR

+\NC \type{char} \NC number \NC the character index of the resultant ligature \NC \NR

+\LL

+\stoptabulate

+

+The \type {char} field in a ligature is required. The \type {type} field inside a

+ligature is the numerical or string value of one of the eight possible ligature

+types supported by \TEX. When \TEX\ inserts a new ligature, it puts the new glyph

+in the middle of the left and right glyphs. The original left and right glyphs

+can optionally be retained, and when at least one of them is kept, it is also

+possible to move the new \quote {insertion point} forward one or two places. The

+glyph that ends up to the right of the insertion point will become the next

+\quote {left}.

+

+\starttabulate[|l|c|l|l|]

+\DB textual (Knuth)       \BC number \BC string        \BC result      \NC\NR

+\TB

+\NC \type{l + r =: n}     \NC 0      \NC \type{=:}     \NC \type{|n}   \NC\NR

+\NC \type{l + r =:| n}    \NC 1      \NC \type{=:|}    \NC \type{|nr}  \NC\NR

+\NC \type{l + r |=: n}    \NC 2      \NC \type{|=:}    \NC \type{|ln}  \NC\NR

+\NC \type{l + r |=:| n}   \NC 3      \NC \type{|=:|}   \NC \type{|lnr} \NC\NR

+\NC \type{l + r  =:|> n}  \NC 5      \NC \type{=:|>}   \NC \type{n|r}  \NC\NR

+\NC \type{l + r |=:> n}   \NC 6      \NC \type{|=:>}   \NC \type{l|n}  \NC\NR

+\NC \type{l + r |=:|> n}  \NC 7      \NC \type{|=:|>}  \NC \type{l|nr} \NC\NR

+\NC \type{l + r |=:|>> n} \NC 11     \NC \type{|=:|>>} \NC \type{ln|r} \NC\NR

+\LL

+\stoptabulate

+

+The default value is~0, and can be left out. That signifies a \quote {normal}

+ligature where the ligature replaces both original glyphs. In this table the~\type {|}

+indicates the final insertion point.

+

+The \type {commands} array is explained below.

+

+\stopsection

+

+\startsection[title={Real fonts}]

+

+\topicindex {fonts+real}

+\topicindex {fonts+virtual}

+

+Whether or not a \TEX\ font is a \quote {real} font that should be written to the

+\PDF\ document is decided by the \type {type} value in the top|-|level font

+structure. If the value is \type {real}, then this is a proper font, and the

+inclusion mechanism will attempt to add the needed font object definitions to the

+\PDF. Values for \type {type} are:

+

+\starttabulate[|l|p|]

+\DB value          \BC description            \NC\NR

+\TB

+\NC \type{real}    \NC this is a base font    \NC\NR

+\NC \type{virtual} \NC this is a virtual font \NC\NR

+\LL

+\stoptabulate

+

+The actions to be taken depend on a number of different variables:

+

+\startitemize[packed]

+\startitem

+    Whether the used font fits in an 8-bit encoding scheme or not. This is true for

+    traditional \TEX\ fonts that communicate via \TFM\ files.

+\stopitem

+\startitem

+    The type of the disk font file, for instance a bitmap file or an outline

+    \TYPEONE, \TRUETYPE\ or \OPENTYPE\ font.

+\stopitem

+\startitem

+    The level of embedding requested, although in most cases a subset of

+    characters is embedded. The times when nothing got embedded are (in our

+    opinion at least) basically gone.

+\stopitem

+\stopitemize

+

+A font that uses anything other than an 8-bit encoding vector has to be written

+to the \PDF\ in a different way. When the font table has \type {encodingbytes}

+set to~2, then it is a wide font, in all other cases it isn't. The value~2 is the

+default for \OPENTYPE\ and \TRUETYPE\ fonts loaded via \LUA. For \TYPEONE\ fonts,

+you have to set \type {encodingbytes} to~2 explicitly. For \PK\ bitmap fonts,

+wide font encoding is not supported at all.

+

+If no special care is needed, \LUATEX\ falls back to the mapfile|-|based solution

+used by \PDFTEX\ and \DVIPS, so that legacy fonts are supported transparently. If

+a \quote {wide} font is used, the new subsystem kicks in, and some extra fields

+have to be present in the font structure. In this case, \LUATEX\ does not use a

+map file at all. These extra fields are: \type {format}, \type {embedding}, \type

+{fullname}, \type {cidinfo} (as explained above), \type {filename}, and the \type

+{index} key in the separate characters.

+

+The \type {format} variable can have the following values. \type {type3} fonts

+are provided for backward compatibility only, and do not support the new wide

+encoding options.

+

+\starttabulate[|l|p|]

+\DB value           \BC description                                               \NC \NR

+\TB

+\NC \type{type1}    \NC this is a \POSTSCRIPT\ \TYPEONE\ font                     \NC \NR

+\NC \type{type3}    \NC this is a bitmapped (\PK) font                            \NC \NR

+\NC \type{truetype} \NC this is a \TRUETYPE\ or \TRUETYPE|-|based \OPENTYPE\ font \NC \NR

+\NC \type{opentype} \NC this is a \POSTSCRIPT|-|based \OPENTYPE\ font             \NC \NR

+\LL

+\stoptabulate

+

+Valid values for the \type {embedding} variable are:

+

+\starttabulate[|l|p|]

+\DB value         \BC description                             \NC \NR

+\TB

+\NC \type{no}     \NC don't embed the font at all             \NC \NR

+\NC \type{subset} \NC include and atttempt to subset the font \NC \NR

+\NC \type{full}   \NC include this font in its entirety       \NC \NR

+\LL

+\stoptabulate

+

+The other fields are used as follows. The \type {fullname} will be the

+\POSTSCRIPT|/|\PDF\ font name. The \type {cidinfo} will be used as the character

+set: the CID \type {/Ordering} and \type {/Registry} keys. The \type {filename}

+points to the actual font file. If you include the full path in the \type

+{filename} or if the file is in the local directory, \LUATEX\ will run a little

+bit more efficient because it will not have to re|-|run the \type {find_*_file}

+callback in that case.

+

+Be careful: when mixing old and new fonts in one document, it is possible to

+create \POSTSCRIPT\ name clashes that can result in printing errors. When this

+happens, you have to change the \type {fullname} of the font to a more unique

+one.

+

+Typeset strings are written out in a wide format using 2~bytes per glyph, using

+the \type {index} key in the character information as value. The overall effect

+is like having an encoding based on numbers instead of traditional (\POSTSCRIPT)

+name|-|based reencoding. One way to get the correct \type {index} numbers for

+\TYPEONE\ fonts is by loading the font via \type {fontloader.open} and use the

+table indices as \type {index} fields.

+

+In order to make sure that cut and paste of the final document works okay you can

+best make sure that there is a \type {tounicode} vector enforced. Not all \PDF\

+viewers handle this right so take \ACROBAT\ as reference.

+

+\stopsection

+

+\startsection[reference=virtualfonts,title={Virtual fonts}]

+

+\subsection{The structure}

+

+\topicindex {fonts+virtual}

+

+You have to take the following steps if you want \LUATEX\ to treat the returned

+table from \cbk {define_font} as a virtual font:

+

+\startitemize[packed]

+\startitem

+    Set the top|-|level key \type {type} to \type {virtual}. In most cases it's

+    optional because we look at the \type {commands} entry anyway.

+\stopitem

+\startitem

+    Make sure there is at least one valid entry in \type {fonts} (see below),

+    although recent versions of \LUATEX\ add a default entry when this table is

+    missing.

+\stopitem

+\startitem

+    Add a \type {commands} array to those characters that matter. A virtual

+    character can itself point to virtual characters but be careful with nesting

+    as you can create loops and overflow the stack (which often indicates an

+    error anyway).

+\stopitem

+\stopitemize

+

+The presence of the toplevel \type {type} key with the specific value \type

+{virtual} will trigger handling of the rest of the special virtual font fields in

+the table, but the mere existence of 'type' is enough to prevent \LUATEX\ from

+looking for a virtual font on its own. This also works \quote {in reverse}: if

+you are absolutely certain that a font is not a virtual font, assigning the value

+\type {real} to \type {type} will inhibit \LUATEX\ from looking for a virtual

+font file, thereby saving you a disk search. This only matters when we load a

+\TFM\ file.

+

+The \type {fonts} is an (indexed) \LUA\ table. The values are one- or two|-|key

+hashes themselves, each entry indicating one of the base fonts in a virtual font.

+In case your font is referring to itself, you can use the \type {font.nextid()}

+function which returns the index of the next to be defined font which is probably

+the currently defined one. So, a table looks like this:

+

+\starttyping

+fonts = {

+  { name = "ptmr8a", size = 655360 },

+  { name = "psyr", size = 600000 },

+  { id = 38 }

+}

+\stoptyping

+

+The first referenced font (at index~1) in this virtual font is \type {ptrmr8a}

+loaded at 10pt, and the second is \type {psyr} loaded at a little over 9pt. The

+third one is a previously defined font that is known to \LUATEX\ as font id~38.

+The array index numbers are used by the character command definitions that are

+part of each character.

+

+The \type {commands} array is a hash where each item is another small array,

+with the first entry representing a command and the extra items being the

+parameters to that command. The allowed commands and their arguments are:

+

+\starttabulate[|l|l|l|p|]

+\DB command        \BC arguments \BC type      \BC description \NC \NR

+\TB

+\NC \type{font}    \NC 1         \NC number    \NC select a new font from the local \type {fonts} table \NC \NR

+\NC \type{char}    \NC 1         \NC number    \NC typeset this character number from the current font,

+                                                   and move right by the character's width \NC \NR

+\NC \type{node}    \NC 1         \NC node      \NC output this node (list), and move right

+                                                   by the width of this list\NC \NR

+\NC \type{slot}    \NC 2         \NC 2 numbers \NC a shortcut for the combination of a font and char command\NC \NR

+\NC \type{push}    \NC 0         \NC           \NC save current position\NC \NR

+\NC \type{nop}     \NC 0         \NC           \NC do nothing \NC \NR

+\NC \type{pop}     \NC 0         \NC           \NC pop position \NC \NR

+\NC \type{rule}    \NC 2         \NC 2 numbers \NC output a rule $ht*wd$, and move right. \NC \NR

+\NC \type{down}    \NC 1         \NC number    \NC move down on the page \NC \NR

+\NC \type{right}   \NC 1         \NC number    \NC move right on the page \NC \NR

+\NC \type{special} \NC 1         \NC string    \NC output a \prm {special} command \NC \NR

+\NC \type{pdf}     \NC 2         \NC 2 strings \NC output a \PDF\ literal, the first string is one of \type {origin},

+                                                   \type {page}, \type {text}, \type {font}, \type {direct} or \type {raw}; if you

+                                                   have one string only \type {origin} is assumed \NC \NR

+\NC \type{lua}     \NC 1         \NC string,

+                                     function  \NC execute a \LUA\ script when the glyph is embedded; in case of a

+                                                   function it gets the font id and character code passed \NC \NR

+\NC \type{image}   \NC 1         \NC image     \NC output an image (the argument can be either an \type {<image>} variable or an \type {image_spec} table) \NC \NR

+\NC \type{comment} \NC any       \NC any       \NC the arguments of this command are ignored \NC \NR

+\LL

+\stoptabulate

+

+When a font id is set to~0 then it will be replaced by the currently assigned

+font id. This prevents the need for hackery with future id's. Normally one could

+use \type {font.nextid} but when more complex fonts are built in the meantime

+other instances could have been loaded.

+

+The \type {pdf} option also accepts a \type {mode} keyword in which case the

+third argument sets the mode. That option will change the mode in an efficient

+way (passing an empty string would result in an extra empty lines in the \PDF\

+file. This option only makes sense for virtual fonts. The \type {font} mode only

+makes sense in virtual fonts. Modes are somewhat fuzzy and partially inherited

+from \PDFTEX.

+

+\starttabulate[|l|p|]

+\DB mode           \BC description \NC \NR

+\TB

+\NC \type {origin} \NC enter page mode and set the position \NC \NR

+\NC \type {page}   \NC enter page mode \NC \NR

+\NC \type {text}   \NC enter text mode \NC \NR

+\NC \type {font}   \NC enter font mode (kind of text mode, only in virtual fonts) \NC \NR

+\NC \type {always} \NC finish the current string and force a transform if needed \NC \NR

+\NC \type {raw}    \NC finish the current string \NC \NR

+\LL

+\stoptabulate

+

+You always need to check what \PDF\ code is generated because there can be all

+kind of interferences with optimization in the backend and fonts are complicated

+anyway. Here is a rather elaborate glyph commands example using such keys:

+

+\starttyping

+...

+commands = {

+    { "push" },                     -- remember where we are

+    { "right", 5000 },              -- move right about 0.08pt

+    { "font", 3 },                  -- select the fonts[3] entry

+    { "char", 97 },                 -- place character 97 (ASCII 'a')

+ -- { "slot", 2, 97 },              -- an alternative for the previous two

+    { "pop" },                      -- go all the way back

+    { "down", -200000 },            -- move upwards by about 3pt

+    { "special", "pdf: 1 0 0 rg" }  -- switch to red color

+ -- { "pdf", "origin", "1 0 0 rg" } -- switch to red color (alternative)

+    { "rule", 500000, 20000 }       -- draw a bar

+    { "special", "pdf: 0 g" }       -- back to black

+ -- { "pdf", "origin", "0 g" }      -- back to black (alternative)

+}

+...

+\stoptyping

+

+The default value for \type {font} is always~1 at the start of the

+\type {commands} array. Therefore, if the virtual font is essentially only a

+re|-|encoding, then you do usually not have created an explicit \quote {font}

+command in the array.

+

+Rules inside of \type {commands} arrays are built up using only two dimensions:

+they do not have depth. For correct vertical placement, an extra \type {down}

+command may be needed.

+

+Regardless of the amount of movement you create within the \type {commands}, the

+output pointer will always move by exactly the width that was given in the \type

+{width} key of the character hash. Any movements that take place inside the \type

+{commands} array are ignored on the upper level.

+

+The special can have a \type {pdf:}, \type {pdf:origin:},  \type {pdf:page:},

+\type {pdf:direct:} or  \type {pdf:raw:} prefix. When you have to concatenate

+strings using the \type {pdf} command might be more efficient.

+

+\subsection{Artificial fonts}

+

+Even in a \quote {real} font, there can be virtual characters. When \LUATEX\

+encounters a \type {commands} field inside a character when it becomes time to

+typeset the character, it will interpret the commands, just like for a true

+virtual character. In this case, if you have created no \quote {fonts} array,

+then the default (and only) \quote {base} font is taken to be the current font

+itself. In practice, this means that you can create virtual duplicates of

+existing characters which is useful if you want to create composite characters.

+

+Note: this feature does {\it not\/} work the other way around. There can not be

+\quote {real} characters in a virtual font! You cannot use this technique for

+font re-encoding either; you need a truly virtual font for that (because

+characters that are already present cannot be altered).

+

+\subsection{Example virtual font}

+

+\topicindex {fonts+virtual}

+

+Finally, here is a plain \TEX\ input file with a virtual font demonstration:

+

+\startbuffer

+\directlua {

+  callback.register('define_font',

+    function (name,size)

+      if name == 'cmr10-red' then

+        local f = font.read_tfm('cmr10',size)

+        f.name  = 'cmr10-red'

+        f.type  = 'virtual'

+        f.fonts = {

+          { name = 'cmr10', size = size }

+        }

+        for i,v in pairs(f.characters) do

+          if string.char(i):find('[tacohanshartmut]') then

+             v.commands = {

+               { "special", "pdf: 1 0 0 rg" },

+               { "char", i },

+               { "special", "pdf: 0 g" },

+              }

+          end

+        end

+        return f

+      else

+        return font.read_tfm(name,size)

+      end

+    end

+  )

+}

+

+\font\myfont  = cmr10-red at 10pt \myfont  This is a line of text \par

+\font\myfontx = cmr10     at 10pt \myfontx Here is another line of text \par

+\stopbuffer

+

+\typebuffer

+

+\stopsection

+

+\startsection[title={The \type {vf} library}]

+

+The \type {vf} library can be used when \LUA\ code, as defined in the \type

+{commands} of the font, is executed. The functions provided are similar as the

+commands: \type {char}, \type {down}, \type {fontid}, \type {image}, \type

+{node}, \type {nop}, \type {pop}, \type {push}, \type {right}, \nod {rule}, \type

+{special} and \type {pdf}. This library has been present for a while but not been

+advertised and tested much, if only because it's easy to define an invalid font

+(or mess up the \PDF\ stream). Keep in mind that the \LUA\ snippets are executed

+each time when a character is output.

+

+\stopsection

+

+\startsection[title={The \type {font} library}]

+

+\topicindex {fonts+library}

+

+The font library provides the interface into the internals of the font system,

+and it also contains helper functions to load traditional \TEX\ font metrics

+formats. Other font loading functionality is provided by the \type {fontloader}

+library that will be discussed in the next section.

+

+\subsection{Loading a \TFM\ file}

+

+\topicindex {fonts+tfm}

+

+The behaviour documented in this subsection is considered stable in the sense that

+there will not be backward|-|incompatible changes any more.

+

+\startfunctioncall

+<table> fnt =

+    font.read_tfm(<string> name, <number> s)

+\stopfunctioncall

+

+The number is a bit special:

+

+\startitemize

+\startitem

+    If it is positive, it specifies an \quote {at size} in scaled points.

+\stopitem

+\startitem

+    If it is negative, its absolute value represents a \quote {scaled}

+    setting relative to the designsize of the font.

+\stopitem

+\stopitemize

+

+\subsection{Loading a \VF\ file}

+

+\topicindex {fonts+vf}

+

+The behavior documented in this subsection is considered stable in the sense that

+there will not be backward-incompatible changes any more.

+

+\startfunctioncall

+<table> vf_fnt =

+    font.read_vf(<string> name, <number> s)

+\stopfunctioncall

+

+The meaning of the number \type {s} and the format of the returned table are

+similar to the ones in the \type {read_tfm} function.

+

+\subsection{The fonts array}

+

+\topicindex {fonts+virtual}

+

+The whole table of \TEX\ fonts is accessible from \LUA\ using a virtual array.

+

+\starttyping

+font.fonts[n] = { ... }

+<table> f = font.fonts[n]

+\stoptyping

+

+Because this is a virtual array, you cannot call \type {pairs} on it, but see

+below for the \type {font.each} iterator.

+

+The two metatable functions implementing the virtual array are:

+

+\startfunctioncall

+<table> f = font.getfont(<number> n)

+font.setfont(<number> n, <table> f)

+\stopfunctioncall

+

+Note that at the moment, each access to the \type {font.fonts} or call to \type

+{font.getfont} creates a \LUA\ table for the whole font unless you cached it.

+This process can be quite slow.

+

+\startfunctioncall

+<table> p = font.getparameters(<number> n)

+\stopfunctioncall

+

+This one will return a table of the parameters as known to \TEX. These can be

+different from the ones in the cached table.

+

+Also note the following: assignments can only be made to fonts that have already

+been defined in \TEX, but have not been accessed {\it at all\/} since that

+definition. This limits the usability of the write access to \type {font.fonts}

+quite a lot, a less stringent ruleset will likely be implemented later.

+

+\subsection{Checking a font's status}

+

+You can test for the status of a font by calling this function:

+

+\startfunctioncall

+<boolean> f =

+    font.frozen(<number> n)

+\stopfunctioncall

+

+The return value is one of \type {true} (unassignable), \type {false} (can be

+changed) or \type {nil} (not a valid font at all).

+

+\subsection{Defining a font directly}

+

+\topicindex {fonts+define}

+

+You can define your own font into \type {font.fonts} by calling this function:

+

+\startfunctioncall

+<number> i =

+    font.define(<table> f)

+\stopfunctioncall

+

+The return value is the internal id number of the defined font (the index into

+\type {font.fonts}). If the font creation fails, an error is raised. The table is

+a font structure. An alternative call is:

+

+\startfunctioncall

+<number> i =

+    font.define(<number> n, <table> f)

+\stopfunctioncall

+

+Where the first argument is a reserved font id (see below).

+

+\subsection{Extending a font}

+

+\topicindex {fonts+extend}

+

+Within reasonable bounds you can extend a font after it has been defined. Because

+some properties are best left unchanged this is limited to adding characters.

+

+\startfunctioncall

+font.addcharacters(<number n>, <table> f)

+\stopfunctioncall

+

+The table passed can have the fields \type {characters} which is a (sub)table

+like the one used in define, and for virtual fonts a \type {fonts} table can be

+added. The characters defined in the \type {characters} table are added (when not

+yet present) or replace an existing entry. Keep in mind that replacing can have

+side effects because a character already can have been used. Instead of posing

+restrictions we expect the user to be careful. (The \type {setfont} helper is

+a more drastic replacer.)

+

+\subsection{Projected next font id}

+

+\topicindex {fonts+id}

+

+\startfunctioncall

+<number> i =

+    font.nextid()

+\stopfunctioncall

+

+This returns the font id number that would be returned by a \type {font.define}

+call if it was executed at this spot in the code flow. This is useful for virtual

+fonts that need to reference themselves. If you pass \type {true} as argument,

+the id gets reserved and you can pass to \type {font.define} as first argument.

+This can be handy when you create complex virtual fonts.

+

+\startfunctioncall

+<number> i =

+    font.nextid(true)

+\stopfunctioncall

+

+\subsection{Font ids}

+

+\topicindex {fonts+id}

+\topicindex {fonts+current}

+

+\startfunctioncall

+<number> i =

+    font.id(<string> csname)

+\stopfunctioncall

+

+This returns the font id associated with \type {csname}, or $-1$ if \type

+{csname} is not defined.

+

+\startfunctioncall

+<number> i =

+    font.max()

+\stopfunctioncall

+

+This is the largest used index in \type {font.fonts}.

+

+\startfunctioncall

+<number> i = font.current()

+font.current(<number> i)

+\stopfunctioncall

+

+This gets or sets the currently used font number.

+

+\subsection{Iterating over all fonts}

+

+\topicindex {fonts+iterate}

+

+\startfunctioncall

+for i,v in font.each() do

+  ...

+end

+\stopfunctioncall

+

+This is an iterator over each of the defined \TEX\ fonts. The first returned

+value is the index in \type {font.fonts}, the second the font itself, as a \LUA\

+table. The indices are listed incrementally, but they do not always form an array

+of consecutive numbers: in some cases there can be holes in the sequence.

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-graphics.tex b/Master/texmf-dist/doc/luatex/base/luatex-graphics.tex
index 56da90071b5..038710cd583 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-graphics.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-graphics.tex
@@ -1,647 +1,831 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-graphics
-
-\startchapter[reference=graphics,title={The graphic libraries}]
-
-\section{The \type {img} library}
-
-The \type {img} library can be used as an alternative to \type {\pdfximage} and
-\type {\pdfrefximage}, and the associated \quote {satellite} commands like \type
-{\pdfximagebbox}. Image objects can also be used within virtual fonts via the
-\type {image} command listed in~\in {section} [virtualfonts].
-
-\subsection{\type {new}}
-
-\startfunctioncall
-<image> var = img.new()
-<image> var = img.new(<table> image_spec)
-\stopfunctioncall
-
-This function creates a userdata object of type \quote {image}. The \type
-{image_spec} argument is optional. If it is given, it must be a table, and that
-table must contain a \type {filename} key. A number of other keys can also be
-useful, these are explained below.
-
-You can either say
-
-\starttyping
-a = img.new()
-\stoptyping
-
-followed by
-
-\starttyping
-a.filename = "foo.png"
-\stoptyping
-
-or you can put the file name (and some or all of the other keys) into a table
-directly, like so:
-
-\starttyping
-a = img.new({filename='foo.pdf', page=1})
-\stoptyping
-
-The generated \type {<image>} userdata object allows access to a set of
-user|-|specified values as well as a set of values that are normally filled in
-and updated automatically by \LUATEX\ itself. Some of those are derived from the
-actual image file, others are updated to reflect the \PDF\ output status of the
-object.
-
-There is one required user-specified field: the file name (\type {filename}). It
-can optionally be augmented by the requested image dimensions (\type {width},
-\type {depth}, \type {height}), user|-|specified image attributes (\type {attr}),
-the requested \PDF\ page identifier (\type {page}), the requested boundingbox
-(\type {pagebox}) for \PDF\ inclusion, the requested color space object (\type
-{colorspace}).
-
-The function \type {img.new} does not access the actual image file, it just
-creates the \type {<image>} userdata object and initializes some memory
-structures. The \type {<image>} object and its internal structures are
-automatically garbage collected.
-
-Once the image is scanned, all the values in the \type {<image>} except \type
-{width}, \type {height} and \type {depth}, become frozen, and you cannot change
-them any more.
-
-You can use \type {pdf.setignoreunknownimages(1)} (or at the \TEX\ end the \type
-{\pdfvariable} \type {ignoreunknownimages}) to get around a quit when no known
-image type is found (based on name or preamble). Beware: this will not catch
-invalid images and we cannot guarantee side effects. A zero dimension image is
-still included when requested. No special flags are set. A proper workflow will
-not rely in such a catch but make sure that images are valid.
-
-\subsection{\type {keys}}
-
-\startfunctioncall
-<table> keys = img.keys()
-\stopfunctioncall
-
-This function returns a list of all the possible \type {image_spec} keys, both
-user-supplied and automatic ones.
-
-% hahe: i need to add r/w ro column...
-\starttabulate[|l|l|p|]
-\BC field name            \BC type     \BC description \NC \NR
-\NC \type{attr}           \NC string   \NC the image attributes for \LUATEX \NC \NR
-\NC \type{bbox}           \NC table    \NC table with 4 boundingbox dimensions \type
-                                           {llx}, \type {lly}, \type {urx} and \type
-                                           {ury} overruling the \type {pagebox} entry \NC \NR
-\NC \type{colordepth}     \NC number   \NC the number of bits used by the color space \NC \NR
-\NC \type{colorspace}     \NC number   \NC the color space object number \NC \NR
-\NC \type{depth}          \NC number   \NC the image depth for \LUATEX \NC \NR
-\NC \type{filename}       \NC string   \NC the image file name \NC \NR
-\NC \type{filepath}       \NC string   \NC the full (expanded) file name of the image\NC \NR
-\NC \type{height}         \NC number   \NC the image height for \LUATEX \NC \NR
-\NC \type{imagetype}      \NC string   \NC one of \type {pdf}, \type {png}, \type {jpg},
-                                           \type {jp2} or \type {jbig2} \NC \NR
-\NC \type{index}          \NC number   \NC the \PDF\ image name suffix \NC \NR
-\NC \type{objnum}         \NC number   \NC the \PDF\ image object number \NC \NR
-\NC \type{page}           \NC number   \NC the identifier for the requested image page \NC \NR
-\NC \type{pagebox}        \NC string   \NC the requested bounding box, one of
-                                           \type {none}, \type {media}, \type {crop},
-                                           \type {bleed}, \type {trim}, \type {art} \NC \NR
-\NC \type{pages}          \NC number   \NC the total number of available pages \NC \NR
-\NC \type{rotation}       \NC number   \NC the image rotation from included \PDF\ file,
-                                           in multiples of 90~deg. \NC \NR
-\NC \type{stream}         \NC string   \NC the raw stream data for an \type {/Xobject}
-                                           \type {/Form} object\NC \NR
-\NC \type{transform}      \NC number   \NC the image transform, integer number 0..7 \NC \NR
-\NC \type{orientation}    \NC number   \NC the (jpeg) image orientation, integer number 1..8 (0 for unset) \NC \NR
-\NC \type{width}          \NC number   \NC the image width for \LUATEX \NC \NR
-\NC \type{xres}           \NC number   \NC the horizontal natural image resolution
-                                           (in \DPI) \NC \NR
-\NC \type{xsize}          \NC number   \NC the natural image width \NC \NR
-\NC \type{yres}           \NC number   \NC the vertical natural image resolution
-                                           (in \DPI) \NC \NR
-\NC \type{ysize}          \NC number   \NC the natural image height \NC \NR
-\NC \type{visiblefileame} \NC string   \NC when set, this name will find its way in the
-                                           \PDF\ file as \type {PTEX} specification; when
-                                           an empty string is assigned nothing is written
-                                           to file; otherwise the natural filename is
-                                           taken \NC \NR
-\stoptabulate
-
-A running (undefined) dimension in \type {width}, \type {height}, or \type
-{depth} is represented as \type {nil} in \LUA, so if you want to load an image at
-its \quote {natural} size, you do not have to specify any of those three fields.
-
-The \type {stream} parameter allows to fabricate an \type {/XObject} \type
-{/Form} object from a string giving the stream contents, e.g., for a filled
-rectangle:
-
-\startfunctioncall
-a.stream = "0 0 20 10 re f"
-\stopfunctioncall
-
-When writing the image, an \type {/Xobject} \type {/Form} object is created, like
-with embedded \PDF\ file writing. The object is written out only once. The \type
-{stream} key requires that also the \type {bbox} table is given. The \type
-{stream} key conflicts with the \type {filename} key. The \type {transform} key
-works as usual also with \type {stream}.
-
-The \type {bbox} key needs a table with four boundingbox values, e.g.:
-
-\startfunctioncall
-a.bbox = { "30bp", 0, "225bp", "200bp" }
-\stopfunctioncall
-
-This replaces and overrules any given \type {pagebox} value; with given \type
-{bbox} the box dimensions coming with an embedded \PDF\ file are ignored. The
-\type {xsize} and \type {ysize} dimensions are set accordingly, when the image is
-scaled. The \type {bbox} parameter is ignored for non-\PDF\ images.
-
-The \type {transform} allows to mirror and rotate the image in steps of 90~deg.
-The default value~$0$ gives an unmirrored, unrotated image. Values $1-3$ give
-counterclockwise rotation by $90$, $180$, or $270$~degrees, whereas with values
-$4-7$ the image is first mirrored and then rotated counterclockwise by $90$,
-$180$, or $270$~degrees. The \type {transform} operation gives the same visual
-result as if you would externally preprocess the image by a graphics tool and
-then use it by \LUATEX. If a \PDF\ file to be embedded already contains a \type
-{/Rotate} specification, the rotation result is the combination of the \type
-{/Rotate} rotation followed by the \type {transform} operation.
-
-\subsection{\type {scan}}
-
-\startfunctioncall
-<image> var = img.scan(<image> var)
-<image> var = img.scan(<table> image_spec)
-\stopfunctioncall
-
-When you say \type {img.scan(a)} for a new image, the file is scanned, and
-variables such as \type {xsize}, \type {ysize}, image \type {type}, number of
-\type {pages}, and the resolution are extracted. Each of the \type {width}, \type
-{height}, \type {depth} fields are set up according to the image dimensions, if
-they were not given an explicit value already. An image file will never be
-scanned more than once for a given image variable. With all subsequent \type
-{img.scan(a)} calls only the dimensions are again set up (if they have been
-changed by the user in the meantime).
-
-For ease of use, you can do right-away a
-
-\starttyping
-<image> a = img.scan { filename = "foo.png" }
-\stoptyping
-
-without a prior \type {img.new}.
-
-Nothing is written yet at this point, so you can do \type {a=img.scan}, retrieve
-the available info like image width and height, and then throw away \type {a}
-again by saying \type {a=nil}. In that case no image object will be reserved in
-the PDF, and the used memory will be cleaned up automatically.
-
-\subsection{\type {copy}}
-
-\startfunctioncall
-<image> var = img.copy(<image> var)
-<image> var = img.copy(<table> image_spec)
-\stopfunctioncall
-
-If you say \type {a = b}, then both variables point to the same \type {<image>}
-object. if you want to write out an image with different sizes, you can do a
-\type {b = img.copy(a)}.
-
-Afterwards, \type {a} and \type {b} still reference the same actual image
-dictionary, but the dimensions for \type {b} can now be changed from their
-initial values that were just copies from \type {a}.
-
-\subsection{\type {write}}
-
-\startfunctioncall
-<image> var = img.write(<image> var)
-<image> var = img.write(<table> image_spec)
-\stopfunctioncall
-
-By \type {img.write(a)} a \PDF\ object number is allocated, and a whatsit node of
-subtype \type {pdf_refximage} is generated and put into the output list. By this
-the image \type {a} is placed into the page stream, and the image file is written
-out into an image stream object after the shipping of the current page is
-finished.
-
-Again you can do a terse call like
-
-\starttyping
-img.write { filename = "foo.png" }
-\stoptyping
-
-The \type {<image>} variable is returned in case you want it for later
-processing.
-
-\subsection{\type {immediatewrite}}
-
-\startfunctioncall
-<image> var = img.immediatewrite(<image> var)
-<image> var = img.immediatewrite(<table> image_spec)
-\stopfunctioncall
-
-By \type {img.immediatewrite(a)} a \PDF\ object number is allocated, and the
-image file for image \type {a} is written out immediately into the \PDF\ file as
-an image stream object (like with \type {\immediate}\type {\pdfximage}). The object
-number of the image stream dictionary is then available by the \type {objnum}
-key. No \type {pdf_refximage} whatsit node is generated. You will need an
-\type {img.write(a)} or \type {img.node(a)} call to let the image appear on the
-page, or reference it by another trick; else you will have a dangling image
-object in the \PDF\ file.
-
-Also here you can do a terse call like
-
-\starttyping
-a = img.immediatewrite { filename = "foo.png" }
-\stoptyping
-
-The \type {<image>} variable is returned and you will most likely need it.
-
-\subsection{\type {node}}
-
-\startfunctioncall
-<node> n = img.node(<image> var)
-<node> n = img.node(<table> image_spec)
-\stopfunctioncall
-
-This function allocates a \PDF\ object number and returns a whatsit node of
-subtype \type {pdf_refximage}, filled with the image parameters \type {width},
-\type {height}, \type {depth}, and \type {objnum}. Also here you can do a terse
-call like:
-
-\starttyping
-n = img.node ({ filename = "foo.png" })
-\stoptyping
-
-This example outputs an image:
-
-\starttyping
-node.write(img.node{filename="foo.png"})
-\stoptyping
-
-\subsection{\type {types}}
-
-\startfunctioncall
-<table> types = img.types()
-\stopfunctioncall
-
-This function returns a list with the supported image file type names, currently
-these are \type {pdf}, \type {png}, \type {jpg}, \type {jp2} (JPEG~2000), and
-\type {jbig2}.
-
-\subsection{\type {boxes}}
-
-\startfunctioncall
-<table> boxes = img.boxes()
-\stopfunctioncall
-
-This function returns a list with the supported \PDF\ page box names, currently
-these are \type {media}, \type {crop}, \type {bleed}, \type {trim}, and \type
-{art}, all in lowercase letters.
-
-\section{The \type {mplib} library}
-
-The \MP\ library interface registers itself in the table \type {mplib}. It is
-based on \MPLIB\ version \ctxlua {context(mplib.version())}.
-
-\subsection{\type {new}}
-
-To create a new \METAPOST\ instance, call
-
-\startfunctioncall
-<mpinstance> mp = mplib.new({...})
-\stopfunctioncall
-
-This creates the \type {mp} instance object. The argument hash can have a number
-of different fields, as follows:
-
-\starttabulate[|l|l|p|p|]
-\BC name               \BC type     \BC description              \BC default           \NC \NR
-\NC \type{error_line}  \NC number   \NC error line width         \NC 79                \NC \NR
-\NC \type{print_line}  \NC number   \NC line length in ps output \NC 100               \NC \NR
-\NC \type{random_seed} \NC number   \NC the initial random seed  \NC variable          \NC \NR
-\NC \type{math_mode}   \NC string   \NC the number system to use:
-                                        \type {double},
-                                        \type {scaled},
-                                        \type {binary} or
-                                        \type {decimal}          \NC \type {scaled}    \NC \NR
-\NC \type{interaction} \NC string   \NC the interaction mode:
-                                        \type {batch},
-                                        \type {nonstop},
-                                        \type {scroll} or
-                                        \type {errorstop}        \NC \type {errorstop} \NC \NR
-\NC \type{job_name}    \NC string   \NC \type {--jobname}        \NC \type {mpout}     \NC \NR
-\NC \type{find_file}   \NC function \NC a function to find files \NC only local files  \NC \NR
-\stoptabulate
-
-The \type {find_file} function should be of this form:
-
-\starttyping
-<string> found = finder (<string> name, <string> mode, <string> type)
-\stoptyping
-
-with:
-
-\starttabulate[|l|p|]
-\BC name        \BC the requested file \NC \NR
-\NC \type{mode} \NC the file mode: \type {r} or \type {w} \NC \NR
-\NC \type{type} \NC the kind of file, one of: \type {mp}, \type {tfm}, \type {map},
-                    \type {pfb}, \type {enc} \NC \NR
-\stoptabulate
-
-Return either the full path name of the found file, or \type {nil} if the file
-cannot be found.
-
-Note that the new version of \MPLIB\ no longer uses binary mem files, so the way
-to preload a set of macros is simply to start off with an \type {input} command
-in the first \type {mp:execute()} call.
-
-\subsection{\type {mp:statistics}}
-
-You can request statistics with:
-
-\startfunctioncall
-<table> stats = mp:statistics()
-\stopfunctioncall
-
-This function returns the vital statistics for an \MPLIB\ instance. There are
-four fields, giving the maximum number of used items in each of four allocated
-object classes:
-
-\starttabulate[|l|l|p|]
-\NC \type{main_memory} \NC number \NC memory size \NC \NR
-\NC \type{hash_size}   \NC number \NC hash size\NC \NR
-\NC \type{param_size}  \NC number \NC simultaneous macro parameters\NC \NR
-\NC \type{max_in_open} \NC number \NC input file nesting levels\NC \NR
-\stoptabulate
-
-Note that in the new version of \MPLIB, this is informational only. The objects
-are all allocated dynamically, so there is no chance of running out of space
-unless the available system memory is exhausted.
-
-\subsection{\type {mp:execute}}
-
-You can ask the \METAPOST\ interpreter to run a chunk of code by calling
-
-\startfunctioncall
-<table> rettable = mp:execute('metapost language chunk')
-\stopfunctioncall
-
-for various bits of \METAPOST\ language input. Be sure to check the \type
-{rettable.status} (see below) because when a fatal \METAPOST\ error occurs the
-\MPLIB\ instance will become unusable thereafter.
-
-Generally speaking, it is best to keep your chunks small, but beware that all
-chunks have to obey proper syntax, like each of them is a small file. For
-instance, you cannot split a single statement over multiple chunks.
-
-In contrast with the normal stand alone \type {mpost} command, there is
-\notabene {no} implied \quote{input} at the start of the first chunk.
-
-\subsection{\type {mp:finish}}
-
-\startfunctioncall
-<table> rettable = mp:finish()
-\stopfunctioncall
-
-If for some reason you want to stop using an \MPLIB\ instance while processing is
-not yet actually done, you can call \type {mp:finish}. Eventually, used memory
-will be freed and open files will be closed by the \LUA\ garbage collector, but
-an explicit \type {mp:finish} is the only way to capture the final part of the
-output streams.
-
-\subsection{Result table}
-
-The return value of \type {mp:execute} and \type {mp:finish} is a table with a
-few possible keys (only \type {status} is always guaranteed to be present).
-
-\starttabulate[|l|l|p|]
-\NC \type{log}    \NC string \NC output to the \quote {log} stream \NC \NR
-\NC \type{term}   \NC string \NC output to the \quote {term} stream \NC \NR
-\NC \type{error}  \NC string \NC output to the \quote {error} stream
-                                 (only used for \quote {out of memory}) \NC \NR
-\NC \type{status} \NC number \NC the return value:
-                                 \type {0} = good,
-                                 \type {1} = warning,
-                                 \type {2} = errors,
-                                 \type {3} = fatal error \NC \NR
-\NC \type{fig}    \NC table  \NC an array of generated figures (if any) \NC \NR
-\stoptabulate
-
-When \type {status} equals~3, you should stop using this \MPLIB\ instance
-immediately, it is no longer capable of processing input.
-
-If it is present, each of the entries in the \type {fig} array is a userdata
-representing a figure object, and each of those has a number of object methods
-you can call:
-
-\starttabulate[|l|l|p|]
-\NC \type{boundingbox}  \NC function \NC returns the bounding box, as an array of 4
-                                         values \NC \NR
-\NC \type{postscript}   \NC function \NC returns a string that is the ps output of the
-                                         \type {fig}. this function accepts two optional
-                                         integer arguments for specifying the values of
-                                         \type {prologues} (first argument) and \type
-                                         {procset} (second argument) \NC \NR
-\NC \type{svg}          \NC function \NC returns a string that is the svg output of the
-                                         \type {fig}. This function accepts an optional
-                                         integer argument for specifying the value of
-                                         \type {prologues} \NC \NR
-\NC \type{objects}      \NC function \NC returns the actual array of graphic objects in
-                                         this \type {fig} \NC \NR
-\NC \type{copy_objects} \NC function \NC returns a deep copy of the array of graphic
-                                         objects in this \type {fig} \NC \NR
-\NC \type{filename}     \NC function \NC the filename this \type {fig}'s \POSTSCRIPT\
-                                         output would have written to in stand alone
-                                         mode \NC \NR
-\NC \type{width}        \NC function \NC the \type {fontcharwd} value \NC \NR
-\NC \type{height}       \NC function \NC the \type {fontcharht} value \NC \NR
-\NC \type{depth}        \NC function \NC the \type {fontchardp} value \NC \NR
-\NC \type{italcorr}     \NC function \NC the \type {fontcharit} value \NC \NR
-\NC \type{charcode}     \NC function \NC the (rounded) \type {charcode} value \NC \NR
-\stoptabulate
-
-Note: you can call \type {fig:objects()} only once for any one \type {fig}
-object!
-
-When the boundingbox represents a \quote {negated rectangle}, i.e.\ when the
-first set of coordinates is larger than the second set, the picture is empty.
-
-Graphical objects come in various types that each has a different list of
-accessible values. The types are: \type {fill}, \type {outline}, \type {text},
-\type {start_clip}, \type {stop_clip}, \type {start_bounds}, \type {stop_bounds},
-\type {special}.
-
-There is helper function (\type {mplib.fields(obj)}) to get the list of
-accessible values for a particular object, but you can just as easily use the
-tables given below.
-
-All graphical objects have a field \type {type} that gives the object type as a
-string value; it is not explicit mentioned in the following tables. In the
-following, \type {number}s are \POSTSCRIPT\ points represented as a floating
-point number, unless stated otherwise. Field values that are of type \type
-{table} are explained in the next section.
-
-\subsubsection{fill}
-
-\starttabulate[|l|l|p|]
-\NC \type{path}       \NC table  \NC the list of knots \NC \NR
-\NC \type{htap}       \NC table  \NC the list of knots for the reversed trajectory \NC \NR
-\NC \type{pen}        \NC table  \NC knots of the pen \NC \NR
-\NC \type{color}      \NC table  \NC the object's color \NC \NR
-\NC \type{linejoin}   \NC number \NC line join style (bare number)\NC \NR
-\NC \type{miterlimit} \NC number \NC miterlimit\NC \NR
-\NC \type{prescript}  \NC string \NC the prescript text \NC \NR
-\NC \type{postscript} \NC string \NC the postscript text \NC \NR
-\stoptabulate
-
-The entries \type {htap} and \type {pen} are optional.
-
-There is helper function (\type {mplib.pen_info(obj)}) that returns a table
-containing a bunch of vital characteristics of the used pen (all values are
-floats):
-
-\starttabulate[|l|l|p|]
-\NC \type{width} \NC number \NC width of the pen \NC \NR
-\NC \type{sx}    \NC number \NC $x$ scale        \NC \NR
-\NC \type{rx}    \NC number \NC $xy$ multiplier  \NC \NR
-\NC \type{ry}    \NC number \NC $yx$ multiplier  \NC \NR
-\NC \type{sy}    \NC number \NC $y$ scale        \NC \NR
-\NC \type{tx}    \NC number \NC $x$ offset       \NC \NR
-\NC \type{ty}    \NC number \NC $y$ offset       \NC \NR
-\stoptabulate
-
-\subsubsection{outline}
-
-\starttabulate[|l|l|p|]
-\NC \type{path}       \NC table  \NC the list of knots \NC \NR
-\NC \type{pen}        \NC table  \NC knots of the pen \NC \NR
-\NC \type{color}      \NC table  \NC the object's color \NC \NR
-\NC \type{linejoin}   \NC number \NC line join style (bare number) \NC \NR
-\NC \type{miterlimit} \NC number \NC miterlimit \NC \NR
-\NC \type{linecap}    \NC number \NC line cap style (bare number) \NC \NR
-\NC \type{dash}       \NC table  \NC representation of a dash list \NC \NR
-\NC \type{prescript}  \NC string \NC the prescript text \NC \NR
-\NC \type{postscript} \NC string \NC the postscript text \NC \NR
-\stoptabulate
-
-The entry \type {dash} is optional.
-
-\subsubsection{text}
-
-\starttabulate[|l|l|p|]
-\NC \type{text}       \NC string \NC the text \NC \NR
-\NC \type{font}       \NC string \NC font tfm name \NC \NR
-\NC \type{dsize}      \NC number \NC font size \NC \NR
-\NC \type{color}      \NC table  \NC the object's color \NC \NR
-\NC \type{width}      \NC number \NC \NC \NR
-\NC \type{height}     \NC number \NC \NC \NR
-\NC \type{depth}      \NC number \NC \NC \NR
-\NC \type{transform}  \NC table  \NC a text transformation \NC \NR
-\NC \type{prescript}  \NC string \NC the prescript text \NC \NR
-\NC \type{postscript} \NC string \NC the postscript text \NC \NR
-\stoptabulate
-
-\subsubsection{special}
-
-\starttabulate[|l|l|p|]
-\NC \type{prescript} \NC string \NC special text \NC \NR
-\stoptabulate
-
-\subsubsection{start_bounds, start_clip}
-
-\starttabulate[|l|l|p|]
-\NC \type{path} \NC table \NC the list of knots \NC \NR
-\stoptabulate
-
-\subsubsection{stop_bounds, stop_clip}
-
-Here are no fields available.
-
-\subsection{Subsidiary table formats}
-
-\subsubsection{Paths and pens}
-
-Paths and pens (that are really just a special type of paths as far as \MPLIB\ is
-concerned) are represented by an array where each entry is a table that
-represents a knot.
-
-\starttabulate[|l|l|p|]
-\NC \type{left_type}  \NC string \NC when present: endpoint, but usually absent \NC \NR
-\NC \type{right_type} \NC string \NC like \type {left_type} \NC \NR
-\NC \type{x_coord}    \NC number \NC X coordinate of this knot \NC \NR
-\NC \type{y_coord}    \NC number \NC Y coordinate of this knot \NC \NR
-\NC \type{left_x}     \NC number \NC X coordinate of the precontrol point of this knot \NC \NR
-\NC \type{left_y}     \NC number \NC Y coordinate of the precontrol point of this knot \NC \NR
-\NC \type{right_x}    \NC number \NC X coordinate of the postcontrol point of this knot \NC \NR
-\NC \type{right_y}    \NC number \NC Y coordinate of the postcontrol point of this knot \NC \NR
-\stoptabulate
-
-There is one special case: pens that are (possibly transformed) ellipses have an
-extra string-valued key \type {type} with value \type {elliptical} besides the
-array part containing the knot list.
-
-\subsubsection{Colors}
-
-A color is an integer array with 0, 1, 3 or 4 values:
-
-\starttabulate[|l|l|p|]
-\NC \type{0} \NC marking only \NC no values                                                     \NC \NR
-\NC \type{1} \NC greyscale    \NC one value in the range $(0,1)$, \quote {black} is $0$         \NC \NR
-\NC \type{3} \NC \RGB         \NC three values in the range $(0,1)$, \quote {black} is $0,0,0$  \NC \NR
-\NC \type{4} \NC \CMYK        \NC four values in the range $(0,1)$, \quote {black} is $0,0,0,1$ \NC \NR
-\stoptabulate
-
-If the color model of the internal object was \type {uninitialized}, then it was
-initialized to the values representing \quote {black} in the colorspace \type
-{defaultcolormodel} that was in effect at the time of the \type {shipout}.
-
-\subsubsection{Transforms}
-
-Each transform is a six|-|item array.
-
-\starttabulate[|l|l|p|]
-\NC \type{1} \NC number \NC represents x  \NC \NR
-\NC \type{2} \NC number \NC represents y  \NC \NR
-\NC \type{3} \NC number \NC represents xx \NC \NR
-\NC \type{4} \NC number \NC represents yx \NC \NR
-\NC \type{5} \NC number \NC represents xy \NC \NR
-\NC \type{6} \NC number \NC represents yy \NC \NR
-\stoptabulate
-
-Note that the translation (index 1 and 2) comes first. This differs from the
-ordering in \POSTSCRIPT, where the translation comes last.
-
-\subsubsection{Dashes}
-
-Each \type {dash} is two-item hash, using the same model as \POSTSCRIPT\ for the
-representation of the dashlist. \type {dashes} is an array of \quote {on} and
-\quote {off}, values, and \type {offset} is the phase of the pattern.
-
-\starttabulate[|l|l|p|]
-\NC \type{dashes} \NC hash   \NC an array of on-off numbers \NC \NR
-\NC \type{offset} \NC number \NC the starting offset value  \NC \NR
-\stoptabulate
-
-\subsection{Character size information}
-
-These functions find the size of a glyph in a defined font. The \type {fontname}
-is the same name as the argument to \type {infont}; the \type {char} is a glyph
-id in the range 0 to 255; the returned \type {w} is in AFM units.
-
-\subsubsection{\type {mp:char_width}}
-
-\startfunctioncall
-<number> w = mp:char_width(<string> fontname, <number> char)
-\stopfunctioncall
-
-\subsubsection{\type {mp:char_height}}
-
-\startfunctioncall
-<number> w = mp:char_height(<string> fontname, <number> char)
-\stopfunctioncall
-
-\subsubsection{\type {mp:char_depth}}
-
-\startfunctioncall
-<number> w = mp:char_depth(<string> fontname, <number> char)
-\stopfunctioncall
-
-\stopchapter
+% language=uk

+

+% mp solve_path

+

+\environment luatex-style

+

+\startcomponent luatex-graphics

+

+\startchapter[reference=graphics,title={The graphic libraries}]

+

+\startsection[title={The \type {img} library}][library=img]

+

+\topicindex {images}

+\topicindex {images+library}

+\topicindex {graphics}

+

+The \type {img} library can be used as an alternative to \orm {pdfximage} and

+\orm {pdfrefximage}, and the associated \quote {satellite} commands like \type

+{\pdfximagebbox}. Image objects can also be used within virtual fonts via the

+\type {image} command listed in~\in {section} [virtualfonts].

+

+\subsection{\type {new}}

+

+\libindex{new}

+

+\startfunctioncall

+<image> var = img.new()

+<image> var = img.new(<table> image_spec)

+\stopfunctioncall

+

+This function creates a userdata object of type \quote {image}. The \type

+{image_spec} argument is optional. If it is given, it must be a table, and that

+table must contain a \type {filename} key. A number of other keys can also be

+useful, these are explained below.

+

+You can either say

+

+\starttyping

+a = img.new()

+\stoptyping

+

+followed by

+

+\starttyping

+a.filename = "foo.png"

+\stoptyping

+

+or you can put the file name (and some or all of the other keys) into a table

+directly, like so:

+

+\starttyping

+a = img.new({filename='foo.pdf', page=1})

+\stoptyping

+

+The generated \type {<image>} userdata object allows access to a set of

+user|-|specified values as well as a set of values that are normally filled in

+and updated automatically by \LUATEX\ itself. Some of those are derived from the

+actual image file, others are updated to reflect the \PDF\ output status of the

+object.

+

+There is one required user-specified field: the file name (\type {filename}). It

+can optionally be augmented by the requested image dimensions (\type {width},

+\type {depth}, \type {height}), user|-|specified image attributes (\type {attr}),

+the requested \PDF\ page identifier (\type {page}), the requested boundingbox

+(\type {pagebox}) for \PDF\ inclusion, the requested color space object (\type

+{colorspace}).

+

+The function \type {img.new} does not access the actual image file, it just

+creates the \type {<image>} userdata object and initializes some memory

+structures. The \type {<image>} object and its internal structures are

+automatically garbage collected.

+

+Once the image is scanned, all the values in the \type {<image>} except \type

+{width}, \type {height} and \type {depth}, become frozen, and you cannot change

+them any more.

+

+You can use \type {pdf.setignoreunknownimages(1)} (or at the \TEX\ end the \lpr

+{pdfvariable} \type {ignoreunknownimages}) to get around a quit when no known

+image type is found (based on name or preamble). Beware: this will not catch

+invalid images and we cannot guarantee side effects. A zero dimension image is

+still included when requested. No special flags are set. A proper workflow will

+not rely in such a catch but make sure that images are valid.

+

+\subsection{\type {fields}}

+

+\libindex{fields}

+

+\startfunctioncall

+<table> keys = img.fields()

+\stopfunctioncall

+

+This function returns a list of all the possible \type {image_spec} keys, both

+user-supplied and automatic ones.

+

+\starttabulate[|l|l|p|]

+\DB field name             \BC type     \BC description \NC \NR

+\TB

+\NC \type{attr}            \NC string   \NC the image attributes for \LUATEX \NC \NR

+\NC \type{bbox}            \NC table    \NC table with 4 boundingbox dimensions \type

+                                            {llx}, \type {lly}, \type {urx} and \type

+                                            {ury} overruling the \type {pagebox} entry \NC \NR

+\NC \type{colordepth}      \NC number   \NC the number of bits used by the color space \NC \NR

+\NC \type{colorspace}      \NC number   \NC the color space object number \NC \NR

+\NC \type{depth}           \NC number   \NC the image depth for \LUATEX \NC \NR

+\NC \type{filename}        \NC string   \NC the image file name \NC \NR

+\NC \type{filepath}        \NC string   \NC the full (expanded) file name of the image\NC \NR

+\NC \type{height}          \NC number   \NC the image height for \LUATEX \NC \NR

+\NC \type{imagetype}       \NC string   \NC one of \type {pdf}, \type {png}, \type {jpg},

+                                            \type {jp2} or \type {jbig2} \NC \NR

+\NC \type{index}           \NC number   \NC the \PDF\ image name suffix \NC \NR

+\NC \type{objnum}          \NC number   \NC the \PDF\ image object number \NC \NR

+\NC \type{page}            \NC number   \NC the identifier for the requested image page \NC \NR

+\NC \type{pagebox}         \NC string   \NC the requested bounding box, one of

+                                            \type {none}, \type {media}, \type {crop},

+                                            \type {bleed}, \type {trim}, \type {art} \NC \NR

+\NC \type{pages}           \NC number   \NC the total number of available pages \NC \NR

+\NC \type{rotation}        \NC number   \NC the image rotation from included \PDF\ file,

+                                            in multiples of 90~deg. \NC \NR

+\NC \type{stream}          \NC string   \NC the raw stream data for an \type {/Xobject}

+                                            \type {/Form} object\NC \NR

+\NC \type{transform}       \NC number   \NC the image transform, integer number 0..7 \NC \NR

+\NC \type{orientation}     \NC number   \NC the (jpeg) image orientation, integer number 1..8

+                                            (0 for unset) \NC \NR

+\NC \type{width}           \NC number   \NC the image width for \LUATEX \NC \NR

+\NC \type{xres}            \NC number   \NC the horizontal natural image resolution

+                                            (in \DPI) \NC \NR

+\NC \type{xsize}           \NC number   \NC the natural image width \NC \NR

+\NC \type{yres}            \NC number   \NC the vertical natural image resolution

+                                            (in \DPI) \NC \NR

+\NC \type{ysize}           \NC number   \NC the natural image height \NC \NR

+\NC \type{visiblefilename} \NC string   \NC when set, this name will find its way in the

+                                            \PDF\ file as \type {PTEX} specification; when

+                                            an empty string is assigned nothing is written

+                                            to file; otherwise the natural filename is

+                                            taken \NC \NR

+\NC \type{userpassword}   \NC string   \NC  the userpassword needed for opening a \PDF\ file \NC \NR

+\NC \type{ownerpassword}  \NC string   \NC  the ownerpassword needed for opening a \PDF\ file \NC \NR

+\NC \type{keepopen}       \NC boolean  \NC  keep the \PDF\ file open \NC \NR

+\NC \type{nobbox}         \NC boolean  \NC  don't add a boundingbox specification for streams \NC \NR

+\NC \type{nolength}       \NC boolean  \NC  don't add length key nor compress for streams \NC \NR

+\NC \type{nosize}         \NC boolean  \NC  don't add size fields for streams \NC \NR

+\LL

+\stoptabulate

+

+A running (undefined) dimension in \type {width}, \type {height}, or \type

+{depth} is represented as \type {nil} in \LUA, so if you want to load an image at

+its \quote {natural} size, you do not have to specify any of those three fields.

+

+The \type {stream} parameter allows to fabricate an \type {/XObject} \type

+{/Form} object from a string giving the stream contents, e.g., for a filled

+rectangle:

+

+\startfunctioncall

+a.stream = "0 0 20 10 re f"

+\stopfunctioncall

+

+When writing the image, an \type {/Xobject} \type {/Form} object is created, like

+with embedded \PDF\ file writing. The object is written out only once. The \type

+{stream} key requires that also the \type {bbox} table is given. The \type

+{stream} key conflicts with the \type {filename} key. The \type {transform} key

+works as usual also with \type {stream}.

+

+The \type {bbox} key needs a table with four boundingbox values, e.g.:

+

+\startfunctioncall

+a.bbox = { "30bp", 0, "225bp", "200bp" }

+\stopfunctioncall

+

+This replaces and overrules any given \type {pagebox} value; with given \type

+{bbox} the box dimensions coming with an embedded \PDF\ file are ignored. The

+\type {xsize} and \type {ysize} dimensions are set accordingly, when the image is

+scaled. The \type {bbox} parameter is ignored for non-\PDF\ images.

+

+The \type {transform} allows to mirror and rotate the image in steps of 90~deg.

+The default value~$0$ gives an unmirrored, unrotated image. Values $1-3$ give

+counterclockwise rotation by $90$, $180$, or $270$~degrees, whereas with values

+$4-7$ the image is first mirrored and then rotated counterclockwise by $90$,

+$180$, or $270$~degrees. The \type {transform} operation gives the same visual

+result as if you would externally preprocess the image by a graphics tool and

+then use it by \LUATEX. If a \PDF\ file to be embedded already contains a \type

+{/Rotate} specification, the rotation result is the combination of the \type

+{/Rotate} rotation followed by the \type {transform} operation.

+

+\subsection{\type {scan}}

+

+\libindex{scan}

+

+\startfunctioncall

+<image> var = img.scan(<image> var)

+<image> var = img.scan(<table> image_spec)

+\stopfunctioncall

+

+When you say \type {img.scan(a)} for a new image, the file is scanned, and

+variables such as \type {xsize}, \type {ysize}, image \type {type}, number of

+\type {pages}, and the resolution are extracted. Each of the \type {width}, \type

+{height}, \type {depth} fields are set up according to the image dimensions, if

+they were not given an explicit value already. An image file will never be

+scanned more than once for a given image variable. With all subsequent \type

+{img.scan(a)} calls only the dimensions are again set up (if they have been

+changed by the user in the meantime).

+

+For ease of use, you can do right-away a

+

+\starttyping

+<image> a = img.scan { filename = "foo.png" }

+\stoptyping

+

+without a prior \type {img.new}.

+

+Nothing is written yet at this point, so you can do \type {a=img.scan}, retrieve

+the available info like image width and height, and then throw away \type {a}

+again by saying \type {a=nil}. In that case no image object will be reserved in

+the PDF, and the used memory will be cleaned up automatically.

+

+\subsection{\type {copy}}

+

+\libindex{copy}

+

+\startfunctioncall

+<image> var = img.copy(<image> var)

+<image> var = img.copy(<table> image_spec)

+\stopfunctioncall

+

+If you say \type {a = b}, then both variables point to the same \type {<image>}

+object. if you want to write out an image with different sizes, you can do

+\type {b = img.copy(a)}.

+

+Afterwards, \type {a} and \type {b} still reference the same actual image

+dictionary, but the dimensions for \type {b} can now be changed from their

+initial values that were just copies from \type {a}.

+

+\subsection{\type {write}, \type {immediatewrite}, \type {immediatewriteobject}}

+

+\topicindex {images+injection}

+\topicindex {images+immediate}

+\topicindex {images+object}

+

+\libindex{write}

+\libindex{immediatewrite}

+\libindex{immediatewriteobject}

+

+\startfunctioncall

+<image> var = img.write(<image> var)

+<image> var = img.write(<table> image_spec)

+\stopfunctioncall

+

+By \type {img.write(a)} a \PDF\ object number is allocated, and a rule node of

+subtype \type {image} is generated and put into the output list. By this the

+image \type {a} is placed into the page stream, and the image file is written out

+into an image stream object after the shipping of the current page is finished.

+

+Again you can do a terse call like

+

+\starttyping

+img.write { filename = "foo.png" }

+\stoptyping

+

+The \type {<image>} variable is returned in case you want it for later

+processing. You can also write an object.

+

+By \type {img.immediatewrite(a)} a \PDF\ object number is allocated, and the

+image file for image \type {a} is written out immediately into the \PDF\ file as

+an image stream object (like with \prm {immediate}\orm {pdfximage}). The object

+number of the image stream dictionary is then available by the \type {objnum}

+key. No \type {pdf_refximage} whatsit node is generated. You will need an

+\type {img.write(a)} or \type {img.node(a)} call to let the image appear on the

+page, or reference it by another trick; else you will have a dangling image

+object in the \PDF\ file.

+

+\startfunctioncall

+<image> var = img.immediatewrite(<image> var)

+<image> var = img.immediatewrite(<table> image_spec)

+\stopfunctioncall

+

+Also here you can do a terse call like

+

+\starttyping

+a = img.immediatewrite { filename = "foo.png" }

+\stoptyping

+

+The \type {<image>} variable is returned and you will most likely need it.

+

+The next function is kind of special as it copies an object from a (\PDF) image

+file. This features is experimental and might disappear.

+

+\startfunctioncall

+<integer> objnum = img.immediatewriteobject(<image> var, <integer> objnum)

+<integer> objnum = img.immediatewriteobject(<table> image_spec, <integer> objnum)

+\stopfunctioncall

+

+\subsection{\type {node}}

+

+\libindex{node}

+

+\startfunctioncall

+<node> n = img.node(<image> var)

+<node> n = img.node(<table> image_spec)

+\stopfunctioncall

+

+This function allocates a \PDF\ object number and returns a whatsit node of

+subtype \type {pdf_refximage}, filled with the image parameters \type {width},

+\type {height}, \type {depth}, and \type {objnum}. Also here you can do a terse

+call like:

+

+\starttyping

+n = img.node ({ filename = "foo.png" })

+\stoptyping

+

+This example outputs an image:

+

+\starttyping

+node.write(img.node{filename="foo.png"})

+\stoptyping

+

+\subsection{\type {types}}

+

+\topicindex {images+types}

+

+\libindex{types}

+

+\startfunctioncall

+<table> types = img.types()

+\stopfunctioncall

+

+This function returns a list with the supported image file type names, currently

+these are \type {pdf}, \type {png}, \type {jpg}, \type {jp2} (JPEG~2000), and

+\type {jbig2}.

+

+\subsection{\type {boxes}}

+

+\libindex{boxes}

+

+\startfunctioncall

+<table> boxes = img.boxes()

+\stopfunctioncall

+

+This function returns a list with the supported \PDF\ page box names, currently

+these are \type {media}, \type {crop}, \type {bleed}, \type {trim}, and \type

+{art}, all in lowercase.

+

+The \PDF\ file is kept open after its properties are determined. After inclusion,

+which happens when the page that references the image is flushed, the file is

+closed. This means that when you have thousands of images on one page, your

+operating system might decide to abort the run. When you include more than one

+page from a \PDF\ file you can set the \type {keepopen} flag when you allocate an

+image object, or pass the \type {keepopen} directive when you refer to the image

+with \lpr {useimageresource}. This only makes sense when you embed many pages.

+An \prm {immediate} applied to \lpr {saveimageresource} will also force a

+close after inclusion.

+

+\starttyping

+\immediate\useimageresource{foo.pdf}%

+          \saveimageresource         \lastsavedimageresourceindex % closed

+          \useimageresource{foo.pdf}%

+          \saveimageresource         \lastsavedimageresourceindex % kept open

+          \useimageresource{foo.pdf}%

+          \saveimageresource keepopen\lastsavedimageresourceindex % kept open

+

+\directlua{img.write(img.scan{ file = "foo.pdf" })}                  % closed

+\directlua{img.write(img.scan{ file = "foo.pdf", keepopen = true })} % kept open

+\stoptyping

+

+\stopsection

+

+\startsection[title={The \type {mplib} library}][library=mplib]

+

+\topicindex {\METAPOST}

+\topicindex {\METAPOST+mplib}

+\topicindex {images+mplib}

+\topicindex {images+\METAPOST}

+

+\libindex{version}

+

+The \MP\ library interface registers itself in the table \type {mplib}. It is

+based on \MPLIB\ version \ctxlua {context(mplib.version())}.

+

+\subsection{\type {new}}

+

+\libindex{new}

+

+To create a new \METAPOST\ instance, call

+

+\startfunctioncall

+<mpinstance> mp = mplib.new({...})

+\stopfunctioncall

+

+This creates the \type {mp} instance object. The argument hash can have a number

+of different fields, as follows:

+

+\starttabulate[|l|l|pl|pl|]

+\DB name               \BC type     \BC description              \BC default           \NC \NR

+\TB

+\NC \type{error_line}  \NC number   \NC error line width         \NC 79                \NC \NR

+\NC \type{print_line}  \NC number   \NC line length in ps output \NC 100               \NC \NR

+\NC \type{random_seed} \NC number   \NC the initial random seed  \NC variable          \NC \NR

+\NC \type{math_mode}   \NC string   \NC the number system to use:

+                                        \type {scaled},

+                                        \type {double} or

+                                      % \type {binary} or

+                                        \type {decimal}          \NC \type {scaled}    \NC \NR

+\NC \type{interaction} \NC string   \NC the interaction mode:

+                                        \type {batch},

+                                        \type {nonstop},

+                                        \type {scroll} or

+                                        \type {errorstop}        \NC \type {errorstop} \NC \NR

+\NC \type{job_name}    \NC string   \NC \type {--jobname}        \NC \type {mpout}     \NC \NR

+\NC \type{find_file}   \NC function \NC a function to find files \NC only local files  \NC \NR

+\LL

+\stoptabulate

+

+The binary mode is no longer available in the \LUATEX\ version of \MPLIB. It

+offers no real advantage and brings a ton of extra libraries with platform

+specific properties that we can now avoid. We might introduce a high resolution

+scaled variant at some point but only when it pays of performance wise.

+

+The \type {find_file} function should be of this form:

+

+\starttyping

+<string> found = finder (<string> name, <string> mode, <string> type)

+\stoptyping

+

+with:

+

+\starttabulate[|l|p|]

+\DB name        \BC the requested file \NC \NR

+\TB

+\NC \type{mode} \NC the file mode: \type {r} or \type {w} \NC \NR

+\NC \type{type} \NC the kind of file, one of: \type {mp}, \type {tfm}, \type {map},

+                    \type {pfb}, \type {enc} \NC \NR

+\LL

+\stoptabulate

+

+Return either the full path name of the found file, or \type {nil} if the file

+cannot be found.

+

+Note that the new version of \MPLIB\ no longer uses binary mem files, so the way

+to preload a set of macros is simply to start off with an \type {input} command

+in the first \type {execute} call.

+

+When you are processing a snippet of text starting with \type {btex} and

+ending with either \type {etex} or \type {verbatimtex}, the \METAPOST\

+\type {texscriptmode} parameter controls how spaces and newlines get honoured.

+The default value is~1. Possible values are:

+

+\starttabulate[|l|p|]

+\DB name      \BC meaning \NC \NR

+\TB

+\NC \type {0} \NC no newlines \NC \NR

+\NC \type {1} \NC newlines in \type {verbatimtex} \NC \NR

+\NC \type {2} \NC newlines in \type {verbatimtex} and \type {etex} \NC \NR

+\NC \type {3} \NC no leading and trailing strip in \type {verbatimtex} \NC \NR

+\NC \type {4} \NC no leading and trailing strip in \type {verbatimtex} and \type {btex} \NC \NR

+\LL

+\stoptabulate

+

+That way the \LUA\ handler (assigned to \type {make_text}) can do what it likes.

+An \type {etex} has to be followed by a space or \type {;} or be at the end of a

+line and preceded by a space or at the beginning of a line.

+

+\subsection{\type {statistics}}

+

+\libindex{statistics}

+

+You can request statistics with:

+

+\startfunctioncall

+<table> stats = mp:statistics()

+\stopfunctioncall

+

+This function returns the vital statistics for an \MPLIB\ instance. There are

+four fields, giving the maximum number of used items in each of four allocated

+object classes:

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{main_memory} \NC number \NC memory size \NC \NR

+\NC \type{hash_size}   \NC number \NC hash size\NC \NR

+\NC \type{param_size}  \NC number \NC simultaneous macro parameters\NC \NR

+\NC \type{max_in_open} \NC number \NC input file nesting levels\NC \NR

+\LL

+\stoptabulate

+

+Note that in the new version of \MPLIB, this is informational only. The objects

+are all allocated dynamically, so there is no chance of running out of space

+unless the available system memory is exhausted.

+

+\subsection{\type {execute}}

+

+\libindex{execute}

+

+You can ask the \METAPOST\ interpreter to run a chunk of code by calling

+

+\startfunctioncall

+<table> rettable = execute(mp,"metapost code")

+\stopfunctioncall

+

+for various bits of \METAPOST\ language input. Be sure to check the \type

+{rettable.status} (see below) because when a fatal \METAPOST\ error occurs the

+\MPLIB\ instance will become unusable thereafter.

+

+Generally speaking, it is best to keep your chunks small, but beware that all

+chunks have to obey proper syntax, like each of them is a small file. For

+instance, you cannot split a single statement over multiple chunks.

+

+In contrast with the normal stand alone \type {mpost} command, there is

+\notabene {no} implied \quote{input} at the start of the first chunk.

+

+\subsection{\type {finish}}

+

+\libindex{finish}

+

+\startfunctioncall

+<table> rettable = finish(mp)

+\stopfunctioncall

+

+If for some reason you want to stop using an \MPLIB\ instance while processing is

+not yet actually done, you can call \type finish}. Eventually, used memory

+will be freed and open files will be closed by the \LUA\ garbage collector, but

+an explicit \type finish} is the only way to capture the final part of the

+output streams.

+

+\subsection{Result table}

+

+\libindex {fields}

+

+The return value of \type {execute} and \type {finish} is a table with a

+few possible keys (only \type {status} is always guaranteed to be present).

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{log}    \NC string \NC output to the \quote {log} stream \NC \NR

+\NC \type{term}   \NC string \NC output to the \quote {term} stream \NC \NR

+\NC \type{error}  \NC string \NC output to the \quote {error} stream

+                                 (only used for \quote {out of memory}) \NC \NR

+\NC \type{status} \NC number \NC the return value:

+                                 \type {0} = good,

+                                 \type {1} = warning,

+                                 \type {2} = errors,

+                                 \type {3} = fatal error \NC \NR

+\NC \type{fig}    \NC table  \NC an array of generated figures (if any) \NC \NR

+\LL

+\stoptabulate

+

+When \type {status} equals~3, you should stop using this \MPLIB\ instance

+immediately, it is no longer capable of processing input.

+

+If it is present, each of the entries in the \type {fig} array is a userdata

+representing a figure object, and each of those has a number of object methods

+you can call:

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{boundingbox}  \NC function \NC returns the bounding box, as an array of 4

+                                         values \NC \NR

+\NC \type{postscript}   \NC function \NC returns a string that is the ps output of the

+                                         \type {fig}. this function accepts two optional

+                                         integer arguments for specifying the values of

+                                         \type {prologues} (first argument) and \type

+                                         {procset} (second argument) \NC \NR

+\NC \type{svg}          \NC function \NC returns a string that is the svg output of the

+                                         \type {fig}. This function accepts an optional

+                                         integer argument for specifying the value of

+                                         \type {prologues} \NC \NR

+\NC \type{objects}      \NC function \NC returns the actual array of graphic objects in

+                                         this \type {fig} \NC \NR

+\NC \type{copy_objects} \NC function \NC returns a deep copy of the array of graphic

+                                         objects in this \type {fig} \NC \NR

+\NC \type{filename}     \NC function \NC the filename this \type {fig}'s \POSTSCRIPT\

+                                         output would have written to in stand alone

+                                         mode \NC \NR

+\NC \type{width}        \NC function \NC the \type {fontcharwd} value \NC \NR

+\NC \type{height}       \NC function \NC the \type {fontcharht} value \NC \NR

+\NC \type{depth}        \NC function \NC the \type {fontchardp} value \NC \NR

+\NC \type{italcorr}     \NC function \NC the \type {fontcharit} value \NC \NR

+\NC \type{charcode}     \NC function \NC the (rounded) \type {charcode} value \NC \NR

+\LL

+\stoptabulate

+

+Note: you can call \type {fig:objects()} only once for any one \type {fig}

+object!

+

+When the boundingbox represents a \quote {negated rectangle}, i.e.\ when the

+first set of coordinates is larger than the second set, the picture is empty.

+

+Graphical objects come in various types that each has a different list of

+accessible values. The types are: \type {fill}, \type {outline}, \type {text},

+\type {start_clip}, \type {stop_clip}, \type {start_bounds}, \type {stop_bounds},

+\type {special}.

+

+There is a helper function (\type {mplib.fields(obj)}) to get the list of

+accessible values for a particular object, but you can just as easily use the

+tables given below.

+

+All graphical objects have a field \type {type} that gives the object type as a

+string value; it is not explicit mentioned in the following tables. In the

+following, \type {number}s are \POSTSCRIPT\ points represented as a floating

+point number, unless stated otherwise. Field values that are of type \type

+{table} are explained in the next section.

+

+\subsubsection{fill}

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{path}       \NC table  \NC the list of knots \NC \NR

+\NC \type{htap}       \NC table  \NC the list of knots for the reversed trajectory \NC \NR

+\NC \type{pen}        \NC table  \NC knots of the pen \NC \NR

+\NC \type{color}      \NC table  \NC the object's color \NC \NR

+\NC \type{linejoin}   \NC number \NC line join style (bare number)\NC \NR

+\NC \type{miterlimit} \NC number \NC miterlimit\NC \NR

+\NC \type{prescript}  \NC string \NC the prescript text \NC \NR

+\NC \type{postscript} \NC string \NC the postscript text \NC \NR

+\LL

+\stoptabulate

+

+The entries \type {htap} and \type {pen} are optional.

+

+\subsubsection{outline}

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{path}       \NC table  \NC the list of knots \NC \NR

+\NC \type{pen}        \NC table  \NC knots of the pen \NC \NR

+\NC \type{color}      \NC table  \NC the object's color \NC \NR

+\NC \type{linejoin}   \NC number \NC line join style (bare number) \NC \NR

+\NC \type{miterlimit} \NC number \NC miterlimit \NC \NR

+\NC \type{linecap}    \NC number \NC line cap style (bare number) \NC \NR

+\NC \type{dash}       \NC table  \NC representation of a dash list \NC \NR

+\NC \type{prescript}  \NC string \NC the prescript text \NC \NR

+\NC \type{postscript} \NC string \NC the postscript text \NC \NR

+\LL

+\stoptabulate

+

+The entry \type {dash} is optional.

+

+\subsubsection{text}

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{text}       \NC string \NC the text \NC \NR

+\NC \type{font}       \NC string \NC font tfm name \NC \NR

+\NC \type{dsize}      \NC number \NC font size \NC \NR

+\NC \type{color}      \NC table  \NC the object's color \NC \NR

+\NC \type{width}      \NC number \NC \NC \NR

+\NC \type{height}     \NC number \NC \NC \NR

+\NC \type{depth}      \NC number \NC \NC \NR

+\NC \type{transform}  \NC table  \NC a text transformation \NC \NR

+\NC \type{prescript}  \NC string \NC the prescript text \NC \NR

+\NC \type{postscript} \NC string \NC the postscript text \NC \NR

+\LL

+\stoptabulate

+

+\subsubsection{special}

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{prescript} \NC string \NC special text \NC \NR

+\LL

+\stoptabulate

+

+\subsubsection{start_bounds, start_clip}

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{path} \NC table \NC the list of knots \NC \NR

+\LL

+\stoptabulate

+

+\subsubsection{stop_bounds, stop_clip}

+

+Here are no fields available.

+

+\subsection{Subsidiary table formats}

+

+\subsubsection{Paths and pens}

+

+Paths and pens (that are really just a special type of paths as far as \MPLIB\ is

+concerned) are represented by an array where each entry is a table that

+represents a knot.

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{left_type}  \NC string \NC when present: endpoint, but usually absent \NC \NR

+\NC \type{right_type} \NC string \NC like \type {left_type} \NC \NR

+\NC \type{x_coord}    \NC number \NC X coordinate of this knot \NC \NR

+\NC \type{y_coord}    \NC number \NC Y coordinate of this knot \NC \NR

+\NC \type{left_x}     \NC number \NC X coordinate of the precontrol point of this knot \NC \NR

+\NC \type{left_y}     \NC number \NC Y coordinate of the precontrol point of this knot \NC \NR

+\NC \type{right_x}    \NC number \NC X coordinate of the postcontrol point of this knot \NC \NR

+\NC \type{right_y}    \NC number \NC Y coordinate of the postcontrol point of this knot \NC \NR

+\LL

+\stoptabulate

+

+There is one special case: pens that are (possibly transformed) ellipses have an

+extra string-valued key \type {type} with value \type {elliptical} besides the

+array part containing the knot list.

+

+\subsubsection{Colors}

+

+A color is an integer array with 0, 1, 3 or 4 values:

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{0} \NC marking only \NC no values                                                     \NC \NR

+\NC \type{1} \NC greyscale    \NC one value in the range $(0,1)$, \quote {black} is $0$         \NC \NR

+\NC \type{3} \NC \RGB         \NC three values in the range $(0,1)$, \quote {black} is $0,0,0$  \NC \NR

+\NC \type{4} \NC \CMYK        \NC four values in the range $(0,1)$, \quote {black} is $0,0,0,1$ \NC \NR

+\LL

+\stoptabulate

+

+If the color model of the internal object was \type {uninitialized}, then it was

+initialized to the values representing \quote {black} in the colorspace \type

+{defaultcolormodel} that was in effect at the time of the \type {shipout}.

+

+\subsubsection{Transforms}

+

+Each transform is a six|-|item array.

+

+\starttabulate[|l|l|p|]

+\DB index  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{1} \NC number \NC represents x  \NC \NR

+\NC \type{2} \NC number \NC represents y  \NC \NR

+\NC \type{3} \NC number \NC represents xx \NC \NR

+\NC \type{4} \NC number \NC represents yx \NC \NR

+\NC \type{5} \NC number \NC represents xy \NC \NR

+\NC \type{6} \NC number \NC represents yy \NC \NR

+\LL

+\stoptabulate

+

+Note that the translation (index 1 and 2) comes first. This differs from the

+ordering in \POSTSCRIPT, where the translation comes last.

+

+\subsubsection{Dashes}

+

+Each \type {dash} is two-item hash, using the same model as \POSTSCRIPT\ for the

+representation of the dashlist. \type {dashes} is an array of \quote {on} and

+\quote {off}, values, and \type {offset} is the phase of the pattern.

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{dashes} \NC hash   \NC an array of on-off numbers \NC \NR

+\NC \type{offset} \NC number \NC the starting offset value  \NC \NR

+\LL

+\stoptabulate

+

+\subsection{Pens and \type {pen_info}}

+

+\libindex{pen_info}

+

+There is helper function (\type {pen_info(obj)}) that returns a table containing

+a bunch of vital characteristics of the used pen (all values are floats):

+

+\starttabulate[|l|l|p|]

+\DB field  \BC type \BC explanation \NC \NR

+\TB

+\NC \type{width} \NC number \NC width of the pen \NC \NR

+\NC \type{sx}    \NC number \NC $x$ scale        \NC \NR

+\NC \type{rx}    \NC number \NC $xy$ multiplier  \NC \NR

+\NC \type{ry}    \NC number \NC $yx$ multiplier  \NC \NR

+\NC \type{sy}    \NC number \NC $y$ scale        \NC \NR

+\NC \type{tx}    \NC number \NC $x$ offset       \NC \NR

+\NC \type{ty}    \NC number \NC $y$ offset       \NC \NR

+\LL

+\stoptabulate

+

+\subsection{Character size information}

+

+These functions find the size of a glyph in a defined font. The \type {fontname}

+is the same name as the argument to \type {infont}; the \type {char} is a glyph

+id in the range 0 to 255; the returned \type {w} is in AFM units.

+

+\subsubsection{\type {char_width}}

+

+\libindex{char_width}

+

+\startfunctioncall

+<number> w = char_width(mp,<string> fontname, <number> char)

+\stopfunctioncall

+

+\subsubsection{\type {char_height}}

+

+\libindex{char_height}

+

+\startfunctioncall

+<number> w = char_height(mp,<string> fontname, <number> char)

+\stopfunctioncall

+

+\subsubsection{\type {char_depth}}

+

+\libindex{char_depth}

+

+\startfunctioncall

+<number> w = char_depth(mp,<string> fontname, <number> char)

+\stopfunctioncall

+

+\subsubsection{\type {get_[boolean|numeric|string|path]}}

+

+\libindex{get_boolean}

+\libindex{get_numeric}

+\libindex{get_path}

+\libindex{get_string}

+

+When a script call brings you from the \METAPOST\ run (temporarily) back to

+\LUA\ you can access variables, but only if they are known (so for instance

+anonymous capsules like loop variables are not accessible).

+

+\startfunctioncall

+<boolean> w = get_boolean(mp,<string> name)

+<number>  n = get_numeric(mp,<string> name)

+<string>  s = get_string (mp,<string> name)

+<table>   p = get_path   (mp,<string> name)

+\stopfunctioncall

+

+The path is returned a a table with subtables that have six numbers: the

+coordinates of the point, pre- and postcontrol. A \type {cycle} fields indicates

+if a path is cyclic.

+

+\stopsection

+

+\stopchapter

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-introduction.tex b/Master/texmf-dist/doc/luatex/base/luatex-introduction.tex
index d0899147da2..0930c532bdf 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-introduction.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-introduction.tex
@@ -1,130 +1,174 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-introduction
-
-\startchapter[title=Introduction]
-
-This is the reference manual of \LUATEX. We don't claim it is complete and we
-assume that the reader knows about \TEX\ as described in \quotation {The \TEX\
-Book}, the \quotation {\ETEX\ manual}, the \quotation {\PDFTEX\ manual}, etc.
-Additional reference material is published in journals of user groups and
-\CONTEXT\ related documentation.
-
-It took about a decade to reach stable version 1.0, but for good reason.
-Successive versions brought new functionality, more control, some cleanup of
-internals and experimental features evolved into stable ones or were dropped.
-Already quite early \LUATEX\ could be used for production and it was used on a
-daily basis by the authors. Successive versions sometimes demanded a adaption to
-the \LUA\ interfacing, but the concepts were unchanged. The current version can
-be considered stable in functionality and there will be no fundamental changes.
-Of course we then can decide to move towards version 2.00 with different
-properties.
-
-Don't expect \LUATEX\ to behave the same as \PDFTEX ! Although the core
-functionality of that 8 bit engine was starting point, it has been combined with
-the directional support of \OMEGA\ (\ALEPH). But, \LUATEX\ can behave different
-due to its wide (32 bit) characters, many registers and large memory support.
-There is native \UTF\ input, support for large (more that 8 bit) fonts, and the
-math machinery is tuned for \OPENTYPE\ math. There is support for directional
-typesetting too. The log output can differ from other engines and will likely
-differ more as we move forward. When you run plain \TEX\ for sure \LUATEX\ runs
-slower than \PDFTEX\ but when you run for instance \CONTEXT\ \MKIV\ in many cases
-it runs faster, especially when you have a bit more complex documents or input.
-Anyway, 32 bit all||over combined with more features has a price, but on a modern
-machine this is no real problem.
-
-Testing is done with \CONTEXT, but \LUATEX\ should work fine with other macro
-packages too. For that purpose we provide generic font handlers that are mostly
-the same as used in \CONTEXT. Discussing specific implementations is beyond this
-manual. Even when we keep \LUATEX\ lean and mean, we already have enough to
-discuss here.
-
-\LUATEX\ consists of a number of interrelated but (still) distinguishable parts.
-The organization of the source code is adapted so that it can glue all these
-components together. We continue cleaning up side effects of the accumulated
-code in \TEX\ engines (especially code that is not needed any longer).
-
-\startitemize[packed]
-    \startitem
-        Most of \PDFTEX\ version 1.40.9, converted to \CCODE. Some experimental
-        features have been removed and some utility macros are not inherited as
-        their functionality can be done in \LUA. The number of backend interface
-        commands has been reduced to a few. The extensions are separated from the
-        core (which we keep close to the original \TEX\ core). Some mechanisms
-        like expansion and protrusion can behave different from the original due
-        to some cleanup and optimization. Some whatsit based functionality (image
-        support and reusable content) is now core functionality.
-    \stopitem
-    \startitem
-        The direction model and some other bits from \ALEPH\ RC4 (derived from
-        \OMEGA) is included. The related primitives are part of core \LUATEX\ but
-        at the node level directional support is no longer based on so called
-        whatsits but on real nodes. In fact, whatsits are now only used for
-        backend specific extensions.
-    \stopitem
-    \startitem
-        Neither \ALEPH's I/O translation processes, nor tcx files, nor \ENCTEX\
-        can be used, these encoding|-|related functions are superseded by a
-        \LUA|-|based solution (reader callbacks). In a similar fashion all file
-        \IO\ can be intercepted.
-    \stopitem
-    \startitem
-        We currently use \LUA\ 5.3.*. There are few \LUA\ libraries that we
-        consider part of the core \LUA\ machinery, for instance \type {lpeg}.
-        There are additional \LUA\ libraries that interface to the internals of
-        \TEX. We also keep the \LUA\ 5.2 \type {bit32} library around.
-    \stopitem
-    \startitem
-        There are various \TEX\ extensions but only those that cannot be done
-        using the \LUA\ interfaces. The math machinery often has two code paths:
-        one traditional and the other more suitable for wide \OPENTYPE\ fonts.
-    \stopitem
-    \startitem
-        The fontloader uses parts of \FONTFORGE\ 2008.11.17 combined with
-        additional code specific for usage in a \TEX\ engine. We try to minimize
-        specific font support to what \TEX\ needs: character references and
-        dimensions and delegate everything else to \LUA. That way we keep \TEX\
-        open for extensions without touching the core.
-    \stopitem
-    \startitem
-        The \METAPOST\ library is integral part of \LUATEX. This gives \TEX\ some
-        graphical capabilities using a relative high speed graphical subsystem.
-        Again \LUA\ is used as glue between the frontend and backend. Further
-        development of \METAPOST\ is closely related to \LUATEX.
-    \stopitem
-\stopitemize
-
-We try to keep upcoming versions compatible but intermediate releases can contain
-experimental features. A general rule is that versions that end up on \TEX live
-and|/|or are released around \CONTEXT\ meetings are stable. Future versions will
-probably become a bit leaner and meaner. Some libraries might become external as
-we don't want to bloat the binary and also don't want to add more hard coded
-solutions. After all, with \LUA\ you can extend the core functionality. The less
-dependencies, the better.
-
-The \TEXLIVE\ version is to be considered the current stable version. Any version
-between the yearly \TEXLIVE\ releases are to be considered beta. The beta
-releases are normally available via the \CONTEXT\ distribution channels (the
-garden and so called minimals).
-
-\blank[1*big]
-
-Hans Hagen, Harmut Henkel, \crlf
-Taco Hoekwater \& Luigi Scarso
-
-\blank[3*big]
-
-\starttabulate
-\NC Version  \EQ \currentdate \NC \NR
-\NC \LUATEX  \EQ version  \cldcontext{status.luatex_version/100},
-                 revision \cldcontext{status.luatex_revision},
-                 number   \cldcontext{environment.luatexversion} \NC \NR
-\NC \CONTEXT \EQ MkIV \contextversion \NC \NR
-\stoptabulate
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-introduction

+

+\startchapter[title=Introduction]

+

+This is the reference manual of \LUATEX. We don't claim it is complete and we

+assume that the reader knows about \TEX\ as described in \quotation {The \TEX\

+Book}, the \quotation {\ETEX\ manual}, the \quotation {\PDFTEX\ manual}, etc.

+Additional reference material is published in journals of user groups and

+\CONTEXT\ related documentation.

+

+It took about a decade to reach stable version 1.0, but for good reason.

+Successive versions brought new functionality, more control, some cleanup of

+internals. Experimental features evolved into stable ones or were dropped.

+Already quite early \LUATEX\ could be used for production and it was used on a

+daily basis by the authors. Successive versions sometimes demanded an adaption to

+the \LUA\ interfacing, but the concepts were unchanged. The current version can

+be considered stable in functionality and there will be no fundamental changes.

+Of course we then can decide to move towards version 2.00 with different

+properties.

+

+Don't expect \LUATEX\ to behave the same as \PDFTEX ! Although the core

+functionality of that 8 bit engine was starting point, it has been combined with

+the directional support of \OMEGA\ (\ALEPH). But, \LUATEX\ can behave different

+due to its wide (32 bit) characters, many registers and large memory support. The

+\PDF\ code produced differs from \PDFTEX\ but users will normally not notice

+that. There is native \UTF\ input, support for large (more than 8 bit) fonts, and

+the math machinery is tuned for \OPENTYPE\ math. There is support for directional

+typesetting too. The log output can differ from other engines and will likely

+differ more as we move forward. When you run plain \TEX\ for sure \LUATEX\ runs

+slower than \PDFTEX\ but when you run for instance \CONTEXT\ \MKIV\ in many cases

+it runs faster, especially when you have a bit more complex documents or input.

+Anyway, 32 bit all||over combined with more features has a price, but on a modern

+machine this is no real problem.

+

+Testing is done with \CONTEXT, but \LUATEX\ should work fine with other macro

+packages too. For that purpose we provide generic font handlers that are mostly

+the same as used in \CONTEXT. Discussing specific implementations is beyond this

+manual. Even when we keep \LUATEX\ lean and mean, we already have enough to

+discuss here.

+

+\LUATEX\ consists of a number of interrelated but (still) distinguishable parts.

+The organization of the source code is adapted so that it can glue all these

+components together. We continue cleaning up side effects of the accumulated

+code in \TEX\ engines (especially code that is not needed any longer).

+

+\startitemize [unpacked]

+    \startitem

+        We started out with most of \PDFTEX\ version 1.40.9. The code base was

+        converted to \CCODE\ and split in modules. Experimental features were

+        removed and utility macros are not inherited because their functionality

+        can be programmed in \LUA. The number of backend interface commands has

+        been reduced to a few. The so called extensions are separated from the

+        core (which we try to keep close to the original \TEX\ core). Some

+        mechanisms like expansion and protrusion can behave different from the

+        original due to some cleanup and optimization. Some whatsit based

+        functionality (image support and reusable content) is now core

+        functionality. We don't stay in sync with \PDFTEX\ development.

+    \stopitem

+    \startitem

+        The direction model from \ALEPH\ RC4 (which is derived from \OMEGA) is

+        included. The related primitives are part of core \LUATEX\ but at the

+        node level directional support is no longer based on so called whatsits

+        but on real nodes with relevant properties. The number of directions is

+        limited to the useful set and the backend has been made direction aware.

+    \stopitem

+    \startitem

+        Neither \ALEPH's I/O translation processes, nor tcx files, nor \ENCTEX\

+        are available. These encoding|-|related functions are superseded by a

+        \LUA|-|based solution (reader callbacks). In a similar fashion all file

+        \IO\ can be intercepted.

+    \stopitem

+    \startitem

+        We currently use \LUA\ 5.3.*. There are few \LUA\ libraries that we

+        consider part of the core \LUA\ machinery, for instance \type {lpeg}.

+        There are additional \LUA\ libraries that interface to the internals of

+        \TEX. We also keep the \LUA\ 5.2 \type {bit32} library around.

+    \stopitem

+    \startitem

+        There are various \TEX\ extensions but only those that cannot be done

+        using the \LUA\ interfaces. The math machinery often has two code paths:

+        one traditional and the other more suitable for wide \OPENTYPE\ fonts. Here

+        we follow the \MICROSOFT\ specifications as much as possible.  Some math

+        functionality has been opened up a bit so that users have more control.

+    \stopitem

+    \startitem

+        The fontloader uses parts of \FONTFORGE\ 2008.11.17 combined with

+        additional code specific for usage in a \TEX\ engine. We try to minimize

+        specific font support to what \TEX\ needs: character references and

+        dimensions and delegate everything else to \LUA. That way we keep \TEX\

+        open for extensions without touching the core. In order to minimize

+        dependencies at some point we may decide to make this an optional

+        library.

+    \stopitem

+    \startitem

+        The \METAPOST\ library is integral part of \LUATEX. This gives \TEX\ some

+        graphical capabilities using a relative high speed graphical subsystem.

+        Again \LUA\ is used as glue between the frontend and backend. Further

+        development of \METAPOST\ is closely related to \LUATEX.

+    \stopitem

+    \startitem

+        The virtual font technology that comes with \TEX\ has been integrated

+        into the font machinery in a way that permits creating virtual fonts

+        at runtime. Because \LUATEX\ can also act as a \LUA\ interpreter this

+        means that a complete \TEX\ workflow can be built without the need for

+        additional programs.

+    \stopitem

+    \startitem

+        The versions starting from 1.09 no longer use the poppler library for

+        inclusion but a lightweight dedicated one. This removes a dependency but

+        also makes the inclusion code of \LUATEX\ different from \PDFTEX. In fact

+        it was already much different due to the \LUA\ image interfacing.

+    \stopitem

+\stopitemize

+

+We try to keep upcoming versions compatible but intermediate releases can contain

+experimental features. A general rule is that versions that end up on \TEXLIVE\

+and|/|or are released around \CONTEXT\ meetings are stable. Any version between

+the yearly \TEXLIVE\ releases are to be considered beta and in the repository end

+up as trunk releases. We have an experimental branch that we use for development

+but there is no support for any of its experimental features. Intermediate

+releases (from trunk) are normally available via the \CONTEXT\ distribution

+channels (the garden and so called minimals).

+

+Version 1.10 is more or less an endpoint in development: this is what you get.

+Because not only \CONTEXT, that we can adapt rather easily, uses \LUATEX, we

+cannot change fundamentals without unforeseen consequences. By now it has been

+proven that \LUA\ can be used to extend the core functionality so there is no

+need to add more, and definitely no hard coded solutions for (not so) common

+problems. Of course there will be bug fixes, maybe some optimization, and there

+might even be some additions or non|-|intrusive improvements, but only after

+testing outside the stable release. After all, the binary is already more than

+large enough and there is not that much to gain.

+

+You might find \LUA\ helpers that are not yet documented. These are considered

+experimental, although when you encounter them in a \CONTEXT\ version that has

+been around for a while you can assume that they will stay. Of course it can just

+be that we forgot to document them yet.

+

+A manual like this is not really meant as tutorial, for that we refer to

+documents that ship with \CONTEXT, articles, etc. It is also never complete

+enough for all readers. We try to keep up but the reader needs to realize that

+it's all volunteer work done in spare time. And for sure, complaining about a bad

+manual or crappy documentation will not really motivate us to spend more time on

+it. That being said, we hope that this document is useful.

+

+\blank[big]

+

+\testpage[5]

+

+\startlines

+Hans Hagen

+Harmut Henkel

+Taco Hoekwater

+Luigi Scarso

+\stoplines

+

+\blank[3*big]

+

+\starttabulate[|||]

+\NC Version  \EQ \currentdate \NC \NR

+\NC \LUATEX  \EQ \cldcontext{LUATEXENGINE} %

+                 \cldcontext{LUATEXVERSION} / %

+                 \cldcontext{LUATEXFUNCTIONALITY}

+                 \NC \NR

+\NC \CONTEXT \EQ MkIV \contextversion \NC \NR

+\stoptabulate

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-languages.tex b/Master/texmf-dist/doc/luatex/base/luatex-languages.tex
index 365e87f266c..98960aa65d4 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-languages.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-languages.tex
@@ -1,966 +1,1113 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-languages
-
-\startchapter[reference=languages,title={Languages, characters, fonts and glyphs}]
-
-\LUATEX's internal handling of the characters and glyphs that eventually become
-typeset is quite different from the way \TEX82 handles those same objects. The
-easiest way to explain the difference is to focus on unrestricted horizontal mode
-(i.e.\ paragraphs) and hyphenation first. Later on, it will be easy to deal
-with the differences that occur in horizontal and math modes.
-
-In \TEX82, the characters you type are converted into \type {char_node} records
-when they are encountered by the main control loop. \TEX\ attaches and processes
-the font information while creating those records, so that the resulting \quote
-{horizontal list} contains the final forms of ligatures and implicit kerning.
-This packaging is needed because we may want to get the effective width of for
-instance a horizontal box.
-
-When it becomes necessary to hyphenate words in a paragraph, \TEX\ converts (one
-word at time) the \type {char_node} records into a string by replacing ligatures
-with their components and ignoring the kerning. Then it runs the hyphenation
-algorithm on this string, and converts the hyphenated result back into a \quote
-{horizontal list} that is consecutively spliced back into the paragraph stream.
-Keep in mind that the paragraph may contain unboxed horizontal material, which
-then already contains ligatures and kerns and the words therein are part of the
-hyphenation process.
-
-Those \type {char_node} records are somewhat misnamed, as they are glyph
-positions in specific fonts, and therefore not really \quote {characters} in the
-linguistic sense. There is no language information inside the \type {char_node}
-records at all. Instead, language information is passed along using \type
-{language whatsit} records inside the horizontal list.
-
-In \LUATEX, the situation is quite different. The characters you type are always
-converted into \type {glyph_node} records with a special subtype to identify them
-as being intended as linguistic characters. \LUATEX\ stores the needed language
-information in those records, but does not do any font|-|related processing at
-the time of node creation. It only stores the index of the current font and a
-reference to a character in that font.
-
-When it becomes necessary to typeset a paragraph, \LUATEX\ first inserts all
-hyphenation points right into the whole node list. Next, it processes all the
-font information in the whole list (creating ligatures and adjusting kerning),
-and finally it adjusts all the subtype identifiers so that the records are \quote
-{glyph nodes} from now on.
-
-\section[charsandglyphs]{Characters and glyphs}
-
-\TEX82 (including \PDFTEX) differentiates between \type {char_node}s and \type
-{lig_node}s. The former are simple items that contained nothing but a \quote
-{character} and a \quote {font} field, and they lived in the same memory as
-tokens did. The latter also contained a list of components, and a subtype
-indicating whether this ligature was the result of a word boundary, and it was
-stored in the same place as other nodes like boxes and kerns and glues.
-
-In \LUATEX, these two types are merged into one, somewhat larger structure called
-a \type {glyph_node}. Besides having the old character, font, and component
-fields, and the new special fields like \quote {attr} (see~\in {section}
-[glyphnodes]), these nodes also contain:
-
-\startitemize
-
-\startitem A subtype, split into four main types:
-
-    \startitemize
-        \startitem
-            \type {character}, for characters to be hyphenated: the lowest bit
-            (bit 0) is set to 1.
-        \stopitem
-        \startitem
-            \type {glyph}, for specific font glyphs: the lowest bit (bit 0) is
-            not set.
-        \stopitem
-        \startitem
-            \type {ligature}, for ligatures (bit 1 is set)
-        \stopitem
-        \startitem
-            \type {ghost}, for \quote {ghost objects} (bit 2 is set)
-        \stopitem
-    \stopitemize
-
-    The latter two make further use of two extra fields (bits 3 and 4):
-
-    \startitemize
-        \startitem
-            \type {left}, for ligatures created from a left word boundary and for
-            ghosts created from \type {\leftghost}
-        \stopitem
-        \startitem
-            \type {right}, for ligatures created from a right word boundary and
-            for ghosts created from \type {\rightghost}
-        \stopitem
-   \stopitemize
-
-   For ligatures, both bits can be set at the same time (in case of a
-   single|-|glyph word).
-
-\stopitem
-
-\startitem
-    \type {glyph_node}s of type \quote {character} also contain language data,
-    split into four items that were current when the node was created: the
-    \type {\setlanguage} (15 bits), \type {\lefthyphenmin} (8 bits), \type
-    {\righthyphenmin} (8 bits), and \type {\uchyph} (1 bit).
-\stopitem
-
-\stopitemize
-
-Incidentally, \LUATEX\ allows 16383 separate languages, and words can be 256
-characters long. The language is stored with each character. You can set
-\type {\firstvalidlanguage} to for instance~1 and make thereby language~0
-an ignored hyphenation language.
-
-The new primitive \type {\hyphenationmin} can be used to signal the minimal length
-of a word. This value stored with the (current) language.
-
-Because the \type {\uchyph} value is saved in the actual nodes, its handling is
-subtly different from \TEX82: changes to \type {\uchyph} become effective
-immediately, not at the end of the current partial paragraph.
-
-Typeset boxes now always have their language information embedded in the nodes
-themselves, so there is no longer a possible dependency on the surrounding
-language settings. In \TEX82, a mid-paragraph statement like \type {\unhbox0} would
-process the box using the current paragraph language unless there was a
-\type {\setlanguage} issued inside the box. In \LUATEX, all language variables are
-already frozen.
-
-In traditional \TEX\ the process of hyphenation is driven by \type {lccode}s. In
-\LUATEX\ we made this dependency less strong. There are several strategies
-possible. When you do nothing, the currently used \type {lccode}s are used, when
-loading patterns, setting exceptions or hyphenating a list.
-
-When you set \type {\savinghyphcodes} to a value larger than zero the current set
-of \type {lccode}s will be saved with the language. In that case changing a \type
-{lccode} afterwards has no effect. However, you can adapt the set with:
-
-\starttyping
-\hjcode`a=`a
-\stoptyping
-
-This change is global which makes sense if you keep in mind that the moment that
-hyphenation happens is (normally) when the paragraph or a horizontal box is
-constructed. When \type {\savinghyphcodes} was zero when the language got
-initialized you start out with nothing, otherwise you already have a set.
-
-When a \type {\hjcode} is larger than $0$ but smaller than $32$ is indicates the
-to be used length. In the following example we map a character (\type {x}) onto
-another one in the patterns and tell the engine that \type {œ} counts as one
-character. Because traditionally zero itself is reserved for inhibiting
-hyphenation, a value of $32$ counts as zero.
-
-\starttyping
-% assuming french patterns:
-foobar % foo-bar
-
-\hjcode`x=`o
-
-fxxbar % fxx-bar
-
-\lefthyphenmin3
-
-œdipus % œdi-pus
-
-\lefthyphenmin4
-
-œdipus % œdipus
-
-\hjcode`œ=2
-
-œdipus % œdi-pus
-
-\hjcode`i=32
-\hjcode`d=32
-
-œdipus % œdipus
-\stoptyping
-
-Carrying all this information with each glyph would give too much overhead and
-also make the process of setting up thee codes more complex. A solution with
-\type {hjcode} sets was considered but rejected because in practice the current
-approach is sufficient and it would not be compatible anyway.
-
-Beware: the values are always saved in the format, independent of the setting
-of \type {\savinghyphcodes} at the moment the format is dumped.
-
-A boundary node normally would mark the end of a word which interferes with for
-instance discretionary injection. For this you can use the \type {\wordboundary}
-as trigger. Here are a few examples of usage:
-
-\startbuffer
-    discrete---discrete
-\stopbuffer
-\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop
-\startbuffer
-    discrete\discretionary{}{}{---}discrete
-\stopbuffer
-\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop
-\startbuffer
-    discrete\wordboundary\discretionary{}{}{---}discrete
-\stopbuffer
-\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop
-\startbuffer
-    discrete\wordboundary\discretionary{}{}{---}\wordboundary discrete
-\stopbuffer
-\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop
-\startbuffer
-    discrete\wordboundary\discretionary{---}{}{}\wordboundary discrete
-\stopbuffer
-\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop
-
-We only accept an explicit hyphen when there is a preceding glyph and we skip a
-sequence of explicit hyphens as that normally indicates a \type {--} or \type
-{---} ligature in which case we can in a worse case usage get bad node lists
-later on due to messed up ligature building as these dashes are ligatures in base
-fonts. This is a side effect of the separating the hyphenation, ligaturing and
-kerning steps.
-
-The start and end of a characters is signalled by a glue, penalty, kern or boundary
-node. But by default also a hlist, vlist, rule, dir, whatsit, ins, and adjust node
-indicate a start or end. You can omit the last set from the test by setting
-\type {\hyphenationbounds} to a non|-|zero value:
-
-\starttabulate[|l|l|]
-\NC \type{0} \NC not strict \NC \NR
-\NC \type{1} \NC strict start \NC \NR
-\NC \type{2} \NC strict end \NC \NR
-\NC \type{3} \NC strict start and strict end \NC \NR
-\stoptabulate
-
-The word start is determined as follows:
-
-\starttabulate[|l|l|]
-\BC boundary  \NC yes when wordboundary \NC \NR
-\BC hlist     \NC when hyphenationbounds 1 or 3 \NC \NR
-\BC vlist     \NC when hyphenationbounds 1 or 3 \NC \NR
-\BC rule      \NC when hyphenationbounds 1 or 3 \NC \NR
-\BC dir       \NC when hyphenationbounds 1 or 3 \NC \NR
-\BC whatsit   \NC when hyphenationbounds 1 or 3 \NC \NR
-\BC glue      \NC yes \NC \NR
-\BC math      \NC skipped \NC \NR
-\BC glyph     \NC exhyphenchar (one only) : yes (so no -- ---) \NC \NR
-\BC otherwise \NC yes \NC \NR
-\stoptabulate
-
-The word end is determined as follows:
-
-\starttabulate[|l|l|]
-\BC boundary  \NC yes \NC \NR
-\BC glyph     \NC yes when different language \NC \NR
-\BC glue      \NC yes \NC \NR
-\BC penalty   \NC yes \NC \NR
-\BC kern      \NC yes when not italic (for some historic reason) \NC \NR
-\BC hlist     \NC when hyphenationbounds 2 or 3 \NC \NR
-\BC vlist     \NC when hyphenationbounds 2 or 3 \NC \NR
-\BC rule      \NC when hyphenationbounds 2 or 3 \NC \NR
-\BC dir       \NC when hyphenationbounds 2 or 3 \NC \NR
-\BC whatsit   \NC when hyphenationbounds 2 or 3 \NC \NR
-\BC ins       \NC when hyphenationbounds 2 or 3 \NC \NR
-\BC adjust    \NC when hyphenationbounds 2 or 3 \NC \NR
-\stoptabulate
-
-\in{Figures}[hb:1] upto \in[hb:5] show some examples. In all cases we set the min
-values to 1 and make sure that the words hyphenate at each character.
-
-\hyphenation{o-n-e t-w-o}
-
-\def\SomeTest#1#2%
-  {\lefthyphenmin  \plusone
-   \righthyphenmin \plusone
-   \parindent      \zeropoint
-   \everypar       \emptytoks
-   \dontcomplain
-   \hbox to 2cm {%
-     \vtop {%
-       \hsize 1pt
-       \hyphenationbounds#1
-       #2
-       \par}}}
-
-\startplacefigure[reference=hb:1,title={\type{one}}]
-    \startcombination[4*1]
-        {\SomeTest{0}{one}}          {\type{0}}
-        {\SomeTest{1}{one}}          {\type{1}}
-        {\SomeTest{2}{one}}          {\type{2}}
-        {\SomeTest{3}{one}}          {\type{3}}
-    \stopcombination
-\stopplacefigure
-\startplacefigure[reference=hb:2,title={\type{one\null two}}]
-    \startcombination[4*1]
-        {\SomeTest{0}{one\null two}} {\type{0}}
-        {\SomeTest{1}{one\null two}} {\type{1}}
-        {\SomeTest{2}{one\null two}} {\type{2}}
-        {\SomeTest{3}{one\null two}} {\type{3}}
-    \stopcombination
-\stopplacefigure
-\startplacefigure[reference=hb:3,title={\type{\null one\null two}}]
-    \startcombination[4*1]
-        {\SomeTest{0}{\null one\null two}} {\type{0}}
-        {\SomeTest{1}{\null one\null two}} {\type{1}}
-        {\SomeTest{2}{\null one\null two}} {\type{2}}
-        {\SomeTest{3}{\null one\null two}} {\type{3}}
-    \stopcombination
-\stopplacefigure
-\startplacefigure[reference=hb:4,title={\type{one\null two\null}}]
-    \startcombination[4*1]
-        {\SomeTest{0}{one\null two\null}} {\type{0}}
-        {\SomeTest{1}{one\null two\null}} {\type{1}}
-        {\SomeTest{2}{one\null two\null}} {\type{2}}
-        {\SomeTest{3}{one\null two\null}} {\type{3}}
-    \stopcombination
-\stopplacefigure
-\startplacefigure[reference=hb:5,title={\type{\null one\null two\null}}]
-    \startcombination[4*1]
-        {\SomeTest{0}{\null one\null two\null}} {\type{0}}
-        {\SomeTest{1}{\null one\null two\null}} {\type{1}}
-        {\SomeTest{2}{\null one\null two\null}} {\type{2}}
-        {\SomeTest{3}{\null one\null two\null}} {\type{3}}
-    \stopcombination
-\stopplacefigure
-
-% (Future versions of \LUATEX\ might provide more granularity.)
-
-In traditional \TEX\ ligature building and hyphenation are interwoven with the
-line break mechanism. In \LUATEX\ these phases are isolated. As a consequence we
-deal differently with (a sequence of) explicit hyphens. We already have added
-some control over aspects of the hyphenation and yet another one concerns
-automatic hyphens (e.g.\ \type {-} characters in the input).
-
-When \type {\automatichyphenmode} has a value of 0, a hyphen will be turned into
-an automatic discretionary. The snippets before and after it will not be
-hyphenated. A side effect is that a leading hyphen can lead to a split but one
-will seldom run into that situation. Setting a pre and post character makes this
-more prominent. A value of 1 will prevent this side effect and a value of 2 will
-not turn the hyphen into a discretionary. Experiments with other options, like
-permitting hyphenation of the words on both sides were discarded.
-
-\startbuffer[a]
-before-after \par
-before--after \par
-before---after \par
-\stopbuffer
-
-\startbuffer[b]
--before \par
-after- \par
---before \par
-after-- \par
----before \par
-after--- \par
-\stopbuffer
-
-\startbuffer[c]
-before-after \par
-before--after \par
-before---after \par
-\stopbuffer
-
-We show three samples:
-
-Input A: \typebuffer[a]
-Input B: \typebuffer[b]
-Input C: \typebuffer[c]
-
-\startbuffer[demo]
-\startcombination[nx=4,ny=3,location=top]
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[a]}} {A~0~6em}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[a]}} {A~0~2pt}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone   \hsize2pt \getbuffer[a]}} {A~1~2pt}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo   \hsize2pt \getbuffer[a]}} {A~2~2pt}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[b]}} {B~0~6em}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[b]}} {B~0~2pt}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone   \hsize2pt \getbuffer[b]}} {B~1~2pt}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo   \hsize2pt \getbuffer[b]}} {B~2~2pt}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[c]}} {C~0~6em}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[c]}} {C~0~2pt}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone   \hsize2pt \getbuffer[c]}} {C~1~2pt}
-    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo   \hsize2pt \getbuffer[c]}} {C~2~2pt}
-\stopcombination
-\stopbuffer
-
-\startplacefigure[reference=automatic:1,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with a \type {\hsize}
-of 6em and 2pt (which triggers a linebreak).}]
-    \dontcomplain \tt \getbuffer[demo]
-\stopplacefigure
-
-\startplacefigure[reference=automatic:2,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with \type
-{\preexhyphenchar} and \type {\postexhyphenchar} set to characters \type {A} and \type {B}.}]
-    \postexhyphenchar`A\relax
-    \preexhyphenchar `B\relax
-    \dontcomplain \tt \getbuffer[demo]
-\stopplacefigure
-
-As with primitive companions of other single character commands, the \type {\-}
-command has a more verbose primitive version in \type {\explicitdiscretionary}
-and the normally intercepted in the hyphenator character \type {-} (or whatever
-is configured) is available as \type {\automaticdiscretionary}.
-
-\section{The main control loop}
-
-In \LUATEX's main loop, almost all input characters that are to be typeset are
-converted into \type {glyph} node records with subtype \quote {character}, but
-there are a few exceptions.
-
-First, the \type {\accent} primitives creates nodes with subtype \quote {glyph}
-instead of \quote {character}: one for the actual accent and one for the
-accentee. The primary reason for this is that \type {\accent} in \TEX82 is
-explicitly dependent on the current font encoding, so it would not make much
-sense to attach a new meaning to the primitive's name, as that would invalidate
-many old documents and macro packages. \footnote {Of course, modern packages will
-not use the \type {\accent} primitive at all but try to map directly on composed
-characters.} A secondary reason is that in \TEX82, \type {\accent} prohibits
-hyphenation of the current word. Since in \LUATEX\ hyphenation only takes place
-on \quote {character} nodes, it is possible to achieve the same effect.
-
-This change of meaning did happen with \type {\char}, that now generates \quote
-{glyph} nodes with a character subtype. In traditional \TEX\ there was a strong
-relationship between the 8|-|bit input encoding, hyphenation and glyphs taken
-from a font. In \LUATEX\ we have \UTF\ input, and in most cases this maps
-directly to a character in a font, apart from glyph replacement in the font
-engine. If you want to access arbitrary glyphs in a font directly you can always
-use \LUA\ to do so, because fonts are available as \LUA\ table.
-
-Second, all the results of processing in math mode eventually become nodes with
-\quote {glyph} subtypes.
-
-Third, the \ALEPH|-|derived commands \type {\leftghost} and \type {\rightghost}
-create nodes of a third subtype: \quote {ghost}. These nodes are ignored
-completely by all further processing until the stage where inter|-|glyph kerning
-is added.
-
-Fourth, automatic discretionaries are handled differently. \TEX82 inserts an
-empty discretionary after sensing an input character that matches the \type
-{\hyphenchar} in the current font. This test is wrong in our opinion: whether or
-not hyphenation takes place should not depend on the current font, it is a
-language property. \footnote {When \TEX\ showed up we didn't have \UNICODE\ yet
-and being limited to eight bits meant that one sometimes had to compromise
-between supporting character input, glyph rendering, hyphenation.}
-
-In \LUATEX, it works like this: if \LUATEX\ senses a string of input characters
-that matches the value of the new integer parameter \type {\exhyphenchar}, it will
-insert an explicit discretionary after that series of nodes. Initex sets the \type
-{\exhyphenchar=`\-}. Incidentally, this is a global parameter instead of a
-language-specific one because it may be useful to change the value depending on
-the document structure instead of the text language.
-
-The insertion of discretionaries after a sequence of explicit hyphens happens at
-the same time as the other hyphenation processing, {\it not\/} inside the main
-control loop.
-
-The only use \LUATEX\ has for \type {\hyphenchar} is at the check whether a word
-should be considered for hyphenation at all. If the \type {\hyphenchar} of the
-font attached to the first character node in a word is negative, then hyphenation
-of that word is abandoned immediately. This behaviour is added for backward
-compatibility only, and the use of \type {\hyphenchar=-1} as a means of
-preventing hyphenation should not be used in new \LUATEX\ documents.
-
-Fifth, \type {\setlanguage} no longer creates whatsits. The meaning of \type
-{\setlanguage} is changed so that it is now an integer parameter like all others.
-That integer parameter is used in \type {\glyph_node} creation to add language
-information to the glyph nodes. In conjunction, the \type {\language} primitive is
-extended so that it always also updates the value of \type {\setlanguage}.
-
-Sixth, the \type {\noboundary} command (that prohibits word boundary processing
-where that would normally take place) now does create nodes. These nodes are
-needed because the exact place of the \type {\noboundary} command in the input
-stream has to be retained until after the ligature and font processing stages.
-
-Finally, there is no longer a \type {main_loop} label in the code. Remember that
-\TEX82 did quite a lot of processing while adding \type {char_nodes} to the
-horizontal list? For speed reasons, it handled that processing code outside of
-the \quote {main control} loop, and only the first character of any \quote {word}
-was handled by that \quote {main control} loop. In \LUATEX, there is no longer a
-need for that (all hard work is done later), and the (now very small) bits of
-character|-|handling code have been moved back inline. When \type
-{\tracingcommands} is on, this is visible because the full word is reported,
-instead of just the initial character.
-
-Because we tend to make hard codes behaviour configurable a few new primitives
-have been added:
-
-\starttyping
-\hyphenpenaltymode
-\automatichyphenpenalty
-\explicithyphenpenalty
-\stoptyping
-
-The first parameter has the following consequences for automatic discs (the ones
-resulting from an \type {\exhyphenchar}:
-
-\starttabulate[|c|l|l|]
-\BC mode     \BC automatic disc \type{-}         \BC explicit disc \type{\-}         \NC \NR
-\HL
-\NC \type{0} \NC \type {\exhyphenpenalty}        \NC \type {\exhyphenpenalty}        \NC \NR
-\NC \type{1} \NC \type {\hyphenpenalty}          \NC \type {\hyphenpenalty}          \NC \NR
-\NC \type{2} \NC \type {\exhyphenpenalty}        \NC \type {\hyphenpenalty}          \NC \NR
-\NC \type{3} \NC \type {\hyphenpenalty}          \NC \type {\exhyphenpenalty}        \NC \NR
-\NC \type{4} \NC \type {\automatichyphenpenalty} \NC \type {\explicithyphenpenalty}  \NC \NR
-\NC \type{5} \NC \type {\exhyphenpenalty}        \NC \type {\explicithyphenpenalty}  \NC \NR
-\NC \type{6} \NC \type {\hyphenpenalty}          \NC \type {\explicithyphenpenalty}  \NC \NR
-\NC \type{7} \NC \type {\automatichyphenpenalty} \NC \type {\exhyphenpenalty}        \NC \NR
-\NC \type{8} \NC \type {\automatichyphenpenalty} \NC \type {\hyphenpenalty}          \NC \NR
-\stoptabulate
-
-other values do what we always did in \LUATEX: insert \type {\exhyphenpenalty}.
-
-\section[patternsexceptions]{Loading patterns and exceptions}
-
-The hyphenation algorithm in \LUATEX\ is quite different from the one in \TEX82,
-although it uses essentially the same user input.
-
-After expansion, the argument for \type {\patterns} has to be proper \UTF8 with
-individual patterns separated by spaces, no \type {\char} or \type {\chardef}d
-commands are allowed. The current implementation quite strict and will reject all
-non|-|\UNICODE\ characters.
-
-Likewise, the expanded argument for \type {\hyphenation} also has to be proper
-\UTF8, but here a bit of extra syntax is provided:
-
-\startitemize[n]
-\startitem
-    Three sets of arguments in curly braces (\type {{}{}{}}) indicates a desired
-    complex discretionary, with arguments as in \type {\discretionary}'s command in
-    normal document input.
-\stopitem
-\startitem
-    A \type {-} indicates a desired simple discretionary, cf.\ \type {\-} and \type
-    {\discretionary{-}{}{}} in normal document input.
-\stopitem
-\startitem
-    Internal command names are ignored. This rule is provided especially for \type
-    {\discretionary}, but it also helps to deal with \type {\relax} commands that
-    may sneak in.
-\stopitem
-\startitem
-    An \type {=} indicates a (non|-|discretionary) hyphen in the document input.
-\stopitem
-\stopitemize
-
-The expanded argument is first converted back to a space-separated string while
-dropping the internal command names. This string is then converted into a
-dictionary by a routine that creates key|-|value pairs by converting the other
-listed items. It is important to note that the keys in an exception dictionary
-can always be generated from the values. Here are a few examples:
-
-\starttabulate[|l|l|l|]
-\BC value                  \BC implied key (input) \NC effect \NC\NR
-\NC \type {ta-ble}         \NC table               \NC \type {ta\-ble} ($=$ \type {ta\discretionary{-}{}{}ble}) \NC\NR
-\NC \type {ba{k-}{}{c}ken} \NC backen              \NC \type {ba\discretionary{k-}{}{c}ken} \NC\NR
-\stoptabulate
-
-The resultant patterns and exception dictionary will be stored under the language
-code that is the present value of \type {\language}.
-
-In the last line of the table, you see there is no \type {\discretionary} command
-in the value: the command is optional in the \TEX-based input syntax. The
-underlying reason for that is that it is conceivable that a whole dictionary of
-words is stored as a plain text file and loaded into \LUATEX\ using one of the
-functions in the \LUA\ \type {lang} library. This loading method is quite a bit
-faster than going through the \TEX\ language primitives, but some (most?) of that
-speed gain would be lost if it had to interpret command sequences while doing so.
-
-It is possible to specify extra hyphenation points in compound words by using
-\type {{-}{}{-}} for the explicit hyphen character (replace \type {-} by the
-actual explicit hyphen character if needed). For example, this matches the word
-\quote {multi|-|word|-|boundaries} and allows an extra break inbetween \quote
-{boun} and \quote {daries}:
-
-\starttyping
-\hyphenation{multi{-}{}{-}word{-}{}{-}boun-daries}
-\stoptyping
-
-The motivation behind the \ETEX\ extension \type {\savinghyphcodes} was that
-hyphenation heavily depended on font encodings. This is no longer true in
-\LUATEX, and the corresponding primitive is basically ignored. Because we now
-have \type {hjcode}, the case relate codes can be used exclusively for \type
-{\uppercase} and \type {\lowercase}.
-
-\section{Applying hyphenation}
-
-The internal structures \LUATEX\ uses for the insertion of discretionaries in
-words is very different from the ones in \TEX82, and that means there are some
-noticeable differences in handling as well.
-
-First and foremost, there is no \quote {compressed trie} involved in hyphenation.
-The algorithm still reads \PATGEN-generated pattern files, but \LUATEX\ uses a
-finite state hash to match the patterns against the word to be hyphenated. This
-algorithm is based on the \quote {libhnj} library used by \OPENOFFICE, which in
-turn is inspired by \TEX.
-
-There are a few differences between \LUATEX\ and \TEX82 that are a direct result
-of the implementation:
-
-\startitemize
-\startitem
-    \LUATEX\ happily hyphenates the full \UNICODE\ character range.
-\stopitem
-\startitem
-    Pattern and exception dictionary size is limited by the available memory
-    only, all allocations are done dynamically. The trie|-|related settings in
-    \type {texmf.cnf} are ignored.
-\stopitem
-\startitem
-    Because there is no \quote {trie preparation} stage, language patterns never
-    become frozen. This means that the primitive \type {\patterns} (and its \LUA\
-    counterpart \type {lang.patterns}) can be used at any time, not only in
-    ini\TEX.
-\stopitem
-\startitem
-    Only the string representation of \type {\patterns} and \type {\hyphenation} is
-    stored in the format file. At format load time, they are simply
-    re|-|evaluated. It follows that there is no real reason to preload languages
-    in the format file. In fact, it is usually not a good idea to do so. It is
-    much smarter to load patterns no sooner than the first time they are actually
-    needed.
-\stopitem
-\startitem
-    \LUATEX\ uses the language-specific variables \type {\prehyphenchar} and \type
-    {\posthyphenchar} in the creation of implicit discretionaries, instead of
-    \TEX82's \type {\hyphenchar}, and the values of the language|-|specific variables
-    \type {\preexhyphenchar} and \type {\postexhyphenchar} for explicit
-    discretionaries (instead of \TEX82's empty discretionary).
-\stopitem
-\startitem
-    The value of the two counters related to hyphenation, \type {\hyphenpenalty}
-    and \type {\exhyphenpenalty}, are now stored in the discretionary nodes. This
-    permits a local overload for explicit \type {\discretionary} commands. The
-    value current when the hyphenation pass is applied is used. When no callbacks
-    are used this is compatible with traditional \TEX. When you apply the \LUA\
-    \type {lang.hyphenate} function the current values are used.
-\stopitem
-\stopitemize
-
-Because we store penalties in the disc node the \type {\discretionary} command has
-been extended to accept an optional penalty specification, so you can do the
-following:
-
-\startbuffer
-\hsize1mm
-1:foo{\hyphenpenalty 10000\discretionary{}{}{}}bar\par
-2:foo\discretionary penalty 10000 {}{}{}bar\par
-3:foo\discretionary{}{}{}bar\par
-\stopbuffer
-
-\typebuffer
-
-This results in:
-
-\blank \start \getbuffer \stop \blank
-
-Inserted characters and ligatures inherit their attributes from the nearest glyph
-node item (usually the preceding one, but the following one for the items
-inserted at the left-hand side of a word).
-
-Word boundaries are no longer implied by font switches, but by language switches.
-One word can have two separate fonts and still be hyphenated correctly (but it
-can not have two different languages, the \type {\setlanguage} command forces a
-word boundary).
-
-All languages start out with \type {\prehyphenchar=`\-}, \type {\posthyphenchar=0},
-\type {\preexhyphenchar=0} and \type {\postexhyphenchar=0}. When you assign the
-values of one of these four parameters, you are actually changing the settings
-for the current \type {\language}, this behaviour is compatible with \type {\patterns}
-and \type {\hyphenation}.
-
-\LUATEX\ also hyphenates the first word in a paragraph. Words can be up to 256
-characters long (up from 64 in \TEX82). Longer words generate an error right now,
-but eventually either the limitation will be removed or perhaps it will become
-possible to silently ignore the excess characters (this is what happens in
-\TEX82, but there the behaviour cannot be controlled).
-
-If you are using the \LUA\ function \type {lang.hyphenate}, you should be aware
-that this function expects to receive a list of \quote {character} nodes. It will
-not operate properly in the presence of \quote {glyph}, \quote {ligature}, or
-\quote {ghost} nodes, nor does it know how to deal with kerning.
-
-The hyphenation exception dictionary is maintained as key|-|value hash, and that
-is also dynamic, so the \type {hyph_size} setting is not used either.
-
-\section{Applying ligatures and kerning}
-
-After all possible hyphenation points have been inserted in the list, \LUATEX\
-will process the list to convert the \quote {character} nodes into \quote {glyph}
-and \quote {ligature} nodes. This is actually done in two stages: first all
-ligatures are processed, then all kerning information is applied to the result
-list. But those two stages are somewhat dependent on each other: If the used font
-makes it possible to do so, the ligaturing stage adds virtual \quote {character}
-nodes to the word boundaries in the list. While doing so, it removes and
-interprets \type {\noboundary} nodes. The kerning stage deletes those word
-boundary items after it is done with them, and it does the same for \quote
-{ghost} nodes. Finally, at the end of the kerning stage, all remaining \quote
-{character} nodes are converted to \quote {glyph} nodes.
-
-This work separation is worth mentioning because, if you overrule from \LUA\ only
-one of the two callbacks related to font handling, then you have to make sure you
-perform the tasks normally done by \LUATEX\ itself in order to make sure that the
-other, non|-|overruled, routine continues to function properly.
-
-Work in this area is not yet complete, but most of the possible cases are handled
-by our rewritten ligaturing engine. At some point all of the possible inputs will
-become supported. \footnote {Not all of this makes sense because we nowadays have
-\OPENTYPE\ fonts and ligature building can happen in ,any different ways there.}
-
-For example, take the word \type {office}, hyphenated \type {of-fice}, using a
-\quote {normal} font with all the \type {f}-\type {f} and \type {f}-\type {i}
-type ligatures:
-
-\starttabulate[|l|l|]
-\NC initial              \NC \type {{o}{f}{f}{i}{c}{e}}             \NC\NR
-\NC after hyphenation    \NC \type {{o}{f}{{-},{},{}}{f}{i}{c}{e}}  \NC\NR
-\NC first ligature stage \NC \type {{o}{{f-},{f},{<ff>}}{i}{c}{e}}  \NC\NR
-\NC final result         \NC \type {{o}{{f-},{<fi>},{<ffi>}}{c}{e}} \NC\NR
-\stoptabulate
-
-That's bad enough, but let us assume that there is also a hyphenation point
-between the \type {f} and the \type {i}, to create \type {of-f-ice}. Then the
-final result should be:
-
-\starttyping
-{o}{{f-},
-    {{f-},
-     {i},
-     {<fi>}},
-    {{<ff>-},
-     {i},
-     {<ffi>}}}{c}{e}
-\stoptyping
-
-with discretionaries in the post-break text as well as in the replacement text of
-the top-level discretionary that resulted from the first hyphenation point.
-
-Here is that nested solution again, in a different representation:
-
-\starttabulate[|l|c|c|c|c|c|c|]
-\NC         \BC pre           \BC     \BC post      \BC       \BC replace       \BC       \NC \NR
-\NC topdisc \NC \type {f-}    \NC (1) \NC           \NC sub 1 \NC               \NC sub 2 \NC \NR
-\NC sub 1   \NC \type {f-}    \NC (2) \NC \type {i} \NC (3)   \NC \type {<fi>}  \NC (4)   \NC \NR
-\NC sub 2   \NC \type {<ff>-} \NC (5) \NC \type {i} \NC (6)   \NC \type {<ffi>} \NC (7)   \NC \NR
-\stoptabulate
-
-When line breaking is choosing its breakpoints, the following fields will
-eventually be selected:
-
-\starttabulate[|l|c|c|]
-\NC \type {of-f-ice} \NC \type {f-}    \NC (1) \NC \NR
-\NC                  \NC \type {f-}    \NC (2) \NC \NR
-\NC                  \NC \type {i}     \NC (3) \NC \NR
-\NC \type {of-fice}  \NC \type {f-}    \NC (1) \NC \NR
-\NC                  \NC \type {<fi>}  \NC (4) \NC \NR
-\NC \type {off-ice}  \NC \type {<ff>-} \NC (5) \NC \NR
-\NC                  \NC \type {i}     \NC (6) \NC \NR
-\NC \type {office}   \NC \type {<ffi>} \NC (7) \NC \NR
-\stoptabulate
-
-The current solution in \LUATEX\ is not able to handle nested discretionaries,
-but it is in fact smart enough to handle this fictional \type {of-f-ice} example.
-It does so by combining two sequential discretionary nodes as if they were a
-single object (where the second discretionary node is treated as an extension of
-the first node).
-
-One can observe that the \type {of-f-ice} and \type {off-ice} cases both end with
-the same actual post replacement list (\type {i}), and that this would be the
-case even if that \type {i} was the first item of a potential following ligature
-like \type {ic}. This allows \LUATEX\ to do away with one of the fields, and thus
-make the whole stuff fit into just two discretionary nodes.
-
-The mapping of the seven list fields to the six fields in this discretionary node
-pair is as follows:
-
-\starttabulate[|l|c|c|]
-\BC field                 \BC description   \NC       \NC \NR
-\NC \type {disc1.pre}     \NC \type {f-}    \NC (1)   \NC \NR
-\NC \type {disc1.post}    \NC \type {<fi>}  \NC (4)   \NC \NR
-\NC \type {disc1.replace} \NC \type {<ffi>} \NC (7)   \NC \NR
-\NC \type {disc2.pre}     \NC \type {f-}    \NC (2)   \NC \NR
-\NC \type {disc2.post}    \NC \type {i}     \NC (3,6) \NC \NR
-\NC \type {disc2.replace} \NC \type {<ff>-} \NC (5)   \NC \NR
-\stoptabulate
-
-What is actually generated after ligaturing has been applied is therefore:
-
-\starttyping
-{o}{{f-},
-    {<fi>},
-    {<ffi>}}
-   {{f-},
-    {i},
-    {<ff>-}}{c}{e}
-\stoptyping
-
-The two discretionaries have different subtypes from a discretionary appearing on
-its own: the first has subtype 4, and the second has subtype 5. The need for
-these special subtypes stems from the fact that not all of the fields appear in
-their \quote {normal} location. The second discretionary especially looks odd,
-with things like the \type {<ff>-} appearing in \type {disc2.replace}. The fact
-that some of the fields have different meanings (and different processing code
-internally) is what makes it necessary to have different subtypes: this enables
-\LUATEX\ to distinguish this sequence of two joined discretionary nodes from the
-case of two standalone discretionaries appearing in a row.
-
-Of course there is still that relationship with fonts: ligatures can be implemented by
-mapping a sequence of glyphs onto one glyph, but also by selective replacement and
-kerning. This means that the above examples are just representing the traditional
-approach.
-
-\section{Breaking paragraphs into lines}
-
-This code is still almost unchanged, but because of the above|-|mentioned changes
-with respect to discretionaries and ligatures, line breaking will potentially be
-different from traditional \TEX. The actual line breaking code is still based on
-the \TEX82 algorithms, and it does not expect there to be discretionaries inside
-of discretionaries.
-
-But that situation is now fairly common in \LUATEX, due to the changes to the
-ligaturing mechanism. And also, the \LUATEX\ discretionary nodes are implemented
-slightly different from the \TEX82 nodes: the \type {no_break} text is now
-embedded inside the disc node, where previously these nodes kept their place in
-the horizontal list. In traditional \TEX\ the discretionary node contains a
-counter indicating how many nodes to skip, but in \LUATEX\ we store the pre, post
-and replace text in the discretionary node.
-
-The combined effect of these two differences is that \LUATEX\ does not always use
-all of the potential breakpoints in a paragraph, especially when fonts with many
-ligatures are used. Of course kerning also complicates matters here.
-
-\section{The \type {lang} library}
-
-This library provides the interface to \LUATEX's structure
-representing a language, and the associated functions.
-
-\startfunctioncall
-<language> l = lang.new()
-<language> l = lang.new(<number> id)
-\stopfunctioncall
-
-This function creates a new userdata object. An object of type \type {<language>}
-is the first argument to most of the other functions in the \type {lang}
-library. These functions can also be used as if they were object methods, using
-the colon syntax.
-
-Without an argument, the next available internal id number will be assigned to
-this object. With argument, an object will be created that links to the internal
-language with that id number.
-
-\startfunctioncall
-<number> n = lang.id(<language> l)
-\stopfunctioncall
-
-returns the internal \type {\language} id number this object refers to.
-
-\startfunctioncall
-<string> n = lang.hyphenation(<language> l)
-lang.hyphenation(<language> l, <string> n)
-\stopfunctioncall
-
-Either returns the current hyphenation exceptions for this language, or adds new
-ones. The syntax of the string is explained in~\in {section}
-[patternsexceptions].
-
-\startfunctioncall
-lang.clear_hyphenation(<language> l)
-\stopfunctioncall
-
-Clears the exception dictionary (string) for this language.
-
-\startfunctioncall
-<string> n = lang.clean(<language> l, <string> o)
-<string> n = lang.clean(<string> o)
-\stopfunctioncall
-
-Creates a hyphenation key from the supplied hyphenation value. The syntax of the
-argument string is explained in~\in {section} [patternsexceptions]. This function
-is useful if you want to do something else based on the words in a dictionary
-file, like spell|-|checking.
-
-\startfunctioncall
-<string> n = lang.patterns(<language> l)
-lang.patterns(<language> l, <string> n)
-\stopfunctioncall
-
-Adds additional patterns for this language object, or returns the current set.
-The syntax of this string is explained in~\in {section} [patternsexceptions].
-
-\startfunctioncall
-lang.clear_patterns(<language> l)
-\stopfunctioncall
-
-Clears the pattern dictionary for this language.
-
-\startfunctioncall
-<number> n = lang.prehyphenchar(<language> l)
-lang.prehyphenchar(<language> l, <number> n)
-\stopfunctioncall
-
-Gets or sets the \quote {pre|-|break} hyphen character for implicit hyphenation
-in this language (initially the hyphen, decimal 45).
-
-\startfunctioncall
-<number> n = lang.posthyphenchar(<language> l)
-lang.posthyphenchar(<language> l, <number> n)
-\stopfunctioncall
-
-Gets or sets the \quote {post|-|break} hyphen character for implicit hyphenation
-in this language (initially null, decimal~0, indicating emptiness).
-
-\startfunctioncall
-<number> n = lang.preexhyphenchar(<language> l)
-lang.preexhyphenchar(<language> l, <number> n)
-\stopfunctioncall
-
-Gets or sets the \quote {pre|-|break} hyphen character for explicit hyphenation
-in this language (initially null, decimal~0, indicating emptiness).
-
-\startfunctioncall
-<number> n = lang.postexhyphenchar(<language> l)
-lang.postexhyphenchar(<language> l, <number> n)
-\stopfunctioncall
-
-Gets or sets the \quote {post|-|break} hyphen character for explicit hyphenation
-in this language (initially null, decimal~0, indicating emptiness).
-
-\startfunctioncall
-<boolean> success = lang.hyphenate(<node> head)
-<boolean> success = lang.hyphenate(<node> head, <node> tail)
-\stopfunctioncall
-
-Inserts hyphenation points (discretionary nodes) in a node list. If \type {tail}
-is given as argument, processing stops on that node. Currently, \type {success}
-is always true if \type {head} (and \type {tail}, if specified) are proper nodes,
-regardless of possible other errors.
-
-Hyphenation works only on \quote {characters}, a special subtype of all the glyph
-nodes with the node subtype having the value \type {1}. Glyph modes with
-different subtypes are not processed. See \in {section~} [charsandglyphs] for
-more details.
-
-The following two commands can be used to set or query hj codes:
-
-\startfunctioncall
-lang.sethjcode(<language> l, <number> char, <number> usedchar)
-<number> usedchar = lang.gethjcode(<language> l, <number> char)
-\stopfunctioncall
-
-When you set a hjcode the current sets get initialized unless the set was already
-initialized due to \type {\savinghyphcodes} being larger than zero.
-
-\stopchapter
-
-\stopcomponent
-
-% \parindent0pt \hsize=1.1cm
-% 12-34-56 \par
-% 12-34-\hbox{56} \par
-% 12-34-\vrule width 1em height 1.5ex \par
-% 12-\hbox{34}-56 \par
-% 12-\vrule width 1em height 1.5ex-56 \par
-% \hjcode`\1=`\1 \hjcode`\2=`\2 \hjcode`\3=`\3 \hjcode`\4=`\4 \vskip.5cm
-% 12-34-56 \par
-% 12-34-\hbox{56} \par
-% 12-34-\vrule width 1em height 1.5ex \par
-% 12-\hbox{34}-56 \par
-% 12-\vrule width 1em height 1.5ex-56 \par
-
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-languages

+

+\startchapter[reference=languages,title={Languages, characters, fonts and glyphs}]

+

+\startsection[title={Introduction}]

+

+\topicindex {languages}

+

+\LUATEX's internal handling of the characters and glyphs that eventually become

+typeset is quite different from the way \TEX82 handles those same objects. The

+easiest way to explain the difference is to focus on unrestricted horizontal mode

+(i.e.\ paragraphs) and hyphenation first. Later on, it will be easy to deal

+with the differences that occur in horizontal and math modes.

+

+In \TEX82, the characters you type are converted into \type {char} node records

+when they are encountered by the main control loop. \TEX\ attaches and processes

+the font information while creating those records, so that the resulting \quote

+{horizontal list} contains the final forms of ligatures and implicit kerning.

+This packaging is needed because we may want to get the effective width of for

+instance a horizontal box.

+

+When it becomes necessary to hyphenate words in a paragraph, \TEX\ converts (one

+word at time) the \type {char} node records into a string by replacing ligatures

+with their components and ignoring the kerning. Then it runs the hyphenation

+algorithm on this string, and converts the hyphenated result back into a \quote

+{horizontal list} that is consecutively spliced back into the paragraph stream.

+Keep in mind that the paragraph may contain unboxed horizontal material, which

+then already contains ligatures and kerns and the words therein are part of the

+hyphenation process.

+

+Those \type {char} node records are somewhat misnamed, as they are glyph

+positions in specific fonts, and therefore not really \quote {characters} in the

+linguistic sense. There is no language information inside the \type {char} node

+records at all. Instead, language information is passed along using \type

+{language whatsit} nodes inside the horizontal list.

+

+In \LUATEX, the situation is quite different. The characters you type are always

+converted into \nod {glyph} node records with a special subtype to identify them

+as being intended as linguistic characters. \LUATEX\ stores the needed language

+information in those records, but does not do any font|-|related processing at

+the time of node creation. It only stores the index of the current font and a

+reference to a character in that font.

+

+When it becomes necessary to typeset a paragraph, \LUATEX\ first inserts all

+hyphenation points right into the whole node list. Next, it processes all the

+font information in the whole list (creating ligatures and adjusting kerning),

+and finally it adjusts all the subtype identifiers so that the records are \quote

+{glyph nodes} from now on.

+

+\stopsection

+

+\startsection[title={Characters, glyphs and discretionaries},reference=charsandglyphs]

+

+\topicindex {characters}

+\topicindex {glyphs}

+\topicindex {hyphenation}

+

+\TEX82 (including \PDFTEX) differentiates between \type {char} nodes and \type

+{lig} nodes. The former are simple items that contained nothing but a \quote

+{character} and a \quote {font} field, and they lived in the same memory as

+tokens did. The latter also contained a list of components, and a subtype

+indicating whether this ligature was the result of a word boundary, and it was

+stored in the same place as other nodes like boxes and kerns and glues.

+

+In \LUATEX, these two types are merged into one, somewhat larger structure called

+a \nod {glyph} node. Besides having the old character, font, and component

+fields there are a few more, like \quote {attr} that we will see in \in {section}

+[glyphnodes], these nodes also contain a subtype, that codes four main types and

+two additional ghost types. For ligatures, multiple bits can be set at the same

+time (in case of a single|-|glyph word).

+

+\startitemize

+    \startitem

+        \type {character}, for characters to be hyphenated: the lowest bit

+        (bit 0) is set to 1.

+    \stopitem

+    \startitem

+        \nod {glyph}, for specific font glyphs: the lowest bit (bit 0) is

+        not set.

+    \stopitem

+    \startitem

+        \type {ligature}, for constructed ligatures bit 1 is set.

+    \stopitem

+    \startitem

+        \type {ghost}, for so called \quote {ghost objects} bit 2 is set.

+    \stopitem

+    \startitem

+        \type {left}, for ligatures created from a left word boundary and for

+        ghosts created from \lpr {leftghost} bit 3 gets set.

+    \stopitem

+    \startitem

+        \type {right}, for ligatures created from a right word boundary and

+        for ghosts created from \lpr {rightghost} bit 4 is set.

+    \stopitem

+\stopitemize

+

+The \nod {glyph} nodes also contain language data, split into four items that

+were current when the node was created: the \prm {setlanguage} (15~bits), \prm

+{lefthyphenmin} (8~bits), \prm {righthyphenmin} (8~bits), and \prm {uchyph}

+(1~bit).

+

+Incidentally, \LUATEX\ allows 16383 separate languages, and words can be 256

+characters long. The language is stored with each character. You can set

+\prm {firstvalidlanguage} to for instance~1 and make thereby language~0

+an ignored hyphenation language.

+

+The new primitive \lpr {hyphenationmin} can be used to signal the minimal length

+of a word. This value is stored with the (current) language.

+

+Because the \prm {uchyph} value is saved in the actual nodes, its handling is

+subtly different from \TEX82: changes to \prm {uchyph} become effective

+immediately, not at the end of the current partial paragraph.

+

+Typeset boxes now always have their language information embedded in the nodes

+themselves, so there is no longer a possible dependency on the surrounding

+language settings. In \TEX82, a mid|-|paragraph statement like \type {\unhbox0}

+would process the box using the current paragraph language unless there was a

+\prm {setlanguage} issued inside the box. In \LUATEX, all language variables

+are already frozen.

+

+In traditional \TEX\ the process of hyphenation is driven by \type {lccode}s. In

+\LUATEX\ we made this dependency less strong. There are several strategies

+possible. When you do nothing, the currently used \type {lccode}s are used, when

+loading patterns, setting exceptions or hyphenating a list.

+

+When you set \prm {savinghyphcodes} to a value greater than zero the current set

+of \type {lccode}s will be saved with the language. In that case changing a \type

+{lccode} afterwards has no effect. However, you can adapt the set with:

+

+\starttyping

+\hjcode`a=`a

+\stoptyping

+

+This change is global which makes sense if you keep in mind that the moment that

+hyphenation happens is (normally) when the paragraph or a horizontal box is

+constructed. When \prm {savinghyphcodes} was zero when the language got

+initialized you start out with nothing, otherwise you already have a set.

+

+When a \lpr {hjcode} is greater than 0 but less than 32 is indicates the

+to be used length. In the following example we map a character (\type {x}) onto

+another one in the patterns and tell the engine that \type {œ} counts as one

+character. Because traditionally zero itself is reserved for inhibiting

+hyphenation, a value of 32 counts as zero.

+

+Here are some examples (we assume that French patterns are used):

+

+\starttabulate[||||]

+\NC                                  \NC \type{foobar} \NC \type{foo-bar} \NC \NR

+\NC \type{\hjcode`x=`o}              \NC \type{fxxbar} \NC \type{fxx-bar} \NC \NR

+\NC \type{\lefthyphenmin3}           \NC \type{œdipus} \NC \type{œdi-pus} \NC \NR

+\NC \type{\lefthyphenmin4}           \NC \type{œdipus} \NC \type{œdipus}  \NC \NR

+\NC \type{\hjcode`œ=2}               \NC \type{œdipus} \NC \type{œdi-pus} \NC \NR

+\NC \type{\hjcode`i=32 \hjcode`d=32} \NC \type{œdipus} \NC \type{œdipus}  \NC \NR

+\NC

+\stoptabulate

+

+Carrying all this information with each glyph would give too much overhead and

+also make the process of setting up these codes more complex. A solution with

+\type {hjcode} sets was considered but rejected because in practice the current

+approach is sufficient and it would not be compatible anyway.

+

+Beware: the values are always saved in the format, independent of the setting

+of \prm {savinghyphcodes} at the moment the format is dumped.

+

+A boundary node normally would mark the end of a word which interferes with for

+instance discretionary injection. For this you can use the \prm {wordboundary}

+as a trigger. Here are a few examples of usage:

+

+\startbuffer

+    discrete---discrete

+\stopbuffer

+\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower

+\startbuffer

+    discrete\discretionary{}{}{---}discrete

+\stopbuffer

+\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower

+\startbuffer

+    discrete\wordboundary\discretionary{}{}{---}discrete

+\stopbuffer

+\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower

+\startbuffer

+    discrete\wordboundary\discretionary{}{}{---}\wordboundary discrete

+\stopbuffer

+\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower

+\startbuffer

+    discrete\wordboundary\discretionary{---}{}{}\wordboundary discrete

+\stopbuffer

+\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower

+

+We only accept an explicit hyphen when there is a preceding glyph and we skip a

+sequence of explicit hyphens since that normally indicates a \type {--} or \type

+{---} ligature in which case we can in a worse case usage get bad node lists

+later on due to messed up ligature building as these dashes are ligatures in base

+fonts. This is a side effect of separating the hyphenation, ligaturing and

+kerning steps.

+

+The start and end of a sequence of characters is signalled by a \nod {glue}, \nod

+{penalty}, \nod {kern} or \nod {boundary} node. But by default also a \nod

+{hlist}, \nod {vlist}, \nod {rule}, \nod {dir}, \nod {whatsit}, \nod {ins}, and

+\nod {adjust} node indicate a start or end. You can omit the last set from the

+test by setting \lpr {hyphenationbounds} to a non|-|zero value:

+

+\starttabulate[|c|l|]

+\DB value    \BC behaviour \NC \NR

+\TB

+\NC \type{0} \NC not strict \NC \NR

+\NC \type{1} \NC strict start \NC \NR

+\NC \type{2} \NC strict end \NC \NR

+\NC \type{3} \NC strict start and strict end \NC \NR

+\LL

+\stoptabulate

+

+The word start is determined as follows:

+

+\starttabulate[|l|l|]

+\DB node      \BC behaviour \NC \NR

+\TB

+\BC boundary  \NC yes when wordboundary \NC \NR

+\BC hlist     \NC when hyphenationbounds 1 or 3 \NC \NR

+\BC vlist     \NC when hyphenationbounds 1 or 3 \NC \NR

+\BC rule      \NC when hyphenationbounds 1 or 3 \NC \NR

+\BC dir       \NC when hyphenationbounds 1 or 3 \NC \NR

+\BC whatsit   \NC when hyphenationbounds 1 or 3 \NC \NR

+\BC glue      \NC yes \NC \NR

+\BC math      \NC skipped \NC \NR

+\BC glyph     \NC exhyphenchar (one only) : yes (so no -- ---) \NC \NR

+\BC otherwise \NC yes \NC \NR

+\LL

+\stoptabulate

+

+The word end is determined as follows:

+

+\starttabulate[|l|l|]

+\DB node      \BC behaviour \NC \NR

+\TB

+\BC boundary  \NC yes \NC \NR

+\BC glyph     \NC yes when different language \NC \NR

+\BC glue      \NC yes \NC \NR

+\BC penalty   \NC yes \NC \NR

+\BC kern      \NC yes when not italic (for some historic reason) \NC \NR

+\BC hlist     \NC when hyphenationbounds 2 or 3 \NC \NR

+\BC vlist     \NC when hyphenationbounds 2 or 3 \NC \NR

+\BC rule      \NC when hyphenationbounds 2 or 3 \NC \NR

+\BC dir       \NC when hyphenationbounds 2 or 3 \NC \NR

+\BC whatsit   \NC when hyphenationbounds 2 or 3 \NC \NR

+\BC ins       \NC when hyphenationbounds 2 or 3 \NC \NR

+\BC adjust    \NC when hyphenationbounds 2 or 3 \NC \NR

+\LL

+\stoptabulate

+

+\in {Figures} [hb:1] upto \in [hb:5] show some examples. In all cases we set the

+min values to 1 and make sure that the words hyphenate at each character.

+

+\hyphenation{o-n-e t-w-o}

+

+\def\SomeTest#1#2%

+  {\lefthyphenmin  \plusone

+   \righthyphenmin \plusone

+   \parindent      \zeropoint

+   \everypar       \emptytoks

+   \dontcomplain

+   \hbox to 2cm {%

+     \vtop {%

+       \hsize 1pt

+       \hyphenationbounds#1

+       #2

+       \par}}}

+

+\startplacefigure[reference=hb:1,title={\type{one}}]

+    \startcombination[4*1]

+        {\SomeTest{0}{one}} {\type{0}}

+        {\SomeTest{1}{one}} {\type{1}}

+        {\SomeTest{2}{one}} {\type{2}}

+        {\SomeTest{3}{one}} {\type{3}}

+    \stopcombination

+\stopplacefigure

+

+\startplacefigure[reference=hb:2,title={\type{one\null two}}]

+    \startcombination[4*1]

+        {\SomeTest{0}{one\null two}} {\type{0}}

+        {\SomeTest{1}{one\null two}} {\type{1}}

+        {\SomeTest{2}{one\null two}} {\type{2}}

+        {\SomeTest{3}{one\null two}} {\type{3}}

+    \stopcombination

+\stopplacefigure

+

+\startplacefigure[reference=hb:3,title={\type{\null one\null two}}]

+    \startcombination[4*1]

+        {\SomeTest{0}{\null one\null two}} {\type{0}}

+        {\SomeTest{1}{\null one\null two}} {\type{1}}

+        {\SomeTest{2}{\null one\null two}} {\type{2}}

+        {\SomeTest{3}{\null one\null two}} {\type{3}}

+    \stopcombination

+\stopplacefigure

+

+\startplacefigure[reference=hb:4,title={\type{one\null two\null}}]

+    \startcombination[4*1]

+        {\SomeTest{0}{one\null two\null}} {\type{0}}

+        {\SomeTest{1}{one\null two\null}} {\type{1}}

+        {\SomeTest{2}{one\null two\null}} {\type{2}}

+        {\SomeTest{3}{one\null two\null}} {\type{3}}

+    \stopcombination

+\stopplacefigure

+

+\startplacefigure[reference=hb:5,title={\type{\null one\null two\null}}]

+    \startcombination[4*1]

+        {\SomeTest{0}{\null one\null two\null}} {\type{0}}

+        {\SomeTest{1}{\null one\null two\null}} {\type{1}}

+        {\SomeTest{2}{\null one\null two\null}} {\type{2}}

+        {\SomeTest{3}{\null one\null two\null}} {\type{3}}

+    \stopcombination

+\stopplacefigure

+

+% (Future versions of \LUATEX\ might provide more granularity.)

+

+In traditional \TEX\ ligature building and hyphenation are interwoven with the

+line break mechanism. In \LUATEX\ these phases are isolated. As a consequence we

+deal differently with (a sequence of) explicit hyphens. We already have added

+some control over aspects of the hyphenation and yet another one concerns

+automatic hyphens (e.g.\ \type {-} characters in the input).

+

+When \lpr {automatichyphenmode} has a value of 0, a hyphen will be turned into

+an automatic discretionary. The snippets before and after it will not be

+hyphenated. A side effect is that a leading hyphen can lead to a split but one

+will seldom run into that situation. Setting a pre and post character makes this

+more prominent. A value of 1 will prevent this side effect and a value of 2 will

+not turn the hyphen into a discretionary. Experiments with other options, like

+permitting hyphenation of the words on both sides were discarded.

+

+\startbuffer[a]

+before-after \par

+before--after \par

+before---after \par

+\stopbuffer

+

+\startbuffer[b]

+-before \par

+after- \par

+--before \par

+after-- \par

+---before \par

+after--- \par

+\stopbuffer

+

+\startbuffer[c]

+before-after \par

+before--after \par

+before---after \par

+\stopbuffer

+

+\startbuffer[demo]

+\startcombination[nx=4,ny=3,location=top]

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[a]}} {A~0~6em}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[a]}} {A~0~2pt}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone   \hsize2pt \getbuffer[a]}} {A~1~2pt}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo   \hsize2pt \getbuffer[a]}} {A~2~2pt}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[b]}} {B~0~6em}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[b]}} {B~0~2pt}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone   \hsize2pt \getbuffer[b]}} {B~1~2pt}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo   \hsize2pt \getbuffer[b]}} {B~2~2pt}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[c]}} {C~0~6em}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[c]}} {C~0~2pt}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone   \hsize2pt \getbuffer[c]}} {C~1~2pt}

+    {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo   \hsize2pt \getbuffer[c]}} {C~2~2pt}

+\stopcombination

+\stopbuffer

+

+\startplacefigure[locationreference=automatichyphenmode:1,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with a \prm {hsize}

+of 6em and 2pt (which triggers a linebreak).}]

+    \dontcomplain \tt \getbuffer[demo]

+\stopplacefigure

+

+\startplacefigure[reference=automatichyphenmode:2,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with \lpr {preexhyphenchar} and \lpr {postexhyphenchar} set to characters \type {A} and \type {B}.}]

+    \postexhyphenchar`A\relax

+    \preexhyphenchar `B\relax

+    \dontcomplain \tt \getbuffer[demo]

+\stopplacefigure

+

+In \in {figure} [automatichyphenmode:1] \in {and} [automatichyphenmode:2] we show

+what happens with three samples:

+

+Input A: \typebuffer[a]

+Input B: \typebuffer[b]

+Input C: \typebuffer[c]

+

+As with primitive companions of other single character commands, the \prm {-}

+command has a more verbose primitive version in \lpr {explicitdiscretionary}

+and the normally intercepted in the hyphenator character \type {-} (or whatever

+is configured) is available as \lpr {automaticdiscretionary}.

+

+\stopsection

+

+\startsection[title={The main control loop}]

+

+\topicindex {main loop}

+\topicindex {hyphenation}

+

+In \LUATEX's main loop, almost all input characters that are to be typeset are

+converted into \nod {glyph} node records with subtype \quote {character}, but

+there are a few exceptions.

+

+\startitemize[n]

+

+\startitem

+    The \prm {accent} primitive creates nodes with subtype \quote {glyph}

+    instead of \quote {character}: one for the actual accent and one for the

+    accentee. The primary reason for this is that \prm {accent} in \TEX82 is

+    explicitly dependent on the current font encoding, so it would not make much

+    sense to attach a new meaning to the primitive's name, as that would

+    invalidate many old documents and macro packages. A secondary reason is that

+    in \TEX82, \prm {accent} prohibits hyphenation of the current word. Since

+    in \LUATEX\ hyphenation only takes place on \quote {character} nodes, it is

+    possible to achieve the same effect. Of course, modern \UNICODE\ aware macro

+    packages will not use the \prm {accent} primitive at all but try to map

+    directly on composed characters.

+

+    This change of meaning did happen with \prm {char}, that now generates

+    \quote {glyph} nodes with a character subtype. In traditional \TEX\ there was

+    a strong relationship between the 8|-|bit input encoding, hyphenation and

+    glyphs taken from a font. In \LUATEX\ we have \UTF\ input, and in most cases

+    this maps directly to a character in a font, apart from glyph replacement in

+    the font engine. If you want to access arbitrary glyphs in a font directly

+    you can always use \LUA\ to do so, because fonts are available as \LUA\

+    table.

+\stopitem

+

+\startitem

+    All the results of processing in math mode eventually become nodes with

+    \quote {glyph} subtypes. In fact, the result of processing math is just

+    a regular list of glyphs, kerns, glue, penalties, boxes etc.

+\stopitem

+

+\startitem

+    The \ALEPH|-|derived commands \lpr {leftghost} and \lpr {rightghost}

+    create nodes of a third subtype: \quote {ghost}. These nodes are ignored

+    completely by all further processing until the stage where inter|-|glyph

+    kerning is added.

+\stopitem

+

+\startitem

+    Automatic discretionaries are handled differently. \TEX82 inserts an empty

+    discretionary after sensing an input character that matches the \prm

+    {hyphenchar} in the current font. This test is wrong in our opinion: whether

+    or not hyphenation takes place should not depend on the current font, it is a

+    language property. \footnote {When \TEX\ showed up we didn't have \UNICODE\

+    yet and being limited to eight bits meant that one sometimes had to

+    compromise between supporting character input, glyph rendering, hyphenation.}

+

+    In \LUATEX, it works like this: if \LUATEX\ senses a string of input

+    characters that matches the value of the new integer parameter \prm

+    {exhyphenchar}, it will insert an explicit discretionary after that series of

+    nodes. Initially \TEX\ sets the \type {\exhyphenchar=`\-}. Incidentally, this

+    is a global parameter instead of a language-specific one because it may be

+    useful to change the value depending on the document structure instead of the

+    text language.

+

+    The insertion of discretionaries after a sequence of explicit hyphens happens

+    at the same time as the other hyphenation processing, {\it not\/} inside the

+    main control loop.

+

+    The only use \LUATEX\ has for \prm {hyphenchar} is at the check whether a

+    word should be considered for hyphenation at all. If the \prm {hyphenchar}

+    of the font attached to the first character node in a word is negative, then

+    hyphenation of that word is abandoned immediately. This behaviour is added

+    for backward compatibility only, and the use of \type {\hyphenchar=-1} as a

+    means of preventing hyphenation should not be used in new \LUATEX\ documents.

+\stopitem

+

+\startitem

+    The \prm {setlanguage} command no longer creates whatsits. The meaning of

+    \prm {setlanguage} is changed so that it is now an integer parameter like all

+    others. That integer parameter is used in \type {\glyph_node} creation to add

+    language information to the glyph nodes. In conjunction, the \prm {language}

+    primitive is extended so that it always also updates the value of \prm

+    {setlanguage}.

+\stopitem

+

+\startitem

+    The \prm {noboundary} command (that prohibits word boundary processing

+    where that would normally take place) now does create nodes. These nodes are

+    needed because the exact place of the \prm {noboundary} command in the

+    input stream has to be retained until after the ligature and font processing

+    stages.

+\stopitem

+

+\startitem

+    There is no longer a \type {main_loop} label in the code. Remember that

+    \TEX82 did quite a lot of processing while adding \type {char_nodes} to the

+    horizontal list? For speed reasons, it handled that processing code outside

+    of the \quote {main control} loop, and only the first character of any \quote

+    {word} was handled by that \quote {main control} loop. In \LUATEX, there is

+    no longer a need for that (all hard work is done later), and the (now very

+    small) bits of character|-|handling code have been moved back inline. When

+    \prm {tracingcommands} is on, this is visible because the full word is

+    reported, instead of just the initial character.

+\stopitem

+

+\stopitemize

+

+Because we tend to make hard coded behaviour configurable a few new primitives

+have been added:

+

+\starttyping

+\hyphenpenaltymode

+\automatichyphenpenalty

+\explicithyphenpenalty

+\stoptyping

+

+The first parameter has the following consequences for automatic discs (the ones

+resulting from an \prm {exhyphenchar}:

+

+\starttabulate[|c|l|l|]

+\DB mode     \BC automatic disc \type {-}      \BC explicit disc \prm{-}         \NC \NR

+\TB

+\NC \type{0} \NC \prm {exhyphenpenalty}        \NC \prm {exhyphenpenalty}        \NC \NR

+\NC \type{1} \NC \prm {hyphenpenalty}          \NC \prm {hyphenpenalty}          \NC \NR

+\NC \type{2} \NC \prm {exhyphenpenalty}        \NC \prm {hyphenpenalty}          \NC \NR

+\NC \type{3} \NC \prm {hyphenpenalty}          \NC \prm {exhyphenpenalty}        \NC \NR

+\NC \type{4} \NC \lpr {automatichyphenpenalty} \NC \lpr {explicithyphenpenalty}  \NC \NR

+\NC \type{5} \NC \prm {exhyphenpenalty}        \NC \lpr {explicithyphenpenalty}  \NC \NR

+\NC \type{6} \NC \prm {hyphenpenalty}          \NC \lpr {explicithyphenpenalty}  \NC \NR

+\NC \type{7} \NC \lpr {automatichyphenpenalty} \NC \prm {exhyphenpenalty}        \NC \NR

+\NC \type{8} \NC \lpr {automatichyphenpenalty} \NC \prm {hyphenpenalty}          \NC \NR

+\LL

+\stoptabulate

+

+other values do what we always did in \LUATEX: insert \prm {exhyphenpenalty}.

+

+\stopsection

+

+\startsection[title={Loading patterns and exceptions},reference=patternsexceptions]

+

+\topicindex {hyphenation}

+\topicindex {hyphenation+patterns}

+\topicindex {hyphenation+exceptions}

+\topicindex {patterns}

+\topicindex {exceptions}

+

+Although we keep the traditional approach towards hyphenation (which is still

+superior) the implementation of the hyphenation algorithm in \LUATEX\ is quite

+different from the one in \TEX82.

+

+After expansion, the argument for \prm {patterns} has to be proper \UTF8 with

+individual patterns separated by spaces, no \prm {char} or \prm {chardef}d

+commands are allowed. The current implementation is quite strict and will reject

+all non|-|\UNICODE\ characters. Likewise, the expanded argument for \prm

+{hyphenation} also has to be proper \UTF8, but here a bit of extra syntax is

+provided:

+

+\startitemize[n]

+\startitem

+    Three sets of arguments in curly braces (\type {{}{}{}}) indicate a desired

+    complex discretionary, with arguments as in \prm {discretionary}'s command in

+    normal document input.

+\stopitem

+\startitem

+    A \type {-} indicates a desired simple discretionary, cf.\ \type {\-} and

+    \type {\discretionary{-}{}{}} in normal document input.

+\stopitem

+\startitem

+    Internal command names are ignored. This rule is provided especially for \prm

+    {discretionary}, but it also helps to deal with \prm {relax} commands that

+    may sneak in.

+\stopitem

+\startitem

+    An \type {=} indicates a (non|-|discretionary) hyphen in the document input.

+\stopitem

+\stopitemize

+

+The expanded argument is first converted back to a space|-|separated string while

+dropping the internal command names. This string is then converted into a

+dictionary by a routine that creates key|-|value pairs by converting the other

+listed items. It is important to note that the keys in an exception dictionary

+can always be generated from the values. Here are a few examples:

+

+\starttabulate[|l|l|l|]

+\DB value                  \BC implied key (input) \BC effect \NC\NR

+\TB

+\NC \type {ta-ble}         \NC table               \NC \type {ta\-ble} ($=$ \type {ta\discretionary{-}{}{}ble}) \NC\NR

+\NC \type {ba{k-}{}{c}ken} \NC backen              \NC \type {ba\discretionary{k-}{}{c}ken} \NC\NR

+\LL

+\stoptabulate

+

+The resultant patterns and exception dictionary will be stored under the language

+code that is the present value of \prm {language}.

+

+In the last line of the table, you see there is no \prm {discretionary} command

+in the value: the command is optional in the \TEX-based input syntax. The

+underlying reason for that is that it is conceivable that a whole dictionary of

+words is stored as a plain text file and loaded into \LUATEX\ using one of the

+functions in the \LUA\ \type {lang} library. This loading method is quite a bit

+faster than going through the \TEX\ language primitives, but some (most?) of that

+speed gain would be lost if it had to interpret command sequences while doing so.

+

+It is possible to specify extra hyphenation points in compound words by using

+\type {{-}{}{-}} for the explicit hyphen character (replace \type {-} by the

+actual explicit hyphen character if needed). For example, this matches the word

+\quote {multi|-|word|-|boundaries} and allows an extra break inbetween \quote

+{boun} and \quote {daries}:

+

+\starttyping

+\hyphenation{multi{-}{}{-}word{-}{}{-}boun-daries}

+\stoptyping

+

+The motivation behind the \ETEX\ extension \prm {savinghyphcodes} was that

+hyphenation heavily depended on font encodings. This is no longer true in

+\LUATEX, and the corresponding primitive is basically ignored. Because we now

+have \lpr {hjcode}, the case relate codes can be used exclusively for \prm

+{uppercase} and \prm {lowercase}.

+

+The three curly brace pair pattern in an exception can be somewhat unexpected so

+we will try to explain it by example. The pattern \type {foo{}{}{x}bar} pattern

+creates a lookup \type {fooxbar} and the pattern \type {foo{}{}{}bar} creates

+\type {foobar}. Then, when a hit happens there is a replacement text (\type {x})

+or none. Because we introduced penalties in discretionary nodes, the exception

+syntax now also can take a penalty specification. The value between square brackets

+is a multiplier for \lpr {exceptionpenalty}. Here we have set it to 10000 so

+effectively we get 30000 in the example.

+

+\def\ShowSample#1#2%

+  {\startlinecorrection[blank]

+   \hyphenation{#1}%

+   \exceptionpenalty=10000

+   \bTABLE[foregroundstyle=type]

+     \bTR

+       \bTD[align=middle,nx=4] \type{#1} \eTD

+     \eTR

+     \bTR

+       \bTD[align=middle] \type{10em} \eTD

+       \bTD[align=middle] \type {3em} \eTD

+       \bTD[align=middle] \type {0em} \eTD

+       \bTD[align=middle] \type {6em} \eTD

+     \eTR

+     \bTR

+       \bTD[width=10em]\vtop{\hsize 10em 123 #2 123\par}\eTD

+       \bTD[width=10em]\vtop{\hsize  3em 123 #2 123\par}\eTD

+       \bTD[width=10em]\vtop{\hsize  0em 123 #2 123\par}\eTD

+       \bTD[width=10em]\vtop{\setupalign[verytolerant,stretch]\rmtf\hsize 6em 123 #2 #2 #2 #2 123\par}\eTD

+     \eTR

+   \eTABLE

+   \stoplinecorrection}

+

+\ShowSample{x{a-}{-b}{}x{a-}{-b}{}x{a-}{-b}{}x{a-}{-b}{}xx}{xxxxxx}

+\ShowSample{x{a-}{-b}{}x{a-}{-b}{}[3]x{a-}{-b}{}[1]x{a-}{-b}{}xx}{xxxxxx}

+

+\ShowSample{z{a-}{-b}{z}{a-}{-b}{z}{a-}{-b}{z}{a-}{-b}{z}z}{zzzzzz}

+\ShowSample{z{a-}{-b}{z}{a-}{-b}{z}[3]{a-}{-b}{z}[1]{a-}{-b}{z}z}{zzzzzz}

+

+\stopsection

+

+\startsection[title={Applying hyphenation}]

+

+\topicindex {hyphenation+how it works}

+\topicindex {hyphenation+discretionaries}

+\topicindex {discretionaries}

+

+The internal structures \LUATEX\ uses for the insertion of discretionaries in

+words is very different from the ones in \TEX82, and that means there are some

+noticeable differences in handling as well.

+

+First and foremost, there is no \quote {compressed trie} involved in hyphenation.

+The algorithm still reads pattern files generated by \PATGEN, but \LUATEX\ uses a

+finite state hash to match the patterns against the word to be hyphenated. This

+algorithm is based on the \quote {libhnj} library used by \OPENOFFICE, which in

+turn is inspired by \TEX.

+

+There are a few differences between \LUATEX\ and \TEX82 that are a direct result

+of the implementation:

+

+\startitemize

+\startitem

+    \LUATEX\ happily hyphenates the full \UNICODE\ character range.

+\stopitem

+\startitem

+    Pattern and exception dictionary size is limited by the available memory

+    only, all allocations are done dynamically. The trie|-|related settings in

+    \type {texmf.cnf} are ignored.

+\stopitem

+\startitem

+    Because there is no \quote {trie preparation} stage, language patterns never

+    become frozen. This means that the primitive \prm {patterns} (and its \LUA\

+    counterpart \type {lang.patterns}) can be used at any time, not only in

+    ini\TEX.

+\stopitem

+\startitem

+    Only the string representation of \prm {patterns} and \prm {hyphenation} is

+    stored in the format file. At format load time, they are simply

+    re|-|evaluated. It follows that there is no real reason to preload languages

+    in the format file. In fact, it is usually not a good idea to do so. It is

+    much smarter to load patterns no sooner than the first time they are actually

+    needed.

+\stopitem

+\startitem

+    \LUATEX\ uses the language-specific variables \lpr {prehyphenchar} and \lpr

+    {posthyphenchar} in the creation of implicit discretionaries, instead of

+    \TEX82's \prm {hyphenchar}, and the values of the language|-|specific

+    variables \lpr {preexhyphenchar} and \lpr {postexhyphenchar} for explicit

+    discretionaries (instead of \TEX82's empty discretionary).

+\stopitem

+\startitem

+    The value of the two counters related to hyphenation, \prm {hyphenpenalty}

+    and \prm {exhyphenpenalty}, are now stored in the discretionary nodes. This

+    permits a local overload for explicit \prm {discretionary} commands. The

+    value current when the hyphenation pass is applied is used. When no callbacks

+    are used this is compatible with traditional \TEX. When you apply the \LUA\

+    \type {lang.hyphenate} function the current values are used.

+\stopitem

+\startitem

+    The hyphenation exception dictionary is maintained as key|-|value hash, and

+    that is also dynamic, so the \type {hyph_size} setting is not used either.

+\stopitem

+\stopitemize

+

+Because we store penalties in the disc node the \prm {discretionary} command has

+been extended to accept an optional penalty specification, so you can do the

+following:

+

+\startbuffer

+\hsize1mm

+1:foo{\hyphenpenalty 10000\discretionary{}{}{}}bar\par

+2:foo\discretionary penalty 10000 {}{}{}bar\par

+3:foo\discretionary{}{}{}bar\par

+\stopbuffer

+

+\typebuffer

+

+This results in:

+

+\blank \start \getbuffer \stop \blank

+

+Inserted characters and ligatures inherit their attributes from the nearest glyph

+node item (usually the preceding one, but the following one for the items

+inserted at the left-hand side of a word).

+

+Word boundaries are no longer implied by font switches, but by language switches.

+One word can have two separate fonts and still be hyphenated correctly (but it

+can not have two different languages, the \prm {setlanguage} command forces a

+word boundary).

+

+All languages start out with \type {\prehyphenchar=`\-}, \type {\posthyphenchar=0},

+\type {\preexhyphenchar=0} and \type {\postexhyphenchar=0}. When you assign the

+values of one of these four parameters, you are actually changing the settings

+for the current \prm {language}, this behaviour is compatible with \prm {patterns}

+and \prm {hyphenation}.

+

+\LUATEX\ also hyphenates the first word in a paragraph. Words can be up to 256

+characters long (up from 64 in \TEX82). Longer words are ignored right now, but

+eventually either the limitation will be removed or perhaps it will become

+possible to silently ignore the excess characters (this is what happens in

+\TEX82, but there the behaviour cannot be controlled).

+

+If you are using the \LUA\ function \type {lang.hyphenate}, you should be aware

+that this function expects to receive a list of \quote {character} nodes. It will

+not operate properly in the presence of \quote {glyph}, \quote {ligature}, or

+\quote {ghost} nodes, nor does it know how to deal with kerning.

+

+\stopsection

+

+\startsection[title={Applying ligatures and kerning}]

+

+\topicindex {ligatures}

+\topicindex {kerning}

+

+After all possible hyphenation points have been inserted in the list, \LUATEX\

+will process the list to convert the \quote {character} nodes into \quote {glyph}

+and \quote {ligature} nodes. This is actually done in two stages: first all

+ligatures are processed, then all kerning information is applied to the result

+list. But those two stages are somewhat dependent on each other: If the used font

+makes it possible to do so, the ligaturing stage adds virtual \quote {character}

+nodes to the word boundaries in the list. While doing so, it removes and

+interprets \prm {noboundary} nodes. The kerning stage deletes those word

+boundary items after it is done with them, and it does the same for \quote

+{ghost} nodes. Finally, at the end of the kerning stage, all remaining \quote

+{character} nodes are converted to \quote {glyph} nodes.

+

+This word separation is worth mentioning because, if you overrule from \LUA\ only

+one of the two callbacks related to font handling, then you have to make sure you

+perform the tasks normally done by \LUATEX\ itself in order to make sure that the

+other, non|-|overruled, routine continues to function properly.

+

+Although we could improve the situation the reality is that in modern \OPENTYPE\

+fonts ligatures can be constructed in many ways: by replacing a sequence of

+characters by one glyph, or by selectively replacing individual glyphs, or by

+kerning, or any combination of this. Add to that contextual analysis and it will

+be clear that we have to let \LUA\ do that job instead. The generic font handler

+that we provide (which is part of \CONTEXT) distinguishes between base mode

+(which essentially is what we describe here and which delegates the task to \TEX)

+and node mode (which deals with more complex fonts.

+

+Let's look at an example. Take the word \type {office}, hyphenated \type

+{of-fice}, using a \quote {normal} font with all the \type {f}-\type {f} and

+\type {f}-\type {i} type ligatures:

+

+\starttabulate[|l|l|]

+\NC initial              \NC \type {{o}{f}{f}{i}{c}{e}}             \NC\NR

+\NC after hyphenation    \NC \type {{o}{f}{{-},{},{}}{f}{i}{c}{e}}  \NC\NR

+\NC first ligature stage \NC \type {{o}{{f-},{f},{<ff>}}{i}{c}{e}}  \NC\NR

+\NC final result         \NC \type {{o}{{f-},{<fi>},{<ffi>}}{c}{e}} \NC\NR

+\stoptabulate

+

+That's bad enough, but let us assume that there is also a hyphenation point

+between the \type {f} and the \type {i}, to create \type {of-f-ice}. Then the

+final result should be:

+

+\starttyping

+{o}{{f-},

+    {{f-},

+     {i},

+     {<fi>}},

+    {{<ff>-},

+     {i},

+     {<ffi>}}}{c}{e}

+\stoptyping

+

+with discretionaries in the post-break text as well as in the replacement text of

+the top-level discretionary that resulted from the first hyphenation point.

+

+Here is that nested solution again, in a different representation:

+

+\testpage[4]

+

+\starttabulate[|l|c|c|c|c|c|c|]

+\DB         \BC pre           \BC     \BC post      \BC       \BC replace       \BC       \NC \NR

+\TB

+\NC topdisc \NC \type {f-}    \NC (1) \NC           \NC sub 1 \NC               \NC sub 2 \NC \NR

+\NC sub 1   \NC \type {f-}    \NC (2) \NC \type {i} \NC (3)   \NC \type {<fi>}  \NC (4)   \NC \NR

+\NC sub 2   \NC \type {<ff>-} \NC (5) \NC \type {i} \NC (6)   \NC \type {<ffi>} \NC (7)   \NC \NR

+\LL

+\stoptabulate

+

+When line breaking is choosing its breakpoints, the following fields will

+eventually be selected:

+

+\starttabulate[|l|c|c|]

+\NC \type {of-f-ice} \NC \type {f-}    \NC (1) \NC \NR

+\NC                  \NC \type {f-}    \NC (2) \NC \NR

+\NC                  \NC \type {i}     \NC (3) \NC \NR

+\NC \type {of-fice}  \NC \type {f-}    \NC (1) \NC \NR

+\NC                  \NC \type {<fi>}  \NC (4) \NC \NR

+\NC \type {off-ice}  \NC \type {<ff>-} \NC (5) \NC \NR

+\NC                  \NC \type {i}     \NC (6) \NC \NR

+\NC \type {office}   \NC \type {<ffi>} \NC (7) \NC \NR

+\stoptabulate

+

+The current solution in \LUATEX\ is not able to handle nested discretionaries,

+but it is in fact smart enough to handle this fictional \type {of-f-ice} example.

+It does so by combining two sequential discretionary nodes as if they were a

+single object (where the second discretionary node is treated as an extension of

+the first node).

+

+One can observe that the \type {of-f-ice} and \type {off-ice} cases both end with

+the same actual post replacement list (\type {i}), and that this would be the

+case even if \type {i} was the first item of a potential following ligature like

+\type {ic}. This allows \LUATEX\ to do away with one of the fields, and thus make

+the whole stuff fit into just two discretionary nodes.

+

+The mapping of the seven list fields to the six fields in this discretionary node

+pair is as follows:

+

+\starttabulate[|l|c|c|]

+\DB field                 \BC description   \NC       \NC \NR

+\TB

+\NC \type {disc1.pre}     \NC \type {f-}    \NC (1)   \NC \NR

+\NC \type {disc1.post}    \NC \type {<fi>}  \NC (4)   \NC \NR

+\NC \type {disc1.replace} \NC \type {<ffi>} \NC (7)   \NC \NR

+\NC \type {disc2.pre}     \NC \type {f-}    \NC (2)   \NC \NR

+\NC \type {disc2.post}    \NC \type {i}     \NC (3,6) \NC \NR

+\NC \type {disc2.replace} \NC \type {<ff>-} \NC (5)   \NC \NR

+\LL

+\stoptabulate

+

+What is actually generated after ligaturing has been applied is therefore:

+

+\starttyping

+{o}{{f-},

+    {<fi>},

+    {<ffi>}}

+   {{f-},

+    {i},

+    {<ff>-}}{c}{e}

+\stoptyping

+

+The two discretionaries have different subtypes from a discretionary appearing on

+its own: the first has subtype 4, and the second has subtype 5. The need for

+these special subtypes stems from the fact that not all of the fields appear in

+their \quote {normal} location. The second discretionary especially looks odd,

+with things like the \type {<ff>-} appearing in \type {disc2.replace}. The fact

+that some of the fields have different meanings (and different processing code

+internally) is what makes it necessary to have different subtypes: this enables

+\LUATEX\ to distinguish this sequence of two joined discretionary nodes from the

+case of two standalone discretionaries appearing in a row.

+

+Of course there is still that relationship with fonts: ligatures can be implemented by

+mapping a sequence of glyphs onto one glyph, but also by selective replacement and

+kerning. This means that the above examples are just representing the traditional

+approach.

+

+\stopsection

+

+\startsection[title={Breaking paragraphs into lines}]

+

+\topicindex {linebreaks}

+\topicindex {paragraphs}

+\topicindex {discretionaries}

+

+This code is almost unchanged, but because of the above|-|mentioned changes

+with respect to discretionaries and ligatures, line breaking will potentially be

+different from traditional \TEX. The actual line breaking code is still based on

+the \TEX82 algorithms, and it does not expect there to be discretionaries inside

+of discretionaries. But, as patterns evolve and font handling can influence

+discretionaries, you need to be aware of the fact that long term consistency is not

+an engine matter only.

+

+But that situation is now fairly common in \LUATEX, due to the changes to the

+ligaturing mechanism. And also, the \LUATEX\ discretionary nodes are implemented

+slightly different from the \TEX82 nodes: the \type {no_break} text is now

+embedded inside the disc node, where previously these nodes kept their place in

+the horizontal list. In traditional \TEX\ the discretionary node contains a

+counter indicating how many nodes to skip, but in \LUATEX\ we store the pre, post

+and replace text in the discretionary node.

+

+The combined effect of these two differences is that \LUATEX\ does not always use

+all of the potential breakpoints in a paragraph, especially when fonts with many

+ligatures are used. Of course kerning also complicates matters here.

+

+\stopsection

+

+\startsection[title={The \type {lang} library}][library=lang]

+

+\subsection {\type {new} and \type {id}}

+

+\topicindex {languages+library}

+

+\libindex {new}

+\libindex {id}

+

+This library provides the interface to \LUATEX's structure representing a

+language, and the associated functions.

+

+\startfunctioncall

+<language> l = lang.new()

+<language> l = lang.new(<number> id)

+\stopfunctioncall

+

+This function creates a new userdata object. An object of type \type {<language>}

+is the first argument to most of the other functions in the \type {lang} library.

+These functions can also be used as if they were object methods, using the colon

+syntax. Without an argument, the next available internal id number will be

+assigned to this object. With argument, an object will be created that links to

+the internal language with that id number.

+

+\startfunctioncall

+<number> n = lang.id(<language> l)

+\stopfunctioncall

+

+The number returned is the internal \prm {language} id number this object refers to.

+

+\subsection {\type {hyphenation}}

+

+\libindex {hyphenation}

+

+You can hyphenate a string directly with:

+

+\startfunctioncall

+<string> n = lang.hyphenation(<language> l)

+lang.hyphenation(<language> l, <string> n)

+\stopfunctioncall

+

+\subsection {\type {clear_hyphenation} and \type {clean}}

+

+\libindex {clear_hyphenation}

+\libindex {clean}

+

+This either returns the current hyphenation exceptions for this language, or adds

+new ones. The syntax of the string is explained in~\in {section}

+[patternsexceptions].

+

+\startfunctioncall

+lang.clear_hyphenation(<language> l)

+\stopfunctioncall

+

+This call clears the exception dictionary (string) for this language.

+

+\startfunctioncall

+<string> n = lang.clean(<language> l, <string> o)

+<string> n = lang.clean(<string> o)

+\stopfunctioncall

+

+This function creates a hyphenation key from the supplied hyphenation value. The

+syntax of the argument string is explained in \in {section} [patternsexceptions].

+This function is useful if you want to do something else based on the words in a

+dictionary file, like spell|-|checking.

+

+\subsection {\type {patterns} and \type {clear_patterns}}

+

+\libindex {patterns}

+\libindex {clear_patterns}

+

+\startfunctioncall

+<string> n = lang.patterns(<language> l)

+lang.patterns(<language> l, <string> n)

+\stopfunctioncall

+

+This adds additional patterns for this language object, or returns the current

+set. The syntax of this string is explained in \in {section}

+[patternsexceptions].

+

+\startfunctioncall

+lang.clear_patterns(<language> l)

+\stopfunctioncall

+

+This can be used to clear the pattern dictionary for a language.

+

+\subsection {\type {hyphenationmin}}

+

+\libindex {hyphenationmin}

+

+This function sets (or gets) the value of the \TEX\ parameter

+\type {\hyphenationmin}.

+

+\startfunctioncall

+n = lang.hyphenationmin(<language> l)

+lang.hyphenationmin(<language> l, <number> n)

+\stopfunctioncall

+

+\subsection {\type {[pre|post][ex|]hyphenchar}}

+

+\libindex {prehyphenchar}

+\libindex {posthyphenchar}

+\libindex {preexhyphenchar}

+\libindex {postexhyphenchar}

+

+\startfunctioncall

+<number> n = lang.prehyphenchar(<language> l)

+lang.prehyphenchar(<language> l, <number> n)

+

+<number> n = lang.posthyphenchar(<language> l)

+lang.posthyphenchar(<language> l, <number> n)

+\stopfunctioncall

+

+These two are used to get or set the \quote {pre|-|break} and \quote

+{post|-|break} hyphen characters for implicit hyphenation in this language. The

+intial values are decimal 45 (hyphen) and decimal~0 (indicating emptiness).

+

+\startfunctioncall

+<number> n = lang.preexhyphenchar(<language> l)

+lang.preexhyphenchar(<language> l, <number> n)

+

+<number> n = lang.postexhyphenchar(<language> l)

+lang.postexhyphenchar(<language> l, <number> n)

+\stopfunctioncall

+

+These gets or set the \quote {pre|-|break} and \quote {post|-|break} hyphen

+characters for explicit hyphenation in this language. Both are initially

+decimal~0 (indicating emptiness).

+

+\subsection {\type {hyphenate}}

+

+\libindex {hyphenate}

+

+The next call inserts hyphenation points (discretionary nodes) in a node list. If

+\type {tail} is given as argument, processing stops on that node. Currently,

+\type {success} is always true if \type {head} (and \type {tail}, if specified)

+are proper nodes, regardless of possible other errors.

+

+\startfunctioncall

+<boolean> success = lang.hyphenate(<node> head)

+<boolean> success = lang.hyphenate(<node> head, <node> tail)

+\stopfunctioncall

+

+Hyphenation works only on \quote {characters}, a special subtype of all the glyph

+nodes with the node subtype having the value \type {1}. Glyph modes with

+different subtypes are not processed. See \in {section} [charsandglyphs] for

+more details.

+

+\subsection {\type {[set|get]hjcode}}

+

+\libindex {sethjcode}

+\libindex {gethjcode}

+

+The following two commands can be used to set or query hj codes:

+

+\startfunctioncall

+lang.sethjcode(<language> l, <number> char, <number> usedchar)

+<number> usedchar = lang.gethjcode(<language> l, <number> char)

+\stopfunctioncall

+

+When you set a hjcode the current sets get initialized unless the set was already

+initialized due to \prm {savinghyphcodes} being larger than zero.

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

+

+% \parindent0pt \hsize=1.1cm

+% 12-34-56 \par

+% 12-34-\hbox{56} \par

+% 12-34-\vrule width 1em height 1.5ex \par

+% 12-\hbox{34}-56 \par

+% 12-\vrule width 1em height 1.5ex-56 \par

+% \hjcode`\1=`\1 \hjcode`\2=`\2 \hjcode`\3=`\3 \hjcode`\4=`\4 \vskip.5cm

+% 12-34-56 \par

+% 12-34-\hbox{56} \par

+% 12-34-\vrule width 1em height 1.5ex \par

+% 12-\hbox{34}-56 \par

+% 12-\vrule width 1em height 1.5ex-56 \par

+

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-logos.tex b/Master/texmf-dist/doc/luatex/base/luatex-logos.tex
index 7406dd60206..d67ce5c0b16 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-logos.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-logos.tex
@@ -1,19 +1,21 @@
-\startenvironment luatex-logos
-
-\logo[DFONT]     {dfont}
-\logo[CFF]       {cff}
-\logo[CMAP]      {CMap}
-\logo[PATGEN]    {patgen}
-\logo[MP]        {MetaPost}
-\logo[METAPOST]  {MetaPost}
-\logo[MPLIB]     {MPlib}
-\logo[COCO]      {coco}
-\logo[SUNOS]     {SunOS}
-\logo[BSD]       {bsd}
-\logo[SYSV]      {sysv}
-\logo[DPI]       {dpi}
-\logo[DLL]       {dll}
-\logo[OPENOFFICE]{OpenOffice}
-\logo[OCP]       {OCP}
-
-\stopenvironment
+\startenvironment luatex-logos

+

+\usemodule[abr-02]

+

+\logo[DFONT]     {dfont}

+\logo[CFF]       {cff}

+\logo[CMAP]      {CMap}

+\logo[PATGEN]    {patgen}

+\logo[MP]        {MetaPost}

+\logo[METAPOST]  {MetaPost}

+\logo[MPLIB]     {MPlib}

+\logo[COCO]      {coco}

+\logo[SUNOS]     {SunOS}

+\logo[BSD]       {bsd}

+\logo[SYSV]      {sysv}

+\logo[DPI]       {dpi}

+\logo[DLL]       {dll}

+\logo[OPENOFFICE]{OpenOffice}

+\logo[OCP]       {OCP}

+

+\stopenvironment

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-lua.tex b/Master/texmf-dist/doc/luatex/base/luatex-lua.tex
index 82b060440b1..3d6d20c35dd 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-lua.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-lua.tex
@@ -1,596 +1,760 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-lua
-
-\startchapter[reference=lua,title={\LUA\ general}]
-
-\section[init]{Initialization}
-
-\subsection{\LUATEX\ as a \LUA\ interpreter}
-
-There are some situations that make \LUATEX\ behave like a standalone \LUA\
-interpreter:
-
-\startitemize[packed]
-\startitem
-    if a \type {--luaonly} option is given on the commandline, or
-\stopitem
-\startitem
-    if the executable is named \type {texlua} or \type {luatexlua}, or
-\stopitem
-\startitem
-    if the only non|-|option argument (file) on the commandline has the extension
-    \type {lua} or \type {luc}.
-\stopitem
-\stopitemize
-
-In this mode, it will set \LUA's \type {arg[0]} to the found script name, pushing
-preceding options in negative values and the rest of the command line in the
-positive values, just like the \LUA\ interpreter.
-
-\LUATEX\ will exit immediately after executing the specified \LUA\ script and is,
-in effect, a somewhat bulky stand alone \LUA\ interpreter with a bunch of extra
-preloaded libraries.
-
-\subsection{\LUATEX\ as a \LUA\ byte compiler}
-
-There are two situations that make \LUATEX\ behave like the \LUA\ byte compiler:
-
-\startitemize[packed]
-\startitem if a \type {--luaconly} option is given on the command line, or \stopitem
-\startitem if the executable is named \type {texluac} \stopitem
-\stopitemize
-
-In this mode, \LUATEX\ is exactly like \type {luac} from the stand alone \LUA\
-distribution, except that it does not have the \type {-l} switch, and that it
-accepts (but ignores) the \type {--luaconly} switch.
-
-\subsection{Other commandline processing}
-
-When the \LUATEX\ executable starts, it looks for the \type {--lua} command line
-option. If there is no \type {--lua} option, the command line is interpreted in a
-similar fashion as the other \TEX\ engines. Some options are accepted but have no
-consequence. The following command|-|line options are understood:
-
-\starttabulate[|l|p|]
-\NC \type{--credits}                    \NC display credits and exit \NC \NR
-\NC \type{--debug-format}               \NC enable format debugging \NC \NR
-\NC \type{--draftmode}                  \NC switch on draft mode i.e.\ generate no output in \PDF\ mode \NC \NR
-\NC \type{--[no-]file-line-error}       \NC disable/enable \type {file:line:error} style messages \NC \NR
-\NC \type{--[no-]file-line-error-style} \NC aliases of \type {--[no-]file-line-error} \NC \NR
-\NC \type{--fmt=FORMAT}                 \NC load the format file \type {FORMAT} \NC\NR
-\NC \type{--halt-on-error}              \NC stop processing at the first error\NC \NR
-\NC \type{--help}                       \NC display help and exit \NC\NR
-\NC \type{--ini}                        \NC be \type {iniluatex}, for dumping formats \NC\NR
-\NC \type{--interaction=STRING}         \NC set interaction mode: \type {batchmode}, \type {nonstopmode}, \type {scrollmode} or \type {errorstopmode} \NC \NR
-\NC \type{--jobname=STRING}             \NC set the job name to \type {STRING} \NC \NR
-\NC \type{--kpathsea-debug=NUMBER}      \NC set path searching debugging flags according to the bits of \type {NUMBER} \NC \NR
-\NC \type{--lua=FILE}                   \NC load and execute a \LUA\ initialization script \NC\NR
-\NC \type{--[no-]mktex=FMT}             \NC disable/enable \type {mktexFMT} generation with \type {FMT} is \type {tex} or \type {tfm} \NC \NR
-\NC \type{--nosocket}                   \NC disable the \LUA\ socket library \NC\NR
-\NC \type{--output-comment=STRING}      \NC use \type {STRING} for \DVI\ file comment instead of date (no effect for \PDF) \NC \NR
-\NC \type{--output-directory=DIR}       \NC use \type {DIR} as the directory to write files to \NC \NR
-\NC \type{--output-format=FORMAT}       \NC use \type {FORMAT} for job output; \type {FORMAT} is \type {dvi} or \type {pdf} \NC \NR
-\NC \type{--progname=STRING}            \NC set the program name to \type {STRING} \NC \NR
-\NC \type{--recorder}                   \NC enable filename recorder \NC \NR
-\NC \type{--safer}                      \NC disable easily exploitable \LUA\ commands \NC\NR
-\NC \type{--[no-]shell-escape}          \NC disable/enable system calls \NC \NR
-\NC \type{--shell-restricted}           \NC restrict system calls to a list of commands given in \type {texmf.cnf} \NC \NR
-\NC \type{--synctex=NUMBER}             \NC enable \type {synctex} \NC \NR
-\NC \type{--utc}                        \NC use utc times when applicable \NC \NR
-\NC \type{--version}                    \NC display version and exit \NC \NR
-\stoptabulate
-
-Some of the traditional flags are just ignored: \type {--etex}, \type
-{--translate-file}, \type {--8bit}. \type {--[no-]parse-first-line}, \type
-{--default-translate-file}. Also, we no longer support write18 because \type
-{os.execute} can do the same.
-
-The value to use for \type {\jobname} is decided as follows:
-
-\startitemize
-\startitem
-    If \type {--jobname} is given on the command line, its argument will be the
-    value for \type {\jobname}, without any changes. The argument will not be
-    used for actual input so it need not exist. The \type {--jobname} switch only
-    controls the \type {\jobname} setting.
-\stopitem
-\startitem
-    Otherwise, \type {\jobname} will be the name of the first file that is read
-    from the file system, with any path components and the last extension (the
-    part following the last \type {.}) stripped off.
-\stopitem
-\startitem
-    An exception to the previous point: if the command line goes into interactive
-    mode (by starting with a command) and there are no files input via \type
-    {\everyjob} either, then the \type {\jobname} is set to \type {texput} as a
-    last resort.
-\stopitem
-\stopitemize
-
-The file names for output files that are generated automatically are created by
-attaching the proper extension (\type {log}, \type {pdf}, etc.) to the found
-\type {\jobname}. These files are created in the directory pointed to by \type
-{--output-directory}, or in the current directory, if that switch is not present.
-
-\blank
-
-Without the \type {--lua} option, command line processing works like it does in
-any other web2c-based typesetting engine, except that \LUATEX\ has a few extra
-switches.
-
-If the \type {--lua} option is present, \LUATEX\ will enter an alternative mode
-of command line processing in comparison to the standard web2c programs.
-
-In this mode, a small series of actions is taken in order. First, it will parse
-the command line as usual, but it will only interpret a small subset of the
-options immediately: \type {--safer}, \type {--nosocket}, \type
-{--[no-]shell-escape}, \type {--enable-write18}, \type {--disable-write18}, \type
-{--shell-restricted}, \type {--help}, \type {--version}, and \type {--credits}.
-
-Next \LUATEX\ searches for the requested \LUA\ initialization script. If it
-cannot be found using the actual name given on the command line, a second attempt
-is made by prepending the value of the environment variable \type {LUATEXDIR}, if
-that variable is defined in the environment.
-
-Then it checks the various safety switches. You can use those to disable some
-\LUA\ commands that can easily be abused by a malicious document. At the moment,
-\type {--safer} \type {nil}s the following functions:
-
-\starttabulate[|l|l|]
-\BC library      \BC functions \NC \NR
-\NC \type {os}   \NC \type {execute} \type {exec} \type {spawn} \type {setenv} \type {rename} \type {remove} \type {tmpdir} \NC \NR
-\NC \type {io}   \NC \type {popen} \type {output} \type {tmpfile} \NC \NR
-\NC \type {lfs}  \NC \type {rmdir} \type {mkdir} \type {chdir} \type {lock} \type {touch} \NC \NR
-\stoptabulate
-
-Furthermore, it disables loading of compiled \LUA\ libraries and it makes \type
-{io.open()} fail on files that are opened for anything besides reading.
-
-When \LUATEX\ starts it set the locale to a neutral value. If for some reason you
-use \type {os.locale}, you need to make sure you \type {nil} it afterwards
-because otherwise it can interfere with code that for instance generates dates.
-You can nil the locale with
-
-\starttyping
-os.setlocale(nil.nil)
-\stoptyping
-
-The \type {--nosocket} option makes the socket library unavailable, so that \LUA\
-cannot use networking.
-
-The switches \type {--[no-]shell-escape}, \type {--[enable|disable]-write18}, and
-\type {--shell-restricted} have the same effects as in \PDFTEX, and additionally
-make \type {io.popen()}, \type {os.execute}, \type {os.exec} and \type {os.spawn}
-adhere to the requested option.
-
-Next the initialization script is loaded and executed. From within the script,
-the entire command line is available in the \LUA\ table \type {arg}, beginning with
-\type {arg[0]}, containing the name of the executable. As consequence warnings
-about unrecognized options are suppressed.
-
-Command line processing happens very early on. So early, in fact, that none of
-\TEX's initializations have taken place yet. For that reason, the tables that
-deal with typesetting, like \type {tex}, \type {token}, \type {node} and
-\type {pdf}, are off|-|limits during the execution of the startup file (they
-are \type {nil}'d). Special care is taken that \type {texio.write} and \type
-{texio.write_nl} function properly, so that you can at least report your actions
-to the log file when (and if) it eventually becomes opened (note that \TEX\ does
-not even know its \type {\jobname} yet at this point).
-
-Everything you do in the \LUA\ initialization script will remain visible during
-the rest of the run, with the exception of the \TEX\ specific libraries like
-\type {tex}, \type {token}, \type {node} and \type {pdf} tables. These will be
-initialized to their documented state after the execution of the script. You
-should not store anything in variables or within tables with these four global
-names, as they will be overwritten completely.
-
-We recommend you use the startup file only for your own \TEX|-|independent
-initializations (if you need any), to parse the command line, set values in the
-\type {texconfig} table, and register the callbacks you need.
-
-\LUATEX\ allows some of the command line options to be overridden by reading
-values from the \type {texconfig} table at the end of script execution (see the
-description of the \type {texconfig} table later on in this document for more
-details on which ones exactly).
-
-Unless the \type {texconfig} table tells \LUATEX\ not to initialize \KPATHSEA\
-at all (set \type {texconfig.kpse_init} to \type {false} for that), \LUATEX\
-acts on some more command line options after the initialization script is
-finished: in order to initialize the built|-|in \KPATHSEA\ library properly,
-\LUATEX\ needs to know the correct program name to use, and for that it needs to
-check \type {--progname}, or \type {--ini} and \type {--fmt}, if \type
-{--progname} is missing.
-
-\section{\LUA\ behaviour}
-
-\LUA s \type {tostring} function (and \type {string.format} may return values in
-scientific notation, thereby confusing the \TEX\ end of things when it is used as
-the right|-|hand side of an assignment to a \type {\dimen} or \type {\count}.
-
-Loading dynamic \LUA\ libraries will fail if there are two \LUA\ libraries loaded
-at the same time (which will typically happen on \type {win32}, because there is
-one \LUA\ 5.3 inside \LUATEX, and another will likely be linked to the \DLL\ file
-of the module itself).
-
-\LUATEX\ is able to use the kpathsea library to find \type {require()}d modules.
-For this purpose, \type {package.searchers[2]} is replaced by a different loader
-function, that decides at runtime whether to use kpathsea or the built|-|in core
-\LUA\ function. It uses \KPATHSEA\ when that is already initialized at that point
-in time, otherwise it reverts to using the normal \type {package.path} loader.
-
-Initialization of \KPATHSEA\ can happen either implicitly (when \LUATEX\ starts
-up and the startup script has not set \type {texconfig.kpse_init} to false), or
-explicitly by calling the \LUA\ function \type {kpse.set_program_name()}.
-
-\LUATEX\ is able to use dynamically loadable \LUA\ libraries, unless
-\type {--safer} was given as an option on the command line. For this purpose,
-\type {package.searchers[3]} is replaced by a different loader function, that
-decides at runtime whether to use \KPATHSEA\ or the built|-|in core \LUA\
-function. It uses \KPATHSEA\ when that is already initialized at that point in
-time, otherwise it reverts to using the normal \type {package.cpath} loader.
-
-This functionality required an extension to kpathsea:
-
-\startnarrower
-There is a new kpathsea file format: \type {kpse_clua_format} that searches for
-files with extension \type {.dll} and \type {.so}. The \type {texmf.cnf} setting
-for this variable is \type {CLUAINPUTS}, and by default it has this value:
-
-\starttyping
-CLUAINPUTS=.:$SELFAUTOLOC/lib/{$progname,$engine,}/lua//
-\stoptyping
-
-This path is imperfect (it requires a \TDS\ subtree below the binaries
-directory), but the architecture has to be in the path somewhere, and the
-currently simplest way to do that is to search below the binaries directory only.
-Of course it no big deal to write an alternative loader and use that in a macro
-package.
-
-One level up (a \type {lib} directory parallel to \type {bin}) would have been
-nicer, but that is not doable because \TEXLIVE\ uses a \type {bin/<arch>}
-structure.
-\stopnarrower
-
-In keeping with the other \TEX|-|like programs in \TEXLIVE, the two \LUA\ functions
-\type {os.execute} and \type {io.popen}, as well as the two new functions \type
-{os.exec} and \type {os.spawn} that are explained below, take the value of \type
-{shell_escape} and|/|or \type {shell_escape_commands} in account. Whenever
-\LUATEX\ is run with the assumed intention to typeset a document (and by that we
-mean that it is called as \type {luatex}, as opposed to \type {texlua}, and that
-the command line option \type {--luaonly} was not given), it will only run the
-four functions above if the matching \type {texmf.cnf} variable(s) or their \type
-{texconfig} (see \in {section} [texconfig]) counterparts allow execution of the
-requested system command. In \quote {script interpreter} runs of \LUATEX, these
-settings have no effect, and all four functions function as normal.
-
-The \type {f:read("*line")} and \type {f:lines()} functions from the io library
-have been adjusted so that they are line|-|ending neutral: any of \type {LF},
-\type {CR} or \type {CR+LF} are acceptable line endings.
-
-\type {luafilesystem} has been extended: there are two extra boolean functions
-(\type {lfs.isdir(filename)} and \type {lfs.isfile(filename)}) and one extra
-string field in its attributes table (\type {permissions}). There is an
-additional function \type {lfs.shortname()} which takes a file name and returns
-its short name on \type {win32} platforms. On other platforms, it just returns
-the given argument. The file name is not tested for existence. Finally, for
-non|-|\type {win32} platforms only, there is the new function \type
-{lfs.readlink()} hat takes an existing symbolic link as argument and returns its
-content. It returns an error on \type {win32}.
-
-The \type {string} library has an extra function: \type {string.explode(s[,m])}.
-This function returns an array containing the string argument \type {s} split
-into sub-strings based on the value of the string argument \type {m}. The second
-argument is a string that is either empty (this splits the string into
-characters), a single character (this splits on each occurrence of that
-character, possibly introducing empty strings), or a single character followed by
-the plus sign \type {+} (this special version does not create empty sub-strings).
-The default value for \type {m} is \quote {\type { +}} (multiple spaces). Note:
-\type {m} is not hidden by surrounding braces as it would be if this function was
-written in \TEX\ macros.
-
-The \type {string} library also has six extra iterators that return strings
-piecemeal:
-
-\startitemize
-\startitem
-    \type {string.utfvalues(s)}: an integer value in the \UNICODE\ range
-\stopitem
-\startitem
-    \type {string.utfcharacters(s)}: a string with a single \UTF-8 token in it
-\stopitem
-\startitem
-    \type {string.characters(s)}: a string containing one byte
-\stopitem
-\startitem
-    \type {string.characterpairs(s)}: two strings each containing one byte or an
-    empty second string if the string length was odd
-\stopitem
-\startitem
-    \type {string.bytes(s)}: a single byte value
-\stopitem
-\startitem
-    \type {string.bytepairs(s)}: two byte values or nil instead of a number as
-    its second return value if the string length was odd
-\stopitem
-\stopitemize
-
-The \type {string.characterpairs()} and \type {string.bytepairs()} iterators
-are useful especially in the conversion of \UTF16 encoded data into \UTF8.
-
-There is also a two|-|argument form of \type {string.dump()}. The second argument
-is a boolean which, if true, strips the symbols from the dumped data. This
-matches an extension made in \type {luajit}.
-
-The \type {string} library functions \type {len}, \type {lower}, \type {sub}
-etc.\ are not \UNICODE|-|aware. For strings in the \UTF8 encoding, i.e., strings
-containing characters above code point 127, the corresponding functions from the
-\type {slnunicode} library can be used, e.g., \type {unicode.utf8.len}, \type
-{unicode.utf8.lower} etc.\ The exceptions are \type {unicode.utf8.find}, that
-always returns byte positions in a string, and \type {unicode.utf8.match} and
-\type {unicode.utf8.gmatch}. While the latter two functions in general {\it
-are} \UNICODE|-|aware, they fall|-|back to non|-|\UNICODE|-|aware behavior when
-using the empty capture \type {()} but other captures work as expected. For the
-interpretation of character classes in \type {unicode.utf8} functions refer to
-the library sources at \hyphenatedurl {http://luaforge.net/projects/sln}.
-
-Version 5.3 of \LUA\ provides some native \UTF8 support but we have added a few
-similar helpers too:
-
-\startitemize
-\startitem
-    \type {string.utfvalue(s)}: returns the codepoints of the characters in the
-    given string
-\stopitem
-\startitem
-    \type {string.utfcharacter(c,...)}: returns a string with the characters of
-    the given code points
-\stopitem
-\startitem
-    \type {string.utflength(s)}: returns the length oif the given string
-\stopitem
-\stopitemize
-
-These three functions are relative fast and don't do much checking. They can be used
-as building blocks for other helpers.
-
-
-\blank
-
-The \type {os} library has a few extra functions and variables:
-
-\startitemize
-
-\startitem
-    \type {os.selfdir} is a variable that holds the directory path of the
-    actual executable. For example: \type {\directlua {tex.sprint(os.selfdir)}}.
-\stopitem
-
-\startitem
-    \type {os.exec(commandline)} is a variation on \type {os.execute}. Here
-    \type {commandline} can be either a single string or a single table.
-
-    If the argument is a table \LUATEX\ first checks if there is a value at
-    integer index zero. If there is, this is the command to be executed.
-    Otherwise, it will use the value at integer index one. If neither are
-    present, nothing at all happens.
-
-    The set of consecutive values starting at integer~1 in the table are the
-    arguments that are passed on to the command (the value at index~1 becomes
-    \type {arg[0]}). The command is searched for in the execution path, so there
-    is normally no need to pass on a fully qualified path name.
-
-    If the argument is a string, then it is automatically converted into a table
-    by splitting on whitespace. In this case, it is impossible for the command
-    and first argument to differ from each other.
-
-    In the string argument format, whitespace can be protected by putting (part
-    of) an argument inside single or double quotes. One layer of quotes is
-    interpreted by \LUATEX, and all occurrences of \type {\"}, \type {\'} or \type
-    {\\} within the quoted text are unescaped. In the table format, there is no
-    string handling taking place.
-
-    This function normally does not return control back to the \LUA\ script: the
-    command will replace the current process. However, it will return the two
-    values \type {nil} and \type {error} if there was a problem while
-    attempting to execute the command.
-
-    On \MSWINDOWS, the current process is actually kept in memory until after the
-    execution of the command has finished. This prevents crashes in situations
-    where \TEXLUA\ scripts are run inside integrated \TEX\ environments.
-
-    The original reason for this command is that it cleans out the current
-    process before starting the new one, making it especially useful for use in
-    \TEXLUA.
-\stopitem
-
-\startitem
-    \type {os.spawn(commandline)} is a returning version of \type {os.exec},
-    with otherwise identical calling conventions.
-
-    If the command ran ok, then the return value is the exit status of the
-    command. Otherwise, it will return the two values \type {nil} and \type
-    {error}.
-\stopitem
-
-\startitem
-    \type {os.setenv(key,value)} sets a variable in the environment. Passing
-    \type {nil} instead of a value string will remove the variable.
-\stopitem
-
-\startitem
-    \type {os.env} is a hash table containing a dump of the variables and
-    values in the process environment at the start of the run. It is writeable,
-    but the actual environment is \notabene {not} updated automatically.
-\stopitem
-
-\startitem
-    \type {os.gettimeofday()} returns the current \quote {\UNIX\ time}, but as a
-    float. This function is not available on the \SUNOS\ platforms, so do not use
-    this function for portable documents.
-\stopitem
-
-\startitem
-    \type {os.times()}returns the current process times according to \ the
-    \UNIX\ C library function \quote {times}. This function is not available on
-    the \MSWINDOWS\ and \SUNOS\ platforms, so do not use this function for
-    portable documents.
-\stopitem
-
-\startitem
-    \type {os.tmpdir()} creates a directory in the \quote {current directory}
-    with the name \type {luatex.XXXXXX} where the \type {X}-es are replaced by a
-    unique string. The function also returns this string, so you can \type
-    {lfs.chdir()} into it, or \type {nil} if it failed to create the directory.
-    The user is responsible for cleaning up at the end of the run, it does not
-    happen automatically.
-\stopitem
-
-\startitem
-    \type {os.type} is a string that gives a global indication of the class of
-    operating system. The possible values are currently \type {windows}, \type
-    {unix}, and \type {msdos} (you are unlikely to find this value \quote {in the
-    wild}).
-\stopitem
-
-\startitem
-    \type {os.name} is a string that gives a more precise indication of the
-    operating system. These possible values are not yet fixed, and for \type
-    {os.type} values \type {windows} and \type {msdos}, the \type {os.name}
-    values are simply \type {windows} and \type {msdos}
-
-    The list for the type \type {unix} is more precise: \type {linux}, \type
-    {freebsd}, \type {kfreebsd}, \type {cygwin}, \type {openbsd}, \type
-    {solaris}, \type {sunos} (pre-solaris), \type {hpux}, \type {irix}, \type
-    {macosx}, \type {gnu} (hurd), \type {bsd} (unknown, but \BSD|-|like), \type
-    {sysv} (unknown, but \SYSV|-|like), \type {generic} (unknown).
-\stopitem
-
-\startitem
-    \type {os.uname()} returns a table with specific operating system
-    information acquired at runtime. The keys in the returned table are all
-    string valued, and their names are: \type {sysname}, \type {machine}, \type
-    {release}, \type {version}, and \type {nodename}.
-\stopitem
-
-\stopitemize
-
-In stock \LUA, many things depend on the current locale. In \LUATEX, we can't do
-that, because it makes documents unportable.  While \LUATEX\ is running if
-forces the following locale settings:
-
-\starttyping
-LC_CTYPE=C
-LC_COLLATE=C
-LC_NUMERIC=C
-\stoptyping
-
-\section {\LUA\ modules}
-
-Some modules that are normally external to \LUA\ are statically linked in with
-\LUATEX, because they offer useful functionality:
-
-\startitemize
-
-\startitem
-    \type {slnunicode}, from the \type {selene} libraries, \hyphenatedurl
-    {http://luaforge.net/projects/sln}. This library has been slightly extended
-    so that the \type {unicode.utf8.*} functions also accept the first 256 values
-    of plane~18. This is the range \LUATEX\ uses for raw binary output, as
-    explained above.
-\stopitem
-
-\startitem
-    \type {luazip}, from the kepler project, \hyphenatedurl
-    {http://www.keplerproject.org/luazip/}.
-\stopitem
-
-\startitem
-    \type {luafilesystem}, also from the kepler project, \hyphenatedurl
-    {http://www.keplerproject.org/luafilesystem/}.
-\stopitem
-
-\startitem
-    \type {lpeg}, by Roberto Ierusalimschy, \hyphenatedurl
-    {http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html}. This library is not
-    \UNICODE|-|aware, but interprets strings on a byte|-|per|-|byte basis. This
-    mainly means that \type {lpeg.S} cannot be used with \UTF8 characters encoded
-    in more than two bytes, and thus \type {lpeg.S} will look for one of those
-    two bytes when matching, not the combination of the two. The same is true for
-    \type {lpeg.R}, although the latter will display an error message if used
-    with multibyte characters. Therefore \type {lpeg.R('aä')} results in the
-    message \type {bad argument #1 to 'R' (range must have two characters)},
-    since to \type {lpeg}, \type {ä} is two 'characters' (bytes), so \type {aä}
-    totals three. In practice this is no real issue.
-\stopitem
-
-\startitem
-    \type {lzlib}, by Tiago Dionizio, \hyphenatedurl
-    {http://luaforge.net/projects/lzlib/}.
-\stopitem
-
-\startitem
-    \type {md5}, by Roberto Ierusalimschy \hyphenatedurl
-    {http://www.inf.puc-rio.br/~roberto/md5/md5-5/md5.html}.
-\stopitem
-
-\startitem
-    \type {luasocket}, by Diego Nehab \hyphenatedurl
-    {http://w3.impa.br/~diego/software/luasocket/}. The \type {.lua} support
-    modules from \type {luasocket} are also preloaded inside the executable,
-    there are no external file dependencies.
-\stopitem
-
-\stopitemize
-
-At some point (this also depends on distributions) \LUATEX\ might have these
-libraries loaded on demand. For this reason you can best use \type {require} to
-make sure they are loaded.
-
-\section{Testing}
-
-For development reasons you can influence the used startup date and time. This can
-be done in two ways.
-
-\startitemize[n]
-
-\startitem
-    By setting the environmment variable \type {SOURCE_DATE_EPOCH}. This will
-    influence the \TEX\ parameters \type {time} and \type {date}, the random seed,
-    the \PDF\ timestamp and the \PDF\ id that is derived from the time as well. This
-    variable is consulted when the \KPSE\ library is enabled. Resolving is
-    delegated to this library.
-\stopitem
-
-\startitem
-    By setting the \type {start_time} variable in the \type {texconfig} table; as
-    with other variables we use the internal name there. For compatibility
-    reasons we also honour a \type {SOURCE_DATE_EPOCH} entry. It should be noted
-    that there are no such variables in other engines and this method is only
-    relevant in case the while setup happens in \LUA.
-\stopitem
-
-\stopitemize
-
-When Universal Time is needed, you can pass the flag \type {utc} to the engine. This
-property also works when the date and time are set by \LUATEX\ itself. It has a
-complementary entry \type {use_utc_time} in the \type {texconfig} table.
-
-\startnotabene
-    To some extend a cleaner solution would be to have a flag that disables all
-    variable data in one go (like filenames and so) but we just follow the method
-    implemented in \PDFTEX\ where primitives are used to influence other
-    properties.
-\stopnotabene
-
-\startnotabene
-    In \CONTEXT\ we provide the command line argument \type {--nodates} that does
-    bit more disabling of dates.
-\stopnotabene
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-lua

+

+\startchapter[reference=lua,title={Using \LUATEX}]

+

+\startsection[title={Initialization},reference=init]

+

+\startsubsection[title={\LUATEX\ as a \LUA\ interpreter}]

+

+\topicindex {initialization}

+\topicindex {\LUA+interpreter}

+

+There are some situations that make \LUATEX\ behave like a standalone \LUA\

+interpreter:

+

+\startitemize[packed]

+\startitem

+    if a \type {--luaonly} option is given on the commandline, or

+\stopitem

+\startitem

+    if the executable is named \type {texlua} or \type {luatexlua}, or

+\stopitem

+\startitem

+    if the only non|-|option argument (file) on the commandline has the extension

+    \type {lua} or \type {luc}.

+\stopitem

+\stopitemize

+

+In this mode, it will set \LUA's \type {arg[0]} to the found script name, pushing

+preceding options in negative values and the rest of the command line in the

+positive values, just like the \LUA\ interpreter.

+

+\LUATEX\ will exit immediately after executing the specified \LUA\ script and is,

+in effect, a somewhat bulky stand alone \LUA\ interpreter with a bunch of extra

+preloaded libraries.

+

+\stopsubsection

+

+\startsubsection[title={\LUATEX\ as a \LUA\ byte compiler}]

+

+\topicindex {\LUA+byte code}

+

+There are two situations that make \LUATEX\ behave like the \LUA\ byte compiler:

+

+\startitemize[packed]

+\startitem if a \type {--luaconly} option is given on the command line, or \stopitem

+\startitem if the executable is named \type {texluac} \stopitem

+\stopitemize

+

+In this mode, \LUATEX\ is exactly like \type {luac} from the stand alone \LUA\

+distribution, except that it does not have the \type {-l} switch, and that it

+accepts (but ignores) the \type {--luaconly} switch. The current version of \LUA\

+can dump bytecode using \type {string.dump} so we might decide to drop this

+version of \LUATEX.

+

+\stopsubsection

+

+\startsubsection[title={Other commandline processing}]

+

+\topicindex {command line}

+

+When the \LUATEX\ executable starts, it looks for the \type {--lua} command line

+option. If there is no \type {--lua} option, the command line is interpreted in a

+similar fashion as the other \TEX\ engines. Some options are accepted but have no

+consequence. The following command|-|line options are understood:

+

+\starttabulate[|l|p|]

+\DB commandline argument                \BC explanation \NC \NR

+\TB

+\NC \type{--credits}                    \NC display credits and exit \NC \NR

+\NC \type{--debug-format}               \NC enable format debugging \NC \NR

+\NC \type{--draftmode}                  \NC switch on draft mode i.e.\ generate no output in \PDF\ mode \NC \NR

+\NC \type{--[no-]file-line-error}       \NC disable/enable \type {file:line:error} style messages \NC \NR

+\NC \type{--[no-]file-line-error-style} \NC aliases of \type {--[no-]file-line-error} \NC \NR

+\NC \type{--fmt=FORMAT}                 \NC load the format file \type {FORMAT} \NC\NR

+\NC \type{--halt-on-error}              \NC stop processing at the first error\NC \NR

+\NC \type{--help}                       \NC display help and exit \NC\NR

+\NC \type{--ini}                        \NC be \type {iniluatex}, for dumping formats \NC\NR

+\NC \type{--interaction=STRING}         \NC set interaction mode: \type {batchmode}, \type {nonstopmode},

+                                            \type {scrollmode} or \type {errorstopmode} \NC \NR

+\NC \type{--jobname=STRING}             \NC set the job name to \type {STRING} \NC \NR

+\NC \type{--kpathsea-debug=NUMBER}      \NC set path searching debugging flags according to the bits of

+                                           \type {NUMBER} \NC \NR

+\NC \type{--lua=FILE}                   \NC load and execute a \LUA\ initialization script \NC\NR

+\NC \type{--[no-]mktex=FMT}             \NC disable/enable \type {mktexFMT} generation with \type {FMT} is

+                                            \type {tex} or \type {tfm} \NC \NR

+\NC \type{--nosocket}                   \NC disable the \LUA\ socket library \NC\NR

+\NC \type{--output-comment=STRING}      \NC use \type {STRING} for \DVI\ file comment instead of date (no

+                                            effect for \PDF) \NC \NR

+\NC \type{--output-directory=DIR}       \NC use \type {DIR} as the directory to write files to \NC \NR

+\NC \type{--output-format=FORMAT}       \NC use \type {FORMAT} for job output; \type {FORMAT} is \type {dvi}

+                                            or \type {pdf} \NC \NR

+\NC \type{--progname=STRING}            \NC set the program name to \type {STRING} \NC \NR

+\NC \type{--recorder}                   \NC enable filename recorder \NC \NR

+\NC \type{--safer}                      \NC disable easily exploitable \LUA\ commands \NC\NR

+\NC \type{--[no-]shell-escape}          \NC disable/enable system calls \NC \NR

+\NC \type{--shell-restricted}           \NC restrict system calls to a list of commands given in \type

+                                            {texmf.cnf} \NC \NR

+\NC \type{--synctex=NUMBER}             \NC enable \type {synctex} \NC \NR

+\NC \type{--utc}                        \NC use utc times when applicable \NC \NR

+\NC \type{--version}                    \NC display version and exit \NC \NR

+\LL

+\stoptabulate

+

+We don't support \prm {write} 18 because \type {os.execute} can do the same. It

+simplifies the code and makes more write targets possible.

+

+The value to use for \prm {jobname} is decided as follows:

+

+\startitemize

+\startitem

+    If \type {--jobname} is given on the command line, its argument will be the

+    value for \prm {jobname}, without any changes. The argument will not be

+    used for actual input so it need not exist. The \type {--jobname} switch only

+    controls the \prm {jobname} setting.

+\stopitem

+\startitem

+    Otherwise, \prm {jobname} will be the name of the first file that is read

+    from the file system, with any path components and the last extension (the

+    part following the last \type {.}) stripped off.

+\stopitem

+\startitem

+    There is an exception to the previous point: if the command line goes into

+    interactive mode (by starting with a command) and there are no files input

+    via \prm {everyjob} either, then the \prm {jobname} is set to \type

+    {texput} as a last resort.

+\stopitem

+\stopitemize

+

+The file names for output files that are generated automatically are created by

+attaching the proper extension (\type {log}, \type {pdf}, etc.) to the found

+\prm {jobname}. These files are created in the directory pointed to by \type

+{--output-directory}, or in the current directory, if that switch is not present.

+

+Without the \type {--lua} option, command line processing works like it does in

+any other \WEBC|-|based typesetting engine, except that \LUATEX\ has a few extra

+switches and lacks some others. Also, if the \type {--lua} option is present,

+\LUATEX\ will enter an alternative mode of command line processing in comparison

+to the standard \WEBC\ programs. In this mode, a small series of actions is taken

+in the following order:

+

+\startitemize[n]

+

+\startitem

+    First, it will parse the command line as usual, but it will only interpret a

+    small subset of the options immediately: \type {--safer}, \type {--nosocket},

+    \type {--[no-]shell-escape}, \type {--enable-write18}, \type

+    {--disable-write18}, \type {--shell-restricted}, \type {--help}, \type

+    {--version}, and \type {--credits}.

+\stopitem

+

+\startitem

+    Next \LUATEX\ searches for the requested \LUA\ initialization script. If it

+    cannot be found using the actual name given on the command line, a second

+    attempt is made by prepending the value of the environment variable \type

+    {LUATEXDIR}, if that variable is defined in the environment.

+\stopitem

+

+\startitem

+    Then it checks the various safety switches. You can use those to disable some

+    \LUA\ commands that can easily be abused by a malicious document. At the

+    moment, \type {--safer} \type {nil}s the following functions:

+

+    \blank

+

+    \starttabulate[|c|l|]

+        \DB library     \BC functions \NC \NR

+        \TB

+        \NC \type {os}  \NC \type {execute} \type {exec} \type {spawn} \type {setenv}

+                            \type {rename} \type {remove} \type {tmpdir} \NC \NR

+        \NC \type {io}  \NC \type {popen} \type {output} \type {tmpfile} \NC \NR

+        \NC \type {lfs} \NC \type {rmdir} \type {mkdir} \type {chdir} \type {lock}

+                            \type {touch} \NC \NR

+        \LL

+    \stoptabulate

+

+    \blank

+

+    Furthermore, it disables loading of compiled \LUA\ libraries and it makes

+    \type {io.open()} fail on files that are opened for anything besides reading.

+\stopitem

+

+\startitem

+    When \LUATEX\ starts it sets the \type {locale} to a neutral value. If for

+    some reason you use \type {os.locale}, you need to make sure you \type {nil}

+    it afterwards because otherwise it can interfere with code that for instance

+    generates dates. You can ignore the \type {locale} with:

+

+    \starttyping

+    os.setlocale(nil,nil)

+    \stoptyping

+

+    The \type {--nosocket} option makes the socket library unavailable, so that \LUA\

+    cannot use networking.

+

+    The switches \type {--[no-]shell-escape}, \type {--[enable|disable]-write18}, and

+    \type {--shell-restricted} have the same effects as in \PDFTEX, and additionally

+    make \type {io.popen()}, \type {os.execute}, \type {os.exec} and \type {os.spawn}

+    adhere to the requested option.

+\stopitem

+

+\startitem

+    Next the initialization script is loaded and executed. From within the

+    script, the entire command line is available in the \LUA\ table \type {arg},

+    beginning with \type {arg[0]}, containing the name of the executable. As

+    consequence warnings about unrecognized options are suppressed.

+\stopitem

+

+\stopitemize

+

+Command line processing happens very early on. So early, in fact, that none of

+\TEX's initializations have taken place yet. For that reason, the tables that

+deal with typesetting, like \type {tex}, \type {token}, \type {node} and

+\type {pdf}, are off|-|limits during the execution of the startup file (they

+are \type {nil}'d). Special care is taken that \type {texio.write} and \type

+{texio.write_nl} function properly, so that you can at least report your actions

+to the log file when (and if) it eventually becomes opened (note that \TEX\ does

+not even know its \prm {jobname} yet at this point).

+

+Everything you do in the \LUA\ initialization script will remain visible during

+the rest of the run, with the exception of the \TEX\ specific libraries like

+\type {tex}, \type {token}, \type {node} and \type {pdf} tables. These will be

+initialized to their documented state after the execution of the script. You

+should not store anything in variables or within tables with these four global

+names, as they will be overwritten completely.

+

+We recommend you use the startup file only for your own \TEX|-|independent

+initializations (if you need any), to parse the command line, set values in the

+\type {texconfig} table, and register the callbacks you need.

+

+\LUATEX\ allows some of the command line options to be overridden by reading

+values from the \type {texconfig} table at the end of script execution (see the

+description of the \type {texconfig} table later on in this document for more

+details on which ones exactly).

+

+Unless the \type {texconfig} table tells \LUATEX\ not to initialize \KPATHSEA\

+at all (set \type {texconfig.kpse_init} to \type {false} for that), \LUATEX\

+acts on some more command line options after the initialization script is

+finished: in order to initialize the built|-|in \KPATHSEA\ library properly,

+\LUATEX\ needs to know the correct program name to use, and for that it needs to

+check \type {--progname}, or \type {--ini} and \type {--fmt}, if \type

+{--progname} is missing.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={\LUA\ behaviour}]

+

+\startsubsection[title={The \LUA\ version}]

+

+\topicindex {\LUA+libraries}

+\topicindex {\LUA+extensions}

+

+We currently use \LUA\ 5.3 and will follow developments of the language but

+normally with some delay. Therefore the user needs to keep an eye on (subtle)

+differences in successive versions of the language. Also, \LUAJITTEX\ lags behind

+in the sense that \LUAJIT\ is not in sync with regular \LUA\ development. Here is

+an example of one aspect.

+

+\LUA s \type {tostring} function (and \type {string.format} may return values in

+scientific notation, thereby confusing the \TEX\ end of things when it is used as

+the right|-|hand side of an assignment to a \prm {dimen} or \prm {count}. The

+output of these serializers also depend on the \LUA\ version, so in \LUA\ 5.3 you

+can get different output than from 5.2.

+

+\stopsubsection

+

+\startsubsection[title={Integration in the \TDS\ ecosystem}]

+

+The main \TEX\ distributions follow the \TEX\ directory structure (\TDS).

+\LUATEX\ is able to use the kpathsea library to find \type {require()}d modules.

+For this purpose, \type {package.searchers[2]} is replaced by a different loader

+function, that decides at runtime whether to use kpathsea or the built|-|in core

+\LUA\ function. It uses \KPATHSEA\ when that is already initialized at that point

+in time, otherwise it reverts to using the normal \type {package.path} loader.

+

+Initialization of \KPATHSEA\ can happen either implicitly (when \LUATEX\ starts

+up and the startup script has not set \type {texconfig.kpse_init} to false), or

+explicitly by calling the \LUA\ function \type {kpse.set_program_name()}.

+

+\stopsubsection

+

+\startsubsection[title={Loading libraries}]

+

+\LUATEX\ is able to use dynamically loadable \LUA\ libraries, unless

+\type {--safer} was given as an option on the command line. For this purpose,

+\type {package.searchers[3]} is replaced by a different loader function, that

+decides at runtime whether to use \KPATHSEA\ or the built|-|in core \LUA\

+function. It uses \KPATHSEA\ when that is already initialized at that point in

+time, otherwise it reverts to using the normal \type {package.cpath} loader.

+

+This functionality required an extension to kpathsea. There is a new kpathsea

+file format: \type {kpse_clua_format} that searches for files with extension

+\type {.dll} and \type {.so}. The \type {texmf.cnf} setting for this variable is

+\type {CLUAINPUTS}, and by default it has this value:

+

+\starttyping

+CLUAINPUTS=.:$SELFAUTOLOC/lib/{$progname,$engine,}/lua//

+\stoptyping

+

+This path is imperfect (it requires a \TDS\ subtree below the binaries

+directory), but the architecture has to be in the path somewhere, and the

+currently simplest way to do that is to search below the binaries directory only.

+Of course it no big deal to write an alternative loader and use that in a macro

+package. One level up (a \type {lib} directory parallel to \type {bin}) would

+have been nicer, but that is not doable because \TEXLIVE\ uses a \type

+{bin/<arch>} structure.

+

+Loading dynamic \LUA\ libraries will fail if there are two \LUA\ libraries loaded

+at the same time (which will typically happen on \type {win32}, because there is

+one \LUA\ 5.3 inside \LUATEX, and another will likely be linked to the \DLL\ file

+of the module itself).

+

+\stopsubsection

+

+\startsubsection[title={Executing programs}]

+

+In keeping with the other \TEX|-|like programs in \TEXLIVE, the two \LUA\ functions

+\type {os.execute} and \type {io.popen}, as well as the two new functions \type

+{os.exec} and \type {os.spawn} that are explained below, take the value of \type

+{shell_escape} and|/|or \type {shell_escape_commands} in account. Whenever

+\LUATEX\ is run with the assumed intention to typeset a document (and by that we

+mean that it is called as \type {luatex}, as opposed to \type {texlua}, and that

+the command line option \type {--luaonly} was not given), it will only run the

+four functions above if the matching \type {texmf.cnf} variable(s) or their \type

+{texconfig} (see \in {section} [texconfig]) counterparts allow execution of the

+requested system command. In \quote {script interpreter} runs of \LUATEX, these

+settings have no effect, and all four functions have their original meaning.

+

+Some libraries have a few more functions, either coded in \CCODE\ or in \LUA. For

+instance, when we started with \LUATEX\ we added some helpers to the \type

+{luafilesystem} namespace \type {lfs}. The two boolean functions \type

+{lfs.isdir} and \type {lfs.isfile} were speedy and better variants of what could

+be done with \type {lfs.attributes}. The additional function \type

+{lfs.shortname} takes a file name and returns its short name on \type {win32}

+platforms. Finally, for non|-|\type {win32} platforms only, we provided \type

+{lfs.readlink} that takes an existing symbolic link as argument and returns its

+name. However, the \type library evolved so we have dropped these in favour of

+pure \LUA\ variants. The \type {shortname} helper is obsolete and now just

+returns the name.

+

+\stopsubsection

+

+\startsubsection[title={Multibyte \type {string} functions}]

+

+The \type {string} library has a few extra functions, for example \libidx

+{string} {explode}. This function takes upto two arguments: \type

+{string.explode(s[,m])} and returns an array containing the string argument \type

+{s} split into sub-strings based on the value of the string argument \type {m}.

+The second argument is a string that is either empty (this splits the string into

+characters), a single character (this splits on each occurrence of that

+character, possibly introducing empty strings), or a single character followed by

+the plus sign \type {+} (this special version does not create empty sub-strings).

+The default value for \type {m} is \quote {\type { +}} (multiple spaces). Note:

+\type {m} is not hidden by surrounding braces as it would be if this function was

+written in \TEX\ macros.

+

+The \type {string} library also has six extra iterators that return strings

+piecemeal: \libidx {string} {utfvalues}, \libidx {string} {utfcharacters},

+\libidx {string} {characters}, \libidx {string} {characterpairs}, \libidx

+{string} {bytes} and \libidx {string} {bytepairs}.

+

+\startitemize

+\startitem

+    \type {string.utfvalues(s)}: an integer value in the \UNICODE\ range

+\stopitem

+\startitem

+    \type {string.utfcharacters(s)}: a string with a single \UTF-8 token in it

+\stopitem

+\startitem

+    \type {string.cWharacters(s)}: a string containing one byte

+\stopitem

+\startitem

+    \type {string.characterpairs(s)}: two strings each containing one byte or an

+    empty second string if the string length was odd

+\stopitem

+\startitem

+    \type {string.bytes(s)}: a single byte value

+\stopitem

+\startitem

+    \type {string.bytepairs(s)}: two byte values or nil instead of a number as

+    its second return value if the string length was odd

+\stopitem

+\stopitemize

+

+The \type {string.characterpairs()} and \type {string.bytepairs()} iterators

+are useful especially in the conversion of \UTF16 encoded data into \UTF8.

+

+There is also a two|-|argument form of \type {string.dump()}. The second argument

+is a boolean which, if true, strips the symbols from the dumped data. This

+matches an extension made in \type {luajit}. This is typically a function that

+gets adapted as \LUA\ itself progresses.

+

+The \type {string} library functions \type {len}, \type {lower}, \type {sub}

+etc.\ are not \UNICODE|-|aware. For strings in the \UTF8 encoding, i.e., strings

+containing characters above code point 127, the corresponding functions from the

+\type {slnunicode} library can be used, e.g., \type {unicode.utf8.len}, \type

+{unicode.utf8.lower} etc.\ The exceptions are \type {unicode.utf8.find}, that

+always returns byte positions in a string, and \type {unicode.utf8.match} and

+\type {unicode.utf8.gmatch}. While the latter two functions in general {\it

+are} \UNICODE|-|aware, they fall|-|back to non|-|\UNICODE|-|aware behavior when

+using the empty capture \type {()} but other captures work as expected. For the

+interpretation of character classes in \type {unicode.utf8} functions refer to

+the library sources at \hyphenatedurl {http://luaforge.net/projects/sln}.

+

+Version 5.3 of \LUA\ provides some native \UTF8 support but we have added a few

+similar helpers too: \libidx {string} {utfvalue}, \libidx {string} {utfcharacter}

+and \libidx {string} {utflength}.

+

+\startitemize

+\startitem

+    \type {string.utfvalue(s)}: returns the codepoints of the characters in the

+    given string

+\stopitem

+\startitem

+    \type {string.utfcharacter(c,...)}: returns a string with the characters of

+    the given code points

+\stopitem

+\startitem

+    \type {string.utflength(s)}: returns the length oif the given string

+\stopitem

+\stopitemize

+

+These three functions are relative fast and don't do much checking. They can be

+used as building blocks for other helpers.

+

+\stopsubsection

+

+\startsubsection[title={Extra \type {os} library functions}]

+

+The \type {os} library has a few extra functions and variables: \libidx {os}

+{selfdir}, \libidx {os} {exec}, \libidx {os} {spawn}, \libidx {os} {setenv},

+\libidx {os} {env}, \libidx {os} {gettimeofday}, \libidx {os} {times}, \libidx

+{os} {tmpdir}, \libidx {os} {type}, \libidx {os} {name} and \libidx {os} {uname},

+that we will discuss here.

+

+\startitemize

+

+\startitem

+    \type {os.selfdir} is a variable that holds the directory path of the

+    actual executable. For example: \type {\directlua {tex.sprint(os.selfdir)}}.

+\stopitem

+

+\startitem

+    \type {os.exec(commandline)} is a variation on \type {os.execute}. Here

+    \type {commandline} can be either a single string or a single table.

+

+    \startitemize

+

+    \startitem

+        If the argument is a table \LUATEX\ first checks if there is a value at

+        integer index zero. If there is, this is the command to be executed.

+        Otherwise, it will use the value at integer index one. If neither are

+        present, nothing at all happens.

+    \stopitem

+

+    \startitem

+        The set of consecutive values starting at integer~1 in the table are the

+        arguments that are passed on to the command (the value at index~1 becomes

+        \type {arg[0]}). The command is searched for in the execution path, so

+        there is normally no need to pass on a fully qualified path name.

+    \stopitem

+

+    \startitem

+        If the argument is a string, then it is automatically converted into a

+        table by splitting on whitespace. In this case, it is impossible for the

+        command and first argument to differ from each other.

+    \stopitem

+

+    \startitem

+        In the string argument format, whitespace can be protected by putting

+        (part of) an argument inside single or double quotes. One layer of quotes

+        is interpreted by \LUATEX, and all occurrences of \type {\"}, \type {\'}

+        or \type {\\} within the quoted text are unescaped. In the table format,

+        there is no string handling taking place.

+    \stopitem

+

+    \stopitemize

+

+    This function normally does not return control back to the \LUA\ script: the

+    command will replace the current process. However, it will return the two

+    values \type {nil} and \type {error} if there was a problem while

+    attempting to execute the command.

+

+    On \MSWINDOWS, the current process is actually kept in memory until after the

+    execution of the command has finished. This prevents crashes in situations

+    where \TEXLUA\ scripts are run inside integrated \TEX\ environments.

+

+    The original reason for this command is that it cleans out the current

+    process before starting the new one, making it especially useful for use in

+    \TEXLUA.

+\stopitem

+

+\startitem

+    \type {os.spawn(commandline)} is a returning version of \type {os.exec},

+    with otherwise identical calling conventions.

+

+    If the command ran ok, then the return value is the exit status of the

+    command. Otherwise, it will return the two values \type {nil} and \type

+    {error}.

+\stopitem

+

+\startitem

+    \type {os.setenv(key,value)} sets a variable in the environment. Passing

+    \type {nil} instead of a value string will remove the variable.

+\stopitem

+

+\startitem

+    \type {os.env} is a hash table containing a dump of the variables and

+    values in the process environment at the start of the run. It is writeable,

+    but the actual environment is \notabene {not} updated automatically.

+\stopitem

+

+\startitem

+    \type {os.gettimeofday()} returns the current \quote {\UNIX\ time}, but as a

+    float. This function is not available on the \SUNOS\ platforms, so do not use

+    this function for portable documents.

+\stopitem

+

+\startitem

+    \type {os.times()}returns the current process times according to \ the

+    \UNIX\ C library function \quote {times}. This function is not available on

+    the \MSWINDOWS\ and \SUNOS\ platforms, so do not use this function for

+    portable documents.

+\stopitem

+

+\startitem

+    \type {os.tmpdir()} creates a directory in the \quote {current directory}

+    with the name \type {luatex.XXXXXX} where the \type {X}-es are replaced by a

+    unique string. The function also returns this string, so you can \type

+    {lfs.chdir()} into it, or \type {nil} if it failed to create the directory.

+    The user is responsible for cleaning up at the end of the run, it does not

+    happen automatically.

+\stopitem

+

+\startitem

+    \type {os.type} is a string that gives a global indication of the class of

+    operating system. The possible values are currently \type {windows}, \type

+    {unix}, and \type {msdos} (you are unlikely to find this value \quote {in the

+    wild}).

+\stopitem

+

+\startitem

+    \type {os.name} is a string that gives a more precise indication of the

+    operating system. These possible values are not yet fixed, and for \type

+    {os.type} values \type {windows} and \type {msdos}, the \type {os.name}

+    values are simply \type {windows} and \type {msdos}

+

+    The list for the type \type {unix} is more precise: \type {linux}, \type

+    {freebsd}, \type {kfreebsd}, \type {cygwin}, \type {openbsd}, \type

+    {solaris}, \type {sunos} (pre-solaris), \type {hpux}, \type {irix}, \type

+    {macosx}, \type {gnu} (hurd), \type {bsd} (unknown, but \BSD|-|like), \type

+    {sysv} (unknown, but \SYSV|-|like), \type {generic} (unknown).

+\stopitem

+

+\startitem

+    \type {os.uname} returns a table with specific operating system

+    information acquired at runtime. The keys in the returned table are all

+    string values, and their names are: \type {sysname}, \type {machine}, \type

+    {release}, \type {version}, and \type {nodename}.

+\stopitem

+

+\stopitemize

+

+\stopsubsection

+

+\startsubsection[title={Binary input from files with \type {fio}}]

+

+There is a whole set of helpers for reading numbers and strings from a file:

+\libidx {fio} {readcardinal1}, \libidx {fio} {readcardinal2}, \libidx {fio}

+{readcardinal3}, \libidx {fio} {readcardinal4}, \libidx {fio}

+{readcardinaltable}, \libidx {fio} {readinteger1}, \libidx {fio} {readinteger2},

+\libidx {fio} {readinteger3}, \libidx {fio} {readinteger4}, \libidx {fio}

+{readintegertable}, \libidx {fio} {readfixed2}, \libidx {fio} {readfixed4},

+\libidx {fio} {read2dot14}, \libidx {fio} {setposition}, \libidx {fio}

+{getposition}, \libidx {fio} {skipposition}, \libidx {fio} {readbytes}, \libidx

+{fio} {readbytetable}. They work on normal \LUA\ file handles.

+

+%libidx{fio}{readline}

+%libidx{fio}{recordfilename}

+%libidx{fio}{checkpermission}

+

+This library provides a set of functions for reading numbers from a file and

+in addition to the regular \type {io} library functions.

+

+\starttabulate

+\NC \type{readcardinal1(f)}         \NC a 1 byte unsigned integer \NC \NR

+\NC \type{readcardinal2(f)}         \NC a 2 byte unsigned integer \NC \NR

+\NC \type{readcardinal3(f)}         \NC a 3 byte unsigned integer \NC \NR

+\NC \type{readcardinal4(f)}         \NC a 4 byte unsigned integer \NC \NR

+\NC \type{readcardinaltable(f,n,b)} \NC \type {n} cardinals of \type {b} bytes \NC \NR

+\NC \type{readinteger1(f)}          \NC a 1 byte signed integer \NC \NR

+\NC \type{readinteger2(f)}          \NC a 2 byte signed integer \NC \NR

+\NC \type{readinteger3(f)}          \NC a 3 byte signed integer \NC \NR

+\NC \type{readinteger4(f)}          \NC a 4 byte signed integer \NC \NR

+\NC \type{readintegertable(f,n,b)}  \NC \type {n} integers of \type {b} bytes \NC \NR

+\NC \type{readfixed2(f)}            \NC a 2 byte float (used in font files) \NC \NR

+\NC \type{readfixed4(f)}            \NC a 4 byte float (used in font files) \NC \NR

+\NC \type{read2dot14(f)}            \NC a 2 byte float (used in font files) \NC \NR

+\NC \type{setposition(f,p)}         \NC goto position \type {p} \NC \NR

+\NC \type{getposition(f)}           \NC get the current position \NC \NR

+\NC \type{skipposition(f,n)}        \NC skip \type {n} positions \NC \NR

+\NC \type{readbytes(f,n)}           \NC \type {n} bytes \NC \NR

+\NC \type{readbytetable(f,n)}       \NC \type {n} bytes\NC \NR

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={Binary input from strings with \type {sio}}]

+

+A similar set of function as in the \type {fio} library is available in the \type

+{sio} library: \libidx {sio} {readcardinal1}, \libidx {sio} {readcardinal2},

+\libidx {sio} {readcardinal3}, \libidx {sio} {readcardinal4}, \libidx {sio}

+{readcardinaltable}, \libidx {sio} {readinteger1}, \libidx {sio} {readinteger2},

+\libidx {sio} {readinteger3}, \libidx {sio} {readinteger4}, \libidx {sio}

+{readintegertable}, \libidx {sio} {readfixed2}, \libidx {sio} {readfixed4},

+\libidx {sio} {read2dot14}, \libidx {sio} {setposition}, \libidx {sio}

+{getposition}, \libidx {sio} {skipposition}, \libidx {sio} {readbytes} and

+\libidx {sio} {readbytetable}. Here the first argument is a string instead of a

+file handle. More details can be found in the previous section.

+

+\stopsubsection

+

+\startsubsection[title={Hashes conform \type {sha2}}]

+

+This library is a side effect of the \type {pdfe} library that needs such

+helpers. The \libidx{sha2}{digest256}, \libidx{sha2}{digest384} and

+\libidx{sha2}{digest512} functions accept a string and return a string with the

+hash.

+

+\stopsubsection

+

+\startsubsection[title={Locales}]

+

+\index {locales}

+

+In stock \LUA, many things depend on the current locale. In \LUATEX, we can't do

+that, because it makes documents unportable.  While \LUATEX\ is running if

+forces the following locale settings:

+

+\starttyping

+LC_CTYPE=C

+LC_COLLATE=C

+LC_NUMERIC=C

+\stoptyping

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={\LUA\ modules}]

+

+\topicindex {\LUA+libraries}

+\topicindex {\LUA+modules}

+

+Some modules that are normally external to \LUA\ are statically linked in with

+\LUATEX, because they offer useful functionality:

+

+\startitemize

+

+\startitem

+    \type {lpeg}, by Roberto Ierusalimschy, \hyphenatedurl

+    {http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html}. This library is not

+    \UNICODE|-|aware, but interprets strings on a byte|-|per|-|byte basis. This

+    mainly means that \type {lpeg.S} cannot be used with \UTF8 characters encoded

+    in more than two bytes, and thus \type {lpeg.S} will look for one of those

+    two bytes when matching, not the combination of the two. The same is true for

+    \type {lpeg.R}, although the latter will display an error message if used

+    with multibyte characters. Therefore \type {lpeg.R('aä')} results in the

+    message \type {bad argument #1 to 'R' (range must have two characters)},

+    since to \type {lpeg}, \type {ä} is two 'characters' (bytes), so \type {aä}

+    totals three. In practice this is no real issue and with some care you can

+    deal with \UNICODE\ just fine.

+\stopitem

+

+\startitem

+    \type {slnunicode}, from the \type {selene} libraries, \hyphenatedurl

+    {http://luaforge.net/projects/sln}. This library has been slightly extended

+    so that the \type {unicode.utf8.*} functions also accept the first 256 values

+    of plane~18. This is the range \LUATEX\ uses for raw binary output, as

+    explained above. We have no plans to provide more like this because you can

+    basically do all that you want in \LUA.

+\stopitem

+

+\startitem

+    \type {luazip}, from the kepler project, \hyphenatedurl

+    {http://www.keplerproject.org/luazip/}.

+\stopitem

+

+\startitem

+    \type {luafilesystem}, also from the kepler project, \hyphenatedurl

+    {http://www.keplerproject.org/luafilesystem/}.

+\stopitem

+

+\startitem

+    \type {lzlib}, by Tiago Dionizio, \hyphenatedurl

+    {http://luaforge.net/projects/lzlib/}.

+\stopitem

+

+\startitem

+    \type {md5}, by Roberto Ierusalimschy \hyphenatedurl

+    {http://www.inf.puc-rio.br/~roberto/md5/md5-5/md5.html}.

+\stopitem

+

+\startitem

+    \type {luasocket}, by Diego Nehab \hyphenatedurl

+    {http://w3.impa.br/~diego/software/luasocket/}. The \type {.lua} support

+    modules from \type {luasocket} are also preloaded inside the executable,

+    there are no external file dependencies.

+\stopitem

+

+\stopitemize

+

+\stopsection

+

+\startsection[title={Testing}]

+

+\topicindex {testing}

+\topicindex {\PDF+date}

+

+For development reasons you can influence the used startup date and time. This can

+be done in two ways.

+

+\startitemize[n]

+

+\startitem

+    By setting the environmment variable \type {SOURCE_DATE_EPOCH}. This will

+    influence the \TEX\ parameters \type {time} and \type {date}, the random seed,

+    the \PDF\ timestamp and the \PDF\ id that is derived from the time as well. This

+    variable is consulted when the \KPSE\ library is enabled. Resolving is

+    delegated to this library.

+\stopitem

+

+\startitem

+    By setting the \type {start_time} variable in the \type {texconfig} table; as

+    with other variables we use the internal name there. For compatibility

+    reasons we also honour a \type {SOURCE_DATE_EPOCH} entry. It should be noted

+    that there are no such variables in other engines and this method is only

+    relevant in case the while setup happens in \LUA.

+\stopitem

+

+\stopitemize

+

+When Universal Time is needed, you can pass the flag \type {utc} to the engine. This

+property also works when the date and time are set by \LUATEX\ itself. It has a

+complementary entry \type {use_utc_time} in the \type {texconfig} table.

+

+There is some control possible, for instance prevent filename to be written to

+the \PDF\ file. This is discussed elsewhere. In \CONTEXT\ we provide the command

+line argument \type {--nodates} that does a bit more disabling of dates.

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-math.tex b/Master/texmf-dist/doc/luatex/base/luatex-math.tex
index f29a3aed0a6..fd038150f59 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-math.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-math.tex
@@ -1,1432 +1,1647 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-math
-
-\startchapter[reference=math,title={Math}]
-
-The handling of mathematics in \LUATEX\ differs quite a bit from how \TEX82 (and
-therefore \PDFTEX) handles math. First, \LUATEX\ adds primitives and extends some
-others so that \UNICODE\ input can be used easily. Second, all of \TEX82's
-internal special values (for example for operator spacing) have been made
-accessible and changeable via control sequences. Third, there are extensions that
-make it easier to use \OPENTYPE\ math fonts. And finally, there are some
-extensions that have been proposed or considered in the past that are now added
-to the engine.
-
-\section{The current math style}
-
-It is possible to discover the math style that will be used for a formula in an
-expandable fashion (while the math list is still being read). To make this
-possible, \LUATEX\ adds the new primitive: \type {\mathstyle}. This is a \quote
-{convert command} like e.g. \type {\romannumeral}: its value can only be read,
-not set.
-
-\subsection{\type {\mathstyle}}
-
-The returned value is between 0 and 7 (in math mode), or $-1$ (all other modes).
-For easy testing, the eight math style commands have been altered so that the can
-be used as numeric values, so you can write code like this:
-
-\starttyping
-\ifnum\mathstyle=\textstyle
-    \message{normal text style}
-\else \ifnum\mathstyle=\crampedtextstyle
-    \message{cramped text style}
-\fi \fi
-\stoptyping
-
-Sometimes you won't get what you expect so a bit of explanation might help to
-understand what happens. When math is parsed and expanded it gets turned into a
-linked list. In a second pass the formula will be build. This has to do with the
-fact that in order to determine the automatically chosen sizes (in for instance
-fractions) following content can influence preceding sizes. A side effect of this
-is for instance that one cannot change the definition of a font family (and
-thereby reusing numbers) because the number that got used is stored and used in
-the second pass (so changing \type {\fam 12} mid|-|formula spoils over to
-preceding use of that family).
-
-The style switching primitives like \type {\textstyle} are turned into nodes so
-the styles set there are frozen. The \type {\mathchoice} primitive results in
-four lists being constructed of which one is used in the second pass. The fact
-that some automatic styles are not yet known also means that the \type
-{\mathstyle} primitive expands to the current style which can of course be
-different from the one really used. It's a snapshot of the first pass state. As a
-consequence in the following example you get a style number (first pass) typeset
-that can actually differ from the used style (second pass). In the case of a math
-choice used ungrouped, the chosen style is used after the choice too, unless you
-group.
-
-\startbuffer[1]
-    [a:\mathstyle]\quad
-    \bgroup
-    \mathchoice
-        {\bf \scriptstyle       (x:d :\mathstyle)}
-        {\bf \scriptscriptstyle (x:t :\mathstyle)}
-        {\bf \scriptscriptstyle (x:s :\mathstyle)}
-        {\bf \scriptscriptstyle (x:ss:\mathstyle)}
-    \egroup
-    \quad[b:\mathstyle]\quad
-    \mathchoice
-        {\bf \scriptstyle       (y:d :\mathstyle)}
-        {\bf \scriptscriptstyle (y:t :\mathstyle)}
-        {\bf \scriptscriptstyle (y:s :\mathstyle)}
-        {\bf \scriptscriptstyle (y:ss:\mathstyle)}
-    \quad[c:\mathstyle]\quad
-    \bgroup
-    \mathchoice
-        {\bf \scriptstyle       (z:d :\mathstyle)}
-        {\bf \scriptscriptstyle (z:t :\mathstyle)}
-        {\bf \scriptscriptstyle (z:s :\mathstyle)}
-        {\bf \scriptscriptstyle (z:ss:\mathstyle)}
-    \egroup
-    \quad[d:\mathstyle]
-\stopbuffer
-
-\startbuffer[2]
-    [a:\mathstyle]\quad
-    \begingroup
-    \mathchoice
-        {\bf \scriptstyle       (x:d :\mathstyle)}
-        {\bf \scriptscriptstyle (x:t :\mathstyle)}
-        {\bf \scriptscriptstyle (x:s :\mathstyle)}
-        {\bf \scriptscriptstyle (x:ss:\mathstyle)}
-    \endgroup
-    \quad[b:\mathstyle]\quad
-    \mathchoice
-        {\bf \scriptstyle       (y:d :\mathstyle)}
-        {\bf \scriptscriptstyle (y:t :\mathstyle)}
-        {\bf \scriptscriptstyle (y:s :\mathstyle)}
-        {\bf \scriptscriptstyle (y:ss:\mathstyle)}
-    \quad[c:\mathstyle]\quad
-    \begingroup
-    \mathchoice
-        {\bf \scriptstyle       (z:d :\mathstyle)}
-        {\bf \scriptscriptstyle (z:t :\mathstyle)}
-        {\bf \scriptscriptstyle (z:s :\mathstyle)}
-        {\bf \scriptscriptstyle (z:ss:\mathstyle)}
-    \endgroup
-    \quad[d:\mathstyle]
-\stopbuffer
-
-\typebuffer[1]
-
-% \typebuffer[2]
-
-This gives:
-
-\blank $\displaystyle \getbuffer[1]$ \blank
-\blank $\textstyle    \getbuffer[1]$ \blank
-
-Using \type {\begingroup} \unknown\ \type {\endgroup} instead gives:
-
-\blank $\displaystyle \getbuffer[2]$ \blank
-\blank $\textstyle    \getbuffer[2]$ \blank
-
-This might look wrong but it's just a side effect of \type {\mathstyle} expanding
-to the current (first pass) style and the number being injected in the list that
-gets converted in the second pass. It all makes sense and it illustrates the
-importance of grouping. In fact, the math choice style being effective afterwards
-has advantages. It would be hard to get it otherwise.
-
-\subsection{\type {\Ustack}}
-
-There are a few math commands in \TEX\ where the style that will be used is not
-known straight from the start. These commands (\type {\over}, \type {\atop},
-\type {\overwithdelims}, \type {\atopwithdelims}) would therefore normally return
-wrong values for \type {\mathstyle}. To fix this, \LUATEX\ introduces a special
-prefix command: \type {\Ustack}:
-
-\starttyping
-$\Ustack {a \over b}$
-\stoptyping
-
-The \type {\Ustack} command will scan the next brace and start a new math group
-with the correct (numerator) math style.
-
-\section{Unicode math characters}
-
-Character handling is now extended up to the full \UNICODE\ range (the \type {\U}
-prefix), which is compatible with \XETEX.
-
-The math primitives from \TEX\ are kept as they are, except for the ones that
-convert from input to math commands: \type {mathcode}, and \type {delcode}. These
-two now allow for a 21-bit character argument on the left hand side of the equals
-sign.
-
-Some of the new \LUATEX\ primitives read more than one separate value. This is
-shown in the tables below by a plus sign in the second column.
-
-The input for such primitives would look like this:
-
-\starttyping
-\def\overbrace{\Umathaccent 0 1 "23DE }
-\stoptyping
-
-The altered \TEX82 primitives are:
-
-\starttabulate[|l|l|r|c|l|r|]
-\BC primitive         \BC min \BC max    \BC \kern 2em \BC min \BC max    \NC \NR
-\NC \type {\mathcode} \NC 0   \NC 10FFFF \NC =         \NC 0   \NC 8000   \NC \NR
-\NC \type {\delcode}  \NC 0   \NC 10FFFF \NC =         \NC 0   \NC FFFFFF \NC \NR
-\stoptabulate
-
-The unaltered ones are:
-
-\starttabulate[|l|l|r|]
-\BC primitive            \BC min \BC max     \NC \NR
-\NC \type {\mathchardef} \NC 0   \NC    8000 \NC \NR
-\NC \type {\mathchar}    \NC 0   \NC    7FFF \NC \NR
-\NC \type {\mathaccent}  \NC 0   \NC    7FFF \NC \NR
-\NC \type {\delimiter}   \NC 0   \NC 7FFFFFF \NC \NR
-\NC \type {\radical}     \NC 0   \NC 7FFFFFF \NC \NR
-\stoptabulate
-
-For practical reasons \type {\mathchardef} will silently accept values larger
-that \type {0x8000} and interpret it as \type {\Umathcharnumdef}. This is needed
-to satisfy older macro packages.
-
-The following new primitives are compatible with \XETEX:
-
-% somewhat fuzzy:
-
-\starttabulate[|l|l|r|c|l|r|]
-\BC primitive                               \BC min       \BC max                          \BC \kern 2em \BC min       \BC max                        \NC \NR
-\NC \type {\Umathchardef}                   \NC 0+0+0     \NC 7+FF+10FFFF\rlap{\high{1}}   \NC           \NC           \NC                            \NC \NR
-\NC \type {\Umathcharnumdef}\rlap{\high{5}} \NC -80000000 \NC    7FFFFFFF\rlap{\high{3}}   \NC           \NC           \NC                            \NC \NR
-\NC \type {\Umathcode}                      \NC 0         \NC      10FFFF                  \NC =         \NC 0+0+0     \NC 7+FF+10FFFF\rlap{\high{1}} \NC \NR
-\NC \type {\Udelcode}                       \NC 0         \NC      10FFFF                  \NC =         \NC 0+0       \NC   FF+10FFFF\rlap{\high{2}} \NC \NR
-\NC \type {\Umathchar}                      \NC 0+0+0     \NC 7+FF+10FFFF                  \NC           \NC           \NC                            \NC \NR
-\NC \type {\Umathaccent}                    \NC 0+0+0     \NC 7+FF+10FFFF\rlap{\high{2,4}} \NC           \NC           \NC                            \NC \NR
-\NC \type {\Udelimiter}                     \NC 0+0+0     \NC 7+FF+10FFFF\rlap{\high{2}}   \NC           \NC           \NC                            \NC \NR
-\NC \type {\Uradical}                       \NC 0+0       \NC   FF+10FFFF\rlap{\high{2}}   \NC           \NC           \NC                            \NC \NR
-\NC \type {\Umathcharnum}                   \NC -80000000 \NC    7FFFFFFF\rlap{\high{3}}   \NC           \NC           \NC                            \NC \NR
-\NC \type {\Umathcodenum}                   \NC 0         \NC      10FFFF                  \NC =         \NC -80000000 \NC    7FFFFFFF\rlap{\high{3}} \NC \NR
-\NC \type {\Udelcodenum}                    \NC 0         \NC      10FFFF                  \NC =         \NC -80000000 \NC    7FFFFFFF\rlap{\high{3}} \NC \NR
-\stoptabulate
-
-Specifications typically look like:
-
-\starttyping
-\Umathchardef\xx="1"0"456
-\Umathcode   123="1"0"789
-\stoptyping
-
-Note 1: The new primitives that deal with delimiter|-|style objects do not set up a
-\quote {large family}. Selecting a suitable size for display purposes is expected
-to be dealt with by the font via the \type {\Umathoperatorsize} parameter (more
-information can be found in a following section).
-
-Note 2: For these three primitives, all information is packed into a single
-signed integer. For the first two (\type {\Umathcharnum} and \type
-{\Umathcodenum}), the lowest 21 bits are the character code, the 3 bits above
-that represent the math class, and the family data is kept in the topmost bits
-(This means that the values for math families 128--255 are actually negative).
-For \type {\Udelcodenum} there is no math class. The math family information is
-stored in the bits directly on top of the character code. Using these three
-commands is not as natural as using the two- and three|-|value commands, so
-unless you know exactly what you are doing and absolutely require the speedup
-resulting from the faster input scanning, it is better to use the verbose
-commands instead.
-
-Note 3: The \type {\Umathaccent} command accepts optional keywords to control
-various details regarding math accents. See \in {section} [mathacc] below for
-details.
-
-New primitives that exist in \LUATEX\ only (all of these will be explained
-in following sections):
-
-\starttabulate[|l|l|]
-\BC primitive                \BC value range (in hex) \NC \NR
-\NC \type {\Uroot}           \NC 0+0--FF+10FFFF$^2$   \NC \NR
-\NC \type {\Uoverdelimiter}  \NC 0+0--FF+10FFFF$^2$   \NC \NR
-\NC \type {\Uunderdelimiter} \NC 0+0--FF+10FFFF$^2$   \NC \NR
-\NC \type {\Udelimiterover}  \NC 0+0--FF+10FFFF$^2$   \NC \NR
-\NC \type {\Udelimiterunder} \NC 0+0--FF+10FFFF$^2$   \NC \NR
-\stoptabulate
-
-\section{Cramped math styles}
-
-\LUATEX\ has four new primitives to set the cramped math styles directly:
-
-\starttyping
-\crampeddisplaystyle
-\crampedtextstyle
-\crampedscriptstyle
-\crampedscriptscriptstyle
-\stoptyping
-
-These additional commands are not all that valuable on their own, but they come
-in handy as arguments to the math parameter settings that will be added shortly.
-
-In Eijkhouts \quotation {\TEX\ by Topic} the rules for handling styles in scripts
-are described as follows:
-
-\startitemize
-\startitem
-    In any style superscripts and subscripts are taken from the next smaller style.
-    Exception: in display style they are in script style.
-\stopitem
-\startitem
-    Subscripts are always in the cramped variant of the style; superscripts are only
-    cramped if the original style was cramped.
-\stopitem
-\startitem
-    In an \type {..\over..} formula in any style the numerator and denominator are
-    taken from the next smaller style.
-\stopitem
-\startitem
-    The denominator is always in cramped style; the numerator is only in cramped
-    style if the original style was cramped.
-\stopitem
-\startitem
-    Formulas under a \type {\sqrt} or \type {\overline} are in cramped style.
-\stopitem
-\stopitemize
-
-In \LUATEX\ one can set the styles in more detail which means that you sometimes
-have to set both normal and cramped styles to get the effect you want. If we
-force styles in the script using \type {\scriptstyle} and \type {\crampedscriptstyle}
-we get this:
-
-\startbuffer[demo]
-\starttabulate
-\NC default       \NC $b_{x=xx}^{x=xx}$ \NC \NR
-\NC script        \NC $b_{\scriptstyle x=xx}^{\scriptstyle x=xx}$ \NC \NR
-\NC crampedscript \NC $b_{\crampedscriptstyle x=xx}^{\crampedscriptstyle x=xx}$ \NC \NR
-\stoptabulate
-\stopbuffer
-
-\getbuffer[demo]
-
-Now we set the following parameters
-
-\startbuffer[setup]
-\Umathordrelspacing\scriptstyle=30mu
-\Umathordordspacing\scriptstyle=30mu
-\stopbuffer
-
-\typebuffer[setup]
-
-This gives:
-
-\start\getbuffer[setup,demo]\stop
-
-But, as this is not what is expected (visually) we should say:
-
-\startbuffer[setup]
-\Umathordrelspacing\scriptstyle=30mu
-\Umathordordspacing\scriptstyle=30mu
-\Umathordrelspacing\crampedscriptstyle=30mu
-\Umathordordspacing\crampedscriptstyle=30mu
-\stopbuffer
-
-\typebuffer[setup]
-
-Now we get:
-
-\start\getbuffer[setup,demo]\stop
-
-\section{Math parameter settings}
-
-In \LUATEX, the font dimension parameters that \TEX\ used in math typesetting are
-now accessible via primitive commands. In fact, refactoring of the math engine
-has resulted in many more parameters than were accessible before.
-
-\starttabulate
-\BC primitive name                   \BC description \NC \NR
-\NC \type {\Umathquad}               \NC the width of 18 mu's \NC \NR
-\NC \type {\Umathaxis}               \NC height of the vertical center axis of
-                                         the math formula above the baseline \NC \NR
-\NC \type {\Umathoperatorsize}       \NC minimum size of large operators in display mode \NC \NR
-\NC \type {\Umathoverbarkern}        \NC vertical clearance above the rule \NC \NR
-\NC \type {\Umathoverbarrule}        \NC the width of the rule \NC \NR
-\NC \type {\Umathoverbarvgap}        \NC vertical clearance below the rule \NC \NR
-\NC \type {\Umathunderbarkern}       \NC vertical clearance below the rule \NC \NR
-\NC \type {\Umathunderbarrule}       \NC the width of the rule \NC \NR
-\NC \type {\Umathunderbarvgap}       \NC vertical clearance above the rule \NC \NR
-\NC \type {\Umathradicalkern}        \NC vertical clearance above the rule \NC \NR
-\NC \type {\Umathradicalrule}        \NC the width of the rule \NC \NR
-\NC \type {\Umathradicalvgap}        \NC vertical clearance below the rule \NC \NR
-\NC \type {\Umathradicaldegreebefore}\NC the forward kern that takes place before placement of
-                                         the radical degree \NC \NR
-\NC \type {\Umathradicaldegreeafter} \NC the backward kern that takes place after placement of
-                                         the radical degree \NC \NR
-\NC \type {\Umathradicaldegreeraise} \NC this is the percentage of the total height and depth of
-                                         the radical sign that the degree is raised by; it is
-                                         expressed in \type {percents}, so 60\% is expressed as the
-                                         integer $60$ \NC \NR
-\NC \type {\Umathstackvgap}          \NC vertical clearance between the two
-                                         elements in a \type {\atop} stack \NC \NR
-\NC \type {\Umathstacknumup}         \NC numerator shift upward in \type {\atop} stack \NC \NR
-\NC \type {\Umathstackdenomdown}     \NC denominator shift downward in \type {\atop} stack \NC \NR
-\NC \type {\Umathfractionrule}       \NC the width of the rule in a \type {\over} \NC \NR
-\NC \type {\Umathfractionnumvgap}    \NC vertical clearance between the numerator and the rule \NC \NR
-\NC \type {\Umathfractionnumup}      \NC numerator shift upward in \type {\over} \NC \NR
-\NC \type {\Umathfractiondenomvgap}  \NC vertical clearance between the denominator and the rule \NC \NR
-\NC \type {\Umathfractiondenomdown}  \NC denominator shift downward in \type {\over} \NC \NR
-\NC \type {\Umathfractiondelsize}    \NC minimum delimiter size for \type {\...withdelims} \NC \NR
-\NC \type {\Umathlimitabovevgap}     \NC vertical clearance for limits above operators \NC \NR
-\NC \type {\Umathlimitabovebgap}     \NC vertical baseline clearance for limits above operators \NC \NR
-\NC \type {\Umathlimitabovekern}     \NC space reserved at the top of the limit \NC \NR
-\NC \type {\Umathlimitbelowvgap}     \NC vertical clearance for limits below operators \NC \NR
-\NC \type {\Umathlimitbelowbgap}     \NC vertical baseline clearance for limits below operators \NC \NR
-\NC \type {\Umathlimitbelowkern}     \NC space reserved at the bottom of the limit \NC \NR
-\NC \type {\Umathoverdelimitervgap}  \NC vertical clearance for limits above delimiters \NC \NR
-\NC \type {\Umathoverdelimiterbgap}  \NC vertical baseline clearance for limits above delimiters \NC \NR
-\NC \type {\Umathunderdelimitervgap} \NC vertical clearance for limits below delimiters \NC \NR
-\NC \type {\Umathunderdelimiterbgap} \NC vertical baseline clearance for limits below delimiters \NC \NR
-\NC \type {\Umathsubshiftdrop}       \NC subscript drop for boxes and subformulas \NC \NR
-\NC \type {\Umathsubshiftdown}       \NC subscript drop for characters \NC \NR
-\NC \type {\Umathsupshiftdrop}       \NC superscript drop (raise, actually) for boxes and subformulas \NC \NR
-\NC \type {\Umathsupshiftup}         \NC superscript raise for characters \NC \NR
-\NC \type {\Umathsubsupshiftdown}    \NC subscript drop in the presence of a superscript \NC \NR
-\NC \type {\Umathsubtopmax}          \NC the top of standalone subscripts cannot be higher than this
-                                         above the baseline \NC \NR
-\NC \type {\Umathsupbottommin}       \NC the bottom of standalone superscripts cannot be less than
-                                         this above the baseline \NC \NR
-\NC \type {\Umathsupsubbottommax}    \NC the bottom of the superscript of a combined super- and subscript
-                                         be at least as high as this above the baseline \NC \NR
-\NC \type {\Umathsubsupvgap}         \NC vertical clearance between super- and subscript \NC \NR
-\NC \type {\Umathspaceafterscript}   \NC additional space added after a super- or subscript \NC \NR
-\NC \type {\Umathconnectoroverlapmin}\NC minimum overlap between parts in an extensible recipe \NC \NR
-\stoptabulate
-
-Each of the parameters in this section can be set by a command like this:
-
-\starttyping
-\Umathquad\displaystyle=1em
-\stoptyping
-
-they obey grouping, and you can use \type {\the\Umathquad\displaystyle} if
-needed.
-
-\section{Skips around display math}
-
-The injection of \type {\abovedisplayskip} and \type {\belowdisplayskip} is not
-symmetrical. An above one is always inserted, also when zero, but the below is
-only inserted when larger than zero. Especially the later makes it sometimes hard
-to fully control spacing. Therefore \LUATEX\ comes with a new directive: \type
-{\mathdisplayskipmode}. The following values apply:
-
-\starttabulate
-\NC 0 \NC normal \TEX\ behaviour \NC \NR
-\NC 1 \NC always (same as 0) \NC \NR
-\NC 2 \NC only when not zero \NC \NR
-\NC 3 \NC never, not even when not zero \NC \NR
-\stoptabulate
-
-\section{Font-based Math Parameters}
-
-While it is nice to have these math parameters available for tweaking, it would
-be tedious to have to set each of them by hand. For this reason, \LUATEX\
-initializes a bunch of these parameters whenever you assign a font identifier to
-a math family based on either the traditional math font dimensions in the font
-(for assignments to math family~2 and~3 using \TFM|-|based fonts like \type
-{cmsy} and \type {cmex}), or based on the named values in a potential \type
-{MathConstants} table when the font is loaded via Lua. If there is a \type
-{MathConstants} table, this takes precedence over font dimensions, and in that
-case no attention is paid to which family is being assigned to: the \type
-{MathConstants} tables in the last assigned family sets all parameters.
-
-In the table below, the one|-|letter style abbreviations and symbolic tfm font
-dimension names match those using in the \TeX book. Assignments to \type
-{\textfont} set the values for the cramped and uncramped display and text styles,
-\type {\scriptfont} sets the script styles, and \type {\scriptscriptfont} sets
-the scriptscript styles, so we have eight parameters for three font sizes. In the
-\TFM\ case, assignments only happen in family~2 and family~3 (and of course only
-for the parameters for which there are font dimensions).
-
-Besides the parameters below, \LUATEX\ also looks at the \quote {space} font
-dimension parameter. For math fonts, this should be set to zero.
-
-\start
-
-\switchtobodyfont[8pt]
-
-\starttabulate[|l|l|l|p|]
-\BC variable                          \BC style                 \BC default value opentype                   \BC default value tfm \NC \NR
-\NC \type {\Umathaxis}                \NC --                    \NC AxisHeight                               \NC axis_height \NC \NR
-\NC \type {\Umathoperatorsize}        \NC D, D'                 \NC DisplayOperatorMinHeight                 \NC $^6$ \NC \NR
-\NC \type {\Umathfractiondelsize}     \NC D, D'                 \NC FractionDelimiterDisplayStyleSize$^9$    \NC delim1 \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC FractionDelimiterSize$^9$                \NC delim2 \NC \NR
-\NC \type {\Umathfractiondenomdown}   \NC D, D'                 \NC FractionDenominatorDisplayStyleShiftDown \NC denom1 \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC FractionDenominatorShiftDown             \NC denom2 \NC \NR
-\NC \type {\Umathfractiondenomvgap}   \NC D, D'                 \NC FractionDenominatorDisplayStyleGapMin    \NC 3*default_rule_thickness \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC FractionDenominatorGapMin                \NC default_rule_thickness \NC \NR
-\NC \type {\Umathfractionnumup}       \NC D, D'                 \NC FractionNumeratorDisplayStyleShiftUp     \NC num1 \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC FractionNumeratorShiftUp                 \NC num2 \NC \NR
-\NC \type {\Umathfractionnumvgap}     \NC D, D'                 \NC FractionNumeratorDisplayStyleGapMin      \NC 3*default_rule_thickness \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC FractionNumeratorGapMin                  \NC default_rule_thickness \NC \NR
-\NC \type {\Umathfractionrule}        \NC --                    \NC FractionRuleThickness                    \NC default_rule_thickness \NC \NR
-\NC \type {\Umathskewedfractionhgap}  \NC --                    \NC SkewedFractionHorizontalGap              \NC math_quad/2 \NC \NR
-\NC \type {\Umathskewedfractionvgap}  \NC --                    \NC SkewedFractionVerticalGap                \NC math_x_height \NC \NR
-\NC \type {\Umathlimitabovebgap}      \NC --                    \NC UpperLimitBaselineRiseMin                \NC big_op_spacing3 \NC \NR
-\NC \type {\Umathlimitabovekern}      \NC --                    \NC 0$^1$                                    \NC big_op_spacing5 \NC \NR
-\NC \type {\Umathlimitabovevgap}      \NC --                    \NC UpperLimitGapMin                         \NC big_op_spacing1 \NC \NR
-\NC \type {\Umathlimitbelowbgap}      \NC --                    \NC LowerLimitBaselineDropMin                \NC big_op_spacing4 \NC \NR
-\NC \type {\Umathlimitbelowkern}      \NC --                    \NC 0$^1$                                    \NC big_op_spacing5 \NC \NR
-\NC \type {\Umathlimitbelowvgap}      \NC --                    \NC LowerLimitGapMin                         \NC big_op_spacing2 \NC \NR
-\NC \type {\Umathoverdelimitervgap}   \NC --                    \NC StretchStackGapBelowMin                  \NC big_op_spacing1 \NC \NR
-\NC \type {\Umathoverdelimiterbgap}   \NC --                    \NC StretchStackTopShiftUp                   \NC big_op_spacing3 \NC \NR
-\NC \type {\Umathunderdelimitervgap}  \NC--                     \NC StretchStackGapAboveMin                  \NC big_op_spacing2 \NC \NR
-\NC \type {\Umathunderdelimiterbgap}  \NC--                     \NC StretchStackBottomShiftDown              \NC big_op_spacing4 \NC \NR
-\NC \type {\Umathoverbarkern}         \NC --                    \NC OverbarExtraAscender                     \NC default_rule_thickness \NC \NR
-\NC \type {\Umathoverbarrule}         \NC --                    \NC OverbarRuleThickness                     \NC default_rule_thickness \NC \NR
-\NC \type {\Umathoverbarvgap}         \NC --                    \NC OverbarVerticalGap                       \NC 3*default_rule_thickness \NC \NR
-\NC \type {\Umathquad}                \NC --                    \NC <font_size(f)>$^1$                       \NC math_quad \NC \NR
-\NC \type {\Umathradicalkern}         \NC --                    \NC RadicalExtraAscender                     \NC default_rule_thickness \NC \NR
-\NC \type {\Umathradicalrule}         \NC --                    \NC RadicalRuleThickness                     \NC <not set>$^2$ \NC \NR
-\NC \type {\Umathradicalvgap}         \NC D, D'                 \NC RadicalDisplayStyleVerticalGap           \NC (default_rule_thickness+\crlf
-                                                                                                                 (abs(math_x_height)/4))$^3$ \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC RadicalVerticalGap                       \NC (default_rule_thickness+\crlf
-                                                                                                                 (abs(default_rule_thickness)/4))$^3$ \NC \NR
-\NC \type {\Umathradicaldegreebefore} \NC --                    \NC RadicalKernBeforeDegree                  \NC <not set>$^2$ \NC \NR
-\NC \type {\Umathradicaldegreeafter}  \NC --                    \NC RadicalKernAfterDegree                   \NC <not set>$^2$ \NC \NR
-\NC \type {\Umathradicaldegreeraise}  \NC --                    \NC RadicalDegreeBottomRaisePercent          \NC <not set>$^{2,7}$ \NC \NR
-\NC \type {\Umathspaceafterscript}    \NC --                    \NC SpaceAfterScript                         \NC script_space$^4$ \NC \NR
-\NC \type {\Umathstackdenomdown}      \NC D, D'                 \NC StackBottomDisplayStyleShiftDown         \NC denom1 \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC StackBottomShiftDown                     \NC denom2 \NC \NR
-\NC \type {\Umathstacknumup}          \NC D, D'                 \NC StackTopDisplayStyleShiftUp              \NC num1 \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC StackTopShiftUp                          \NC num3 \NC \NR
-\NC \type {\Umathstackvgap}           \NC D, D'                 \NC StackDisplayStyleGapMin                  \NC 7*default_rule_thickness \NC \NR
-\NC                                   \NC T, T', S, S', SS, SS' \NC StackGapMin                              \NC 3*default_rule_thickness \NC \NR
-\NC \type {\Umathsubshiftdown}        \NC --                    \NC SubscriptShiftDown                       \NC sub1 \NC \NR
-\NC \type {\Umathsubshiftdrop}        \NC --                    \NC SubscriptBaselineDropMin                 \NC sub_drop \NC \NR
-\NC \type {\Umathsubsupshiftdown}     \NC --                    \NC SubscriptShiftDownWithSuperscript$^8$    \NC \NC \NR
-\NC                                   \NC                       \NC \quad\ or SubscriptShiftDown             \NC sub2 \NC \NR
-\NC \type {\Umathsubtopmax}           \NC --                    \NC SubscriptTopMax                          \NC (abs(math_x_height * 4) / 5) \NC \NR
-\NC \type {\Umathsubsupvgap}          \NC --                    \NC SubSuperscriptGapMin                     \NC 4*default_rule_thickness \NC \NR
-\NC \type {\Umathsupbottommin}        \NC --                    \NC SuperscriptBottomMin                     \NC (abs(math_x_height) / 4) \NC \NR
-\NC \type {\Umathsupshiftdrop}        \NC --                    \NC SuperscriptBaselineDropMax               \NC sup_drop \NC \NR
-\NC \type {\Umathsupshiftup}          \NC D                     \NC SuperscriptShiftUp                       \NC sup1 \NC \NR
-\NC                                   \NC T, S, SS,             \NC SuperscriptShiftUp                       \NC sup2 \NC \NR
-\NC                                   \NC D', T', S', SS'       \NC SuperscriptShiftUpCramped                \NC sup3 \NC \NR
-\NC \type {\Umathsupsubbottommax}     \NC --                    \NC SuperscriptBottomMaxWithSubscript        \NC (abs(math_x_height * 4) / 5) \NC \NR
-\NC \type {\Umathunderbarkern}        \NC --                    \NC UnderbarExtraDescender                   \NC default_rule_thickness \NC \NR
-\NC \type {\Umathunderbarrule}        \NC --                    \NC UnderbarRuleThickness                    \NC default_rule_thickness \NC \NR
-\NC \type {\Umathunderbarvgap}        \NC --                    \NC UnderbarVerticalGap                      \NC 3*default_rule_thickness \NC \NR
-\NC \type {\Umathconnectoroverlapmin} \NC --                    \NC MinConnectorOverlap                      \NC 0$^5$ \NC \NR
-\stoptabulate
-
-\stop
-
-Note 1: \OPENTYPE\ fonts set \type {\Umathlimitabovekern} and \type
-{\Umathlimitbelowkern} to zero and set \type {\Umathquad} to the font size of the
-used font, because these are not supported in the \type {MATH} table,
-
-Note 2: Traditional \TFM\ fonts do not set \type {\Umathradicalrule} because
-\TEX82\ uses the height of the radical instead. When this parameter is indeed not
-set when \LUATEX\ has to typeset a radical, a backward compatibility mode will
-kick in that assumes that an oldstyle \TEX\ font is used. Also, they do not set
-\type {\Umathradicaldegreebefore}, \type {\Umathradicaldegreeafter}, and \type
-{\Umathradicaldegreeraise}. These are then automatically initialized to
-$5/18$quad, $-10/18$quad, and 60.
-
-Note 3: If \TFM\ fonts are used, then the \type {\Umathradicalvgap} is not set
-until the first time \LUATEX\ has to typeset a formula because this needs
-parameters from both family~2 and family~3. This provides a partial backward
-compatibility with \TEX82, but that compatibility is only partial: once the \type
-{\Umathradicalvgap} is set, it will not be recalculated any more.
-
-Note 4: When \TFM\ fonts are used a similar situation arises with respect to
-\type {\Umathspaceafterscript}: it is not set until the first time \LUATEX\ has
-to typeset a formula. This provides some backward compatibility with \TEX82. But
-once the \type {\Umathspaceafterscript} is set, \type {\scriptspace} will never
-be looked at again.
-
-Note 5: Traditional \TFM\ fonts set \type {\Umathconnectoroverlapmin} to zero
-because \TEX82\ always stacks extensibles without any overlap.
-
-Note 6: The \type {\Umathoperatorsize} is only used in \type {\displaystyle}, and
-is only set in \OPENTYPE\ fonts. In \TFM\ font mode, it is artificially set to
-one scaled point more than the initial attempt's size, so that always the \quote
-{first next} will be tried, just like in \TEX82.
-
-Note 7: The \type {\Umathradicaldegreeraise} is a special case because it is the
-only parameter that is expressed in a percentage instead of as a number of scaled
-points.
-
-Note 8: \type {SubscriptShiftDownWithSuperscript} does not actually exist in the
-\quote {standard} \OPENTYPE\ math font Cambria, but it is useful enough to be
-added.
-
-Note 9: \type {FractionDelimiterDisplayStyleSize} and \type
-{FractionDelimiterSize} do not actually exist in the \quote {standard} \OPENTYPE\
-math font Cambria, but were useful enough to be added.
-
-\section{Nolimit correction}
-
-There are two extra math parameters \type {\Umathnolimitsupfactor} and \type
-{\Umathnolimitsubfactor} that were added to provide some control over how limits
-are spaced (for example the position of super and subscripts after integral
-operators). They relate to an extra parameter \type {\mathnolimitsmode}. The half
-corrections are what happens when scripts are placed on above and below. The
-problem with italic corrections is that officially that correction italic is used
-for above|/|below placement while advanced kerns are used for placement at the
-right end. The question is: how often is this implemented, and if so, does the
-kerns assume correction too. Anyway, with this parameter one can control it.
-
-\starttabulate[|l|ck1|ck1|ck1|ck1|ck1|ck1|]
-    \NC
-        \NC \mathnolimitsmode0    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode1    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode2    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode3    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode4    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode8000 $\displaystyle\int\nolimits^0_1$
-    \NC \NR
-    \TB
-    \BC mode
-        \NC \tttf 0
-        \NC \tttf 1
-        \NC \tttf 2
-        \NC \tttf 3
-        \NC \tttf 4
-        \NC \tttf 8000
-    \NC \NR
-    \BC superscript
-        \NC 0
-        \NC font
-        \NC 0
-        \NC 0
-        \NC +ic/2
-        \NC 0
-    \NC \NR
-    \BC subscript
-        \NC -ic
-        \NC font
-        \NC 0
-        \NC -ic/2
-        \NC -ic/2
-        \NC 8000ic/1000
-    \NC \NR
-\stoptabulate
-
-When the mode is set to one, the math parameters are used. This way a macro
-package writer can decide what looks best. Given the current state of fonts in
-\CONTEXT\ we currently use mode 1 with factor 0 for the superscript and 750 for
-the subscripts. Positive values are used for both parameters but the subscript
-shifts to the left. A \type {\mathnolimitsmode} larger that 15 is considered to
-be a factor for the subscript correction. This feature can be handy when
-experimenting.
-
-\section{Math italic mess}
-
-The \type {\mathitalicsmode} parameter can be set to~1 to force italic correction
-before noads that represent some more complex structure (read: everything
-that is not an ord, bin, rel, open, close, punct or inner). We show a Cambria
-example.
-
-\starttexdefinition Whatever #1
-    \NC \type{\mathitalicsmode = #1}
-    \NC \mathitalicsmode#1\ruledhbox{$\left|T^1\right|$}
-    \NC \mathitalicsmode#1\ruledhbox{$\left|T\right|$}
-    \NC \mathitalicsmode#1\ruledhbox{$T+1$}
-    \NC \mathitalicsmode#1\ruledhbox{$T{1\over2}$}
-    \NC \mathitalicsmode#1\ruledhbox{$T\sqrt{1}$}
-    \NC \NR
-\stoptexdefinition
-
-\start
-    \switchtobodyfont[cambria]
-    \starttabulate[|c|c|c|c|c|c|]
-        \Whatever{0}%
-        \Whatever{1}%
-    \stoptabulate
-\stop
-
-This kind of parameters relate to the fact that italic correction in \OPENTYPE\
-math is bound to fuzzy rules. So, control is the solution.
-
-\section{Script boxes}
-
-If you want typeset text in math macro packages often provide something \type
-{\text} which obeys the script sizes. As the definition can be anything there is
-a good change that the kerning doesn't come out well when used in a script. Given
-that the first glyph ends up in an \type {\hbox} we have some control over this.
-And, as a bonus we also added control over the normal sublist kerning. The \type
-{\mathscriptboxmode} parameter defaults to~1.
-
-\starttabulate[|l|l|]
-\NC \type {0} \NC forget about kerning \NC \NR
-\NC \type {1} \NC kern math sub lists with a valid glyph \NC \NR
-\NC \type {2} \NC also kern math sub boxes that have a valid glyph \NC \NR
-\NC \type {2} \NC only kern math sub boxes with a boundary node present\NC \NR
-\stoptabulate
-
-Here we show some examples. Of course this doesn't solve all our problems, if
-only because some fonts have characters with bounding boxes that compensate for
-italics, while other fonts can lack kerns.
-
-\startbuffer[1]
-    $T_{\tf fluff}$
-\stopbuffer
-
-\startbuffer[2]
-    $T_{\text{fluff}}$
-\stopbuffer
-
-\startbuffer[3]
-    $T_{\text{\boundary1 fluff}}$
-\stopbuffer
-
-\unexpanded\def\Show#1#2#3%
-  {\doifelsenothing{#3}
-     {\small\typeinlinebuffer[#1]}
-     {\doifelse{#3}{-}
-        {\small\type{mode #2}}
-        {\switchtobodyfont[#3]\showfontkerns\showglyphs\mathscriptboxmode#2\relax\inlinebuffer[#1]}}}
-
-\starttabulate[|lT|c|c|c|c|c|]
-    \NC          \NC \Show{1}{0}{}         \NC\Show{1}{1}{}         \NC \Show{2}{1}{}         \NC \Show{2}{2}{}         \NC \Show{3}{3}{}         \NC \NR
-    \NC          \NC \Show{1}{0}{-}        \NC\Show{1}{1}{-}        \NC \Show{2}{1}{-}        \NC \Show{2}{2}{-}        \NC \Show{3}{3}{-}        \NC \NR
-    \NC modern   \NC \Show{1}{0}{modern}   \NC\Show{1}{1}{modern}   \NC \Show{2}{1}{modern}   \NC \Show{2}{2}{modern}   \NC \Show{3}{3}{modern}   \NC \NR
-    \NC lucidaot \NC \Show{1}{0}{lucidaot} \NC\Show{1}{1}{lucidaot} \NC \Show{2}{1}{lucidaot} \NC \Show{2}{2}{lucidaot} \NC \Show{3}{3}{lucidaot} \NC \NR
-    \NC pagella  \NC \Show{1}{0}{pagella}  \NC\Show{1}{1}{pagella}  \NC \Show{2}{1}{pagella}  \NC \Show{2}{2}{pagella}  \NC \Show{3}{3}{pagella}  \NC \NR
-    \NC cambria  \NC \Show{1}{0}{cambria}  \NC\Show{1}{1}{cambria}  \NC \Show{2}{1}{cambria}  \NC \Show{2}{2}{cambria}  \NC \Show{3}{3}{cambria}  \NC \NR
-    \NC dejavu   \NC \Show{1}{0}{dejavu}   \NC\Show{1}{1}{dejavu}   \NC \Show{2}{1}{dejavu}   \NC \Show{2}{2}{dejavu}   \NC \Show{3}{3}{dejavu}   \NC \NR
-\stoptabulate
-
-\section{Unscaled fences}
-
-The \type {\mathdelimitersmode} primitive is experimental and deals with the
-following (potential) problems. Three bits can be set. The first bit prevents
-an unwanted shift when the fence symbol is not scaled (a cambria side effect). The
-second bit forces italic correction between a preceding character ordinal and
-the fenced subformula, while the third bit turns that subformula into a ordinary
-so that the same spacing applies as with unfenced variants. Here we show Cambria
-(with \type {\mathitalicsmode} enabled).
-
-\starttexdefinition Whatever #1
-    \NC \type{\mathdelimitersmode = #1}
-    \NC \mathitalicsmode1\mathdelimitersmode#1\ruledhbox{\showglyphs\showfontkerns\showfontitalics$f(x)$}
-    \NC \mathitalicsmode1\mathdelimitersmode#1\ruledhbox{\showglyphs\showfontkerns\showfontitalics$f\left(x\right)$}
-    \NC \NR
-\stoptexdefinition
-
-\start
-    \switchtobodyfont[cambria]
-    \starttabulate[|l|l|l|]
-        \Whatever{0}\Whatever{1}\Whatever{2}\Whatever{3}%
-        \Whatever{4}\Whatever{5}\Whatever{6}\Whatever{7}%
-    \stoptabulate
-\stop
-
-So, when set to 7 fenced subformulas with unscaled delimiters come out the same
-as unfenced ones. This can be handy for cases where one is forced to use \type
-{\left} and \type {\right} always because of unpredictable content. As said, it's
-an experimental features (which somehow fits in the exceptional way fences are
-dealt with in the engine).
-
-\section{Math spacing setting}
-
-Besides the parameters mentioned in the previous sections, there are also 64 new
-primitives to control the math spacing table (as explained in Chapter~18 of the
-\TEX book). The primitive names are a simple matter of combining two math atom
-types, but for completeness' sake, here is the whole list:
-
-\starttwocolumns
-\starttyping
-\Umathordordspacing
-\Umathordopspacing
-\Umathordbinspacing
-\Umathordrelspacing
-\Umathordopenspacing
-\Umathordclosespacing
-\Umathordpunctspacing
-\Umathordinnerspacing
-\Umathopordspacing
-\Umathopopspacing
-\Umathopbinspacing
-\Umathoprelspacing
-\Umathopopenspacing
-\Umathopclosespacing
-\Umathoppunctspacing
-\Umathopinnerspacing
-\Umathbinordspacing
-\Umathbinopspacing
-\Umathbinbinspacing
-\Umathbinrelspacing
-\Umathbinopenspacing
-\Umathbinclosespacing
-\Umathbinpunctspacing
-\Umathbininnerspacing
-\Umathrelordspacing
-\Umathrelopspacing
-\Umathrelbinspacing
-\Umathrelrelspacing
-\Umathrelopenspacing
-\Umathrelclosespacing
-\Umathrelpunctspacing
-\Umathrelinnerspacing
-\Umathopenordspacing
-\Umathopenopspacing
-\Umathopenbinspacing
-\Umathopenrelspacing
-\Umathopenopenspacing
-\Umathopenclosespacing
-\Umathopenpunctspacing
-\Umathopeninnerspacing
-\Umathcloseordspacing
-\Umathcloseopspacing
-\Umathclosebinspacing
-\Umathcloserelspacing
-\Umathcloseopenspacing
-\Umathcloseclosespacing
-\Umathclosepunctspacing
-\Umathcloseinnerspacing
-\Umathpunctordspacing
-\Umathpunctopspacing
-\Umathpunctbinspacing
-\Umathpunctrelspacing
-\Umathpunctopenspacing
-\Umathpunctclosespacing
-\Umathpunctpunctspacing
-\Umathpunctinnerspacing
-\Umathinnerordspacing
-\Umathinneropspacing
-\Umathinnerbinspacing
-\Umathinnerrelspacing
-\Umathinneropenspacing
-\Umathinnerclosespacing
-\Umathinnerpunctspacing
-\Umathinnerinnerspacing
-\stoptyping
-\stoptwocolumns
-
-These parameters are of type \type {\muskip}, so setting a parameter can be done
-like this:
-
-\starttyping
-\Umathopordspacing\displaystyle=4mu plus 2mu
-\stoptyping
-
-They are all initialized by \type {initex} to the values mentioned in the table
-in Chapter~18 of the \TEX book.
-
-Note 1: for ease of use as well as for backward compatibility, \type
-{\thinmuskip}, \type {\medmuskip} and \type {\thickmuskip} are treated
-especially. In their case a pointer to the corresponding internal parameter is
-saved, not the actual \type {\muskip} value. This means that any later changes to
-one of these three parameters will be taken into account.
-
-Note 2: Careful readers will realise that there are also primitives for the items
-marked \type {*} in the \TEX book. These will not actually be used as those
-combinations of atoms cannot actually happen, but it seemed better not to break
-orthogonality. They are initialized to zero.
-
-\section[mathacc]{Math accent handling}
-
-\LUATEX\ supports both top accents and bottom accents in math mode, and math
-accents stretch automatically (if this is supported by the font the accent comes
-from, of course). Bottom and combined accents as well as fixed-width math accents
-are controlled by optional keywords following \type {\Umathaccent}.
-
-The keyword \type {bottom} after \type {\Umathaccent} signals that a bottom accent
-is needed, and the keyword \type {both} signals that both a top and a bottom
-accent are needed (in this case two accents need to be specified, of course).
-
-Then the set of three integers defining the accent is read. This set of integers
-can be prefixed by the \type {fixed} keyword to indicate that a non-stretching
-variant is requested (in case of both accents, this step is repeated).
-
-A simple example:
-
-\starttyping
-\Umathaccent both fixed 0 0 "20D7 fixed 0 0 "20D7 {example}
-\stoptyping
-
-If a math top accent has to be placed and the accentee is a character and has a
-non-zero \type {top_accent} value, then this value will be used to place the
-accent instead of the \type {\skewchar} kern used by \TEX82.
-
-The \type {top_accent} value represents a vertical line somewhere in the
-accentee. The accent will be shifted horizontally such that its own \type
-{top_accent} line coincides with the one from the accentee. If the \type
-{top_accent} value of the accent is zero, then half the width of the accent
-followed by its italic correction is used instead.
-
-The vertical placement of a top accent depends on the \type {x_height} of the
-font of the accentee (as explained in the \TEX book), but if value that turns out
-to be zero and the font had a \type {MathConstants} table, then \type
-{AccentBaseHeight} is used instead.
-
-The vertical placement of a bottom accent is straight below the accentee, no
-correction takes place.
-
-Possible locations are \type {top}, \type {bottom}, \type {both} and \type
-{center}. When no location is given \type {top} is assumed. An additional
-parameter \type {fraction} can be specified followed by a number; a value of for
-instance 1200 means that the criterium is 1.2 times the width of the nucleus. The
-fraction only applies to the stepwise selected shapes and is mostly meant for the
-\type {overlay} location. It also works for the other locations but then it
-concerns the width.
-
-\section{Math root extension}
-
-The new primitive \type {\Uroot} allows the construction of a radical noad
-including a degree field. Its syntax is an extension of \type {\Uradical}:
-
-\starttyping
-\Uradical <fam integer> <char integer> <radicand>
-\Uroot    <fam integer> <char integer> <degree> <radicand>
-\stoptyping
-
-The placement of the degree is controlled by the math parameters \type
-{\Umathradicaldegreebefore}, \type {\Umathradicaldegreeafter}, and \type
-{\Umathradicaldegreeraise}. The degree will be typeset in \type
-{\scriptscriptstyle}.
-
-\section{Math kerning in super- and subscripts}
-
-The character fields in a \LUA|-|loaded \OPENTYPE\ math font can have a \quote
-{mathkern} table. The format of this table is the same as the \quote {mathkern}
-table that is returned by the \type {fontloader} library, except that all height
-and kern values have to be specified in actual scaled points.
-
-When a super- or subscript has to be placed next to a math item, \LUATEX\ checks
-whether the super- or subscript and the nucleus are both simple character items.
-If they are, and if the fonts of both character items are \OPENTYPE\ fonts (as
-opposed to legacy \TEX\ fonts), then \LUATEX\ will use the \OPENTYPE\ math
-algorithm for deciding on the horizontal placement of the super- or subscript.
-
-This works as follows:
-
-\startitemize
-    \startitem
-        The vertical position of the script is calculated.
-    \stopitem
-    \startitem
-        The default horizontal position is flat next to the base character.
-    \stopitem
-    \startitem
-        For superscripts, the italic correction of the base character is added.
-    \stopitem
-    \startitem
-        For a superscript, two vertical values are calculated: the bottom of the
-        script (after shifting up), and the top of the base. For a subscript, the two
-        values are the top of the (shifted down) script, and the bottom of the base.
-    \stopitem
-    \startitem
-        For each of these two locations:
-        \startitemize
-            \startitem
-                find the math kern value at this height for the base (for a subscript
-                placement, this is the bottom_right corner, for a superscript
-                placement the top_right corner)
-            \stopitem
-            \startitem
-                find the math kern value at this height for the script (for a
-                subscript placement, this is the top_left corner, for a superscript
-                placement the bottom_left corner)
-            \stopitem
-            \startitem
-                add the found values together to get a preliminary result.
-            \stopitem
-        \stopitemize
-    \stopitem
-    \startitem
-        The horizontal kern to be applied is the smallest of the two results from
-        previous step.
-    \stopitem
-\stopitemize
-
-The math kern value at a specific height is the kern value that is specified by the
-next higher height and kern pair, or the highest one in the character (if there is no
-value high enough in the character), or simply zero (if the character has no math kern
-pairs at all).
-
-\section{Scripts on horizontally extensible items like arrows}
-
-The primitives \type {\Uunderdelimiter} and \type {\Uoverdelimiter} allow the
-placement of a subscript or superscript on an automatically extensible item and
-\type {\Udelimiterunder} and \type {\Udelimiterover} allow the placement of an
-automatically extensible item as a subscript or superscript on a nucleus. The
-input:
-
-% these produce radical noads .. in fact the code base has the numbers wrong for
-% quite a while, so no one seems to use this
-
-\startbuffer
-$\Uoverdelimiter  0 "2194 {\hbox{\strut  overdelimiter}}$
-$\Uunderdelimiter 0 "2194 {\hbox{\strut underdelimiter}}$
-$\Udelimiterover  0 "2194 {\hbox{\strut  delimiterover}}$
-$\Udelimiterunder 0 "2194 {\hbox{\strut delimiterunder}}$
-\stopbuffer
-
-\typebuffer will render this:
-
-\blank \startnarrower \getbuffer \stopnarrower \blank
-
-The vertical placements are controlled by \type {\Umathunderdelimiterbgap}, \type
-{\Umathunderdelimitervgap}, \type {\Umathoverdelimiterbgap}, and \type
-{\Umathoverdelimitervgap} in a similar way as limit placements on large operators.
-The superscript in \type {\Uoverdelimiter} is typeset in a suitable scripted style,
-the subscript in \type {\Uunderdelimiter} is cramped as well.
-
-These primitives accepts an option \type {width} specification. When used the
-also optional keywords \type {left}, \type {middle} and \type {right} will
-determine what happens when a requested size can't be met (which can happen when
-we step to successive larger variants).
-
-An extra primitive \type {\Uhextensible} is available that can be used like this:
-
-\startbuffer
-$\Uhextensible width 10cm 0 "2194$
-\stopbuffer
-
-\typebuffer This will render this:
-
-\blank \startnarrower \getbuffer \stopnarrower \blank
-
-Here you can also pass options, like:
-
-\startbuffer
-$\Uhextensible width 1pt middle 0 "2194$
-\stopbuffer
-
-\typebuffer This gives:
-
-\blank \startnarrower \getbuffer \stopnarrower \blank
-
-\LUATEX\ internally uses a structure that supports \OPENTYPE\ \quote
-{MathVariants} as well as \TFM\ \quote {extensible recipes}. In most cases where
-font metrics are involved we have a different code path for traditional fonts end
-\OPENTYPE\ fonts.
-
-\section {Extracting values}
-
-You can extract the components of a math character. Say that we have defined:
-
-\starttyping
-\Umathcode 1 2 3 4
-\stoptyping
-
-then
-
-\starttyping
-[\Umathcharclass1] [\Umathcharfam1] [\Umathcharslot1]
-\stoptyping
-
-will return:
-
-\starttyping
-[2] [3] [4]
-\stoptyping
-
-These commands are provides as convenience. Before they came available you could
-do the following:
-
-\starttyping
-\def\Umathcharclass{\directlua{tex.print(tex.getmathcode(token.scan_int())[1])}}
-\def\Umathcharfam  {\directlua{tex.print(tex.getmathcode(token.scan_int())[2])}}
-\def\Umathcharslot {\directlua{tex.print(tex.getmathcode(token.scan_int())[3])}}
-\stoptyping
-
-\section{fractions}
-
-The \type {\abovewithdelims} command accepts a keyword \type {exact}. When issued
-the extra space relative to the rule thickness is not added. One can of course
-use the \type {\Umathfraction..gap} commands to influence the spacing. Also the
-rule is still positioned around the math axis.
-
-\starttyping
-$$ { {a} \abovewithdelims() exact 4pt {b} }$$
-\stoptyping
-
-The math parameter table contains some parameters that specify a horizontal and
-vertical gap for skewed fractions. Of course some guessing is needed in order to
-implement something that uses them. And so we now provide a primitive similar to the
-other fraction related ones but with a few options so that one can influence the
-rendering. Of course a user can also mess around a bit with the parameters
-\type {\Umathskewedfractionhgap} and \type {\Umathskewedfractionvgap}.
-
-The syntax used here is:
-
-\starttyping
-{ {1} \Uskewed / <options> {2} }
-{ {1} \Uskewedwithdelims / () <options> {2} }
-\stoptyping
-
-where the options can be \type {noaxis} and \type {exact}. By default we add half
-the axis to the shifts and by default we zero the width of the middle character.
-For Latin Modern The result looks as follows:
-
-\def\ShowA#1#2#3{$x + { {#1} \Uskewed           /    #3 {#2} } + x$}
-\def\ShowB#1#2#3{$x + { {#1} \Uskewedwithdelims / () #3 {#2} } + x$}
-
-\start
-    \switchtobodyfont[modern]
-    \starttabulate[||||||]
-        \NC \NC
-            \ShowA{a}{b}{} \NC
-            \ShowA{1}{2}{} \NC
-            \ShowB{a}{b}{} \NC
-            \ShowB{1}{2}{} \NC
-        \NR
-        \NC \type{exact} \NC
-            \ShowA{a}{b}{exact} \NC
-            \ShowA{1}{2}{exact} \NC
-            \ShowB{a}{b}{exact} \NC
-            \ShowB{1}{2}{exact} \NC
-        \NR
-        \NC \type{noaxis} \NC
-            \ShowA{a}{b}{noaxis} \NC
-            \ShowA{1}{2}{noaxis} \NC
-            \ShowB{a}{b}{noaxis} \NC
-            \ShowB{1}{2}{noaxis} \NC
-        \NR
-        \NC \type{exact noaxis} \NC
-            \ShowA{a}{b}{exact noaxis} \NC
-            \ShowA{1}{2}{exact noaxis} \NC
-            \ShowB{a}{b}{exact noaxis} \NC
-            \ShowB{1}{2}{exact noaxis} \NC
-        \NR
-    \stoptabulate
-\stop
-
-\section {Last lines}
-
-There is a new primitive to control the overshoot in the calculation of the
-previous line in mid|-|paragraph display math. The default value is 2 times
-the em width of the current font:
-
-\starttyping
-\predisplaygapfactor=2000
-\stoptyping
-
-If you want to have the length of the last line independent of math i.e.\ you don't
-want to revert to a hack where you insert a fake display math formula in order to
-get the length of the last line, the following will often work too:
-
-\starttyping
-\def\lastlinelength{\dimexpr
-    \directlua {tex.sprint (
-        (nodes.dimensions(node.tail(tex.lists.page_head).list))
-    )}sp
-\relax}
-\stoptyping
-
-\section {Other Math changes}
-
-\subsection {Verbose versions of single-character math commands}
-
-\LUATEX\ defines six new primitives that have the same function as
-\type {^}, \type {_}, \type {$}, and \type {$$}:
-
-\starttabulate[|l|l|]
-\BC primitive                  \BC explanation \NC \NR
-\NC \type {\Usuperscript}      \NC Duplicates the functionality of \type {^} \NC \NR
-\NC \type {\Usubscript}        \NC Duplicates the functionality of \type {_} \NC \NR
-\NC \type {\Ustartmath}        \NC Duplicates the functionality of \type {$}, % $
-                                   when used in non-math mode. \NC \NR
-\NC \type {\Ustopmath}         \NC Duplicates the functionality of \type {$}, % $
-                                   when used in inline math mode. \NC \NR
-\NC \type {\Ustartdisplaymath} \NC Duplicates the functionality of \type {$$}, % $$
-                                   when used in non-math mode. \NC \NR
-\NC \type {\Ustopdisplaymath}  \NC Duplicates the functionality of \type {$$}, % $$
-                                   when used in display math mode. \NC \NR
-\stoptabulate
-
-The \type {\Ustopmath} and \type {\Ustopdisplaymath} primitives check if the current
-math mode is the correct one (inline vs.\ displayed), but you can freely intermix
-the four mathon|/|mathoff commands with explicit dollar sign(s).
-
-\subsection{Script commands \type {\Unosuperscript} and \type {\Unosubscript}}
-
-These two commands result in super- and subscripts but with the current style (at the
-time of rendering). So,
-
-\startbuffer[script]
-$
-    x\Usuperscript  {1}\Usubscript  {2} =
-    x\Unosuperscript{1}\Unosubscript{2} =
-    x\Usuperscript  {1}\Unosubscript{2} =
-    x\Unosuperscript{1}\Usubscript  {2}
-$
-\stopbuffer
-
-\typebuffer
-
-results in \inlinebuffer[script].
-
-
-\subsection{Allowed math commands in non-math modes}
-
-The commands \type {\mathchar}, and \type {\Umathchar} and control sequences that
-are the result of \type {\mathchardef} or \type {\Umathchardef} are also
-acceptable in the horizontal and vertical modes. In those cases, the \type
-{\textfont} from the requested math family is used.
-
-\section{Math surrounding skips}
-
-Inline math is surrounded by (optional) \type {\mathsurround} spacing but that is fixed
-dimension. There is now an additional parameter \type {\mathsurroundskip}. When set to a
-non|-|zero value (or zero with some stretch or shrink) this parameter will replace
-\type {\mathsurround}. By using an additional parameter instead of changing the nature
-of \type {\mathsurround}, we can remain compatible. In the meantime a bit more
-control has been added via \type {\mathsurroundmode}. This directive can take 6 values
-with zero being the default behaviour.
-
-\start
-
-\def\OneLiner#1#2%
-  {\NC \type{#1}
-   \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt   x$x$x}
-   \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x $x$ x}
-   \NC #2
-   \NC \NR}
-
-\startbuffer
-\mathsurround    10pt
-\mathsurroundskip20pt
-\stopbuffer
-
-\typebuffer \getbuffer
-
-\starttabulate[|c|c|c|pl|]
-\HL
-\BC mode \BC \type {x$x$x} \BC \type {x $x$ x} \BC effect \NC \NR
-\HL
-\OneLiner{0}{obey \type {\mathsurround} when \type {\mathsurroundskip} is 0pt}
-\OneLiner{1}{only add skip to the left}
-\OneLiner{2}{only add skip to the right}
-\OneLiner{3}{add skip to the left and right}
-\OneLiner{4}{ignore the skip setting, obey \type {\mathsurround}}
-\OneLiner{5}{disable all spacing around math}
-\OneLiner{6}{only apply \type {\mathsurroundskip} when also spacing}
-\OneLiner{7}{only apply \type {\mathsurroundskip} when no spacing}
-\HL
-\stoptabulate
-
-\stop
-
-Method six omits the surround glue when there is (x)spacing glue present while
-method seven does the opposite, the glue is only applied when there is (x)space
-glue present too. Anything more fancy, like checking the begining or end of a
-paragraph (or edges of a box) would not be robust anyway. If you want that you
-can write a callback that runs over a list and analyzes a paragraph. Actually, in
-that case you could also inject glue (or set the properties of a math node)
-explicitly. So, these modes are in practice mostly useful for special purposes
-and experiments (they originate in a tracker item). Keep in mind that this glue
-is part of the math node and not always treated as normal glue: it travels with
-the begin and end math nodes. Also, method 6 and 7 will zero the skip related
-fields in a node when applicable in the first occasion that checks them
-(linebreaking or packaging).
-
-% \section{Math todo}
-%
-% The following items are still todo.
-%
-% \startitemize
-% \startitem
-%     Pre-scripts.
-% \stopitem
-% \startitem
-%     Multi-story stacks.
-% \stopitem
-% \startitem
-%     Flattened accents for high characters (maybe).
-% \stopitem
-% \startitem
-%     Better control over the spacing around displays and handling of equation numbers.
-% \stopitem
-% \startitem
-%     Support for multi|-|line displays using \MATHML\ style alignment points.
-% \stopitem
-% \stopitemize
-
-\subsection {Delimiters: \type{\Uleft}, \type {\Umiddle} and \type {\Uright}}
-
-Normally you will force delimiters to certain sizes by putting an empty box or
-rule next to it. The resulting delimiter will either be a character from the
-stepwise size range or an extensible. The latter can be quite differently
-positioned that the characters as it depends on the fit as well as the fact if
-the used characters in the font have depth or height. Commands like (plain \TEX
-s) \type {\big} need use this feature. In \LUATEX\ we provide a bit more control
-by three variants that supporting optional parameters \type {height}, \type
-{depth} and \type {axis}. The following example uses this:
-
-\startbuffer
-\Uleft   height 30pt depth 10pt      \Udelimiter "0 "0 "000028
-\quad x\quad
-\Umiddle height 40pt depth 15pt      \Udelimiter "0 "0 "002016
-\quad x\quad
-\Uright  height 30pt depth 10pt      \Udelimiter "0 "0 "000029
-\quad \quad \quad
-\Uleft   height 30pt depth 10pt axis \Udelimiter "0 "0 "000028
-\quad x\quad
-\Umiddle height 40pt depth 15pt axis \Udelimiter "0 "0 "002016
-\quad x\quad
-\Uright  height 30pt depth 10pt axis \Udelimiter "0 "0 "000029
-\stopbuffer
-
-\typebuffer
-
-\startlinecorrection
-\ruledhbox{\mathematics{\getbuffer}}
-\stoplinecorrection
-
-The keyword \type {exact} can be used as directive that the real dimensions
-should be applied when the criteria can't be met which can happen when we're
-still stepping through the successively larger variants. When no dimensions are
-given the \type {noaxis} command can be used to prevent shifting over the axis.
-
-You can influence the final class with the keyword \type {class} which will
-influence the spacing. The numbers are the same as for character classes.
-
-\subsection{Fixed scripts}
-
-We have three parameters that are used for this fixed anchoring:
-
-\starttabulate[|l|l|]
-\NC $d$ \NC \type {\Umathsubshiftdown}    \NC \NR
-\NC $u$ \NC \type {\Umathsupshiftup}      \NC \NR
-\NC $s$ \NC \type {\Umathsubsupshiftdown} \NC \NR
-\stoptabulate
-
-When we set \type {\mathscriptsmode} to a value other than zero these are used
-for calculating fixed positions. This is something that is needed for instance
-for chemistry. You can manipulate the mentioned variables to achive different
-effects.
-
-\def\SampleMath#1%
-  {$\mathscriptsmode#1\mathupright CH_2 + CH^+_2 + CH^2_2$}
-
-\starttabulate[|c|c|c|l|]
-\BC mode \BC down          \BC up            \BC                \NC \NR
-\NC 0    \NC dynamic       \NC dynamic       \NC \SampleMath{0} \NC \NR
-\NC 1    \NC $d$           \NC $u$           \NC \SampleMath{1} \NC \NR
-\NC 2    \NC $s$           \NC $u$           \NC \SampleMath{2} \NC \NR
-\NC 3    \NC $s$           \NC $u + s - d$   \NC \SampleMath{3} \NC \NR
-\NC 4    \NC $d + (s-d)/2$ \NC $u + (s-d)/2$ \NC \SampleMath{4} \NC \NR
-\NC 5    \NC $d$           \NC $u + s - d$   \NC \SampleMath{5} \NC \NR
-\stoptabulate
-
-The value of this parameter obeys grouping but applies to the whole current
-formula.
-
-% if needed we can put the value in stylenodes but maybe more should go there
-
-\subsection{Penalties: \type {\mathpenaltiesmode}}
-
-Only in inline math penalties will be added in a math list. You can force
-penalties (also in display math) by setting:
-
-\starttyping
-\mathpenaltiesmode = 1
-\stoptyping
-
-This primnitive is not really needed in \LUATEX\ because you can use the callback
-\type {mlist_to_hlist} to force penalties by just calling the regular routine
-with forced penalties. However, as part of opening up and control this primitive
-makes sense. As a bonus we also provide two extra penalties:
-
-\starttyping
-\prebinoppenalty = -100 % example value
-\prerelpenalty   =  900 % example value
-\stoptyping
-
-They default to inifinite which signals that they don't need to be inserted. When
-set they are injected before a binop or rel noad. This is an experimental feature.
-
-\subsection {Tracing}
-
-Because there are quite some math related parameters and values, it is possible
-to limit tracing. Only when \type {tracingassigns} and|/|or \type
-{tracingrestores} are set to~2 or more they will be traced.
-
-\subsection {Math options}
-
-The logic in the math engine is rather complex and there are often no universal
-solutions (read: what works out well for one font, fails for another). Therefore
-some variations in the implementation will be driven by options for which a new
-primitive \type {\mathoption} has been introduced (so that we don't end up with
-many new commands). The approach of options also permits us to see what effect a
-specific solution has.
-
-\subsubsection {\type {\mathoption old}}
-
-This option was introduced for testing purposes when the math engine got split
-code paths and it forces the engine to treat new fonts as old ones with respect
-to italic correction etc. There are no guarantees given with respect to the final
-result and unexpected side effects are not seens as bugs as they relate to font
-properties.
-
-\startbuffer
-\mathoption old 1
-\stopbuffer
-
-The \type {oldmath} boolean flag in the \LUA\ font table is the official way to
-force old treatment as it's bound to fonts.
-
-\subsubsection {\type {\mathoption noitaliccompensation}}
-
-This option compensates placement for characters with a built|-|in italic
-correction.
-
-\startbuffer
-{\showboxes\int}\quad
-{\showboxes\int_{|}^{|}}\quad
-{\showboxes\int\limits_{|}^{|}}
-\stopbuffer
-
-\typebuffer
-
-Gives (with computer modern that has such italics):
-
-\startlinecorrection[blank]
-    \switchtobodyfont[modern]
-    \startcombination[nx=2,ny=2,distance=5em]
-        {\mathoption noitaliccompensation 0\relax \mathematics{\getbuffer}}
-            {\nohyphens\type{0:inline}}
-        {\mathoption noitaliccompensation 0\relax \mathematics{\displaymath\getbuffer}}
-            {\nohyphens\type{0:display}}
-        {\mathoption noitaliccompensation 1\relax \mathematics{\getbuffer}}
-            {\nohyphens\type{1:inline}}
-        {\mathoption noitaliccompensation 1\relax \mathematics{\displaymath\getbuffer}}
-            {\nohyphens\type{1:display}}
-    \stopcombination
-\stoplinecorrection
-
-\subsubsection {\type {\mathoption nocharitalic}}
-
-When two characters follow each other italic correction can interfere. The
-following example shows what this option does:
-
-\startbuffer
-\catcode"1D443=11
-\catcode"1D444=11
-\catcode"1D445=11
-P( PP PQR
-\stopbuffer
-
-\typebuffer
-
-Gives (with computer modern that has such italics):
-
-\startlinecorrection[blank]
-    \switchtobodyfont[modern]
-    \startcombination[nx=2,ny=2,distance=5em]
-        {\mathoption nocharitalic 0\relax \mathematics{\getbuffer}}
-            {\nohyphens\type{0:inline}}
-        {\mathoption nocharitalic 0\relax \mathematics{\displaymath\getbuffer}}
-            {\nohyphens\type{0:display}}
-        {\mathoption nocharitalic 1\relax \mathematics{\getbuffer}}
-            {\nohyphens\type{1:inline}}
-        {\mathoption nocharitalic 1\relax \mathematics{\displaymath\getbuffer}}
-            {\nohyphens\type{1:display}}
-    \stopcombination
-\stoplinecorrection
-
-\subsubsection {\type {\mathoption useoldfractionscaling}}
-
-This option has been introduced as solution for tracker item 604 for fuzzy cases
-around either or not present fraction related settings for new fonts.
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-math

+

+\startchapter[reference=math,title={Math}]

+

+\startsection[title={Traditional alongside \OPENTYPE}]

+

+\topicindex {math}

+

+The handling of mathematics in \LUATEX\ differs quite a bit from how \TEX82 (and

+therefore \PDFTEX) handles math. First, \LUATEX\ adds primitives and extends some

+others so that \UNICODE\ input can be used easily. Second, all of \TEX82's

+internal special values (for example for operator spacing) have been made

+accessible and changeable via control sequences. Third, there are extensions that

+make it easier to use \OPENTYPE\ math fonts. And finally, there are some

+extensions that have been proposed or considered in the past that are now added

+to the engine.

+

+\stopsection

+

+\startsection[title={Unicode math characters}]

+

+\topicindex {math+\UNICODE}

+\topicindex {\UNICODE+math}

+

+Character handling is now extended up to the full \UNICODE\ range (the \type {\U}

+prefix), which is compatible with \XETEX.

+

+The math primitives from \TEX\ are kept as they are, except for the ones that

+convert from input to math commands: \type {mathcode}, and \type {delcode}. These

+two now allow for a 21-bit character argument on the left hand side of the equals

+sign.

+

+Some of the new \LUATEX\ primitives read more than one separate value. This is

+shown in the tables below by a plus sign.

+

+The input for such primitives would look like this:

+

+\starttyping

+\def\overbrace{\Umathaccent 0 1 "23DE }

+\stoptyping

+

+The altered \TEX82 primitives are:

+

+\starttabulate[|l|l|r|c|l|r|]

+\DB primitive       \BC min \BC max    \BC \kern 2em \BC min \BC max    \NC \NR

+\TB

+\NC \prm {mathcode} \NC 0   \NC 10FFFF \NC =         \NC 0   \NC 8000   \NC \NR

+\NC \prm {delcode}  \NC 0   \NC 10FFFF \NC =         \NC 0   \NC FFFFFF \NC \NR

+\LL

+\stoptabulate

+

+The unaltered ones are:

+

+\starttabulate[|l|l|r|]

+\DB primitive          \BC min \BC max     \NC \NR

+\TB

+\NC \prm {mathchardef} \NC 0   \NC    8000 \NC \NR

+\NC \prm {mathchar}    \NC 0   \NC    7FFF \NC \NR

+\NC \prm {mathaccent}  \NC 0   \NC    7FFF \NC \NR

+\NC \prm {delimiter}   \NC 0   \NC 7FFFFFF \NC \NR

+\NC \prm {radical}     \NC 0   \NC 7FFFFFF \NC \NR

+\LL

+\stoptabulate

+

+For practical reasons \prm {mathchardef} will silently accept values larger

+that \type {0x8000} and interpret it as \lpr {Umathcharnumdef}. This is needed

+to satisfy older macro packages.

+

+The following new primitives are compatible with \XETEX:

+

+% somewhat fuzzy:

+

+\starttabulate[|l|l|r|c|l|r|]

+\DB primitive                             \BC min       \BC max         \BC \kern 2em \BC min       \BC max         \NC \NR

+\TB

+\NC \lpr {Umathchardef}                   \NC 0+0+0     \NC 7+FF+10FFFF \NC           \NC           \NC             \NC \NR

+\NC \lpr {Umathcharnumdef}\rlap{\high{5}} \NC -80000000 \NC    7FFFFFFF \NC           \NC           \NC             \NC \NR

+\NC \lpr {Umathcode}                      \NC 0         \NC      10FFFF \NC =         \NC 0+0+0     \NC 7+FF+10FFFF \NC \NR

+\NC \lpr {Udelcode}                       \NC 0         \NC      10FFFF \NC =         \NC 0+0       \NC   FF+10FFFF \NC \NR

+\NC \lpr {Umathchar}                      \NC 0+0+0     \NC 7+FF+10FFFF \NC           \NC           \NC             \NC \NR

+\NC \lpr {Umathaccent}                    \NC 0+0+0     \NC 7+FF+10FFFF \NC           \NC           \NC             \NC \NR

+\NC \lpr {Udelimiter}                     \NC 0+0+0     \NC 7+FF+10FFFF \NC           \NC           \NC             \NC \NR

+\NC \lpr {Uradical}                       \NC 0+0       \NC   FF+10FFFF \NC           \NC           \NC             \NC \NR

+\NC \lpr {Umathcharnum}                   \NC -80000000 \NC    7FFFFFFF \NC           \NC           \NC             \NC \NR

+\NC \lpr {Umathcodenum}                   \NC 0         \NC      10FFFF \NC =         \NC -80000000 \NC    7FFFFFFF \NC \NR

+\NC \lpr {Udelcodenum}                    \NC 0         \NC      10FFFF \NC =         \NC -80000000 \NC    7FFFFFFF \NC \NR

+\LL

+\stoptabulate

+

+Specifications typically look like:

+

+\starttyping

+\Umathchardef\xx="1"0"456

+\Umathcode   123="1"0"789

+\stoptyping

+

+The new primitives that deal with delimiter|-|style objects do not set up a

+\quote {large family}. Selecting a suitable size for display purposes is expected

+to be dealt with by the font via the \lpr {Umathoperatorsize} parameter.

+

+For some of these primitives, all information is packed into a single signed

+integer. For the first two (\lpr {Umathcharnum} and \lpr {Umathcodenum}), the

+lowest 21 bits are the character code, the 3 bits above that represent the math

+class, and the family data is kept in the topmost bits. This means that the values

+for math families 128--255 are actually negative. For \lpr {Udelcodenum} there

+is no math class. The math family information is stored in the bits directly on

+top of the character code. Using these three commands is not as natural as using

+the two- and three|-|value commands, so unless you know exactly what you are

+doing and absolutely require the speedup resulting from the faster input

+scanning, it is better to use the verbose commands instead.

+

+The \lpr {Umathaccent} command accepts optional keywords to control various

+details regarding math accents. See \in {section} [mathacc] below for details.

+

+There are more new primitives and all of these will be explained in following

+sections:

+

+\starttabulate[|l|l|]

+\DB primitive                \BC value range (in hex) \NC \NR

+\TB

+\NC \lpr {Uroot}           \NC 0 + 0--FF + 10FFFF   \NC \NR

+\NC \lpr {Uoverdelimiter}  \NC 0 + 0--FF + 10FFFF   \NC \NR

+\NC \lpr {Uunderdelimiter} \NC 0 + 0--FF + 10FFFF   \NC \NR

+\NC \lpr {Udelimiterover}  \NC 0 + 0--FF + 10FFFF   \NC \NR

+\NC \lpr {Udelimiterunder} \NC 0 + 0--FF + 10FFFF   \NC \NR

+\LL

+\stoptabulate

+

+\stopsection

+

+\startsection[title={Math styles}]

+

+\subsection{\lpr {mathstyle}}

+

+\topicindex {math+styles}

+

+It is possible to discover the math style that will be used for a formula in an

+expandable fashion (while the math list is still being read). To make this

+possible, \LUATEX\ adds the new primitive: \lpr {mathstyle}. This is a \quote

+{convert command} like e.g. \prm {romannumeral}: its value can only be read,

+not set.

+

+The returned value is between 0 and 7 (in math mode), or $-1$ (all other modes).

+For easy testing, the eight math style commands have been altered so that they can

+be used as numeric values, so you can write code like this:

+

+\starttyping

+\ifnum\mathstyle=\textstyle

+    \message{normal text style}

+\else \ifnum\mathstyle=\crampedtextstyle

+    \message{cramped text style}

+\fi \fi

+\stoptyping

+

+Sometimes you won't get what you expect so a bit of explanation might help to

+understand what happens. When math is parsed and expanded it gets turned into a

+linked list. In a second pass the formula will be build. This has to do with the

+fact that in order to determine the automatically chosen sizes (in for instance

+fractions) following content can influence preceding sizes. A side effect of this

+is for instance that one cannot change the definition of a font family (and

+thereby reusing numbers) because the number that got used is stored and used in

+the second pass (so changing \type {\fam 12} mid|-|formula spoils over to

+preceding use of that family).

+

+The style switching primitives like \prm {textstyle} are turned into nodes so the

+styles set there are frozen. The \prm {mathchoice} primitive results in four

+lists being constructed of which one is used in the second pass. The fact that

+some automatic styles are not yet known also means that the \lpr {mathstyle}

+primitive expands to the current style which can of course be different from the

+one really used. It's a snapshot of the first pass state. As a consequence in the

+following example you get a style number (first pass) typeset that can actually

+differ from the used style (second pass). In the case of a math choice used

+ungrouped, the chosen style is used after the choice too, unless you group.

+

+\startbuffer[1]

+    [a:\mathstyle]\quad

+    \bgroup

+    \mathchoice

+        {\bf \scriptstyle       (x:d :\mathstyle)}

+        {\bf \scriptscriptstyle (x:t :\mathstyle)}

+        {\bf \scriptscriptstyle (x:s :\mathstyle)}

+        {\bf \scriptscriptstyle (x:ss:\mathstyle)}

+    \egroup

+    \quad[b:\mathstyle]\quad

+    \mathchoice

+        {\bf \scriptstyle       (y:d :\mathstyle)}

+        {\bf \scriptscriptstyle (y:t :\mathstyle)}

+        {\bf \scriptscriptstyle (y:s :\mathstyle)}

+        {\bf \scriptscriptstyle (y:ss:\mathstyle)}

+    \quad[c:\mathstyle]\quad

+    \bgroup

+    \mathchoice

+        {\bf \scriptstyle       (z:d :\mathstyle)}

+        {\bf \scriptscriptstyle (z:t :\mathstyle)}

+        {\bf \scriptscriptstyle (z:s :\mathstyle)}

+        {\bf \scriptscriptstyle (z:ss:\mathstyle)}

+    \egroup

+    \quad[d:\mathstyle]

+\stopbuffer

+

+\startbuffer[2]

+    [a:\mathstyle]\quad

+    \begingroup

+    \mathchoice

+        {\bf \scriptstyle       (x:d :\mathstyle)}

+        {\bf \scriptscriptstyle (x:t :\mathstyle)}

+        {\bf \scriptscriptstyle (x:s :\mathstyle)}

+        {\bf \scriptscriptstyle (x:ss:\mathstyle)}

+    \endgroup

+    \quad[b:\mathstyle]\quad

+    \mathchoice

+        {\bf \scriptstyle       (y:d :\mathstyle)}

+        {\bf \scriptscriptstyle (y:t :\mathstyle)}

+        {\bf \scriptscriptstyle (y:s :\mathstyle)}

+        {\bf \scriptscriptstyle (y:ss:\mathstyle)}

+    \quad[c:\mathstyle]\quad

+    \begingroup

+    \mathchoice

+        {\bf \scriptstyle       (z:d :\mathstyle)}

+        {\bf \scriptscriptstyle (z:t :\mathstyle)}

+        {\bf \scriptscriptstyle (z:s :\mathstyle)}

+        {\bf \scriptscriptstyle (z:ss:\mathstyle)}

+    \endgroup

+    \quad[d:\mathstyle]

+\stopbuffer

+

+\typebuffer[1]

+

+% \typebuffer[2]

+

+This gives:

+

+\blank $\displaystyle \getbuffer[1]$ \blank

+\blank $\textstyle    \getbuffer[1]$ \blank

+

+Using \prm {begingroup} \unknown\ \prm {endgroup} instead gives:

+

+\blank $\displaystyle \getbuffer[2]$ \blank

+\blank $\textstyle    \getbuffer[2]$ \blank

+

+This might look wrong but it's just a side effect of \lpr {mathstyle} expanding

+to the current (first pass) style and the number being injected in the list that

+gets converted in the second pass. It all makes sense and it illustrates the

+importance of grouping. In fact, the math choice style being effective afterwards

+has advantages. It would be hard to get it otherwise.

+

+\subsection{\lpr {Ustack}}

+

+\topicindex {math+stacks}

+

+There are a few math commands in \TEX\ where the style that will be used is not

+known straight from the start. These commands (\prm {over}, \prm {atop},

+\prm {overwithdelims}, \prm {atopwithdelims}) would therefore normally return

+wrong values for \lpr {mathstyle}. To fix this, \LUATEX\ introduces a special

+prefix command: \lpr {Ustack}:

+

+\starttyping

+$\Ustack {a \over b}$

+\stoptyping

+

+The \lpr {Ustack} command will scan the next brace and start a new math group

+with the correct (numerator) math style.

+

+\subsection{Cramped math styles}

+

+\topicindex {math+styles}

+\topicindex {math+spacing}

+\topicindex {math+cramped}

+

+\LUATEX\ has four new primitives to set the cramped math styles directly:

+

+\starttyping

+\crampeddisplaystyle

+\crampedtextstyle

+\crampedscriptstyle

+\crampedscriptscriptstyle

+\stoptyping

+

+These additional commands are not all that valuable on their own, but they come

+in handy as arguments to the math parameter settings that will be added shortly.

+

+In Eijkhouts \quotation {\TEX\ by Topic} the rules for handling styles in scripts

+are described as follows:

+

+\startitemize

+\startitem

+    In any style superscripts and subscripts are taken from the next smaller style.

+    Exception: in display style they are in script style.

+\stopitem

+\startitem

+    Subscripts are always in the cramped variant of the style; superscripts are only

+    cramped if the original style was cramped.

+\stopitem

+\startitem

+    In an \type {..\over..} formula in any style the numerator and denominator are

+    taken from the next smaller style.

+\stopitem

+\startitem

+    The denominator is always in cramped style; the numerator is only in cramped

+    style if the original style was cramped.

+\stopitem

+\startitem

+    Formulas under a \type {\sqrt} or \prm {overline} are in cramped style.

+\stopitem

+\stopitemize

+

+In \LUATEX\ one can set the styles in more detail which means that you sometimes

+have to set both normal and cramped styles to get the effect you want. (Even) if

+we force styles in the script using \prm {scriptstyle} and \lpr

+{crampedscriptstyle} we get this:

+

+\startbuffer[demo]

+\starttabulate

+\DB style         \BC example \NC \NR

+\TB

+\NC default       \NC $b_{x=xx}^{x=xx}$ \NC \NR

+\NC script        \NC $b_{\scriptstyle x=xx}^{\scriptstyle x=xx}$ \NC \NR

+\NC crampedscript \NC $b_{\crampedscriptstyle x=xx}^{\crampedscriptstyle x=xx}$ \NC \NR

+\LL

+\stoptabulate

+\stopbuffer

+

+\getbuffer[demo]

+

+Now we set the following parameters

+

+\startbuffer[setup]

+\Umathordrelspacing\scriptstyle=30mu

+\Umathordordspacing\scriptstyle=30mu

+\stopbuffer

+

+\typebuffer[setup]

+

+This gives a different result:

+

+\start\getbuffer[setup,demo]\stop

+

+But, as this is not what is expected (visually) we should say:

+

+\startbuffer[setup]

+\Umathordrelspacing\scriptstyle=30mu

+\Umathordordspacing\scriptstyle=30mu

+\Umathordrelspacing\crampedscriptstyle=30mu

+\Umathordordspacing\crampedscriptstyle=30mu

+\stopbuffer

+

+\typebuffer[setup]

+

+Now we get:

+

+\start\getbuffer[setup,demo]\stop

+

+\stopsection

+

+\startsection[title={Math parameter settings}]

+

+\subsection {Many new \lpr {Umath*} primitives}

+

+\topicindex {math+parameters}

+

+In \LUATEX, the font dimension parameters that \TEX\ used in math typesetting are

+now accessible via primitive commands. In fact, refactoring of the math engine

+has resulted in many more parameters than were not accessible before.

+

+\starttabulate

+\DB primitive name                   \BC description \NC \NR

+\TB

+\NC \lpr {Umathquad}               \NC the width of 18 mu's \NC \NR

+\NC \lpr {Umathaxis}               \NC height of the vertical center axis of

+                                         the math formula above the baseline \NC \NR

+\NC \lpr {Umathoperatorsize}       \NC minimum size of large operators in display mode \NC \NR

+\NC \lpr {Umathoverbarkern}        \NC vertical clearance above the rule \NC \NR

+\NC \lpr {Umathoverbarrule}        \NC the width of the rule \NC \NR

+\NC \lpr {Umathoverbarvgap}        \NC vertical clearance below the rule \NC \NR

+\NC \lpr {Umathunderbarkern}       \NC vertical clearance below the rule \NC \NR

+\NC \lpr {Umathunderbarrule}       \NC the width of the rule \NC \NR

+\NC \lpr {Umathunderbarvgap}       \NC vertical clearance above the rule \NC \NR

+\NC \lpr {Umathradicalkern}        \NC vertical clearance above the rule \NC \NR

+\NC \lpr {Umathradicalrule}        \NC the width of the rule \NC \NR

+\NC \lpr {Umathradicalvgap}        \NC vertical clearance below the rule \NC \NR

+\NC \lpr {Umathradicaldegreebefore}\NC the forward kern that takes place before placement of

+                                       the radical degree \NC \NR

+\NC \lpr {Umathradicaldegreeafter} \NC the backward kern that takes place after placement of

+                                       the radical degree \NC \NR

+\NC \lpr {Umathradicaldegreeraise} \NC this is the percentage of the total height and depth of

+                                       the radical sign that the degree is raised by; it is

+                                       expressed in \type {percents}, so 60\% is expressed as the

+                                       integer $60$ \NC \NR

+\NC \lpr {Umathstackvgap}          \NC vertical clearance between the two

+                                       elements in a \prm {atop} stack \NC \NR

+\NC \lpr {Umathstacknumup}         \NC numerator shift upward in \prm {atop} stack \NC \NR

+\NC \lpr {Umathstackdenomdown}     \NC denominator shift downward in \prm {atop} stack \NC \NR

+\NC \lpr {Umathfractionrule}       \NC the width of the rule in a \prm {over} \NC \NR

+\NC \lpr {Umathfractionnumvgap}    \NC vertical clearance between the numerator and the rule \NC \NR

+\NC \lpr {Umathfractionnumup}      \NC numerator shift upward in \prm {over} \NC \NR

+\NC \lpr {Umathfractiondenomvgap}  \NC vertical clearance between the denominator and the rule \NC \NR

+\NC \lpr {Umathfractiondenomdown}  \NC denominator shift downward in \prm {over} \NC \NR

+\NC \lpr {Umathfractiondelsize}    \NC minimum delimiter size for \type {\...withdelims} \NC \NR

+\NC \lpr {Umathlimitabovevgap}     \NC vertical clearance for limits above operators \NC \NR

+\NC \lpr {Umathlimitabovebgap}     \NC vertical baseline clearance for limits above operators \NC \NR

+\NC \lpr {Umathlimitabovekern}     \NC space reserved at the top of the limit \NC \NR

+\NC \lpr {Umathlimitbelowvgap}     \NC vertical clearance for limits below operators \NC \NR

+\NC \lpr {Umathlimitbelowbgap}     \NC vertical baseline clearance for limits below operators \NC \NR

+\NC \lpr {Umathlimitbelowkern}     \NC space reserved at the bottom of the limit \NC \NR

+\NC \lpr {Umathoverdelimitervgap}  \NC vertical clearance for limits above delimiters \NC \NR

+\NC \lpr {Umathoverdelimiterbgap}  \NC vertical baseline clearance for limits above delimiters \NC \NR

+\NC \lpr {Umathunderdelimitervgap} \NC vertical clearance for limits below delimiters \NC \NR

+\NC \lpr {Umathunderdelimiterbgap} \NC vertical baseline clearance for limits below delimiters \NC \NR

+\NC \lpr {Umathsubshiftdrop}       \NC subscript drop for boxes and subformulas \NC \NR

+\NC \lpr {Umathsubshiftdown}       \NC subscript drop for characters \NC \NR

+\NC \lpr {Umathsupshiftdrop}       \NC superscript drop (raise, actually) for boxes and subformulas \NC \NR

+\NC \lpr {Umathsupshiftup}         \NC superscript raise for characters \NC \NR

+\NC \lpr {Umathsubsupshiftdown}    \NC subscript drop in the presence of a superscript \NC \NR

+\NC \lpr {Umathsubtopmax}          \NC the top of standalone subscripts cannot be higher than this

+                                       above the baseline \NC \NR

+\NC \lpr {Umathsupbottommin}       \NC the bottom of standalone superscripts cannot be less than

+                                       this above the baseline \NC \NR

+\NC \lpr {Umathsupsubbottommax}    \NC the bottom of the superscript of a combined super- and subscript

+                                       be at least as high as this above the baseline \NC \NR

+\NC \lpr {Umathsubsupvgap}         \NC vertical clearance between super- and subscript \NC \NR

+\NC \lpr {Umathspaceafterscript}   \NC additional space added after a super- or subscript \NC \NR

+\NC \lpr {Umathconnectoroverlapmin}\NC minimum overlap between parts in an extensible recipe \NC \NR

+\LL

+\stoptabulate

+

+Each of the parameters in this section can be set by a command like this:

+

+\starttyping

+\Umathquad\displaystyle=1em

+\stoptyping

+

+they obey grouping, and you can use \type {\the\Umathquad\displaystyle} if

+needed.

+

+\subsection{Font|-|based math parameters}

+

+\topicindex {math+parameters}

+

+While it is nice to have these math parameters available for tweaking, it would

+be tedious to have to set each of them by hand. For this reason, \LUATEX\

+initializes a bunch of these parameters whenever you assign a font identifier to

+a math family based on either the traditional math font dimensions in the font

+(for assignments to math family~2 and~3 using \TFM|-|based fonts like \type

+{cmsy} and \type {cmex}), or based on the named values in a potential \type

+{MathConstants} table when the font is loaded via Lua. If there is a \type

+{MathConstants} table, this takes precedence over font dimensions, and in that

+case no attention is paid to which family is being assigned to: the \type

+{MathConstants} tables in the last assigned family sets all parameters.

+

+In the table below, the one|-|letter style abbreviations and symbolic tfm font

+dimension names match those used in the \TeX book. Assignments to \prm

+{textfont} set the values for the cramped and uncramped display and text styles,

+\prm {scriptfont} sets the script styles, and \prm {scriptscriptfont} sets the

+scriptscript styles, so we have eight parameters for three font sizes. In the

+\TFM\ case, assignments only happen in family~2 and family~3 (and of course only

+for the parameters for which there are font dimensions).

+

+Besides the parameters below, \LUATEX\ also looks at the \quote {space} font

+dimension parameter. For math fonts, this should be set to zero.

+

+\def\MathLine#1#2#3#4#5%

+  {\TB

+   \NC \llap{\high{\tx #2\enspace}}\ttbf \string #1 \NC \tt #5 \NC \NR

+   \NC \tx #3 \NC \tt #4 \NC \NR}

+

+\starttabulate[|l|l|]

+\DB variable / style \BC tfm / opentype \NC \NR

+\MathLine{\Umathaxis}               {}   {}                     {AxisHeight}                              {axis_height}

+\MathLine{\Umathoperatorsize}       {6}  {D, D'}                {DisplayOperatorMinHeight}                {\emdash}

+\MathLine{\Umathfractiondelsize}    {9}  {D, D'}                {FractionDelimiterDisplayStyleSize}       {delim1}

+\MathLine{\Umathfractiondelsize}    {9}  {T, T', S, S', SS, SS'}{FractionDelimiterSize}                   {delim2}

+\MathLine{\Umathfractiondenomdown}  {}   {D, D'}                {FractionDenominatorDisplayStyleShiftDown}{denom1}

+\MathLine{\Umathfractiondenomdown}  {}   {T, T', S, S', SS, SS'}{FractionDenominatorShiftDown}            {denom2}

+\MathLine{\Umathfractiondenomvgap}  {}   {D, D'}                {FractionDenominatorDisplayStyleGapMin}   {3*default_rule_thickness}

+\MathLine{\Umathfractiondenomvgap}  {}   {T, T', S, S', SS, SS'}{FractionDenominatorGapMin}               {default_rule_thickness}

+\MathLine{\Umathfractionnumup}      {}   {D, D'}                {FractionNumeratorDisplayStyleShiftUp}    {num1}

+\MathLine{\Umathfractionnumup}      {}   {T, T', S, S', SS, SS'}{FractionNumeratorShiftUp}                {num2}

+\MathLine{\Umathfractionnumvgap}    {}   {D, D'}                {FractionNumeratorDisplayStyleGapMin}     {3*default_rule_thickness}

+\MathLine{\Umathfractionnumvgap}    {}   {T, T', S, S', SS, SS'}{FractionNumeratorGapMin}                 {default_rule_thickness}

+\MathLine{\Umathfractionrule}       {}   {}                     {FractionRuleThickness}                   {default_rule_thickness}

+\MathLine{\Umathskewedfractionhgap} {}   {}                     {SkewedFractionHorizontalGap}             {math_quad/2}

+\MathLine{\Umathskewedfractionvgap} {}   {}                     {SkewedFractionVerticalGap}               {math_x_height}

+\MathLine{\Umathlimitabovebgap}     {}   {}                     {UpperLimitBaselineRiseMin}               {big_op_spacing3}

+\MathLine{\Umathlimitabovekern}     {1}  {}                     {0}                                       {big_op_spacing5}

+\MathLine{\Umathlimitabovevgap}     {}   {}                     {UpperLimitGapMin}                        {big_op_spacing1}

+\MathLine{\Umathlimitbelowbgap}     {}   {}                     {LowerLimitBaselineDropMin}               {big_op_spacing4}

+\MathLine{\Umathlimitbelowkern}     {1}  {}                     {0}                                       {big_op_spacing5}

+\MathLine{\Umathlimitbelowvgap}     {}   {}                     {LowerLimitGapMin}                        {big_op_spacing2}

+\MathLine{\Umathoverdelimitervgap}  {}   {}                     {StretchStackGapBelowMin}                 {big_op_spacing1}

+\MathLine{\Umathoverdelimiterbgap}  {}   {}                     {StretchStackTopShiftUp}                  {big_op_spacing3}

+\MathLine{\Umathunderdelimitervgap} {}   {}                     {StretchStackGapAboveMin}                 {big_op_spacing2}

+\MathLine{\Umathunderdelimiterbgap} {}   {}                     {StretchStackBottomShiftDown}             {big_op_spacing4}

+\MathLine{\Umathoverbarkern}        {}   {}                     {OverbarExtraAscender}                    {default_rule_thickness}

+\MathLine{\Umathoverbarrule}        {}   {}                     {OverbarRuleThickness}                    {default_rule_thickness}

+\MathLine{\Umathoverbarvgap}        {}   {}                     {OverbarVerticalGap}                      {3*default_rule_thickness}

+\MathLine{\Umathquad}               {1}  {}                     {<font_size(f)>}                          {math_quad}

+\MathLine{\Umathradicalkern}        {}   {}                     {RadicalExtraAscender}                    {default_rule_thickness}

+\MathLine{\Umathradicalrule}        {2}  {}                     {RadicalRuleThickness}                    {<not set>}

+\MathLine{\Umathradicalvgap}        {3}  {D, D'}                {RadicalDisplayStyleVerticalGap}          {default_rule_thickness+abs(math_x_height)/4}

+\MathLine{\Umathradicalvgap}        {3}  {T, T', S, S', SS, SS'}{RadicalVerticalGap}                      {default_rule_thickness+abs(default_rule_thickness)/4}

+\MathLine{\Umathradicaldegreebefore}{2}  {}                     {RadicalKernBeforeDegree}                 {<not set>}

+\MathLine{\Umathradicaldegreeafter} {2}  {}                     {RadicalKernAfterDegree}                  {<not set>}

+\MathLine{\Umathradicaldegreeraise} {2,7}{}                     {RadicalDegreeBottomRaisePercent}         {<not set>}

+\MathLine{\Umathspaceafterscript}   {4}  {}                     {SpaceAfterScript}                        {script_space}

+\MathLine{\Umathstackdenomdown}     {}   {D, D'}                {StackBottomDisplayStyleShiftDown}        {denom1}

+\MathLine{\Umathstackdenomdown}     {}   {T, T', S, S', SS, SS'}{StackBottomShiftDown}                    {denom2}

+\MathLine{\Umathstacknumup}         {}   {D, D'}                {StackTopDisplayStyleShiftUp}             {num1}

+\MathLine{\Umathstacknumup}         {}   {T, T', S, S', SS, SS'}{StackTopShiftUp}                         {num3}

+\MathLine{\Umathstackvgap}          {}   {D, D'}                {StackDisplayStyleGapMin}                 {7*default_rule_thickness}

+\MathLine{\Umathstackvgap}          {}   {T, T', S, S', SS, SS'}{StackGapMin}                             {3*default_rule_thickness}

+\MathLine{\Umathsubshiftdown}       {}   {}                     {SubscriptShiftDown}                      {sub1}

+\MathLine{\Umathsubshiftdrop}       {}   {}                     {SubscriptBaselineDropMin}                {sub_drop}

+\MathLine{\Umathsubsupshiftdown}    {8}  {}                     {SubscriptShiftDownWithSuperscript}       {\emdash}

+\MathLine{\Umathsubtopmax}          {}   {}                     {SubscriptTopMax}                         {abs(math_x_height*4)/5}

+\MathLine{\Umathsubsupvgap}         {}   {}                     {SubSuperscriptGapMin}                    {4*default_rule_thickness}

+\MathLine{\Umathsupbottommin}       {}   {}                     {SuperscriptBottomMin}                    {abs(math_x_height/4)}

+\MathLine{\Umathsupshiftdrop}       {}   {}                     {SuperscriptBaselineDropMax}              {sup_drop}

+\MathLine{\Umathsupshiftup}         {}   {D}                    {SuperscriptShiftUp}                      {sup1}

+\MathLine{\Umathsupshiftup}         {}   {T, S, SS,}            {SuperscriptShiftUp}                      {sup2}

+\MathLine{\Umathsupshiftup}         {}   {D', T', S', SS'}      {SuperscriptShiftUpCramped}               {sup3}

+\MathLine{\Umathsupsubbottommax}    {}   {}                     {SuperscriptBottomMaxWithSubscript}       {abs(math_x_height*4)/5}

+\MathLine{\Umathunderbarkern}       {}   {}                     {UnderbarExtraDescender}                  {default_rule_thickness}

+\MathLine{\Umathunderbarrule}       {}   {}                     {UnderbarRuleThickness}                   {default_rule_thickness}

+\MathLine{\Umathunderbarvgap}       {}   {}                     {UnderbarVerticalGap}                     {3*default_rule_thickness}

+\MathLine{\Umathconnectoroverlapmin}{5}  {}                     {MinConnectorOverlap}                     {0}

+\LL

+\stoptabulate

+

+Note 1: \OPENTYPE\ fonts set \lpr {Umathlimitabovekern} and \lpr

+{Umathlimitbelowkern} to zero and set \lpr {Umathquad} to the font size of the

+used font, because these are not supported in the \type {MATH} table,

+

+Note 2: Traditional \TFM\ fonts do not set \lpr {Umathradicalrule} because

+\TEX82\ uses the height of the radical instead. When this parameter is indeed not

+set when \LUATEX\ has to typeset a radical, a backward compatibility mode will

+kick in that assumes that an oldstyle \TEX\ font is used. Also, they do not set

+\lpr {Umathradicaldegreebefore}, \lpr {Umathradicaldegreeafter}, and \lpr

+{Umathradicaldegreeraise}. These are then automatically initialized to

+$5/18$quad, $-10/18$quad, and 60.

+

+Note 3: If \TFM\ fonts are used, then the \lpr {Umathradicalvgap} is not set

+until the first time \LUATEX\ has to typeset a formula because this needs

+parameters from both family~2 and family~3. This provides a partial backward

+compatibility with \TEX82, but that compatibility is only partial: once the \lpr

+{Umathradicalvgap} is set, it will not be recalculated any more.

+

+Note 4: When \TFM\ fonts are used a similar situation arises with respect to \lpr

+{Umathspaceafterscript}: it is not set until the first time \LUATEX\ has to

+typeset a formula. This provides some backward compatibility with \TEX82. But

+once the \lpr {Umathspaceafterscript} is set, \prm {scriptspace} will never be

+looked at again.

+

+Note 5: Traditional \TFM\ fonts set \lpr {Umathconnectoroverlapmin} to zero

+because \TEX82\ always stacks extensibles without any overlap.

+

+Note 6: The \lpr {Umathoperatorsize} is only used in \prm {displaystyle}, and is

+only set in \OPENTYPE\ fonts. In \TFM\ font mode, it is artificially set to one

+scaled point more than the initial attempt's size, so that always the \quote

+{first next} will be tried, just like in \TEX82.

+

+Note 7: The \lpr {Umathradicaldegreeraise} is a special case because it is the

+only parameter that is expressed in a percentage instead of a number of scaled

+points.

+

+Note 8: \type {SubscriptShiftDownWithSuperscript} does not actually exist in the

+\quote {standard} \OPENTYPE\ math font Cambria, but it is useful enough to be

+added.

+

+Note 9: \type {FractionDelimiterDisplayStyleSize} and \type

+{FractionDelimiterSize} do not actually exist in the \quote {standard} \OPENTYPE\

+math font Cambria, but were useful enough to be added.

+

+\stopsection

+

+\startsection[title={Math spacing}]

+

+\subsection{Inline surrounding space}

+

+\topicindex {math+spacing}

+

+Inline math is surrounded by (optional) \prm {mathsurround} spacing but that is a fixed

+dimension. There is now an additional parameter \lpr {mathsurroundskip}. When set to a

+non|-|zero value (or zero with some stretch or shrink) this parameter will replace

+\prm {mathsurround}. By using an additional parameter instead of changing the nature

+of \prm {mathsurround}, we can remain compatible. In the meantime a bit more

+control has been added via \lpr {mathsurroundmode}. This directive can take 6 values

+with zero being the default behaviour.

+

+\start

+

+\def\OneLiner#1#2%

+  {\NC \type{#1}

+   \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt   x$x$x}

+   \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x $x$ x}

+   \NC #2

+   \NC \NR}

+

+\startbuffer

+\mathsurround    10pt

+\mathsurroundskip20pt

+\stopbuffer

+

+\typebuffer \getbuffer

+

+\starttabulate[|c|c|c|pl|]

+\DB mode \BC x\$x\$x \BC x \$x\$ x \BC effect \NC \NR

+\TB

+\OneLiner{0}{obey \prm {mathsurround} when \lpr {mathsurroundskip} is 0pt}

+\OneLiner{1}{only add skip to the left}

+\OneLiner{2}{only add skip to the right}

+\OneLiner{3}{add skip to the left and right}

+\OneLiner{4}{ignore the skip setting, obey \prm {mathsurround}}

+\OneLiner{5}{disable all spacing around math}

+\OneLiner{6}{only apply \lpr {mathsurroundskip} when also spacing}

+\OneLiner{7}{only apply \lpr {mathsurroundskip} when no spacing}

+\LL

+\stoptabulate

+

+\stop

+

+Method six omits the surround glue when there is (x)spacing glue present while

+method seven does the opposite, the glue is only applied when there is (x)space

+glue present too. Anything more fancy, like checking the begining or end of a

+paragraph (or edges of a box) would not be robust anyway. If you want that you

+can write a callback that runs over a list and analyzes a paragraph. Actually, in

+that case you could also inject glue (or set the properties of a math node)

+explicitly. So, these modes are in practice mostly useful for special purposes

+and experiments (they originate in a tracker item). Keep in mind that this glue

+is part of the math node and not always treated as normal glue: it travels with

+the begin and end math nodes. Also, method 6 and 7 will zero the skip related

+fields in a node when applicable in the first occasion that checks them

+(linebreaking or packaging).

+

+\subsection{Pairwise spacing}

+

+\topicindex {math+spacing}

+

+Besides the parameters mentioned in the previous sections, there are also 64 new

+primitives to control the math spacing table (as explained in Chapter~18 of the

+\TEX book). The primitive names are a simple matter of combining two math atom

+types, but for completeness' sake, here is the whole list:

+

+\starttwocolumns

+\startlines

+\lpr {Umathordordspacing}

+\lpr {Umathordopspacing}

+\lpr {Umathordbinspacing}

+\lpr {Umathordrelspacing}

+\lpr {Umathordopenspacing}

+\lpr {Umathordclosespacing}

+\lpr {Umathordpunctspacing}

+\lpr {Umathordinnerspacing}

+\lpr {Umathopordspacing}

+\lpr {Umathopopspacing}

+\lpr {Umathopbinspacing}

+\lpr {Umathoprelspacing}

+\lpr {Umathopopenspacing}

+\lpr {Umathopclosespacing}

+\lpr {Umathoppunctspacing}

+\lpr {Umathopinnerspacing}

+\lpr {Umathbinordspacing}

+\lpr {Umathbinopspacing}

+\lpr {Umathbinbinspacing}

+\lpr {Umathbinrelspacing}

+\lpr {Umathbinopenspacing}

+\lpr {Umathbinclosespacing}

+\lpr {Umathbinpunctspacing}

+\lpr {Umathbininnerspacing}

+\lpr {Umathrelordspacing}

+\lpr {Umathrelopspacing}

+\lpr {Umathrelbinspacing}

+\lpr {Umathrelrelspacing}

+\lpr {Umathrelopenspacing}

+\lpr {Umathrelclosespacing}

+\lpr {Umathrelpunctspacing}

+\lpr {Umathrelinnerspacing}

+\lpr {Umathopenordspacing}

+\lpr {Umathopenopspacing}

+\lpr {Umathopenbinspacing}

+\lpr {Umathopenrelspacing}

+\lpr {Umathopenopenspacing}

+\lpr {Umathopenclosespacing}

+\lpr {Umathopenpunctspacing}

+\lpr {Umathopeninnerspacing}

+\lpr {Umathcloseordspacing}

+\lpr {Umathcloseopspacing}

+\lpr {Umathclosebinspacing}

+\lpr {Umathcloserelspacing}

+\lpr {Umathcloseopenspacing}

+\lpr {Umathcloseclosespacing}

+\lpr {Umathclosepunctspacing}

+\lpr {Umathcloseinnerspacing}

+\lpr {Umathpunctordspacing}

+\lpr {Umathpunctopspacing}

+\lpr {Umathpunctbinspacing}

+\lpr {Umathpunctrelspacing}

+\lpr {Umathpunctopenspacing}

+\lpr {Umathpunctclosespacing}

+\lpr {Umathpunctpunctspacing}

+\lpr {Umathpunctinnerspacing}

+\lpr {Umathinnerordspacing}

+\lpr {Umathinneropspacing}

+\lpr {Umathinnerbinspacing}

+\lpr {Umathinnerrelspacing}

+\lpr {Umathinneropenspacing}

+\lpr {Umathinnerclosespacing}

+\lpr {Umathinnerpunctspacing}

+\lpr {Umathinnerinnerspacing}

+\stoplines

+\stoptwocolumns

+

+These parameters are of type \prm {muskip}, so setting a parameter can be done

+like this:

+

+\starttyping

+\Umathopordspacing\displaystyle=4mu plus 2mu

+\stoptyping

+

+They are all initialized by \type {initex} to the values mentioned in the table

+in Chapter~18 of the \TEX book.

+

+Note 1: for ease of use as well as for backward compatibility, \prm {thinmuskip},

+\prm {medmuskip} and \prm {thickmuskip} are treated specially. In their case a

+pointer to the corresponding internal parameter is saved, not the actual \prm

+{muskip} value. This means that any later changes to one of these three

+parameters will be taken into account.

+

+Note 2: Careful readers will realise that there are also primitives for the items

+marked \type {*} in the \TEX book. These will not actually be used as those

+combinations of atoms cannot actually happen, but it seemed better not to break

+orthogonality. They are initialized to zero.

+

+\subsection{Skips around display math}

+

+\topicindex {math+spacing}

+

+The injection of \prm {abovedisplayskip} and \prm {belowdisplayskip} is not

+symmetrical. An above one is always inserted, also when zero, but the below is

+only inserted when larger than zero. Especially the latter makes it sometimes hard

+to fully control spacing. Therefore \LUATEX\ comes with a new directive: \lpr

+{mathdisplayskipmode}. The following values apply:

+

+\starttabulate[|c|l|]

+\DB value  \BC meaning \NC \NR

+\TB

+\NC 0 \NC normal \TEX\ behaviour \NC \NR

+\NC 1 \NC always (same as 0) \NC \NR

+\NC 2 \NC only when not zero \NC \NR

+\NC 3 \NC never, not even when not zero \NC \NR

+\LL

+\stoptabulate

+

+\subsection {Nolimit correction}

+

+\topicindex {math+limits}

+

+There are two extra math parameters \lpr {Umathnolimitsupfactor} and \lpr

+{Umathnolimitsubfactor} that were added to provide some control over how limits

+are spaced (for example the position of super and subscripts after integral

+operators). They relate to an extra parameter \lpr {mathnolimitsmode}. The half

+corrections are what happens when scripts are placed above and below. The

+problem with italic corrections is that officially that correction italic is used

+for above|/|below placement while advanced kerns are used for placement at the

+right end. The question is: how often is this implemented, and if so, do the

+kerns assume correction too. Anyway, with this parameter one can control it.

+

+\starttabulate[|l|ck1|ck1|ck1|ck1|ck1|ck1|]

+    \NC

+        \NC \mathnolimitsmode0    $\displaystyle\int\nolimits^0_1$

+        \NC \mathnolimitsmode1    $\displaystyle\int\nolimits^0_1$

+        \NC \mathnolimitsmode2    $\displaystyle\int\nolimits^0_1$

+        \NC \mathnolimitsmode3    $\displaystyle\int\nolimits^0_1$

+        \NC \mathnolimitsmode4    $\displaystyle\int\nolimits^0_1$

+        \NC \mathnolimitsmode8000 $\displaystyle\int\nolimits^0_1$

+    \NC \NR

+    \TB

+    \BC mode

+        \NC \tttf 0

+        \NC \tttf 1

+        \NC \tttf 2

+        \NC \tttf 3

+        \NC \tttf 4

+        \NC \tttf 8000

+    \NC \NR

+    \BC superscript

+        \NC 0

+        \NC font

+        \NC 0

+        \NC 0

+        \NC +ic/2

+        \NC 0

+    \NC \NR

+    \BC subscript

+        \NC -ic

+        \NC font

+        \NC 0

+        \NC -ic/2

+        \NC -ic/2

+        \NC 8000ic/1000

+    \NC \NR

+\stoptabulate

+

+When the mode is set to one, the math parameters are used. This way a macro

+package writer can decide what looks best. Given the current state of fonts in

+\CONTEXT\ we currently use mode 1 with factor 0 for the superscript and 750 for

+the subscripts. Positive values are used for both parameters but the subscript

+shifts to the left. A \lpr {mathnolimitsmode} larger that 15 is considered to

+be a factor for the subscript correction. This feature can be handy when

+experimenting.

+

+\subsection {Math italic mess}

+

+\topicindex {math+italics}

+

+The \lpr {mathitalicsmode} parameter can be set to~1 to force italic correction

+before noads that represent some more complex structure (read: everything

+that is not an ord, bin, rel, open, close, punct or inner). We show a Cambria

+example.

+

+\starttexdefinition Whatever #1

+    \NC \type{\mathitalicsmode = #1}

+    \NC \mathitalicsmode#1\ruledhbox{$\left|T^1\right|$}

+    \NC \mathitalicsmode#1\ruledhbox{$\left|T\right|$}

+    \NC \mathitalicsmode#1\ruledhbox{$T+1$}

+    \NC \mathitalicsmode#1\ruledhbox{$T{1\over2}$}

+    \NC \mathitalicsmode#1\ruledhbox{$T\sqrt{1}$}

+    \NC \NR

+\stoptexdefinition

+

+\start

+    \switchtobodyfont[cambria]

+    \starttabulate[|c|c|c|c|c|c|]

+        \Whatever{0}%

+        \Whatever{1}%

+    \stoptabulate

+\stop

+

+This kind of parameters relate to the fact that italic correction in \OPENTYPE\

+math is bound to fuzzy rules. So, control is the solution.

+

+\subsection {Script and kerning}

+

+\topicindex {math+kerning}

+\topicindex {math+scripts}

+

+If you want to typeset text in math macro packages often provide something \type

+{\text} which obeys the script sizes. As the definition can be anything there is

+a good chance that the kerning doesn't come out well when used in a script. Given

+that the first glyph ends up in a \prm {hbox} we have some control over this.

+And, as a bonus we also added control over the normal sublist kerning. The \lpr

+{mathscriptboxmode} parameter defaults to~1.

+

+\starttabulate[|c|l|]

+\DB value     \BC meaning \NC \NR

+\TB

+\NC \type {0} \NC forget about kerning \NC \NR

+\NC \type {1} \NC kern math sub lists with a valid glyph \NC \NR

+\NC \type {2} \NC also kern math sub boxes that have a valid glyph \NC \NR

+\NC \type {2} \NC only kern math sub boxes with a boundary node present\NC \NR

+\LL

+\stoptabulate

+

+Here we show some examples. Of course this doesn't solve all our problems, if

+only because some fonts have characters with bounding boxes that compensate for

+italics, while other fonts can lack kerns.

+

+\startbuffer[1]

+    $T_{\tf fluff}$

+\stopbuffer

+

+\startbuffer[2]

+    $T_{\text{fluff}}$

+\stopbuffer

+

+\startbuffer[3]

+    $T_{\text{\boundary1 fluff}}$

+\stopbuffer

+

+\unexpanded\def\Show#1#2#3%

+  {\doifelsenothing{#3}

+     {\small\tx\typeinlinebuffer[#1]}

+     {\doifelse{#3}{-}

+        {\small\bf\tt mode #2}

+        {\switchtobodyfont[#3]\showfontkerns\showglyphs\mathscriptboxmode#2\relax\inlinebuffer[#1]}}}

+

+\starttabulate[|lBT|c|c|c|c|c|]

+    \NC          \NC \Show{1}{0}{}         \NC\Show{1}{1}{}         \NC \Show{2}{1}{}         \NC \Show{2}{2}{}         \NC \Show{3}{3}{}         \NC \NR

+    \NC          \NC \Show{1}{0}{-}        \NC\Show{1}{1}{-}        \NC \Show{2}{1}{-}        \NC \Show{2}{2}{-}        \NC \Show{3}{3}{-}        \NC \NR

+    \NC modern   \NC \Show{1}{0}{modern}   \NC\Show{1}{1}{modern}   \NC \Show{2}{1}{modern}   \NC \Show{2}{2}{modern}   \NC \Show{3}{3}{modern}   \NC \NR

+    \NC lucidaot \NC \Show{1}{0}{lucidaot} \NC\Show{1}{1}{lucidaot} \NC \Show{2}{1}{lucidaot} \NC \Show{2}{2}{lucidaot} \NC \Show{3}{3}{lucidaot} \NC \NR

+    \NC pagella  \NC \Show{1}{0}{pagella}  \NC\Show{1}{1}{pagella}  \NC \Show{2}{1}{pagella}  \NC \Show{2}{2}{pagella}  \NC \Show{3}{3}{pagella}  \NC \NR

+    \NC cambria  \NC \Show{1}{0}{cambria}  \NC\Show{1}{1}{cambria}  \NC \Show{2}{1}{cambria}  \NC \Show{2}{2}{cambria}  \NC \Show{3}{3}{cambria}  \NC \NR

+    \NC dejavu   \NC \Show{1}{0}{dejavu}   \NC\Show{1}{1}{dejavu}   \NC \Show{2}{1}{dejavu}   \NC \Show{2}{2}{dejavu}   \NC \Show{3}{3}{dejavu}   \NC \NR

+\stoptabulate

+

+Kerning between a character subscript is controlled by \lpr {mathscriptcharmode}

+which also defaults to~1.

+

+Here is another example. Internally we tag kerns as italic kerns or font kerns

+where font kerns result from the staircase kern tables. In 2018 fonts like Latin

+Modern and Pagella rely on cheats with the boundingbox, Cambria uses staircase

+kerns and Lucida a mixture. Depending on how fonts evolve we might add some more

+control over what one can turn on and off.

+

+\def\MathSample#1#2#3%

+  {\NC

+   #1 \NC

+   #2 \NC

+   \showglyphdata \switchtobodyfont[#2,17.3pt]$#3T_{f}$         \NC

+   \showglyphdata \switchtobodyfont[#2,17.3pt]$#3\gamma_{e}$    \NC

+   \showglyphdata \switchtobodyfont[#2,17.3pt]$#3\gamma_{ee}$   \NC

+   \showglyphdata \switchtobodyfont[#2,17.3pt]$#3T_{\tf fluff}$ \NC

+   \NR}

+

+\starttabulate[|Tl|Tl|l|l|l|l|]

+    \FL

+    \MathSample{normal}{modern}  {\mr}

+    \MathSample{}      {pagella} {\mr}

+    \MathSample{}      {cambria} {\mr}

+    \MathSample{}      {lucidaot}{\mr}

+    \ML

+    \MathSample{bold}  {modern}  {\mb}

+    \MathSample{}      {pagella} {\mb}

+    \MathSample{}      {cambria} {\mb}

+    \MathSample{}      {lucidaot}{\mb}

+    \LL

+\stoptabulate

+

+\subsection{Fixed scripts}

+

+We have three parameters that are used for this fixed anchoring:

+

+\starttabulate[|c|l|]

+\DB parameter \BC register \NC \NR

+\NC $d$ \NC \lpr {Umathsubshiftdown}    \NC \NR

+\NC $u$ \NC \lpr {Umathsupshiftup}      \NC \NR

+\NC $s$ \NC \lpr {Umathsubsupshiftdown} \NC \NR

+\LL

+\stoptabulate

+

+When we set \lpr {mathscriptsmode} to a value other than zero these are used

+for calculating fixed positions. This is something that is needed for instance

+for chemistry. You can manipulate the mentioned variables to achieve different

+effects.

+

+\def\SampleMath#1%

+  {$\mathscriptsmode#1\mathupright CH_2 + CH^+_2 + CH^2_2$}

+

+\starttabulate[|c|c|c|p|]

+\DB mode \BC down          \BC up            \BC example        \NC \NR

+\TB

+\NC 0    \NC dynamic       \NC dynamic       \NC \SampleMath{0} \NC \NR

+\NC 1    \NC $d$           \NC $u$           \NC \SampleMath{1} \NC \NR

+\NC 2    \NC $s$           \NC $u$           \NC \SampleMath{2} \NC \NR

+\NC 3    \NC $s$           \NC $u + s - d$   \NC \SampleMath{3} \NC \NR

+\NC 4    \NC $d + (s-d)/2$ \NC $u + (s-d)/2$ \NC \SampleMath{4} \NC \NR

+\NC 5    \NC $d$           \NC $u + s - d$   \NC \SampleMath{5} \NC \NR

+\LL

+\stoptabulate

+

+The value of this parameter obeys grouping but applies to the whole current

+formula.

+

+% if needed we can put the value in stylenodes but maybe more should go there

+

+\subsection{Penalties: \lpr {mathpenaltiesmode}}

+

+\topicindex {math+penalties}

+

+Only in inline math penalties will be added in a math list. You can force

+penalties (also in display math) by setting:

+

+\starttyping

+\mathpenaltiesmode = 1

+\stoptyping

+

+This primnitive is not really needed in \LUATEX\ because you can use the callback

+\cbk {mlist_to_hlist} to force penalties by just calling the regular routine

+with forced penalties. However, as part of opening up and control this primitive

+makes sense. As a bonus we also provide two extra penalties:

+

+\starttyping

+\prebinoppenalty = -100 % example value

+\prerelpenalty   =  900 % example value

+\stoptyping

+

+They default to inifinite which signals that they don't need to be inserted. When

+set they are injected before a binop or rel noad. This is an experimental feature.

+

+\subsection{Equation spacing: \lpr {matheqnogapstep}}

+

+By default \TEX\ will add one quad between the equation and the number. This is

+hard coded. A new primitive can control this:

+

+\startsyntax

+\matheqnogapstep = 1000

+\stopsyntax

+

+Because a math quad from the math text font is used instead of a dimension, we

+use a step to control the size. A value of zero will suppress the gap. The step

+is divided by 1000 which is the usual way to mimmick floating point factors in

+\TEX.

+

+\stopsection

+

+\startsection[title={Math constructs}]

+

+\subsection {Unscaled fences}

+

+\topicindex {math+fences}

+

+The \lpr {mathdelimitersmode} primitive is experimental and deals with the

+following (potential) problems. Three bits can be set. The first bit prevents an

+unwanted shift when the fence symbol is not scaled (a cambria side effect). The

+second bit forces italic correction between a preceding character ordinal and the

+fenced subformula, while the third bit turns that subformula into an ordinary so

+that the same spacing applies as with unfenced variants. Here we show Cambria

+(with \lpr {mathitalicsmode} enabled).

+

+\starttexdefinition Whatever #1

+    \NC \type{\mathdelimitersmode = #1}

+    \NC \mathitalicsmode1\mathdelimitersmode#1\ruledhbox{\showglyphs\showfontkerns\showfontitalics$f(x)$}

+    \NC \mathitalicsmode1\mathdelimitersmode#1\ruledhbox{\showglyphs\showfontkerns\showfontitalics$f\left(x\right)$}

+    \NC \NR

+\stoptexdefinition

+

+\start

+    \switchtobodyfont[cambria]

+    \starttabulate[|l|l|l|]

+        \Whatever{0}\Whatever{1}\Whatever{2}\Whatever{3}%

+        \Whatever{4}\Whatever{5}\Whatever{6}\Whatever{7}%

+    \stoptabulate

+\stop

+

+So, when set to 7 fenced subformulas with unscaled delimiters come out the same

+as unfenced ones. This can be handy for cases where one is forced to use \prm

+{left} and \prm {right} always because of unpredictable content. As said, it's an

+experimental feature (which somehow fits in the exceptional way fences are dealt

+with in the engine). The full list of flags is given in the next table:

+

+\starttabulate[|c|l|]

+\DB value  \BC meaning \NC \NR

+\TB

+\NC \type{"01} \NC don't apply the usual shift \NC \NR

+\NC \type{"02} \NC apply italic correction when possible \NC \NR

+\NC \type{"04} \NC force an ordinary subformula \NC \NR

+\NC \type{"08} \NC no shift when a base character \NC \NR

+\NC \type{"10} \NC only shift when an extensible \NC \NR

+\LL

+\stoptabulate

+

+The effect can depend on the font (and for Cambria one can use for instance \type {"16}).

+

+\subsection[mathacc]{Accent handling}

+

+\topicindex {math+accents}

+

+\LUATEX\ supports both top accents and bottom accents in math mode, and math

+accents stretch automatically (if this is supported by the font the accent comes

+from, of course). Bottom and combined accents as well as fixed-width math accents

+are controlled by optional keywords following \lpr {Umathaccent}.

+

+The keyword \type {bottom} after \lpr {Umathaccent} signals that a bottom accent

+is needed, and the keyword \type {both} signals that both a top and a bottom

+accent are needed (in this case two accents need to be specified, of course).

+

+Then the set of three integers defining the accent is read. This set of integers

+can be prefixed by the \type {fixed} keyword to indicate that a non-stretching

+variant is requested (in case of both accents, this step is repeated).

+

+A simple example:

+

+\starttyping

+\Umathaccent both fixed 0 0 "20D7 fixed 0 0 "20D7 {example}

+\stoptyping

+

+If a math top accent has to be placed and the accentee is a character and has a

+non-zero \type {top_accent} value, then this value will be used to place the

+accent instead of the \prm {skewchar} kern used by \TEX82.

+

+The \type {top_accent} value represents a vertical line somewhere in the

+accentee. The accent will be shifted horizontally such that its own \type

+{top_accent} line coincides with the one from the accentee. If the \type

+{top_accent} value of the accent is zero, then half the width of the accent

+followed by its italic correction is used instead.

+

+The vertical placement of a top accent depends on the \type {x_height} of the

+font of the accentee (as explained in the \TEX book), but if a value turns out

+to be zero and the font had a \type {MathConstants} table, then \type

+{AccentBaseHeight} is used instead.

+

+The vertical placement of a bottom accent is straight below the accentee, no

+correction takes place.

+

+Possible locations are \type {top}, \type {bottom}, \type {both} and \type

+{center}. When no location is given \type {top} is assumed. An additional

+parameter \nod {fraction} can be specified followed by a number; a value of for

+instance 1200 means that the criterium is 1.2 times the width of the nucleus. The

+fraction only applies to the stepwise selected shapes and is mostly meant for the

+\type {overlay} location. It also works for the other locations but then it

+concerns the width.

+

+\subsection{Radical extensions}

+

+\topicindex {math+radicals}

+

+The new primitive \lpr {Uroot} allows the construction of a radical noad

+including a degree field. Its syntax is an extension of \lpr {Uradical}:

+

+\starttyping

+\Uradical <fam integer> <char integer> <radicand>

+\Uroot    <fam integer> <char integer> <degree> <radicand>

+\stoptyping

+

+The placement of the degree is controlled by the math parameters \lpr

+{Umathradicaldegreebefore}, \lpr {Umathradicaldegreeafter}, and \lpr

+{Umathradicaldegreeraise}. The degree will be typeset in \prm

+{scriptscriptstyle}.

+

+\subsection{Super- and subscripts}

+

+The character fields in a \LUA|-|loaded \OPENTYPE\ math font can have a \quote

+{mathkern} table. The format of this table is the same as the \quote {mathkern}

+table that is returned by the \type {fontloader} library, except that all height

+and kern values have to be specified in actual scaled points.

+

+When a super- or subscript has to be placed next to a math item, \LUATEX\ checks

+whether the super- or subscript and the nucleus are both simple character items.

+If they are, and if the fonts of both character items are \OPENTYPE\ fonts (as

+opposed to legacy \TEX\ fonts), then \LUATEX\ will use the \OPENTYPE\ math

+algorithm for deciding on the horizontal placement of the super- or subscript.

+

+This works as follows:

+

+\startitemize

+    \startitem

+        The vertical position of the script is calculated.

+    \stopitem

+    \startitem

+        The default horizontal position is flat next to the base character.

+    \stopitem

+    \startitem

+        For superscripts, the italic correction of the base character is added.

+    \stopitem

+    \startitem

+        For a superscript, two vertical values are calculated: the bottom of the

+        script (after shifting up), and the top of the base. For a subscript, the two

+        values are the top of the (shifted down) script, and the bottom of the base.

+    \stopitem

+    \startitem

+        For each of these two locations:

+        \startitemize

+            \startitem

+                find the math kern value at this height for the base (for a subscript

+                placement, this is the bottom_right corner, for a superscript

+                placement the top_right corner)

+            \stopitem

+            \startitem

+                find the math kern value at this height for the script (for a

+                subscript placement, this is the top_left corner, for a superscript

+                placement the bottom_left corner)

+            \stopitem

+            \startitem

+                add the found values together to get a preliminary result.

+            \stopitem

+        \stopitemize

+    \stopitem

+    \startitem

+        The horizontal kern to be applied is the smallest of the two results from

+        previous step.

+    \stopitem

+\stopitemize

+

+The math kern value at a specific height is the kern value that is specified by the

+next higher height and kern pair, or the highest one in the character (if there is no

+value high enough in the character), or simply zero (if the character has no math kern

+pairs at all).

+

+\subsection{Scripts on extensibles}

+

+\topicindex {math+scripts}

+\topicindex {math+delimiters}

+\topicindex {math+extensibles}

+

+The primitives \lpr {Uunderdelimiter} and \lpr {Uoverdelimiter} allow the

+placement of a subscript or superscript on an automatically extensible item and

+\lpr {Udelimiterunder} and \lpr {Udelimiterover} allow the placement of an

+automatically extensible item as a subscript or superscript on a nucleus. The

+input:

+

+% these produce radical noads .. in fact the code base has the numbers wrong for

+% quite a while, so no one seems to use this

+

+\startbuffer

+$\Uoverdelimiter  0 "2194 {\hbox{\strut  overdelimiter}}$

+$\Uunderdelimiter 0 "2194 {\hbox{\strut underdelimiter}}$

+$\Udelimiterover  0 "2194 {\hbox{\strut  delimiterover}}$

+$\Udelimiterunder 0 "2194 {\hbox{\strut delimiterunder}}$

+\stopbuffer

+

+\typebuffer will render this:

+

+\blank \startnarrower \getbuffer \stopnarrower \blank

+

+The vertical placements are controlled by \lpr {Umathunderdelimiterbgap}, \lpr

+{Umathunderdelimitervgap}, \lpr {Umathoverdelimiterbgap}, and \lpr

+{Umathoverdelimitervgap} in a similar way as limit placements on large operators.

+The superscript in \lpr {Uoverdelimiter} is typeset in a suitable scripted style,

+the subscript in \lpr {Uunderdelimiter} is cramped as well.

+

+These primitives accepts an option \type {width} specification. When used the

+also optional keywords \type {left}, \type {middle} and \type {right} will

+determine what happens when a requested size can't be met (which can happen when

+we step to successive larger variants).

+

+An extra primitive \lpr {Uhextensible} is available that can be used like this:

+

+\startbuffer

+$\Uhextensible width 10cm 0 "2194$

+\stopbuffer

+

+\typebuffer This will render this:

+

+\blank \startnarrower \getbuffer \stopnarrower \blank

+

+Here you can also pass options, like:

+

+\startbuffer

+$\Uhextensible width 1pt middle 0 "2194$

+\stopbuffer

+

+\typebuffer This gives:

+

+\blank \startnarrower \getbuffer \stopnarrower \blank

+

+\LUATEX\ internally uses a structure that supports \OPENTYPE\ \quote

+{MathVariants} as well as \TFM\ \quote {extensible recipes}. In most cases where

+font metrics are involved we have a different code path for traditional fonts end

+\OPENTYPE\ fonts.

+

+\subsection{Fractions}

+

+\topicindex {math+fractions}

+

+The \prm {abovewithdelims} command accepts a keyword \type {exact}. When issued

+the extra space relative to the rule thickness is not added. One can of course

+use the \type {\Umathfraction..gap} commands to influence the spacing. Also the

+rule is still positioned around the math axis.

+

+\starttyping

+$$ { {a} \abovewithdelims() exact 4pt {b} }$$

+\stoptyping

+

+The math parameter table contains some parameters that specify a horizontal and

+vertical gap for skewed fractions. Of course some guessing is needed in order to

+implement something that uses them. And so we now provide a primitive similar to the

+other fraction related ones but with a few options so that one can influence the

+rendering. Of course a user can also mess around a bit with the parameters

+\lpr {Umathskewedfractionhgap} and \lpr {Umathskewedfractionvgap}.

+

+The syntax used here is:

+

+\starttyping

+{ {1} \Uskewed / <options> {2} }

+{ {1} \Uskewedwithdelims / () <options> {2} }

+\stoptyping

+

+where the options can be \type {noaxis} and \type {exact}. By default we add half

+the axis to the shifts and by default we zero the width of the middle character.

+For Latin Modern the result looks as follows:

+

+\def\ShowA#1#2#3{$x + { {#1} \Uskewed           /    #3 {#2} } + x$}

+\def\ShowB#1#2#3{$x + { {#1} \Uskewedwithdelims / () #3 {#2} } + x$}

+

+\start

+    \switchtobodyfont[modern]

+    \starttabulate[||||||]

+        \NC \NC

+            \ShowA{a}{b}{} \NC

+            \ShowA{1}{2}{} \NC

+            \ShowB{a}{b}{} \NC

+            \ShowB{1}{2}{} \NC

+        \NR

+        \NC \type{exact} \NC

+            \ShowA{a}{b}{exact} \NC

+            \ShowA{1}{2}{exact} \NC

+            \ShowB{a}{b}{exact} \NC

+            \ShowB{1}{2}{exact} \NC

+        \NR

+        \NC \type{noaxis} \NC

+            \ShowA{a}{b}{noaxis} \NC

+            \ShowA{1}{2}{noaxis} \NC

+            \ShowB{a}{b}{noaxis} \NC

+            \ShowB{1}{2}{noaxis} \NC

+        \NR

+        \NC \type{exact noaxis} \NC

+            \ShowA{a}{b}{exact noaxis} \NC

+            \ShowA{1}{2}{exact noaxis} \NC

+            \ShowB{a}{b}{exact noaxis} \NC

+            \ShowB{1}{2}{exact noaxis} \NC

+        \NR

+    \stoptabulate

+\stop

+

+\subsection {Delimiters: \type{\Uleft}, \prm {Umiddle} and \prm {Uright}}

+

+\topicindex {math+delimiters}

+

+Normally you will force delimiters to certain sizes by putting an empty box or

+rule next to it. The resulting delimiter will either be a character from the

+stepwise size range or an extensible. The latter can be quite differently

+positioned than the characters as it depends on the fit as well as the fact if

+the used characters in the font have depth or height. Commands like (plain \TEX

+s) \type {\big} need use this feature. In \LUATEX\ we provide a bit more control

+by three variants that support optional parameters \type {height}, \type {depth}

+and \type {axis}. The following example uses this:

+

+\startbuffer

+\Uleft   height 30pt depth 10pt      \Udelimiter "0 "0 "000028

+\quad x\quad

+\Umiddle height 40pt depth 15pt      \Udelimiter "0 "0 "002016

+\quad x\quad

+\Uright  height 30pt depth 10pt      \Udelimiter "0 "0 "000029

+\quad \quad \quad

+\Uleft   height 30pt depth 10pt axis \Udelimiter "0 "0 "000028

+\quad x\quad

+\Umiddle height 40pt depth 15pt axis \Udelimiter "0 "0 "002016

+\quad x\quad

+\Uright  height 30pt depth 10pt axis \Udelimiter "0 "0 "000029

+\stopbuffer

+

+\typebuffer

+

+\startlinecorrection

+\ruledhbox{\mathematics{\getbuffer}}

+\stoplinecorrection

+

+The keyword \type {exact} can be used as directive that the real dimensions

+should be applied when the criteria can't be met which can happen when we're

+still stepping through the successively larger variants. When no dimensions are

+given the \type {noaxis} command can be used to prevent shifting over the axis.

+

+You can influence the final class with the keyword \type {class} which will

+influence the spacing. The numbers are the same as for character classes.

+

+\stopsection

+

+\startsection[title={Extracting values}]

+

+\subsection{Codes}

+

+\topicindex {math+codes}

+

+You can extract the components of a math character. Say that we have defined:

+

+\starttyping

+\Umathcode 1 2 3 4

+\stoptyping

+

+then

+

+\starttyping

+[\Umathcharclass1] [\Umathcharfam1] [\Umathcharslot1]

+\stoptyping

+

+will return:

+

+\starttyping

+[2] [3] [4]

+\stoptyping

+

+These commands are provides as convenience. Before they come available you could

+do the following:

+

+\starttyping

+\def\Umathcharclass{\directlua{tex.print(tex.getmathcode(token.scan_int())[1])}}

+\def\Umathcharfam  {\directlua{tex.print(tex.getmathcode(token.scan_int())[2])}}

+\def\Umathcharslot {\directlua{tex.print(tex.getmathcode(token.scan_int())[3])}}

+\stoptyping

+

+\subsection {Last lines}

+

+\topicindex {math+last line}

+

+There is a new primitive to control the overshoot in the calculation of the

+previous line in mid|-|paragraph display math. The default value is 2 times

+the em width of the current font:

+

+\starttyping

+\predisplaygapfactor=2000

+\stoptyping

+

+If you want to have the length of the last line independent of math i.e.\ you don't

+want to revert to a hack where you insert a fake display math formula in order to

+get the length of the last line, the following will often work too:

+

+\starttyping

+\def\lastlinelength{\dimexpr

+    \directlua {tex.sprint (

+        (nodes.dimensions(node.tail(tex.lists.page_head).list))

+    )}sp

+\relax}

+\stoptyping

+

+\stopsection

+

+\startsection[title={Math mode}]

+

+\subsection {Verbose versions of single|-|character math commands}

+

+\topicindex {math+styles}

+

+\LUATEX\ defines six new primitives that have the same function as

+\type {^}, \type {_}, \type {$}, and \type {$$}:

+

+\starttabulate[|l|l|]

+\DB primitive                  \BC explanation \NC \NR

+\TB

+\NC \lpr {Usuperscript}      \NC duplicates the functionality of \type {^} \NC \NR

+\NC \lpr {Usubscript}        \NC duplicates the functionality of \type {_} \NC \NR

+\NC \lpr {Ustartmath}        \NC duplicates the functionality of \type {$}, % $

+                                   when used in non-math mode. \NC \NR

+\NC \lpr {Ustopmath}         \NC duplicates the functionality of \type {$}, % $

+                                   when used in inline math mode. \NC \NR

+\NC \lpr {Ustartdisplaymath} \NC duplicates the functionality of \type {$$}, % $$

+                                   when used in non-math mode. \NC \NR

+\NC \lpr {Ustopdisplaymath}  \NC duplicates the functionality of \type {$$}, % $$

+                                   when used in display math mode. \NC \NR

+\LL

+\stoptabulate

+

+The \lpr {Ustopmath} and \lpr {Ustopdisplaymath} primitives check if the current

+math mode is the correct one (inline vs.\ displayed), but you can freely intermix

+the four mathon|/|mathoff commands with explicit dollar sign(s).

+

+\subsection{Script commands \lpr {Unosuperscript} and \lpr {Unosubscript}}

+

+\topicindex {math+styles}

+\topicindex {math+scripts}

+

+These two commands result in super- and subscripts but with the current style (at the

+time of rendering). So,

+

+\startbuffer[script]

+$

+    x\Usuperscript  {1}\Usubscript  {2} =

+    x\Unosuperscript{1}\Unosubscript{2} =

+    x\Usuperscript  {1}\Unosubscript{2} =

+    x\Unosuperscript{1}\Usubscript  {2}

+$

+\stopbuffer

+

+\typebuffer

+

+results in \inlinebuffer[script].

+

+\subsection{Allowed math commands in non|-|math modes}

+

+\topicindex {math+text}

+\topicindex {text+math}

+

+The commands \prm {mathchar}, and \lpr {Umathchar} and control sequences that are

+the result of \prm {mathchardef} or \lpr {Umathchardef} are also acceptable in

+the horizontal and vertical modes. In those cases, the \prm {textfont} from the

+requested math family is used.

+

+% \startsection[title={Math todo}]

+%

+% The following items are still todo.

+%

+% \startitemize

+% \startitem

+%     Pre-scripts.

+% \stopitem

+% \startitem

+%     Multi-story stacks.

+% \stopitem

+% \startitem

+%     Flattened accents for high characters (maybe).

+% \stopitem

+% \startitem

+%     Better control over the spacing around displays and handling of equation numbers.

+% \stopitem

+% \startitem

+%     Support for multi|-|line displays using \MATHML\ style alignment points.

+% \stopitem

+% \stopitemize

+%

+% \stopsection

+

+\stopsection

+

+\startsection[title={Goodies}]

+

+\subsection {Flattening: \lpr {mathflattenmode}}

+

+\topicindex {math+flattening}

+

+The \TEX\ math engine collapses \type {ord} noads without sub- and superscripts

+and a character as nucleus. and which has the side effect that in \OPENTYPE\ mode

+italic corrections are applied (given that they are enabled).

+

+\startbuffer[sample]

+\switchtobodyfont[modern]

+$V \mathbin{\mathbin{v}} V$\par

+$V \mathord{\mathord{v}} V$\par

+\stopbuffer

+

+\typebuffer[sample]

+

+This renders as:

+

+\blank \start \mathflattenmode\plusone \getbuffer[sample] \stop \blank

+

+When we set \lpr {mathflattenmode} to 31 we get:

+

+\blank \start \mathflattenmode\numexpr1+2+4+8+16\relax \getbuffer[sample] \stop \blank

+

+When you see no difference, then the font probably has the proper character

+dimensions and no italic correction is needed. For Latin Modern (at least till

+2018) there was a visual difference. In that respect this parameter is not always

+needed unless of course you want efficient math lists anyway.

+

+You can influence flattening by adding the appropriate number to the value of the

+mode parameter. The default value is~1.

+

+\starttabulate[|Tc|c|]

+\DB mode \BC class \NC \NR

+\TB

+\NC  1   \NC ord   \NC \NR

+\NC  2   \NC bin   \NC \NR

+\NC  4   \NC rel   \NC \NR

+\NC  8   \NC punct \NC \NR

+\NC 16   \NC inner \NC \NR

+\LL

+\stoptabulate

+

+\subsection {Less Tracing}

+

+\topicindex {math+tracing}

+

+Because there are quite some math related parameters and values, it is possible

+to limit tracing. Only when \type {tracingassigns} and|/|or \type

+{tracingrestores} are set to~2 or more they will be traced.

+

+\subsection {Math options with \lpr {mathoption}}

+

+The logic in the math engine is rather complex and there are often no universal

+solutions (read: what works out well for one font, fails for another). Therefore

+some variations in the implementation are driven by parameters (modes). In

+addition there is a new primitive \lpr {mathoption} which will be used for

+testing. Don't rely on any option to be there in a production version as they are

+meant for development.

+

+This option was introduced for testing purposes when the math engine got split

+code paths and it forces the engine to treat new fonts as old ones with respect

+to italic correction etc. There are no guarantees given with respect to the final

+result and unexpected side effects are not seen as bugs as they relate to font

+properties. Ther eis currently only one option:

+

+\startbuffer

+\mathoption old 1

+\stopbuffer

+

+The \type {oldmath} boolean flag in the \LUA\ font table is the official way to

+force old treatment as it's bound to fonts. Like with all options we may

+temporarily introduce with this command this feature is not meant for production.

+

+% % obsolete:

+%

+% \subsubsection {\type {\mathoption noitaliccompensation}}

+%

+% This option compensates placement for characters with a built|-|in italic

+% correction.

+%

+% \startbuffer

+% {\showboxes\int}\quad

+% {\showboxes\int_{|}^{|}}\quad

+% {\showboxes\int\limits_{|}^{|}}

+% \stopbuffer

+%

+% \typebuffer

+%

+% Gives (with computer modern that has such italics):

+%

+% \startlinecorrection[blank]

+%     \switchtobodyfont[modern]

+%     \startcombination[nx=2,ny=2,distance=5em]

+%         {\mathoption noitaliccompensation 0\relax \mathematics{\getbuffer}}

+%             {\nohyphens\type{0:inline}}

+%         {\mathoption noitaliccompensation 0\relax \mathematics{\displaymath\getbuffer}}

+%             {\nohyphens\type{0:display}}

+%         {\mathoption noitaliccompensation 1\relax \mathematics{\getbuffer}}

+%             {\nohyphens\type{1:inline}}

+%         {\mathoption noitaliccompensation 1\relax \mathematics{\displaymath\getbuffer}}

+%             {\nohyphens\type{1:display}}

+%     \stopcombination

+% \stoplinecorrection

+

+% % obsolete:

+%

+% \subsubsection {\type {\mathoption nocharitalic}}

+%

+% When two characters follow each other italic correction can interfere. The

+% following example shows what this option does:

+%

+% \startbuffer

+% \catcode"1D443=11

+% \catcode"1D444=11

+% \catcode"1D445=11

+% P( PP PQR

+% \stopbuffer

+%

+% \typebuffer

+%

+% Gives (with computer modern that has such italics):

+%

+% \startlinecorrection[blank]

+%     \switchtobodyfont[modern]

+%     \startcombination[nx=2,ny=2,distance=5em]

+%         {\mathoption nocharitalic 0\relax \mathematics{\getbuffer}}

+%             {\nohyphens\type{0:inline}}

+%         {\mathoption nocharitalic 0\relax \mathematics{\displaymath\getbuffer}}

+%             {\nohyphens\type{0:display}}

+%         {\mathoption nocharitalic 1\relax \mathematics{\getbuffer}}

+%             {\nohyphens\type{1:inline}}

+%         {\mathoption nocharitalic 1\relax \mathematics{\displaymath\getbuffer}}

+%             {\nohyphens\type{1:display}}

+%     \stopcombination

+% \stoplinecorrection

+

+% % obsolete:

+%

+% \subsubsection {\type {\mathoption useoldfractionscaling}}

+%

+% This option has been introduced as solution for tracker item 604 for fuzzy cases

+% around either or not present fraction related settings for new fonts.

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-modifications.tex b/Master/texmf-dist/doc/luatex/base/luatex-modifications.tex
index ee9a51de6e7..294f3fc3f8e 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-modifications.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-modifications.tex
@@ -1,1283 +1,1380 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-modifications
-
-\startchapter[reference=modifications,title={Modifications}]
-
-\startsection[title=The merged engines]
-
-\startsubsection[title=The need for change]
-
-The first version of \LUATEX\ only had a few extra primitives and it was largely
-the same as \PDFTEX. Then we merged substantial parts of \ALEPH\ into the code
-and got more primitives. When we got more stable the decision was made to clean
-up the rather hybrid nature of the program. This means that some primitives have
-been promoted to core primitives, often with a different name, and that others
-were removed. This made it possible to start cleaning up the code base. In \in
-{chapter} [enhancements] we discussed some new primitives, here we will cover
-most of the adapted ones.
-
-Besides the expected changes caused by new functionality, there are a number of
-not|-|so|-|expected changes. These are sometimes a side|-|effect of a new
-(conflicting) feature, or, more often than not, a change neccessary to clean up
-the internal interfaces. These will also be mentioned.
-
-\stopsubsection
-
-\startsubsection[title=Changes from \TEX\ 3.1415926]
-
-Of course it all starts with traditional \TEX. Even if we started with \PDFTEX,
-most still comes from the original. But we divert a bit.
-
-\startitemize
-
-\startitem
-    The current code base is written in \CCODE, not \PASCAL. We use \CWEB\ when
-    possible. As a consequence instead of one large file plus change files, we
-    now have multiple files organized in categories like \type {tex}, \type
-    {pdf}, \type {lang}, \type {font}, \type {lua}, etc. There are some artefacts
-    of the conversion to \CCODE, but in due time we will clean up the source code
-    and make sure that the documentation is done right. Many files are in the
-    \CWEB\ format, but others, like those interfacing to \LUA, are \CCODE\ files.
-    Of course we want to stay as close as possible to the original so that the
-    documentation of the fundamentals behind \TEX\ by Don Knuth still applies.
-\stopitem
-
-\startitem
-    See \in {chapter} [languages] for many small changes related to paragraph
-    building, language handling and hyphenation. The most important change is
-    that adding a brace group in the middle of a word (like in \type {of{}fice})
-    does not prevent ligature creation.
-\stopitem
-
-\startitem
-    There is no pool file, all strings are embedded during compilation.
-\stopitem
-
-\startitem
-    The specifier \type {plus 1 fillll} does not generate an error. The extra
-    \quote{l} is simply typeset.
-\stopitem
-
-\startitem
-    The upper limit to \type {\endlinechar} and \type {\newlinechar} is 127.
-\stopitem
-
-\startitem
-    Magnification (\type {\mag}) is only supported in \DVI\ output mode. You can
-    set this parameter and it even works with \type {true} units till you switch
-    to \PDF\ output mode. When you use \PDF\ output you can best not touch the
-    \type {\mag} variable. This fuzzy behaviour is not much different from using
-    \PDF\ backend related functionality while eventually \DVI\ output is
-    required.
-
-    After the output mode has been frozen (normally that happens when the first
-    page is shipped out) or when \PDF\ output is enabled, the \type {true}
-    specification is ignored. When you preload a plain format adapted to
-    \LUATEX\ it can be that the \type {\mag} parameter already has been set.
-\stopitem
-
-\stopitemize
-
-\stopsubsection
-
-\startsubsection[title=Changes from \ETEX\ 2.2]
-
-Being the de factor standard extension of course we provide the \ETEX\
-functionality, but with a few small adaptations.
-
-\startitemize
-
-\startitem
-    The \ETEX\ functionality is always present and enabled so the prepended
-    asterisk or \type {-etex} switch for \INITEX\ is not needed.
-\stopitem
-
-\startitem
-    The \TEXXET\ extension is not present, so the primitives \type
-    {\TeXXeTstate}, \type {\beginR}, \type {\beginL}, \type {\endR} and \type
-    {\endL} are missing. Instead we use the \OMEGA\ approach to directionality.
-\stopitem
-
-\startitem
-    Some of the tracing information that is output by \ETEX's \type
-    {\tracingassigns} and \type {\tracingrestores} is not there.
-\stopitem
-
-\startitem
-    Register management in \LUATEX\ uses the \ALEPH\ model, so the maximum value
-    is 65535 and the implementation uses a flat array instead of the mixed
-    flat|\&|sparse model from \ETEX.
-\stopitem
-
-\startitem
-    When kpathsea is used to find files, \LUATEX\ uses the \type {ofm} file
-    format to search for font metrics. In turn, this means that \LUATEX\ looks at
-    the \type {OFMFONTS} configuration variable (like \OMEGA\ and \ALEPH) instead
-    of \type {TFMFONTS} (like \TEX\ and \PDFTEX). Likewise for virtual fonts
-    (\LUATEX\ uses the variable \type {OVFFONTS} instead of \type {VFFONTS}).
-\stopitem
-
-\stopitemize
-
-\stopsubsection
-
-\startsubsection[title=Changes from \PDFTEX\ 1.40]
-
-Because we want to produce \PDF\ the most natural starting point was the popular
-\PDFTEX\ program. We inherit the stable features, dropped most of the
-experimental code and promoted some functionality to core \LUATEX\ functionality
-which in turn triggered renaming primitives.
-
-For compatibility reasons we still refer to \type {\pdf...} commands but \LUATEX\
-has a different backend interface. Instead of these primitives there are three
-interfacing primitives: \type {\pdfextension}, \type {\pdfvariable} and
-\type {\pdffeedback} that take keywords and optional further arguments. This way
-we can extend the features when needed but don't need to adapt the core engine.
-The front- and backend are decoupled as much as possible.
-
-\startitemize
-
-\startitem
-    The (experimental) support for snap nodes has been removed, because it is
-    much more natural to build this functionality on top of node processing and
-    attributes. The associated primitives that are now gone are: \type
-    {\pdfsnaprefpoint}, \type {\pdfsnapy}, and \type {\pdfsnapycomp}.
-\stopitem
-
-\startitem
-    The (experimental) support for specialized spacing around nodes has also been
-    removed. The associated primitives that are now gone are: \type
-    {\pdfadjustinterwordglue}, \type {\pdfprependkern}, and \type
-    {\pdfappendkern}, as well as the five supporting primitives \type
-    {\knbscode}, \type {\stbscode}, \type {\shbscode}, \type {\knbccode}, and
-    \type {\knaccode}.
-\stopitem
-
-\startitem
-    A number of \quote {\PDFTEX\ primitives} have been removed as they can be
-    implemented using \LUA:
-
-    \start \raggedright
-    \type {\pdfelapsedtime}, \type {\pdfescapehex}, \type {\pdfescapename}, \type
-    {\pdfescapestring}, \type {\pdffiledump}, \type {\pdffilemoddate}, \type
-    {\pdffilesize}, \type {\pdfforcepagebox}, \type {\pdflastmatch}, \type
-    {\pdfmatch}, \type {\pdfmdfivesum}, \type {\pdfmovechars}, \type
-    {\pdfoptionalwaysusepdfpagebox}, \type {\pdfoptionpdfinclusionerrorlevel},
-    \type {\pdfresettimer}, \type {\pdfshellescape}, \type {\pdfstrcmp} and \type
-    {\pdfunescapehex}
-    \par \stop
-\stopitem
-
-\startitem
-    The version related primitives \type {\pdftexbanner}, \type {\pdftexversion}
-    and \type {\pdftexrevision} are no longer present as there is no longer a
-    relationship with \PDFTEX\ development.
-\stopitem
-
-\startitem
-    The experimental snapper mechanism has been removed and therefore also the
-    primitives:
-
-    \start \raggedright
-    \type {\pdfignoreddimen}, \type {\pdffirstlineheight}, \type
-    {\pdfeachlineheight}, \type {\pdfeachlinedepth} and \type
-    {\pdflastlinedepth}
-    \par \stop
-\stopitem
-
-\startitem
-    The experimental primitives \type {\primitive}, \type {\ifprimitive}, \type
-    {\ifabsnum} and \type {\ifabsdim} are promoted to core primitives. The \type
-    {\pdf*} prefixed originals are not available.
-\stopitem
-
-\startitem
-    The \PNG\ transparency fix from 1.40.6 is not applied as high|-|level support
-    is pending. Because \LUATEX\ has a different subsystem for managing images,
-    more diversion from its ancestor happened in the meantime.
-\stopitem
-
-\startitem
-    Two extra token lists are provides, \type {\pdfxformresources} and \type
-    {\pdfxformattr}, as an alternative to \type {\pdfxform} keywords.
-\stopitem
-
-\startitem
-    The current version of \LUATEX\ no longer replaces and|/|or merges fonts in
-    embedded \PDF\ files with fonts of the enveloping \PDF\ document. This
-    regression may be temporary, depending on how the rewritten font backend will
-    look like.
-\stopitem
-
-\startitem
-    The primitives \type {\pdfpagewidth} and \type {\pdfpageheight} have been removed
-    because \type {\pagewidth} and \type {\pageheight} have that purpose.
-\stopitem
-
-\startitem
-    The primitives \type {\pdfnormaldeviate}, \type {\pdfuniformdeviate}, \type
-    {\pdfsetrandomseed} and \type {\pdfrandomseed} have been promoted to core
-    primitives without \type {pdf} prefix so the original commands are no longer
-    recognized.
-\stopitem
-
-\startitem
-    The primitives \type {\ifincsname}, \type {\expanded} and \type {\quitvmode}
-    are now core primitives.
-\stopitem
-
-\startitem
-    As the hz and protrusion mechanism are part of the core the related
-    primitives \type {\lpcode}, \type {\rpcode}, \type {\efcode}, \type
-    {\leftmarginkern}, \type {\rightmarginkern} are promoted to core primitives. The
-    two commands \type {\protrudechars} and \type {\adjustspacing} replace their
-    prefixed with \type {\pdf} originals.
-\stopitem
-
-\startitem
-    The hz optimization code has been partially redone so that we no longer need
-    to create extra font instances. The front- and backend have been decoupled
-    and more efficient (\PDF) code is generated.
-\stopitem
-
-\startitem
-    When \type {\adjustspacing} has value~2, hz optimization will be applied to
-    glyphs and kerns. When the value is~3, only glyphs will be treated. A value
-    smaller than~2 disables this feature.
-\stopitem
-
-\startitem
-    The \type {\tagcode} primitive is promoted to core primitive.
-\stopitem
-
-\startitem
-    The \type {\letterspacefont} feature is now part of the core but will not be
-    changed (improved). We just provide it for legacy use.
-\stopitem
-
-\startitem
-    The \type {\pdfnoligatures} primitive is now \type {\ignoreligaturesinfont}.
-\stopitem
-
-\startitem
-    The \type {\pdfcopyfont} primitive is now \type {\copyfont}.
-\stopitem
-
-\startitem
-    The \type {\pdffontexpand} primitive is now \type {\expandglyphsinfont}.
-\stopitem
-
-\startitem
-    Because position tracking is also available in \DVI\ mode the \type
-    {\savepos}, \type {\lastxpos} and \type {\lastypos} commands now replace
-    their \type {pdf} prefixed originals.
-\stopitem
-
-\startitem
-    The introspective primitives \type {\pdflastximagecolordepth} and \type
-    {\pdfximagebbox} have been removed. One can use external applications to
-    determine these properties or use the built|-|in \type {img} library.
-\stopitem
-
-\startitem
-    The initializers \type {\pdfoutput} has been replaced by \type {\outputmode} and
-    \type {\pdfdraftmode} is now \type {\draftmode}.
-\stopitem
-
-\startitem
-    The pixel multiplier dimension \type {\pdfpxdimen} lots its prefix and is now calles
-    \type {\pxdimen}.
-\stopitem
-
-\startitem
-    An extra \type {\pdfimageaddfilename} option has been added that can be used to block
-    writing the filename to the \PDF\ file.
-\stopitem
-
-\startitem
-    The primitive \type {\pdftracingfonts} is now \type {\tracingfonts} as it
-    doesn't relate to the backend.
-\stopitem
-
-\startitem
-    The experimental primitive \type {\pdfinsertht} is kept as \type {\insertht}.
-\stopitem
-
-\startitem
-    The promotion of primitives to core primitives as well as the separation of
-    font- and backend means that the initialization namespace \type {pdftex} is
-    gone.
-\stopitem
-
-\stopitemize
-
-One change involves the so called xforms and ximages. In \PDFTEX\ these are
-implemented as so called whatsits. But contrary to other whatsits they have
-dimensions that need to be taken into account when for instance calculating
-optimal line breaks. In \LUATEX\ these are now promoted to normal nodes, which
-simplifies code that needs those dimensions.
-
-Another reason for promotion is that these are useful concepts. Backends can
-provide the ability to use content that has been rendered in several places,
-and images are also common. For that reason we also changed the names:
-
-\starttabulate[|l|l|]
-\BC new name                             \BC old name                    \NC \NR
-\NC \type {\saveboxresource}             \NC \type {\pdfxform}           \NC \NR
-\NC \type {\saveimageresource}           \NC \type {\pdfximage}          \NC \NR
-\NC \type {\useboxresource}              \NC \type {\pdfrefxform}        \NC \NR
-\NC \type {\useimageresource}            \NC \type {\pdfrefximage}       \NC \NR
-\NC \type {\lastsavedboxresourceindex}   \NC \type {\pdflastxform}       \NC \NR
-\NC \type {\lastsavedimageresourceindex} \NC \type {\pdflastximage}      \NC \NR
-\NC \type {\lastsavedimageresourcepages} \NC \type {\pdflastximagepages} \NC \NR
-\stoptabulate
-
-There are a few \type {\pdffeedback} features that relate to this but these are
-typical backend specific ones. The index that gets returned is to be considered
-as \quote {just a number} and although it still has the same meaning (object
-related) as before, you should not depend on that.
-
-The protrusion detection mechanism is enhanced a bit to enable a bit more complex
-situations. When protrusion characters are identified some nodes are skipped:
-
-\startitemize[packed]
-\startitem zero glue \stopitem
-\startitem penalties \stopitem
-\startitem empty discretionaries \stopitem
-\startitem normal zero kerns \stopitem
-\startitem rules with zero dimensions \stopitem
-\startitem math nodes with a surround of zero \stopitem
-\startitem dir nodes \stopitem
-\startitem empty horizontal lists \stopitem
-\startitem local par nodes \stopitem
-\startitem inserts, marks and adjusts \stopitem
-\startitem boundaries \stopitem
-\startitem whatsits \stopitem
-\stopitemize
-
-Because this can not be enough, you can also use a protrusion boundary node to
-make the next node being ignored. When the value is~1 or~3, the next node will be
-ignored in the test when locating a left boundary condition. When the value is~2
-or~3, the previous node will be ignored when locating a right boundary condition
-(the search goes from right to left). This permits protrusion combined with for
-instance content moved into the margin:
-
-\starttyping
-\protrusionboundary1\llap{!\quad}«Who needs protrusion?»
-\stoptyping
-
-\stopsubsection
-
-\startsubsection[title=Changes from \ALEPH\ RC4]
-
-Because we wanted proper directional typesetting the \ALEPH\ mechanisms looked
-most attractive. These are rather close to the ones provided by \OMEGA, so what
-we say next applies to both these programs.
-
-\startitemize
-
-\startitem
-    The extended 16-bit math primitives (\type {\omathcode} etc.) have been
-    removed.
-\stopitem
-
-\startitem
-    The \OCP\ processing has been removed completely and as a consequence, the
-    following primitives have been removed:
-
-    \start \raggedright
-    \type {\ocp}, \type {\externalocp}, \type {\ocplist}, \type {\pushocplist},
-    \type {\popocplist}, \type {\clearocplists}, \type {\addbeforeocplist}, \type
-    {\addafterocplist}, \type {\removebeforeocplist}, \type {\removeafterocplist}
-    and \type {\ocptracelevel}
-    \par \stop
-\stopitem
-
-\startitem
-    \LUATEX\ only understands 4~of the 16~direction specifiers of \ALEPH: \type
-    {TLT} (latin), \type {TRT} (arabic), \type {RTT} (cjk), \type {LTL}
-    (mongolian). All other direction specifiers generate an error.
-\stopitem
-
-\startitem
-    The input translations from \ALEPH\ are not implemented, the related
-    primitives are not available:
-
-    \start \raggedright
-    \type {\DefaultInputMode}, \type {\noDefaultInputMode}, \type {\noInputMode},
-    \type {\InputMode}, \type {\DefaultOutputMode}, \type {\noDefaultOutputMode},
-    \type {\noOutputMode}, \type {\OutputMode}, \type {\DefaultInputTranslation},
-    \type {\noDefaultInputTranslation}, \type {\noInputTranslation}, \type
-    {\InputTranslation}, \type {\DefaultOutputTranslation}, \type
-    {\noDefaultOutputTranslation}, \type {\noOutputTranslation} and \type
-    {\OutputTranslation}
-    \par \stop
-\stopitem
-
-\startitem
-    Several bugs have been fixed an confusing implementation details have been sorted
-    out.
-\stopitem
-
-\startitem
-    The scanner for direction specifications now allows an optional space after
-    the direction is completely parsed.
-\stopitem
-
-\startitem
-    The \type {^^} notation has been extended: after \type {^^^^} four hexadecimal
-    characters are expected and after \type {^^^^^^} six hexadecimal characters
-    have to be given. The original \TEX\ interpretation is still valid for the
-    \type {^^} case but the four and six variants do no backtracking, i.e.\ when
-    they are not followed by the right number of hexadecimal digits they issue an
-    error message. Because \type{^^^} is a normal \TEX\ case, we don't support the
-    odd number of \type {^^^^^} either.
-\stopitem
-
-\startitem
-    Glues {\it immediately after} direction change commands are not legal
-    breakpoints.
-\stopitem
-
-\startitem
-    Several mechanisms that need to be right|-|to|-|left aware have been
-    improved. For instance placement of formula numbers.
-\stopitem
-
-\startitem
-    The page dimension related primitives \type {\pagewidth} and \type
-    {\pageheight} have been promoted to core primitives. The \type {\hoffset} and
-    \type {\voffset} primitives have been fixed.
-\stopitem
-
-\startitem
-    The primitives \type {\charwd}, \type {\charht}, \type {\chardp} and \type
-    {\charit} have been removed as we have the \ETEX\ variants \type
-    {\fontchar*}.
-\stopitem
-
-\startitem
-    The two dimension registers \type {\pagerightoffset} and \type
-    {\pagebottomoffset} are now core primitives.
-\stopitem
-
-\startitem
-    The direction related primitives \type {\pagedir}, \type {\bodydir}, \type
-    {\pardir}, \type {\textdir}, \type {\mathdir} and \type {\boxdir} are now
-    core primitives.
-\stopitem
-
-\startitem
-    The promotion of primitives to core primitives as well as the removed of all
-    others means that the initialization namespace \type {aleph} is gone.
-\stopitem
-
-\stopitemize
-
-The above let's itself summarize as: we took the 32 bit aspects and much of the
-directional mechanisms.
-
-\stopsubsection
-
-\startsubsection[title=Changes from standard \WEBC]
-
-The compilation framework is \WEBC\ and we keep using that but without the
-\PASCAL\ to \CCODE\ step. This framework also provides some common features that
-deal with reading bytes from files and locating files in \TDS. This is what we do
-different:
-
-\startitemize
-
-\startitem
-    There is no mltex support.
-\stopitem
-
-\startitem
-    There is no enctex support.
-\stopitem
-
-\startitem
-    The following encoding related command line switches are silently ignored,
-    even in non|-|\LUA\ mode: \type {-8bit}, \type {-translate-file}, \type
-    {-mltex}, \type {-enc} and \type {-etex}.
-\stopitem
-
-\startitem
-    The \type {\openout} whatsits are not written to the log file.
-\stopitem
-
-\startitem
-    Some of the so|-|called \WEBC\ extensions are hard to set up in non|-|\KPSE\
-    mode because \type {texmf.cnf} is not read: \type {shell-escape} is off (but
-    that is not a problem because of \LUA's \type {os.execute}), and the paranoia
-    checks on \type {openin} and \type {openout} do not happen. However, it is
-    easy for a \LUA\ script to do this itself by overloading \type {io.open}.
-\stopitem
-
-\startitem
-    The \quote{E} option does not do anything useful.
-\stopitem
-
-\stopitemize
-
-\stopsubsection
-
-\stopsection
-
-\startsection[reference=backendprimitives,title=The backend primitives \type {\pdf*}]
-
-In a previous section we mentioned that some \PDFTEX\ primitives were removed and
-others promoted to core \LUATEX\ primitives. That is only part of the story. In
-order to separate the backend specific primitives in de code these commands are
-now replaced by only a few. In traditional \TEX\ we only had the \DVI\ backend
-but now we have two: \DVI\ and \PDF. Additional functionality is implemented as
-\quote {extensions} in \TEX speak. By separating more strickly we are able to
-keep the core (fontend) clean and stable. If for some reason an extra backend
-option is needed, it can be implemented without touching the core. The three
-\PDF\ backend related primitives are
-
-\starttyping
-\pdfextension command [specification]
-\pdfvariable  name
-\pdffeedback  name
-\stoptyping
-
-An extension triggers further parsing, depending on the command given. A variable is
-a (kind of) register and can be read and written, while a feedback is reporting
-something (as it comes from the backend it's normally a sequence of tokens).
-
-In order for \LUATEX\ to be more than just \TEX\ you need to enable primitives. That
-has already be the case right from the start. If you want the traditional \PDFTEX\
-primitives (for as far their functionality is still around) you now can do this:
-
-\starttyping
-\protected\def\pdfliteral             {\pdfextension literal}
-\protected\def\pdfcolorstack          {\pdfextension colorstack}
-\protected\def\pdfsetmatrix           {\pdfextension setmatrix}
-\protected\def\pdfsave                {\pdfextension save\relax}
-\protected\def\pdfrestore             {\pdfextension restore\relax}
-\protected\def\pdfobj                 {\pdfextension obj }
-\protected\def\pdfrefobj              {\pdfextension refobj }
-\protected\def\pdfannot               {\pdfextension annot }
-\protected\def\pdfstartlink           {\pdfextension startlink }
-\protected\def\pdfendlink             {\pdfextension endlink\relax}
-\protected\def\pdfoutline             {\pdfextension outline }
-\protected\def\pdfdest                {\pdfextension dest }
-\protected\def\pdfthread              {\pdfextension thread }
-\protected\def\pdfstartthread         {\pdfextension startthread }
-\protected\def\pdfendthread           {\pdfextension endthread\relax}
-\protected\def\pdfinfo                {\pdfextension info }
-\protected\def\pdfcatalog             {\pdfextension catalog }
-\protected\def\pdfnames               {\pdfextension names }
-\protected\def\pdfincludechars        {\pdfextension includechars }
-\protected\def\pdffontattr            {\pdfextension fontattr }
-\protected\def\pdfmapfile             {\pdfextension mapfile }
-\protected\def\pdfmapline             {\pdfextension mapline }
-\protected\def\pdftrailer             {\pdfextension trailer }
-\protected\def\pdfglyphtounicode      {\pdfextension glyphtounicode }
-\stoptyping
-
-The introspective primitives can be defined as:
-
-\starttyping
-\def\pdftexversion    {\numexpr\pdffeedback version\relax}
-\def\pdftexrevision           {\pdffeedback revision}
-\def\pdflastlink      {\numexpr\pdffeedback lastlink\relax}
-\def\pdfretval        {\numexpr\pdffeedback retval\relax}
-\def\pdflastobj       {\numexpr\pdffeedback lastobj\relax}
-\def\pdflastannot     {\numexpr\pdffeedback lastannot\relax}
-\def\pdfxformname     {\numexpr\pdffeedback xformname\relax}
-\def\pdfcreationdate          {\pdffeedback creationdate}
-\def\pdffontname      {\numexpr\pdffeedback fontname\relax}
-\def\pdffontobjnum    {\numexpr\pdffeedback fontobjnum\relax}
-\def\pdffontsize      {\dimexpr\pdffeedback fontsize\relax}
-\def\pdfpageref       {\numexpr\pdffeedback pageref\relax}
-\def\pdfcolorstackinit        {\pdffeedback colorstackinit}
-\stoptyping
-
-The configuration related registers have become:
-
-\starttyping
-\edef\pdfcompresslevel            {\pdfvariable compresslevel}
-\edef\pdfobjcompresslevel         {\pdfvariable objcompresslevel}
-\edef\pdfdecimaldigits            {\pdfvariable decimaldigits}
-\edef\pdfgamma                    {\pdfvariable gamma}
-\edef\pdfimageresolution          {\pdfvariable imageresolution}
-\edef\pdfimageapplygamma          {\pdfvariable imageapplygamma}
-\edef\pdfimagegamma               {\pdfvariable imagegamma}
-\edef\pdfimagehicolor             {\pdfvariable imagehicolor}
-\edef\pdfimageaddfilename         {\pdfvariable imageaddfilename}
-\edef\pdfpkresolution             {\pdfvariable pkresolution}
-\edef\pdfpkfixeddpi               {\pdfvariable pkfixeddpi}
-\edef\pdfinclusioncopyfonts       {\pdfvariable inclusioncopyfonts}
-\edef\pdfinclusionerrorlevel      {\pdfvariable inclusionerrorlevel}
-\edef\pdfignoreunknownimages      {\pdfvariable ignoreunknownimages}
-\edef\pdfgentounicode             {\pdfvariable gentounicode}
-\edef\pdfomitcidset               {\pdfvariable omitcidset}
-\edef\pdfpagebox                  {\pdfvariable pagebox}
-\edef\pdfminorversion             {\pdfvariable minorversion}
-\edef\pdfuniqueresname            {\pdfvariable uniqueresname}
-
-\edef\pdfhorigin                  {\pdfvariable horigin}
-\edef\pdfvorigin                  {\pdfvariable vorigin}
-\edef\pdflinkmargin               {\pdfvariable linkmargin}
-\edef\pdfdestmargin               {\pdfvariable destmargin}
-\edef\pdfthreadmargin             {\pdfvariable threadmargin}
-\edef\pdfxformmargin              {\pdfvariable xformmargin}
-
-\edef\pdfpagesattr                {\pdfvariable pagesattr}
-\edef\pdfpageattr                 {\pdfvariable pageattr}
-\edef\pdfpageresources            {\pdfvariable pageresources}
-\edef\pdfxformattr                {\pdfvariable xformattr}
-\edef\pdfxformresources           {\pdfvariable xformresources}
-\edef\pdfpkmode                   {\pdfvariable pkmode}
-
-\edef\pdfsuppressoptionalinfo     {\pdfvariable suppressoptionalinfo }
-\edef\pdftrailerid                {\pdfvariable trailerid }
-\stoptyping
-
-The variables are internal ones, so they are anonymous. When you ask for the
-meaning of a few previously defined ones:
-
-\starttyping
-\meaning\pdfhorigin
-\meaning\pdfcompresslevel
-\meaning\pdfpageattr
-\stoptyping
-
-you will get:
-
-\starttyping
-macro:->[internal backend dimension]
-macro:->[internal backend integer]
-macro:->[internal backend tokenlist]
-\stoptyping
-
-The \type {\edef} can also be an \type {\def} but it's a bit more efficient
-to expand the lookup related register beforehand. After that you can adapt
-the defaults; these are:
-
-\starttyping
-\pdfcompresslevel         9
-\pdfobjcompresslevel      1 % used: (0,9)
-\pdfdecimaldigits         4 % used: (3,6)
-\pdfgamma              1000
-\pdfimageresolution      71
-\pdfimageapplygamma       0
-\pdfimagegamma         2200
-\pdfimagehicolor          1
-\pdfimageaddfilename      1
-\pdfpkresolution         72
-\pdfpkfixeddpi            0
-\pdfinclusioncopyfonts    0
-\pdfinclusionerrorlevel   0
-\pdfignoreunknownimages   0
-\pdfgentounicode          0
-\pdfomitcidset            0
-\pdfpagebox               0
-\pdfminorversion          4
-\pdfuniqueresname         0
-
-\pdfhorigin             1in
-\pdfvorigin             1in
-\pdflinkmargin          0pt
-\pdfdestmargin          0pt
-\pdfthreadmargin        0pt
-\pdfxformmargin         0pt
-\stoptyping
-
-If you also want some backward compatibility, you can add:
-
-\starttyping
-\let\pdfpagewidth      \pagewidth
-\let\pdfpageheight     \pageheight
-
-\let\pdfadjustspacing  \adjustspacing
-\let\pdfprotrudechars  \protrudechars
-\let\pdfnoligatures    \ignoreligaturesinfont
-\let\pdffontexpand     \expandglyphsinfont
-\let\pdfcopyfont       \copyfont
-
-\let\pdfxform          \saveboxresource
-\let\pdflastxform      \lastsavedboxresourceindex
-\let\pdfrefxform       \useboxresource
-
-\let\pdfximage         \saveimageresource
-\let\pdflastximage     \lastsavedimageresourceindex
-\let\pdflastximagepages\lastsavedimageresourcepages
-\let\pdfrefximage      \useimageresource
-
-\let\pdfsavepos        \savepos
-\let\pdflastxpos       \lastxpos
-\let\pdflastypos       \lastypos
-
-\let\pdfoutput         \outputmode
-\let\pdfdraftmode      \draftmode
-
-\let\pdfpxdimen        \pxdimen
-
-\let\pdfinsertht       \insertht
-
-\let\pdfnormaldeviate  \normaldeviate
-\let\pdfuniformdeviate \uniformdeviate
-\let\pdfsetrandomseed  \setrandomseed
-\let\pdfrandomseed     \randomseed
-
-\let\pdfprimitive      \primitive
-\let\ifpdfprimitive    \ifprimitive
-
-\let\ifpdfabsnum       \ifabsnum
-\let\ifpdfabsdim       \ifabsdim
-\stoptyping
-
-And even:
-
-\starttyping
-\newdimen\pdfeachlineheight
-\newdimen\pdfeachlinedepth
-\newdimen\pdflastlinedepth
-\newdimen\pdffirstlineheight
-\newdimen\pdfignoreddimen
-\stoptyping
-
-The backend is derived from \PDFTEX\ so the same syntax applies. However, the
-\type {outline} command accepts a \type {objnum} followed by a number. No
-checking takes place so when this is used it had better be a valid (flushed)
-object.
-
-In order to be (more or less) compatible with \PDFTEX\ we also support the
-option to suppress some info:
-
-\starttyping
-\pdfvariable suppressoptionalinfo \numexpr
-        0
-    +   1   % PTEX.FullBanner
-    +   2   % PTEX.FileName
-    +   4   % PTEX.PageNumber
-    +   8   % PTEX.InfoDict
-    +  16   % Creator
-    +  32   % CreationDate
-    +  64   % ModDate
-    + 128   % Producer
-    + 256   % Trapped
-    + 512   % ID
-\relax
-\stoptyping
-
-In addition you can overload the trailer id, but we don't do any checking on
-validity, so you have to pass a valid array. The following is like the ones
-normally generated by the engine:
-
-\starttyping
-\pdfvariable trailerid {[
-    <FA052949448907805BA83C1E78896398>
-    <FA052949448907805BA83C1E78896398>
-]}
-\stoptyping
-
-So, you even need to include the brackets!
-
-Although we started from a merge of \PDFTEX\ and \ALEPH, by now the code base as
-well as functionality has diverted from those parents. Here we show the options
-that can be passed to the extensions.
-
-\starttexsyntax
-\pdfextension literal
-    [ direct | page | raw ] { tokens }
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension dest
-    num integer | name { tokens }!crlf
-    [ fitbh | fitbv | fitb | fith| fitv | fit |
-      fitr <rule spec> | xyz [ zoom <integer> ]
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension annot
-    reserveobjnum | useobjnum <integer>
-    { tokens }
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension save
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension restore
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension setmatrix
-    { tokens }
-\stoptexsyntax
-
-\starttexsyntax
-[ \immediate ] \pdfextension obj
-    reserveobjnum
-\stoptexsyntax
-
-\starttexsyntax
-[ \immediate ] \pdfextension obj
-    [ useobjnum <integer> ]
-    [ uncompressed ]
-    [ stream  [ attr { tokens } ] ]
-    [ file ]
-    { tokens }
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension refobj
-    <integer>
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension colorstack
-    <integer>
-    set { tokens } | push { tokens } | pop | current
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension startlink
-    [ attr { tokens } ]
-    user { tokens } | goto | thread
-    [ file { tokens } ]
-    [ page <integer> { tokens } | name { tokens } | num  integer ]
-    [ newwindow  | nonewwindow ]
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension endlink
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension startthread
-    num <integer> | name { tokens }
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension endthread
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension thread
-    num <integer> | name { tokens }
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension outline
-    [ attr { tokens } ]
-    [ useobjnum <integer> ]
-    [ count <integer> ]
-    { tokens }
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension glyphtounicode
-    { tokens }
-    { tokens }
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension catalog
-    { tokens }
-    [ openaction
-      user { tokens } | goto | thread
-      [ file { tokens } ]
-      [ page <integer> { tokens } | name { tokens } | num <integer> ]
-      [ newwindow  | nonewwindow ] ]
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension fontattr
-    <integer>
-    {tokens}
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension mapfile
-    {tokens}
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension mapline
-    {tokens}
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension includechars
-    {tokens}
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension info
-    {tokens}
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension names
-    {tokens}
-\stoptexsyntax
-
-\starttexsyntax
-\pdfextension trailer
-    {tokens}
-\stoptexsyntax
-
-\stopsection
-
-\startsection[title=Directions]
-
-The directional model in \LUATEX\ is inherited from \OMEGA|/|\ALEPH\ but we tried
-to improve it a bit. At some point we played with recovery of modes but that was
-disabled later on when we found that it interfered with nested directions. That
-itself had as side effect that the node list was no longer balanced with respect
-to directional nodes which in turn can give side effects when a series of dir
-changes happens without grouping.
-
-The current (0.97 onward) approach is that we again make the list balanced but
-try to avoid some side effects. What happens is quite intuitive if we forget
-about spaces (turned into glue) but even there what happens makes sense if you
-look at it in detail. However that logic makes in|-|group switching kind of
-useless when no proper nested grouping is used: switching from right to left
-several times nested, results in spacing ending up after each other due to nested
-mirroring. Of course a sane macro package will manage this for the user but here
-we are discussing the low level dir injection.
-
-This is what happens:
-
-\starttyping
-\textdir TRT nur {\textdir TLT run \textdir TRT NUR} nur
-\stoptyping
-
-This becomes stepwise:
-
-\startnarrower
-\starttyping
-injected: [+TRT]nur {[+TLT]run [+TRT]NUR} nur
-balanced: [+TRT]nur {[+TLT]run [-TLT][+TRT]NUR[-TRT]} nur[-TRT]
-result  : run {RUNrun } run
-\stoptyping
-\stopnarrower
-
-And this:
-
-\starttyping
-\textdir TRT nur {nur \textdir TLT run \textdir TRT NUR} nur
-\stoptyping
-
-becomes:
-
-\startnarrower
-\starttyping
-injected: [+TRT]nur {nur [+TLT]run [+TRT]NUR} nur
-balanced: [+TRT]nur {nur [+TLT]run [-TLT][+TRT]NUR[-TRT]} nur[-TRT]
-result  : run {run RUNrun } run
-\stoptyping
-\stopnarrower
-
-Now, in the following examples watch where we put the braces:
-
-\startbuffer
-\textdir TRT nur {{\textdir TLT run} {\textdir TRT NUR}} nur
-\stopbuffer
-
-\typebuffer
-
-This becomes:
-
-\startnarrower
-\getbuffer
-\stopnarrower
-
-Compare this to:
-
-\startbuffer
-\textdir TRT nur {{\textdir TLT run }{\textdir TRT NUR}} nur
-\stopbuffer
-
-\typebuffer
-
-Which renders as:
-
-\startnarrower
-\getbuffer
-\stopnarrower
-
-So how do we deal with the next?
-
-\startbuffer
-\def\ltr{\textdir TLT\relax}
-\def\rtl{\textdir TRT\relax}
-
-run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}
-run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}
-\stopbuffer
-
-\typebuffer
-
-It gets typeset as:
-
-\startnarrower
-\startlines
-\getbuffer
-\stoplines
-\stopnarrower
-
-We could define the two helpers to look back, pick up a skip, remove it and
-inject it after the dir node. But that way we loose the subtype information that
-for some applications can be handy to be kept as|-|is. This is why we now have a
-variant of \type {\textdir} which injects the balanced node before the skip.
-Instead of the previous definition we can use:
-
-\startbuffer[def]
-\def\ltr{\linedir TLT\relax}
-\def\rtl{\linedir TRT\relax}
-\stopbuffer
-
-\typebuffer[def]
-
-and this time:
-
-\startbuffer[txt]
-run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}
-run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}
-\stopbuffer
-
-\typebuffer[txt]
-
-comes out as a properly spaced:
-
-\startnarrower
-\startlines
-\getbuffer[def,txt]
-\stoplines
-\stopnarrower
-
-Anything more complex that this, like combination of skips and penalties, or
-kerns, should be handled in the input or macro package because there is no way we
-can predict the expected behaviour. In fact, the \type {\linedir} is just a
-convenience extra which could also have been implemented using node list parsing.
-
-Glue after a dir node is ignored in the linebreak decision but you can bypass that
-by setting \type {\breakafterdirmode} to~\type {1}. The following table shows the
-difference. Watch your spaces.
-
-\def\ShowSome#1{%
-    \BC
-        \type{#1}
-    \NC
-        \breakafterdirmode = 0
-        \hsize 0pt
-        #1
-    \NC
-    \NC
-        \breakafterdirmode = 1
-        \hsize 0pt
-        #1
-    \NC
-    \NC \NR \HL
-}
-\starttabulate[|l|Tp(0pt)|w(5em)|Tp(0pt)|p|]
-    \HL
-    \BC \type{\breakafterdirmode}
-    \BC \type{0}
-    \NC
-    \BC \type{1}
-    \NC
-    \NC \NR
-    \HL
-    \ShowSome{pre {\textdir TLT xxx} post}
-    \ShowSome{pre {\textdir TLT xxx }post}
-    \ShowSome{pre{ \textdir TLT xxx} post}
-    \ShowSome{pre{ \textdir TLT xxx }post}
-    \ShowSome{pre { \textdir TLT xxx } post}
-    \ShowSome{pre {\textdir TLT\relax\space xxx} post}
-\stoptabulate
-
-
-Another adaptation to the \ALEPH\ directional model is control over shapes driven
-by \type {\hangindent} and \type {\parshape}. This is controlled by a new parameter
-\type {\shapemode}:
-
-\starttabulate[|c|c|c|]
-\BC          \BC \type {\hangindent} \BC \type {\parshape} \NC \NR
-\BC \type{0} \NC  normal             \NC normal            \NC \NR
-\BC \type{1} \NC  mirrored           \NC normal            \NC \NR
-\BC \type{2} \NC  normal             \NC mirrored          \NC \NR
-\BC \type{3} \NC  mirrored           \NC mirrored          \NC \NR
-\stoptabulate
-
-The value is reset to zero (like \type {\hangindent} and \type {\parshape})
-after the paragraph is done with. You can use negative values to prevent
-this.
-
-In \in {figure} [fig:shapemode] a few examples are given.
-
-\startplacefigure[reference=fig:shapemode,title={The effect of \type {shapemode}.}]
-    \startcombination[2*3]
-        {\ruledvbox \bgroup \setuptolerance[verytolerant]
-            \hsize .45\textwidth \switchtobodyfont[6pt]
-                \pardir TLT \textdir TLT
-                \hangindent 40pt \hangafter -3
-                \leftskip10pt \input tufte \par
-         \egroup} {TLT: hangindent}
-        {\ruledvbox \bgroup \setuptolerance[verytolerant]
-            \hsize .45\textwidth \switchtobodyfont[6pt]
-            \pardir TLT \textdir TLT
-            \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
-            \input tufte \par
-         \egroup} {TLT: parshape}
-        {\ruledvbox \bgroup \setuptolerance[verytolerant]
-            \hsize .45\textwidth \switchtobodyfont[6pt]
-            \pardir TRT \textdir TRT
-            \hangindent 40pt \hangafter -3
-            \leftskip10pt \input tufte \par
-         \egroup} {TRT: hangindent mode 0}
-        {\ruledvbox \bgroup \setuptolerance[verytolerant]
-            \hsize .45\textwidth \switchtobodyfont[6pt]
-            \pardir TRT \textdir TRT
-            \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
-            \input tufte \par
-         \egroup} {TRT: parshape mode 0}
-        {\ruledvbox \bgroup \setuptolerance[verytolerant]
-            \hsize .45\textwidth \switchtobodyfont[6pt]
-            \shapemode=3
-            \pardir TRT \textdir TRT
-            \hangindent 40pt \hangafter -3
-            \leftskip10pt \input tufte \par
-         \egroup} {TRT: hangindent mode 1 & 3}
-        {\ruledvbox \bgroup \setuptolerance[verytolerant]
-            \hsize .45\textwidth \switchtobodyfont[6pt]
-            \shapemode=3
-            \pardir TRT \textdir TRT
-            \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
-            \input tufte \par
-         \egroup} {TRT: parshape mode 2 & 3}
-    \stopcombination
-\stopplacefigure
-
-\stopsection
-
-\startsection[title=Implementation notes]
-
-\startsubsection[title=Memory allocation]
-
-The single internal memory heap that traditional \TEX\ used for tokens and nodes
-is split into two separate arrays. Each of these will grow dynamically when
-needed.
-
-The \type {texmf.cnf} settings related to main memory are no longer used (these
-are: \type {main_memory}, \type {mem_bot}, \type {extra_mem_top} and \type
-{extra_mem_bot}). \quote {Out of main memory} errors can still occur, but the
-limiting factor is now the amount of RAM in your system, not a predefined limit.
-
-Also, the memory (de)allocation routines for nodes are completely rewritten. The
-relevant code now lives in the C file \type {texnode.c}, and basically uses a
-dozen or so \quote {avail} lists instead of a doubly|-|linked model. An extra
-function layer is added so that the code can ask for nodes by type instead of
-directly requisitioning a certain amount of memory words.
-
-Because of the split into two arrays and the resulting differences in the data
-structures, some of the macros have been duplicated. For instance, there are now
-\type {vlink} and \type {vinfo} as well as \type {token_link} and \type
-{token_info}. All access to the variable memory array is now hidden behind a
-macro called \type {vmem}. We mention this because using the \TEX book as
-reference is still quite valid but not for memory related details. Another
-significate detail is that we have double linked node lists and that some nodes
-carry more data.
-
-The implementation of the growth of two arrays (via reallocation) introduces a
-potential pitfall: the memory arrays should never be used as the left hand side
-of a statement that can modify the array in question. Details like this are
-of no concern to users.
-
-The input line buffer and pool size are now also reallocated when needed, and the
-\type {texmf.cnf} settings \type {buf_size} and \type {pool_size} are silently
-ignored.
-
-\stopsubsection
-
-\startsubsection[title=Sparse arrays]
-
-The \type {\mathcode}, \type {\delcode}, \type {\catcode}, \type {\sfcode}, \type
-{\lccode} and \type {\uccode} (and the new \type {\hjcode}) tables are now sparse
-arrays that are implemented in~\CCODE. They are no longer part of the \TEX\
-\quote {equivalence table} and because each had 1.1 million entries with a few
-memory words each, this makes a major difference in memory usage.
-
-The \type {\catcode}, \type {\sfcode}, \type {\lccode}, \type {\uccode} and \type
-{\hjcode} assignments do not yet show up when using the \ETEX\ tracing routines
-\type {\tracingassigns} and \type {\tracingrestores}.
-
-A side|-|effect of the current implementation is that \type {\global} is now more
-expensive in terms of processing than non|-|global assignments.
-
-The glyph ids within a font are also managed by means of a sparse array as glyph
-ids can go up to index $2^{21}-1$.
-
-\stopsubsection
-
-\startsubsection[title=Simple single-character csnames]
-
-Single|-|character commands are no longer treated specially in the internals,
-they are stored in the hash just like the multiletter csnames.
-
-The code that displays control sequences explicitly checks if the length is one
-when it has to decide whether or not to add a trailing space.
-
-Active characters are internally implemented as a special type of multi|-|letter
-control sequences that uses a prefix that is otherwise impossible to obtain.
-
-\stopsubsection
-
-\startsubsection[title=Compressed format]
-
-The format is passed through \type {zlib}, allowing it to shrink to roughly half
-of the size it would have had in uncompressed form. This takes a bit more \CPU\
-cycles but much less disk \IO, so it should still be faster.
-
-\stopsubsection
-
-\startsubsection[title=Binary file reading]
-
-All of the internal code is changed in such a way that if one of the \type
-{read_xxx_file} callbacks is not set, then the file is read by a \CCODE\ function
-using basically the same convention as the callback: a single read into a buffer
-big enough to hold the entire file contents. While this uses more memory than the
-previous code (that mostly used \type {getc} calls), it can be quite a bit faster
-(depending on your \IO\ subsystem).
-
-\stopsubsection
-
-\startsubsection[title=Tabs and spaces]
-
-We conform to the way other \TEX\ engines handle trailing tabs and spaces. For
-decades trailing tabs and spaces (before a newline) were removed from the input
-but this behaviour was changed in September 2017 to only handle spaces. We are
-aware that this can introduce compatibility issues in existing workflows but
-because we don't want too many differences with upstream \TEXLIVE\ we just follow
-up on that patch (which is a functional one and not really a fix). It is up to
-macro packages maintainers to deal with possible compatibility issues and in
-\LUATEX\ they can do so via the callbacks that deal with reading from files.
-
-The previous behaviour was a known side effect and (as that kind of input
-normally comes from generated sources) it was normally dealt with by adding a
-comment token to the line in case the spaces and|/|or tabs were intentional and
-to be kept. We are aware of the fact that this contradicts some of our other
-choices but consistency with other engines and the fact that in \KPSE\ mode a
-common file \IO\ layer is used can have a side effect of breaking compatibility.
-We still stick to our view that at the log level we can (and might be) more
-incompatible.
-
-\stopsubsection
-
-\stopsection
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-modifications

+

+\startchapter[reference=modifications,title={Modifications}]

+

+\startsection[title=The merged engines]

+

+\startsubsection[title=The need for change]

+

+\topicindex {engines}

+\topicindex {history}

+

+The first version of \LUATEX\ only had a few extra primitives and it was largely

+the same as \PDFTEX. Then we merged substantial parts of \ALEPH\ into the code

+and got more primitives. When we got more stable the decision was made to clean

+up the rather hybrid nature of the program. This means that some primitives have

+been promoted to core primitives, often with a different name, and that others

+were removed. This made it possible to start cleaning up the code base. In \in

+{chapter} [enhancements] we discussed some new primitives, here we will cover

+most of the adapted ones.

+

+Besides the expected changes caused by new functionality, there are a number of

+not|-|so|-|expected changes. These are sometimes a side|-|effect of a new

+(conflicting) feature, or, more often than not, a change necessary to clean up

+the internal interfaces. These will also be mentioned.

+

+\stopsubsection

+

+\startsubsection[title=Changes from \TEX\ 3.1415926]

+

+\topicindex {\TEX}

+

+Of course it all starts with traditional \TEX. Even if we started with \PDFTEX,

+most still comes from the original. But we divert a bit.

+

+\startitemize

+

+\startitem

+    The current code base is written in \CCODE, not \PASCAL. We use \CWEB\ when

+    possible. As a consequence instead of one large file plus change files, we

+    now have multiple files organized in categories like \type {tex}, \type

+    {pdf}, \type {lang}, \type {font}, \type {lua}, etc. There are some artifacts

+    of the conversion to \CCODE, but in due time we will clean up the source code

+    and make sure that the documentation is done right. Many files are in the

+    \CWEB\ format, but others, like those interfacing to \LUA, are \CCODE\ files.

+    Of course we want to stay as close as possible to the original so that the

+    documentation of the fundamentals behind \TEX\ by Don Knuth still applies.

+\stopitem

+

+\startitem

+    See \in {chapter} [languages] for many small changes related to paragraph

+    building, language handling and hyphenation. The most important change is

+    that adding a brace group in the middle of a word (like in \type {of{}fice})

+    does not prevent ligature creation.

+\stopitem

+

+\startitem

+    There is no pool file, all strings are embedded during compilation.

+\stopitem

+

+\startitem

+    The specifier \type {plus 1 fillll} does not generate an error. The extra

+    \quote{l} is simply typeset.

+\stopitem

+

+\startitem

+    The upper limit to \prm {endlinechar} and \prm {newlinechar} is 127.

+\stopitem

+

+\startitem

+    Magnification (\prm {mag}) is only supported in \DVI\ output mode. You can

+    set this parameter and it even works with \type {true} units till you switch

+    to \PDF\ output mode. When you use \PDF\ output you can best not touch the

+    \prm {mag} variable. This fuzzy behaviour is not much different from using

+    \PDF\ backend related functionality while eventually \DVI\ output is

+    required.

+

+    After the output mode has been frozen (normally that happens when the first

+    page is shipped out) or when \PDF\ output is enabled, the \type {true}

+    specification is ignored. When you preload a plain format adapted to

+    \LUATEX\ it can be that the \prm {mag} parameter already has been set.

+\stopitem

+

+\stopitemize

+

+\stopsubsection

+

+\startsubsection[title=Changes from \ETEX\ 2.2]

+

+\topicindex {\ETEX}

+

+Being the de factor standard extension of course we provide the \ETEX\

+functionality, but with a few small adaptations.

+

+\startitemize

+

+\startitem

+    The \ETEX\ functionality is always present and enabled so the prepended

+    asterisk or \type {-etex} switch for \INITEX\ is not needed.

+\stopitem

+

+\startitem

+    The \TEXXET\ extension is not present, so the primitives \type

+    {\TeXXeTstate}, \type {\beginR}, \type {\beginL}, \type {\endR} and \type

+    {\endL} are missing. Instead we used the \OMEGA/\ALEPH\ approach to

+    directionality as starting point.

+\stopitem

+

+\startitem

+    Some of the tracing information that is output by \ETEX's \prm

+    {tracingassigns} and \prm {tracingrestores} is not there.

+\stopitem

+

+\startitem

+    Register management in \LUATEX\ uses the \OMEGA/\ALEPH\ model, so the maximum

+    value is 65535 and the implementation uses a flat array instead of the mixed

+    flat & sparse model from \ETEX.

+\stopitem

+

+\startitem

+    When kpathsea is used to find files, \LUATEX\ uses the \type {ofm} file

+    format to search for font metrics. In turn, this means that \LUATEX\ looks at

+    the \type {OFMFONTS} configuration variable (like \OMEGA\ and \ALEPH) instead

+    of \type {TFMFONTS} (like \TEX\ and \PDFTEX). Likewise for virtual fonts

+    (\LUATEX\ uses the variable \type {OVFFONTS} instead of \type {VFFONTS}).

+\stopitem

+

+\stopitemize

+

+\stopsubsection

+

+\startsubsection[title=Changes from \PDFTEX\ 1.40]

+

+\topicindex {\PDFTEX}

+

+Because we want to produce \PDF\ the most natural starting point was the popular

+\PDFTEX\ program. We inherit the stable features, dropped most of the

+experimental code and promoted some functionality to core \LUATEX\ functionality

+which in turn triggered renaming primitives.

+

+For compatibility reasons we still refer to \type {\pdf...} commands but \LUATEX\

+has a different backend interface. Instead of these primitives there are three

+interfacing primitives: \lpr {pdfextension}, \lpr {pdfvariable} and \lpr

+{pdffeedback} that take keywords and optional further arguments (below we will

+still use the \tex {pdf} prefix names as reference). This way we can extend the

+features when needed but don't need to adapt the core engine. The front- and

+backend are decoupled as much as possible.

+

+\startitemize

+

+\startitem

+    The (experimental) support for snap nodes has been removed, because it is

+    much more natural to build this functionality on top of node processing and

+    attributes. The associated primitives that are gone are: \orm

+    {pdfsnaprefpoint}, \orm {pdfsnapy}, and \orm {pdfsnapycomp}.

+\stopitem

+

+\startitem

+    The (experimental) support for specialized spacing around nodes has also been

+    removed. The associated primitives that are gone are: \orm

+    {pdfadjustinterwordglue}, \orm {pdfprependkern}, and \orm {pdfappendkern}, as

+    well as the five supporting primitives \orm {knbscode}, \orm {stbscode}, \orm

+    {shbscode}, \orm {knbccode}, and \orm {knaccode}.

+\stopitem

+

+\startitem

+    A number of \quote {\PDFTEX\ primitives} have been removed as they can be

+    implemented using \LUA: \orm {pdfelapsedtime}, \orm {pdfescapehex}, \orm

+    {pdfescapename}, \orm {pdfescapestring}, \orm {pdffiledump}, \orm

+    {pdffilemoddate}, \orm {pdffilesize}, \orm {pdfforcepagebox}, \orm

+    {pdflastmatch}, \orm {pdfmatch}, \orm {pdfmdfivesum}, \orm {pdfmovechars},

+    \orm {pdfoptionalwaysusepdfpagebox}, \orm {pdfoptionpdfinclusionerrorlevel},

+    \orm {pdfresettimer}, \orm {pdfshellescape}, \orm {pdfstrcmp} and \orm

+    {pdfunescapehex}.

+\stopitem

+

+\startitem

+    The version related primitives \orm {pdftexbanner}, \orm {pdftexversion}

+    and \orm {pdftexrevision} are no longer present as there is no longer a

+    relationship with \PDFTEX\ development.

+\stopitem

+

+\startitem

+    The experimental snapper mechanism has been removed and therefore also the

+    primitives \orm {pdfignoreddimen}, \orm {pdffirstlineheight}, \orm

+    {pdfeachlineheight}, \orm {pdfeachlinedepth} and \orm {pdflastlinedepth}.

+\stopitem

+

+\startitem

+    The experimental primitives \lpr {primitive}, \lpr {ifprimitive}, \lpr

+    {ifabsnum} and \lpr {ifabsdim} are promoted to core primitives. The \type

+    {\pdf*} prefixed originals are not available.

+\stopitem

+

+\startitem

+    Because \LUATEX\ has a different subsystem for managing images, more

+    diversion from its ancestor happened in the meantime. We don't adapt to

+    changes in \PDFTEX.

+\stopitem

+

+\startitem

+    Two extra token lists are provided, \orm {pdfxformresources} and \orm

+    {pdfxformattr}, as an alternative to \orm {pdfxform} keywords.

+\stopitem

+

+\startitem

+    Image specifications also support \type {visiblefilename}, \type

+    {userpassword} and \type {ownerpassword}. The password options are only

+    relevant for encrypted \PDF\ files.

+\stopitem

+

+\startitem

+    The current version of \LUATEX\ no longer replaces and|/|or merges fonts in

+    embedded \PDF\ files with fonts of the enveloping \PDF\ document. This

+    regression may be temporary, depending on how the rewritten font backend will

+    look like.

+\stopitem

+

+\startitem

+    The primitives \orm {pdfpagewidth} and \orm {pdfpageheight} have been removed

+    because \lpr {pagewidth} and \lpr {pageheight} have that purpose.

+\stopitem

+

+\startitem

+    The primitives \orm {pdfnormaldeviate}, \orm {pdfuniformdeviate}, \orm

+    {pdfsetrandomseed} and \orm {pdfrandomseed} have been promoted to core

+    primitives without \type {pdf} prefix so the original commands are no longer

+    recognized.

+\stopitem

+

+\startitem

+    The primitives \lpr {ifincsname}, \lpr {expanded} and \lpr {quitvmode}

+    are now core primitives.

+\stopitem

+

+\startitem

+    As the hz and protrusion mechanism are part of the core the related

+    primitives \lpr {lpcode}, \lpr {rpcode}, \lpr {efcode}, \lpr

+    {leftmarginkern}, \lpr {rightmarginkern} are promoted to core primitives. The

+    two commands \lpr {protrudechars} and \lpr {adjustspacing} replace their

+    prefixed with \type {\pdf} originals.

+\stopitem

+

+\startitem

+    The hz optimization code has been partially redone so that we no longer need

+    to create extra font instances. The front- and backend have been decoupled

+    and more efficient (\PDF) code is generated.

+\stopitem

+

+\startitem

+    When \lpr {adjustspacing} has value~2, hz optimization will be applied to

+    glyphs and kerns. When the value is~3, only glyphs will be treated. A value

+    smaller than~2 disables this feature.

+\stopitem

+

+\startitem

+    The \lpr {tagcode} primitive is promoted to core primitive.

+\stopitem

+

+\startitem

+    The \lpr {letterspacefont} feature is now part of the core but will not be

+    changed (improved). We just provide it for legacy use.

+\stopitem

+

+\startitem

+    The \orm {pdfnoligatures} primitive is now \lpr {ignoreligaturesinfont}.

+\stopitem

+

+\startitem

+    The \orm {pdfcopyfont} primitive is now \lpr {copyfont}.

+\stopitem

+

+\startitem

+    The \orm {pdffontexpand} primitive is now \lpr {expandglyphsinfont}.

+\stopitem

+

+\startitem

+    Because position tracking is also available in \DVI\ mode the \lpr {savepos},

+    \lpr {lastxpos} and \lpr {lastypos} commands now replace their \type {pdf}

+    prefixed originals.

+\stopitem

+

+\startitem

+    The introspective primitives \type {\pdflastximagecolordepth} and \type

+    {\pdfximagebbox} have been removed. One can use external applications to

+    determine these properties or use the built|-|in \type {img} library.

+\stopitem

+

+\startitem

+    The initializers \orm {pdfoutput} has been replaced by \lpr {outputmode} and

+    \orm {pdfdraftmode} is now \lpr {draftmode}.

+\stopitem

+

+\startitem

+    The pixel multiplier dimension \orm {pdfpxdimen} lost its prefix and is now

+    called \lpr {pxdimen}.

+\stopitem

+

+\startitem

+    An extra \orm {pdfimageaddfilename} option has been added that can be used to

+    block writing the filename to the \PDF\ file.

+\stopitem

+

+\startitem

+    The primitive \orm {pdftracingfonts} is now \lpr {tracingfonts} as it

+    doesn't relate to the backend.

+\stopitem

+

+\startitem

+    The experimental primitive \orm {pdfinsertht} is kept as \lpr {insertht}.

+\stopitem

+

+\startitem

+    There is some more control over what metadata goes into the \PDF\ file.

+\stopitem

+

+\startitem

+    The promotion of primitives to core primitives as well as the separation of

+    font- and backend means that the initialization namespace \type {pdftex} is

+    gone.

+\stopitem

+

+\stopitemize

+

+One change involves the so called xforms and ximages. In \PDFTEX\ these are

+implemented as so called whatsits. But contrary to other whatsits they have

+dimensions that need to be taken into account when for instance calculating

+optimal line breaks. In \LUATEX\ these are now promoted to a special type of rule

+nodes, which simplifies code that needs those dimensions.

+

+Another reason for promotion is that these are useful concepts. Backends can

+provide the ability to use content that has been rendered in several places, and

+images are also common. As already mentioned in \in {section}

+[sec:imagedandforms], we now have:

+

+\starttabulate[|l|l|]

+\DB \LUATEX \BC \PDFTEX \NC \NR

+\TB

+\NC \lpr {saveboxresource}             \NC \orm {pdfxform}           \NC \NR

+\NC \lpr {saveimageresource}           \NC \orm {pdfximage}          \NC \NR

+\NC \lpr {useboxresource}              \NC \orm {pdfrefxform}        \NC \NR

+\NC \lpr {useimageresource}            \NC \orm {pdfrefximage}       \NC \NR

+\NC \lpr {lastsavedboxresourceindex}   \NC \orm {pdflastxform}       \NC \NR

+\NC \lpr {lastsavedimageresourceindex} \NC \orm {pdflastximage}      \NC \NR

+\NC \lpr {lastsavedimageresourcepages} \NC \orm {pdflastximagepages} \NC \NR

+\LL

+\stoptabulate

+

+There are a few \lpr {pdffeedback} features that relate to this but these are

+typical backend specific ones. The index that gets returned is to be considered

+as \quote {just a number} and although it still has the same meaning (object

+related) as before, you should not depend on that.

+

+The protrusion detection mechanism is enhanced a bit to enable a bit more complex

+situations. When protrusion characters are identified some nodes are skipped:

+

+\startitemize[packed,columns,two]

+\startitem zero glue \stopitem

+\startitem penalties \stopitem

+\startitem empty discretionaries \stopitem

+\startitem normal zero kerns \stopitem

+\startitem rules with zero dimensions \stopitem

+\startitem math nodes with a surround of zero \stopitem

+\startitem dir nodes \stopitem

+\startitem empty horizontal lists \stopitem

+\startitem local par nodes \stopitem

+\startitem inserts, marks and adjusts \stopitem

+\startitem boundaries \stopitem

+\startitem whatsits \stopitem

+\stopitemize

+

+Because this can not be enough, you can also use a protrusion boundary node to

+make the next node being ignored. When the value is~1 or~3, the next node will be

+ignored in the test when locating a left boundary condition. When the value is~2

+or~3, the previous node will be ignored when locating a right boundary condition

+(the search goes from right to left). This permits protrusion combined with for

+instance content moved into the margin:

+

+\starttyping

+\protrusionboundary1\llap{!\quad}«Who needs protrusion?»

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title=Changes from \ALEPH\ RC4]

+

+\topicindex {\ALEPH}

+

+Because we wanted proper directional typesetting the \ALEPH\ mechanisms looked

+most attractive. These are rather close to the ones provided by \OMEGA, so what

+we say next applies to both these programs.

+

+\startitemize

+

+\startitem

+    The extended 16-bit math primitives (\orm {omathcode} etc.) have been

+    removed.

+\stopitem

+

+\startitem

+    The \OCP\ processing has been removed completely and as a consequence, the

+    following primitives have been removed: \orm {ocp}, \orm {externalocp}, \orm

+    {ocplist}, \orm {pushocplist}, \orm {popocplist}, \orm {clearocplists}, \orm

+    {addbeforeocplist}, \orm {addafterocplist}, \orm {removebeforeocplist}, \orm

+    {removeafterocplist} and \orm {ocptracelevel}.

+\stopitem

+

+\startitem

+    \LUATEX\ only understands 4~of the 16~direction specifiers of \ALEPH: \type

+    {TLT} (latin), \type {TRT} (arabic), \type {RTT} (cjk), \type {LTL} (mongolian).

+    All other direction specifiers generate an error. In addition to a keyword

+    driven model we also provide an integer driven one.

+\stopitem

+

+\startitem

+    The input translations from \ALEPH\ are not implemented, the related

+    primitives are not available: \orm {DefaultInputMode}, \orm

+    {noDefaultInputMode}, \orm {noInputMode}, \orm {InputMode}, \orm

+    {DefaultOutputMode}, \orm {noDefaultOutputMode}, \orm {noOutputMode}, \orm

+    {OutputMode}, \orm {DefaultInputTranslation}, \orm

+    {noDefaultInputTranslation}, \orm {noInputTranslation}, \orm

+    {InputTranslation}, \orm {DefaultOutputTranslation}, \orm

+    {noDefaultOutputTranslation}, \orm {noOutputTranslation} and \orm

+    {OutputTranslation}.

+\stopitem

+

+\startitem

+    Several bugs have been fixed and confusing implementation details have been

+    sorted out.

+\stopitem

+

+\startitem

+    The scanner for direction specifications now allows an optional space after

+    the direction is completely parsed.

+\stopitem

+

+\startitem

+    The \type {^^} notation has been extended: after \type {^^^^} four

+    hexadecimal characters are expected and after \type {^^^^^^} six hexadecimal

+    characters have to be given. The original \TEX\ interpretation is still valid

+    for the \type {^^} case but the four and six variants do no backtracking,

+    i.e.\ when they are not followed by the right number of hexadecimal digits

+    they issue an error message. Because \type{^^^} is a normal \TEX\ case, we

+    don't support the odd number of \type {^^^^^} either.

+\stopitem

+

+\startitem

+    Glues {\it immediately after} direction change commands are not legal

+    breakpoints.

+\stopitem

+

+\startitem

+    Several mechanisms that need to be right|-|to|-|left aware have been

+    improved. For instance placement of formula numbers.

+\stopitem

+

+\startitem

+    The page dimension related primitives \lpr {pagewidth} and \lpr {pageheight}

+    have been promoted to core primitives. The \prm {hoffset} and \prm {voffset}

+    primitives have been fixed.

+\stopitem

+

+\startitem

+    The primitives \type {\charwd}, \type {\charht}, \type {\chardp} and \type

+    {\charit} have been removed as we have the \ETEX\ variants \type

+    {\fontchar*}.

+\stopitem

+

+\startitem

+    The two dimension registers \lpr {pagerightoffset} and \lpr

+    {pagebottomoffset} are now core primitives.

+\stopitem

+

+\startitem

+    The direction related primitives \lpr {pagedir}, \lpr {bodydir}, \lpr

+    {pardir}, \lpr {textdir}, \lpr {mathdir} and \lpr {boxdir} are now core

+    primitives.

+\stopitem

+

+\startitem

+    The promotion of primitives to core primitives as well as removing of all

+    others means that the initialization namespace \type {aleph} that early

+    versions of \LUATEX\ provided is gone.

+\stopitem

+

+\stopitemize

+

+The above let's itself summarize as: we took the 32 bit aspects and much of the

+directional mechanisms and merged it into the \PDFTEX\ code base as starting

+point for further development. Then we simplified directionality, fixed it and

+opened it up.

+

+\stopsubsection

+

+\startsubsection[title=Changes from standard \WEBC]

+

+\topicindex {\WEBC}

+

+The compilation framework is \WEBC\ and we keep using that but without the

+\PASCAL\ to \CCODE\ step. This framework also provides some common features that

+deal with reading bytes from files and locating files in \TDS. This is what we do

+different:

+

+\startitemize

+

+\startitem

+    There is no mltex support.

+\stopitem

+

+\startitem

+    There is no enctex support.

+\stopitem

+

+\startitem

+    The following encoding related command line switches are silently ignored,

+    even in non|-|\LUA\ mode: \type {-8bit}, \type {-translate-file}, \type

+    {-mltex}, \type {-enc} and \type {-etex}.

+\stopitem

+

+\startitem

+    The \prm {openout} whatsits are not written to the log file.

+\stopitem

+

+\startitem

+    Some of the so|-|called \WEBC\ extensions are hard to set up in non|-|\KPSE\

+    mode because \type {texmf.cnf} is not read: \type {shell-escape} is off (but

+    that is not a problem because of \LUA's \type {os.execute}), and the paranoia

+    checks on \type {openin} and \type {openout} do not happen. However, it is

+    easy for a \LUA\ script to do this itself by overloading \type {io.open} and

+    alike.

+\stopitem

+

+\startitem

+    The \quote{E} option does not do anything useful.

+\stopitem

+

+\stopitemize

+

+\stopsubsection

+

+\stopsection

+

+\startsection[reference=backendprimitives,title=The backend primitives]

+

+\startsubsection[title={Less primitives}]

+

+\topicindex {backend}

+\topicindex {\PDF+backend}

+

+In a previous section we mentioned that some \PDFTEX\ primitives were removed and

+others promoted to core \LUATEX\ primitives. That is only part of the story. In

+order to separate the backend specific primitives in de code these commands are

+now replaced by only a few. In traditional \TEX\ we only had the \DVI\ backend

+but now we have two: \DVI\ and \PDF. Additional functionality is implemented as

+\quote {extensions} in \TEX\ speak. By separating more strickly we are able to

+keep the core (frontend) clean and stable and isolate these extensions. If for

+some reason an extra backend option is needed, it can be implemented without

+touching the core. The three \PDF\ backend related primitives are:

+

+\starttyping

+\pdfextension command [specification]

+\pdfvariable  name

+\pdffeedback  name

+\stoptyping

+

+An extension triggers further parsing, depending on the command given. A variable is

+a (kind of) register and can be read and written, while a feedback is reporting

+something (as it comes from the backend it's normally a sequence of tokens).

+

+\stopsubsection

+

+\startsubsection[title={\lpr{pdfextension}, \lpr {pdfvariable} and \lpr {pdffeedback}},reference=sec:pdfextensions]

+

+In order for \LUATEX\ to be more than just \TEX\ you need to enable primitives. That

+has already been the case right from the start. If you want the traditional \PDFTEX\

+primitives (for as far their functionality is still around) you now can do this:

+

+\starttyping

+\protected\def\pdfliteral             {\pdfextension literal}

+\protected\def\pdfcolorstack          {\pdfextension colorstack}

+\protected\def\pdfsetmatrix           {\pdfextension setmatrix}

+\protected\def\pdfsave                {\pdfextension save\relax}

+\protected\def\pdfrestore             {\pdfextension restore\relax}

+\protected\def\pdfobj                 {\pdfextension obj }

+\protected\def\pdfrefobj              {\pdfextension refobj }

+\protected\def\pdfannot               {\pdfextension annot }

+\protected\def\pdfstartlink           {\pdfextension startlink }

+\protected\def\pdfendlink             {\pdfextension endlink\relax}

+\protected\def\pdfoutline             {\pdfextension outline }

+\protected\def\pdfdest                {\pdfextension dest }

+\protected\def\pdfthread              {\pdfextension thread }

+\protected\def\pdfstartthread         {\pdfextension startthread }

+\protected\def\pdfendthread           {\pdfextension endthread\relax}

+\protected\def\pdfinfo                {\pdfextension info }

+\protected\def\pdfcatalog             {\pdfextension catalog }

+\protected\def\pdfnames               {\pdfextension names }

+\protected\def\pdfincludechars        {\pdfextension includechars }

+\protected\def\pdffontattr            {\pdfextension fontattr }

+\protected\def\pdfmapfile             {\pdfextension mapfile }

+\protected\def\pdfmapline             {\pdfextension mapline }

+\protected\def\pdftrailer             {\pdfextension trailer }

+\protected\def\pdfglyphtounicode      {\pdfextension glyphtounicode }

+\stoptyping

+

+The introspective primitives can be defined as:

+

+\starttyping

+\def\pdftexversion    {\numexpr\pdffeedback version\relax}

+\def\pdftexrevision           {\pdffeedback revision}

+\def\pdflastlink      {\numexpr\pdffeedback lastlink\relax}

+\def\pdfretval        {\numexpr\pdffeedback retval\relax}

+\def\pdflastobj       {\numexpr\pdffeedback lastobj\relax}

+\def\pdflastannot     {\numexpr\pdffeedback lastannot\relax}

+\def\pdfxformname     {\numexpr\pdffeedback xformname\relax}

+\def\pdfcreationdate          {\pdffeedback creationdate}

+\def\pdffontname      {\numexpr\pdffeedback fontname\relax}

+\def\pdffontobjnum    {\numexpr\pdffeedback fontobjnum\relax}

+\def\pdffontsize      {\dimexpr\pdffeedback fontsize\relax}

+\def\pdfpageref       {\numexpr\pdffeedback pageref\relax}

+\def\pdfcolorstackinit        {\pdffeedback colorstackinit}

+\stoptyping

+

+The configuration related registers have become:

+

+\starttyping

+\edef\pdfcompresslevel            {\pdfvariable compresslevel}

+\edef\pdfobjcompresslevel         {\pdfvariable objcompresslevel}

+\edef\pdfrecompress               {\pdfvariable recompress}

+\edef\pdfdecimaldigits            {\pdfvariable decimaldigits}

+\edef\pdfgamma                    {\pdfvariable gamma}

+\edef\pdfimageresolution          {\pdfvariable imageresolution}

+\edef\pdfimageapplygamma          {\pdfvariable imageapplygamma}

+\edef\pdfimagegamma               {\pdfvariable imagegamma}

+\edef\pdfimagehicolor             {\pdfvariable imagehicolor}

+\edef\pdfimageaddfilename         {\pdfvariable imageaddfilename}

+\edef\pdfpkresolution             {\pdfvariable pkresolution}

+\edef\pdfpkfixeddpi               {\pdfvariable pkfixeddpi}

+\edef\pdfinclusioncopyfonts       {\pdfvariable inclusioncopyfonts}

+\edef\pdfinclusionerrorlevel      {\pdfvariable inclusionerrorlevel}

+\edef\pdfignoreunknownimages      {\pdfvariable ignoreunknownimages}

+\edef\pdfgentounicode             {\pdfvariable gentounicode}

+\edef\pdfomitcidset               {\pdfvariable omitcidset}

+\edef\pdfomitcharset              {\pdfvariable omitcharset}

+\edef\pdfpagebox                  {\pdfvariable pagebox}

+\edef\pdfminorversion             {\pdfvariable minorversion}

+\edef\pdfuniqueresname            {\pdfvariable uniqueresname}

+

+\edef\pdfhorigin                  {\pdfvariable horigin}

+\edef\pdfvorigin                  {\pdfvariable vorigin}

+\edef\pdflinkmargin               {\pdfvariable linkmargin}

+\edef\pdfdestmargin               {\pdfvariable destmargin}

+\edef\pdfthreadmargin             {\pdfvariable threadmargin}

+\edef\pdfxformmargin              {\pdfvariable xformmargin}

+

+\edef\pdfpagesattr                {\pdfvariable pagesattr}

+\edef\pdfpageattr                 {\pdfvariable pageattr}

+\edef\pdfpageresources            {\pdfvariable pageresources}

+\edef\pdfxformattr                {\pdfvariable xformattr}

+\edef\pdfxformresources           {\pdfvariable xformresources}

+\edef\pdfpkmode                   {\pdfvariable pkmode}

+

+\edef\pdfsuppressoptionalinfo     {\pdfvariable suppressoptionalinfo }

+\edef\pdftrailerid                {\pdfvariable trailerid }

+\stoptyping

+

+The variables are internal ones, so they are anonymous. When you ask for the

+meaning of a few previously defined ones:

+

+\starttyping

+\meaning\pdfhorigin

+\meaning\pdfcompresslevel

+\meaning\pdfpageattr

+\stoptyping

+

+you will get:

+

+\starttyping

+macro:->[internal backend dimension]

+macro:->[internal backend integer]

+macro:->[internal backend tokenlist]

+\stoptyping

+

+The \prm {edef} can also be a \prm {def} but it's a bit more efficient to expand

+the lookup related register beforehand.

+

+The backend is derived from \PDFTEX\ so the same syntax applies. However, the

+\type {outline} command accepts a \type {objnum} followed by a number. No

+checking takes place so when this is used it had better be a valid (flushed)

+object.

+

+In order to be (more or less) compatible with \PDFTEX\ we also support the option

+to suppress some info but we do so via a bitset:

+

+\starttyping

+\pdfvariable suppressoptionalinfo \numexpr

+        0

+    +   1   % PTEX.FullBanner

+    +   2   % PTEX.FileName

+    +   4   % PTEX.PageNumber

+    +   8   % PTEX.InfoDict

+    +  16   % Creator

+    +  32   % CreationDate

+    +  64   % ModDate

+    + 128   % Producer

+    + 256   % Trapped

+    + 512   % ID

+\relax

+\stoptyping

+

+In addition you can overload the trailer id, but we don't do any checking on

+validity, so you have to pass a valid array. The following is like the ones

+normally generated by the engine. You even need to include the brackets here!

+

+\starttyping

+\pdfvariable trailerid {[

+    <FA052949448907805BA83C1E78896398>

+    <FA052949448907805BA83C1E78896398>

+]}

+\stoptyping

+

+Although we started from a merge of \PDFTEX\ and \ALEPH, by now the code base as

+well as functionality has diverted from those parents. Here we show the options

+that can be passed to the extensions.

+

+\starttexsyntax

+\pdfextension literal

+    [ direct | page | raw ] { tokens }

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension dest

+    num integer | name { tokens }!crlf

+    [ fitbh | fitbv | fitb | fith| fitv | fit |

+      fitr <rule spec> | xyz [ zoom <integer> ]

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension annot

+    reserveobjnum | useobjnum <integer>

+    { tokens }

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension save

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension restore

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension setmatrix

+    { tokens }

+\stoptexsyntax

+

+\starttexsyntax

+[ \immediate ] \pdfextension obj

+    reserveobjnum

+\stoptexsyntax

+

+\starttexsyntax

+[ \immediate ] \pdfextension obj

+    [ useobjnum <integer> ]

+    [ uncompressed ]

+    [ stream  [ attr { tokens } ] ]

+    [ file ]

+    { tokens }

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension refobj

+    <integer>

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension colorstack

+    <integer>

+    set { tokens } | push { tokens } | pop | current

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension startlink

+    [ attr { tokens } ]

+    user { tokens } | goto | thread

+    [ file { tokens } ]

+    [ page <integer> { tokens } | name { tokens } | num  integer ]

+    [ newwindow  | nonewwindow ]

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension endlink

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension startthread

+    num <integer> | name { tokens }

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension endthread

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension thread

+    num <integer> | name { tokens }

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension outline

+    [ attr { tokens } ]

+    [ useobjnum <integer> ]

+    [ count <integer> ]

+    { tokens }

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension glyphtounicode

+    { tokens }

+    { tokens }

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension catalog

+    { tokens }

+    [ openaction

+      user { tokens } | goto | thread

+      [ file { tokens } ]

+      [ page <integer> { tokens } | name { tokens } | num <integer> ]

+      [ newwindow  | nonewwindow ] ]

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension fontattr

+    <integer>

+    {tokens}

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension mapfile

+    {tokens}

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension mapline

+    {tokens}

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension includechars

+    {tokens}

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension info

+    {tokens}

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension names

+    {tokens}

+\stoptexsyntax

+

+\starttexsyntax

+\pdfextension trailer

+    {tokens}

+\stoptexsyntax

+

+\stopsubsection

+

+\startsubsection[title={Defaults}]

+

+The engine sets the following defaults.

+

+\starttyping

+\pdfcompresslevel         9

+\pdfobjcompresslevel      1 % used: (0,9)

+\pdfrecompress            0 % mostly for debugging

+\pdfdecimaldigits         4 % used: (3,6)

+\pdfgamma              1000

+\pdfimageresolution      71

+\pdfimageapplygamma       0

+\pdfimagegamma         2200

+\pdfimagehicolor          1

+\pdfimageaddfilename      1

+\pdfpkresolution         72

+\pdfpkfixeddpi            0

+\pdfinclusioncopyfonts    0

+\pdfinclusionerrorlevel   0

+\pdfignoreunknownimages   0

+\pdfgentounicode          0

+\pdfomitcidset            0

+\pdfomitcharset           0

+\pdfpagebox               0

+\pdfminorversion          4

+\pdfuniqueresname         0

+

+\pdfhorigin             1in

+\pdfvorigin             1in

+\pdflinkmargin          0pt

+\pdfdestmargin          0pt

+\pdfthreadmargin        0pt

+\pdfxformmargin         0pt

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={Backward compatibility}]

+

+If you also want some backward compatibility, you can add:

+

+\starttyping

+\let\pdfpagewidth      \pagewidth

+\let\pdfpageheight     \pageheight

+

+\let\pdfadjustspacing  \adjustspacing

+\let\pdfprotrudechars  \protrudechars

+\let\pdfnoligatures    \ignoreligaturesinfont

+\let\pdffontexpand     \expandglyphsinfont

+\let\pdfcopyfont       \copyfont

+

+\let\pdfxform          \saveboxresource

+\let\pdflastxform      \lastsavedboxresourceindex

+\let\pdfrefxform       \useboxresource

+

+\let\pdfximage         \saveimageresource

+\let\pdflastximage     \lastsavedimageresourceindex

+\let\pdflastximagepages\lastsavedimageresourcepages

+\let\pdfrefximage      \useimageresource

+

+\let\pdfsavepos        \savepos

+\let\pdflastxpos       \lastxpos

+\let\pdflastypos       \lastypos

+

+\let\pdfoutput         \outputmode

+\let\pdfdraftmode      \draftmode

+

+\let\pdfpxdimen        \pxdimen

+

+\let\pdfinsertht       \insertht

+

+\let\pdfnormaldeviate  \normaldeviate

+\let\pdfuniformdeviate \uniformdeviate

+\let\pdfsetrandomseed  \setrandomseed

+\let\pdfrandomseed     \randomseed

+

+\let\pdfprimitive      \primitive

+\let\ifpdfprimitive    \ifprimitive

+

+\let\ifpdfabsnum       \ifabsnum

+\let\ifpdfabsdim       \ifabsdim

+\stoptyping

+

+And even:

+

+\starttyping

+\newdimen\pdfeachlineheight

+\newdimen\pdfeachlinedepth

+\newdimen\pdflastlinedepth

+\newdimen\pdffirstlineheight

+\newdimen\pdfignoreddimen

+\stoptyping

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title=Directions]

+

+\topicindex {\OMEGA}

+\topicindex {\ALEPH}

+\topicindex {directions}

+

+\startsubsection[title={Four directions}]

+

+The directional model in \LUATEX\ is inherited from \OMEGA|/|\ALEPH\ but we tried

+to improve it a bit. At some point we played with recovery of modes but that was

+disabled later on when we found that it interfered with nested directions. That

+itself had as side effect that the node list was no longer balanced with respect

+to directional nodes which in turn can give side effects when a series of dir

+changes happens without grouping.

+

+When extending the \PDF\ backend to support directions some inconsistencies were

+found and as a result we decided to support only the four models that make sense

+\type {TLT} (latin), \type {TRT} (arabic), \type {RTT} (cjk) and \type {LTL}

+(mongolian).

+

+\stopsubsection

+

+\startsubsection[title={How it works}]

+

+The approach is that we again make the list balanced but try to avoid some side

+effects. What happens is quite intuitive if we forget about spaces (turned into

+glue) but even there what happens makes sense if you look at it in detail.

+However that logic makes in|-|group switching kind of useless when no proper

+nested grouping is used: switching from right to left several times nested,

+results in spacing ending up after each other due to nested mirroring. Of course

+a sane macro package will manage this for the user but here we are discussing the

+low level dir injection.

+

+This is what happens:

+

+\starttyping

+\textdir TRT nur {\textdir TLT run \textdir TRT NUR} nur

+\stoptyping

+

+This becomes stepwise:

+

+\startnarrower

+\starttyping

+injected: [+TRT]nur {[+TLT]run [+TRT]NUR} nur

+balanced: [+TRT]nur {[+TLT]run [-TLT][+TRT]NUR[-TRT]} nur[-TRT]

+result  : run {RUNrun } run

+\stoptyping

+\stopnarrower

+

+And this:

+

+\starttyping

+\textdir TRT nur {nur \textdir TLT run \textdir TRT NUR} nur

+\stoptyping

+

+becomes:

+

+\startnarrower

+\starttyping

+injected: [+TRT]nur {nur [+TLT]run [+TRT]NUR} nur

+balanced: [+TRT]nur {nur [+TLT]run [-TLT][+TRT]NUR[-TRT]} nur[-TRT]

+result  : run {run RUNrun } run

+\stoptyping

+\stopnarrower

+

+Now, in the following examples watch where we put the braces:

+

+\startbuffer

+\textdir TRT nur {{\textdir TLT run} {\textdir TRT NUR}} nur

+\stopbuffer

+

+\typebuffer

+

+This becomes:

+

+\startnarrower

+\getbuffer

+\stopnarrower

+

+Compare this to:

+

+\startbuffer

+\textdir TRT nur {{\textdir TLT run }{\textdir TRT NUR}} nur

+\stopbuffer

+

+\typebuffer

+

+Which renders as:

+

+\startnarrower

+\getbuffer

+\stopnarrower

+

+So how do we deal with the next?

+

+\startbuffer

+\def\ltr{\textdir TLT\relax}

+\def\rtl{\textdir TRT\relax}

+

+run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}

+run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}

+\stopbuffer

+

+\typebuffer

+

+It gets typeset as:

+

+\startnarrower

+\startlines

+\getbuffer

+\stoplines

+\stopnarrower

+

+We could define the two helpers to look back, pick up a skip, remove it and

+inject it after the dir node. But that way we loose the subtype information that

+for some applications can be handy to be kept as|-|is. This is why we now have a

+variant of \lpr {textdir} which injects the balanced node before the skip.

+Instead of the previous definition we can use:

+

+\startbuffer[def]

+\def\ltr{\linedir TLT\relax}

+\def\rtl{\linedir TRT\relax}

+\stopbuffer

+

+\typebuffer[def]

+

+and this time:

+

+\startbuffer[txt]

+run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}

+run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}

+\stopbuffer

+

+\typebuffer[txt]

+

+comes out as a properly spaced:

+

+\startnarrower

+\startlines

+\getbuffer[def,txt]

+\stoplines

+\stopnarrower

+

+Anything more complex that this, like combination of skips and penalties, or

+kerns, should be handled in the input or macro package because there is no way we

+can predict the expected behaviour. In fact, the \lpr {linedir} is just a

+convenience extra which could also have been implemented using node list parsing.

+

+\stopsubsection

+

+\startsubsection[title={Controlling glue with \lpr {breakafterdirmode}}]

+

+Glue after a dir node is ignored in the linebreak decision but you can bypass that

+by setting \lpr {breakafterdirmode} to~\type {1}. The following table shows the

+difference. Watch your spaces.

+

+\def\ShowSome#1{%

+    \BC \type{#1}

+    \NC \breakafterdirmode\zerocount\hsize\zeropoint#1

+    \NC

+    \NC \breakafterdirmode\plusone\hsize\zeropoint#1

+    \NC

+    \NC \NR

+}

+

+\starttabulate[|l|Tp(1pt)|w(5em)|Tp(1pt)|w(5em)|]

+    \DB

+    \BC \type{0}

+    \NC

+    \BC \type{1}

+    \NC

+    \NC \NR

+    \TB

+    \ShowSome{pre {\textdir TLT xxx} post}

+    \ShowSome{pre {\textdir TLT xxx }post}

+    \ShowSome{pre{ \textdir TLT xxx} post}

+    \ShowSome{pre{ \textdir TLT xxx }post}

+    \ShowSome{pre { \textdir TLT xxx } post}

+    \ShowSome{pre {\textdir TLT\relax\space xxx} post}

+    \LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={Controling parshapes with \lpr {shapemode}}]

+

+Another adaptation to the \ALEPH\ directional model is control over shapes driven

+by \prm {hangindent} and \prm {parshape}. This is controlled by a new parameter

+\lpr {shapemode}:

+

+\starttabulate[|c|l|l|]

+\DB value    \BC \prm {hangindent} \BC \prm {parshape} \NC \NR

+\TB

+\BC \type{0} \NC  normal             \NC normal            \NC \NR

+\BC \type{1} \NC  mirrored           \NC normal            \NC \NR

+\BC \type{2} \NC  normal             \NC mirrored          \NC \NR

+\BC \type{3} \NC  mirrored           \NC mirrored          \NC \NR

+\LL

+\stoptabulate

+

+The value is reset to zero (like \prm {hangindent} and \prm {parshape})

+after the paragraph is done with. You can use negative values to prevent

+this. In \in {figure} [fig:shapemode] a few examples are given.

+

+\startplacefigure[reference=fig:shapemode,title={The effect of \type {shapemode}.}]

+    \startcombination[2*3]

+        {\ruledvbox \bgroup \setuptolerance[verytolerant]

+            \hsize .45\textwidth \switchtobodyfont[6pt]

+                \pardir TLT \textdir TLT

+                \hangindent 40pt \hangafter -3

+                \leftskip10pt \input tufte \par

+         \egroup} {TLT: hangindent}

+        {\ruledvbox \bgroup \setuptolerance[verytolerant]

+            \hsize .45\textwidth \switchtobodyfont[6pt]

+            \pardir TLT \textdir TLT

+            \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize

+            \input tufte \par

+         \egroup} {TLT: parshape}

+        {\ruledvbox \bgroup \setuptolerance[verytolerant]

+            \hsize .45\textwidth \switchtobodyfont[6pt]

+            \pardir TRT \textdir TRT

+            \hangindent 40pt \hangafter -3

+            \leftskip10pt \input tufte \par

+         \egroup} {TRT: hangindent mode 0}

+        {\ruledvbox \bgroup \setuptolerance[verytolerant]

+            \hsize .45\textwidth \switchtobodyfont[6pt]

+            \pardir TRT \textdir TRT

+            \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize

+            \input tufte \par

+         \egroup} {TRT: parshape mode 0}

+        {\ruledvbox \bgroup \setuptolerance[verytolerant]

+            \hsize .45\textwidth \switchtobodyfont[6pt]

+            \shapemode=3

+            \pardir TRT \textdir TRT

+            \hangindent 40pt \hangafter -3

+            \leftskip10pt \input tufte \par

+         \egroup} {TRT: hangindent mode 1 & 3}

+        {\ruledvbox \bgroup \setuptolerance[verytolerant]

+            \hsize .45\textwidth \switchtobodyfont[6pt]

+            \shapemode=3

+            \pardir TRT \textdir TRT

+            \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize

+            \input tufte \par

+         \egroup} {TRT: parshape mode 2 & 3}

+    \stopcombination

+\stopplacefigure

+

+\stopsubsection

+

+\startsubsection[title={Symbols or numbers}]

+

+Internally the implementation is different from \ALEPH. First of all we use no

+whatsits but dedicated nodes, but also we have only 4 directions that are mapped

+onto 4 numbers. A text direction node can mark the start or end of a sequence of

+nodes, and therefore has two states. At the \TEX\ end we don't see these states

+because \TEX\ itself will add proper end state nodes if needed.

+

+The symbolic names \type {TLT}, \type {TRT}, etc.\ originate in \OMEGA. In

+\LUATEX\ we also have a number based model which sometimes makes more sense.

+

+\starttabulate[|c|l|l|]

+\DB value     \BC equivalent \NC \NR

+\TB

+\BC \type {0} \NC TLT \NC \NR

+\BC \type {1} \NC TRT \NC \NR

+\BC \type {2} \NC LTL \NC \NR

+\BC \type {3} \NC RTT \NC \NR

+\LL

+\stoptabulate

+

+We support the \OMEGA\ primitives \orm {textdir}, \orm {pardir}, \orm {pagedir},

+\orm {pardir} and \orm {mathdir}. These accept three character keywords. The

+primitives that set the direction by number are: \lpr {textdirection}, \lpr

+{pardirection}, \lpr {pagedirection} and \lpr {bodydirection} and \lpr

+{mathdirection}. When specifying a direction for a box you can use \type {bdir}

+instead of \type {dir}.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title=Implementation notes]

+

+\startsubsection[title=Memory allocation]

+

+\topicindex {memory}

+

+The single internal memory heap that traditional \TEX\ used for tokens and nodes

+is split into two separate arrays. Each of these will grow dynamically when

+needed.

+

+The \type {texmf.cnf} settings related to main memory are no longer used (these

+are: \type {main_memory}, \type {mem_bot}, \type {extra_mem_top} and \type

+{extra_mem_bot}). \quote {Out of main memory} errors can still occur, but the

+limiting factor is now the amount of RAM in your system, not a predefined limit.

+

+Also, the memory (de)allocation routines for nodes are completely rewritten. The

+relevant code now lives in the C file \type {texnode.c}, and basically uses a

+dozen or so \quote {avail} lists instead of a doubly|-|linked model. An extra

+function layer is added so that the code can ask for nodes by type instead of

+directly requisitioning a certain amount of memory words.

+

+Because of the split into two arrays and the resulting differences in the data

+structures, some of the macros have been duplicated. For instance, there are now

+\type {vlink} and \type {vinfo} as well as \type {token_link} and \type

+{token_info}. All access to the variable memory array is now hidden behind a

+macro called \type {vmem}. We mention this because using the \TEX book as

+reference is still quite valid but not for memory related details. Another

+significant detail is that we have double linked node lists and that most nodes

+carry more data.

+

+The input line buffer and pool size are now also reallocated when needed, and the

+\type {texmf.cnf} settings \type {buf_size} and \type {pool_size} are silently

+ignored.

+

+\stopsubsection

+

+\startsubsection[title=Sparse arrays]

+

+The \prm {mathcode}, \prm {delcode}, \prm {catcode}, \prm {sfcode}, \prm {lccode}

+and \prm {uccode} (and the new \lpr {hjcode}) tables are now sparse arrays that

+are implemented in~\CCODE. They are no longer part of the \TEX\ \quote

+{equivalence table} and because each had 1.1 million entries with a few memory

+words each, this makes a major difference in memory usage. Performance is not

+really hurt by this.

+

+The \prm {catcode}, \prm {sfcode}, \prm {lccode}, \prm {uccode} and \lpr {hjcode}

+assignments don't show up when using the \ETEX\ tracing routines \prm

+{tracingassigns} and \prm {tracingrestores} but we don't see that as a real

+limitation.

+

+A side|-|effect of the current implementation is that \prm {global} is now more

+expensive in terms of processing than non|-|global assignments but not many users

+will notice that.

+

+The glyph ids within a font are also managed by means of a sparse array as glyph

+ids can go up to index $2^{21}-1$ but these are never accessed directly so again

+users will not notice this.

+

+\stopsubsection

+

+\startsubsection[title=Simple single|-|character csnames]

+

+\topicindex {csnames}

+

+Single|-|character commands are no longer treated specially in the internals,

+they are stored in the hash just like the multiletter csnames.

+

+The code that displays control sequences explicitly checks if the length is one

+when it has to decide whether or not to add a trailing space.

+

+Active characters are internally implemented as a special type of multi|-|letter

+control sequences that uses a prefix that is otherwise impossible to obtain.

+

+\stopsubsection

+

+\startsubsection[title=The compressed format file]

+

+\topicindex {format}

+

+The format is passed through \type {zlib}, allowing it to shrink to roughly half

+of the size it would have had in uncompressed form. This takes a bit more \CPU\

+cycles but much less disk \IO, so it should still be faster. We use a level~3

+compression which we found to be the optimal trade|-|off between filesize and

+decompression speed.

+

+\stopsubsection

+

+\startsubsection[title=Binary file reading]

+

+\topicindex {files+binary}

+

+All of the internal code is changed in such a way that if one of the \type

+{read_xxx_file} callbacks is not set, then the file is read by a \CCODE\ function

+using basically the same convention as the callback: a single read into a buffer

+big enough to hold the entire file contents. While this uses more memory than the

+previous code (that mostly used \type {getc} calls), it can be quite a bit faster

+(depending on your \IO\ subsystem).

+

+\stopsubsection

+

+\startsubsection[title=Tabs and spaces]

+

+\topicindex {space}

+\topicindex {newline}

+

+We conform to the way other \TEX\ engines handle trailing tabs and spaces. For

+decades trailing tabs and spaces (before a newline) were removed from the input

+but this behaviour was changed in September 2017 to only handle spaces. We are

+aware that this can introduce compatibility issues in existing workflows but

+because we don't want too many differences with upstream \TEXLIVE\ we just follow

+up on that patch (which is a functional one and not really a fix). It is up to

+macro packages maintainers to deal with possible compatibility issues and in

+\LUATEX\ they can do so via the callbacks that deal with reading from files.

+

+The previous behaviour was a known side effect and (as that kind of input

+normally comes from generated sources) it was normally dealt with by adding a

+comment token to the line in case the spaces and|/|or tabs were intentional and

+to be kept. We are aware of the fact that this contradicts some of our other

+choices but consistency with other engines and the fact that in \KPSE\ mode a

+common file \IO\ layer is used can have a side effect of breaking compatibility.

+We still stick to our view that at the log level we can (and might be) more

+incompatible. We already expose some more details.

+

+\stopsubsection

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-nodes.tex b/Master/texmf-dist/doc/luatex/base/luatex-nodes.tex
index 16488dbbb85..573bfd8d80d 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-nodes.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-nodes.tex
@@ -1,2060 +1,2664 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-nodes
-
-\startchapter[reference=nodes,title={Nodes}]
-
-\section{\LUA\ node representation}
-
-\TEX's nodes are represented in \LUA\ as userdata object with a variable set of
-fields. In the following syntax tables, such the type of such a userdata object
-is represented as \syntax {<node>}.
-
-The current return value of \type {node.types()} is:
-\startluacode
-    for id, name in table.sortedhash(node.types()) do
-        context.type(name)
-        context(" (%s), ",id)
-    end
-    context.removeunwantedspaces()
-    context.removepunctuation()
-\stopluacode
-. % period
-
-The \type {\lastnodetype} primitive is \ETEX\ compliant. The valid range is still
-$[-1,15]$ and glyph nodes (formerly known as char nodes) have number~0 while
-ligature nodes are mapped to~7. That way macro packages can use the same symbolic
-names as in traditional \ETEX. Keep in mind that these \ETEX\ node numbers are
-different from the real internal ones and that there are more \ETEX\ node types
-than~15.
-
-You can ask for a list of fields with the \type {node.fields} (which takes an id)
-and for valid subtypes with \type {node.subtypes} (which takes a string because
-eventually we might support more used enumerations).
-
-\subsection{Attributes}
-
-The newly introduced attribute registers are non|-|trivial, because the value
-that is attached to a node is essentially a sparse array of key|-|value pairs. It
-is generally easiest to deal with attribute lists and attributes by using the
-dedicated functions in the \type {node} library, but for completeness, here is
-the low|-|level interface.
-
-\subsubsection{attribute_list nodes}
-
-An \type {attribute_list} item is used as a head pointer for a list of attribute
-items. It has only one user-visible field:
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type \BC explanation \NC \NR
-\NC \type{next} \NC node \NC pointer to the first attribute \NC \NR
-\stoptabulate
-
-\subsubsection{attribute nodes}
-
-A normal node's attribute field will point to an item of type \type
-{attribute_list}, and the \type {next} field in that item will point to the first
-defined \quote {attribute} item, whose \type {next} will point to the second
-\quote {attribute} item, etc.
-
-\starttabulate[|l|l|p|]
-\BC field         \BC type   \BC explanation \NC \NR
-\NC \type{next}   \NC node   \NC pointer to the next attribute \NC \NR
-\NC \type{number} \NC number \NC the attribute type id \NC \NR
-\NC \type{value}  \NC number \NC the attribute value \NC \NR
-\stoptabulate
-
-As mentioned it's better to use the official helpers rather than edit these
-fields directly. For instance the \type {prev} field is used for other purposes
-and there is no double linked list.
-
-\subsection{Main text nodes}
-
-These are the nodes that comprise actual typesetting commands. A few fields are
-present in all nodes regardless of their type, these are:
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{next}    \NC node   \NC the next node in a list, or nil \NC \NR
-\NC \type{id}      \NC number \NC the node's type (\type {id}) number \NC \NR
-\NC \type{subtype} \NC number \NC the node \type {subtype} identifier \NC \NR
-\stoptabulate
-
-The \type {subtype} is sometimes just a stub entry. Not all nodes actually use
-the \type {subtype}, but this way you can be sure that all nodes accept it as a
-valid field name, and that is often handy in node list traversal. In the
-following tables \type {next} and \type {id} are not explicitly mentioned.
-
-Besides these three fields, almost all nodes also have an \type {attr} field, and
-there is a also a field called \type {prev}. That last field is always present,
-but only initialized on explicit request: when the function \type {node.slide()}
-is called, it will set up the \type {prev} fields to be a backwards pointer in
-the argument node list. By now most of \TEX's node processing makes sure that the
-\type {prev} nodes are valid but there can be exceptions, especially when the
-internal magic uses a leading \type {temp} nodes to temporarily store a state.
-
-\subsubsection{hlist nodes}
-
-\starttabulate[|l|l|p|]
-\BC field             \BC type   \BC explanation \NC \NR
-\NC \type{subtype}    \NC number \NC \showsubtypes{list} \NC \NR
-\NC \type{attr}       \NC node   \NC list of attributes \NC \NR
-\NC \type{width}      \NC number \NC the width of the box \NC \NR
-\NC \type{height}     \NC number \NC the height of the box \NC \NR
-\NC \type{depth}      \NC number \NC the depth of the box \NC \NR
-\NC \type{shift}      \NC number \NC a displacement perpendicular to the character progression direction \NC \NR
-\NC \type{glue_order} \NC number \NC a number in the range $[0,4]$, indicating the glue order \NC \NR
-\NC \type{glue_set}   \NC number \NC the calculated glue ratio \NC \NR
-\NC \type{glue_sign}  \NC number \NC 0 = \type {normal}, 1 = \type {stretching}, 2 = \type {shrinking} \NC \NR
-\NC \type{head/list}  \NC node   \NC the first node of the body of this list \NC \NR
-\NC \type{dir}        \NC string \NC the direction of this box, see~\in[dirnodes] \NC \NR
-\stoptabulate
-
-A warning: never assign a node list to the \type {head} field unless you are sure
-its internal link structure is correct, otherwise an error may result.
-
-Note: the field name \type {head} and \type {list} are both valid. Sometimes it
-makes more sense to refer to a list by \type {head}, sometimes \type {list} makes
-more sense.
-
-\subsubsection{vlist nodes}
-
-This node is similar to \type {hlist}, except that \quote {shift} is a displacement
-perpendicular to the line progression direction, and \quote {subtype} only has
-the values 0, 4, and~5.
-
-\subsubsection{rule nodes}
-
-Contrary to traditional \TEX, \LUATEX\ has more subtypes because we also use
-rules to store reuseable objects and images. User nodes are invisible and can be
-intercepted by a callback.
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{subtype} \NC number \NC \showsubtypes{rule} \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{width}   \NC number \NC the width of the rule where the special value $-1073741824$ is used for \quote {running} glue dimensions \NC \NR
-\NC \type{height}  \NC number \NC the height of the rule (can be negative) \NC \NR
-\NC \type{depth}   \NC number \NC the depth of the rule (can be negative) \NC \NR
-\NC \type{dir}     \NC string \NC the direction of this rule, see~\in[dirnodes] \NC \NR
-\NC \type{index}   \NC number \NC an optional index that can be referred to \NC \NR
-\stoptabulate
-
-\subsubsection{ins nodes}
-
-\starttabulate[|l|l|p|]
-\BC field            \BC type   \BC explanation \NC \NR
-\NC \type{subtype}   \NC number \NC the insertion class \NC \NR
-\NC \type{attr}      \NC node   \NC list of attributes \NC \NR
-\NC \type{cost}      \NC number \NC the penalty associated with this insert \NC \NR
-\NC \type{height}    \NC number \NC height of the insert \NC \NR
-\NC \type{depth}     \NC number \NC depth of the insert \NC \NR
-\NC \type{head/list} \NC node   \NC the first node of the body of this insert \NC \NR
-\stoptabulate
-
-There is a set of extra fields that concern the associated glue: \type {width},
-\type {stretch}, \type {stretch_order}, \type {shrink} and \type {shrink_order}.
-These are all numbers.
-
-A warning: never assign a node list to the \type {head} field unless you are sure
-its internal link structure is correct, otherwise an error may be result. You can use
-\type {list} instead (often in functions you want to use local variable swith similar
-names and both names are equally sensible).
-
-\subsubsection{mark nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{subtype} \NC number \NC unused \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{class}   \NC number \NC the mark class \NC \NR
-\NC \type{mark}    \NC table  \NC a table representing a token list \NC \NR
-\stoptabulate
-
-\subsubsection{adjust nodes}
-
-\starttabulate[|l|l|p|]
-\BC field            \BC type   \BC explanation \NC \NR
-\NC \type{subtype}   \NC number \NC \showsubtypes{adjust} \NC \NR
-\NC \type{attr}      \NC node   \NC list of attributes \NC \NR
-\NC \type{head/list} \NC node   \NC adjusted material \NC \NR
-\stoptabulate
-
-A warning: never assign a node list to the \type {head} field unless you are sure
-its internal link structure is correct, otherwise an error may be result.
-
-\subsubsection{disc nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{subtype} \NC number \NC \showsubtypes{disc} \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{pre}     \NC node   \NC pointer to the pre|-|break text \NC \NR
-\NC \type{post}    \NC node   \NC pointer to the post|-|break text \NC \NR
-\NC \type{replace} \NC node   \NC pointer to the no|-|break text \NC \NR
-\NC \type{penalty} \NC number \NC the penalty associated with the break, normally \type {\hyphenpenalty} or \type {\exhyphenpenalty} \NC \NR
-\stoptabulate
-
-The subtype numbers~4 and~5 belong to the \quote {of-f-ice} explanation given
-elsewhere.
-
-These disc nodes are kind of special as at some point they also keep information
-about breakpoints and nested ligatures. The \type {pre}, \type {post} and \type
-{replace} fields at the \LUA\ end are in fact indirectly accessed and have a
-\type {prev} pointer that is not \type {nil}. This means that when you mess
-around with the head of these (three) lists, you also need to reassign them
-because that will restore the proper \type {prev} pointer, so:
-
-\starttyping
-pre = d.pre
--- change the list starting with pre
-d.pre = pre
-\stoptyping
-
-Otherwise you can end up with an invalid internal perception of reality and
-\LUATEX\ might even decide to crash on you. It also means that running forward
-over for instance \type {pre} is ok but backward you need to stop at \type {pre}.
-And you definitely must not mess with the node that \type {prev} points to, if
-only because it is not really an node but part of the disc data structure (so
-freeing it again might crash \LUATEX).
-
-\subsubsection{math nodes}
-
-\starttabulate[|l|l|p|]
-\BC field           \BC type   \BC explanation \NC \NR
-\NC \type{subtype}  \NC number \NC \showsubtypes{math} \NC \NR
-\NC \type{attr}     \NC node   \NC list of attributes \NC \NR
-\NC \type{surround} \NC number \NC width of the \type {\mathsurround} kern \NC \NR
-\stoptabulate
-
-There is a set of extra fields that concern the associated glue: \type {width},
-\type {stretch}, \type {stretch_order}, \type {shrink} and \type {shrink_order}.
-These are all numbers.
-
-\subsubsection{glue nodes}
-
-Skips are about the only type of data objects in traditional \TEX\ that are not a
-simple value. The structure that represents the glue components of a skip is
-called a \type {glue_spec}, and it has the following accessible fields:
-
-\starttabulate[|l|l|p|]
-\BC key                  \BC type   \BC explanation \NC \NR
-\NC \type{width}         \NC number \NC the horizontal or vertical displacement \NC \NR
-\NC \type{stretch}       \NC number \NC extra (positive) displacement or stretch amount \NC \NR
-\NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR
-\NC \type{shrink}        \NC number \NC extra (negative) displacement or shrink amount\NC \NR
-\NC \type{shrink_order}  \NC number \NC factor applied to shrink amount \NC \NR
-\stoptabulate
-
-The effective width of some glue subtypes depends on the stretch or shrink needed
-to make the encapsulating box fit its dimensions. For instance, in a paragraph
-lines normally have glue representing spaces and these stretch of shrink to make
-the content fit in the available space. The \type {effective_glue} function that
-takes a glue node and a parent (hlist or vlist) returns the effective width of
-that glue item.
-
-A gluespec node is a special kind of node that is used for storing a set of glue
-values in registers. Originally they were also used to store properties of glue
-nodes (using a system of reference counts) but we now keep these properties in
-the glue nodes themselves, which gives a cleaner interface to \LUA.
-
-The indirect spec approach was in fact an optimization in the original \TEX\
-code. First of all it can save quite some memory because all these spaces that
-become glue now share the same specification (only the reference count is
-incremented), and zero testing is also a bit faster because only the pointer has
-to be checked (this is no longer true for engines that implement for instance
-protrusion where we really need to ensure that zero is zero when we test for
-bounds). Another side effect is that glue specifications are read|-|only, so in
-the end copies need to be made when they are used from \LUA\ (each assignment to
-a field can result in a new copy). So in the end the advantages of sharing are
-not that high (and nowadays memory is less an issue, also given that a glue node
-is only a few memory words larger than a spec).
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{subtype} \NC number \NC \showsubtypes{glue} \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{leader}  \NC node   \NC pointer to a box or rule for leaders \NC \NR
-\stoptabulate
-
-In addition there are the \type {width}, \type {stretch} \type {stretch_order},
-\type {shrink}, and \type {shrink_order} fields. Note that we use the key \type
-{width} in both horizontal and vertical glue. This suits the \TEX\ internals well
-so we decided to stick to that naming.
-
-A regular word space also results in a \type {spaceskip} subtype (this used to be
-a \type {userskip} with subtype zero).
-
-\subsubsection{kern nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{subtype} \NC number \NC \showsubtypes{kern} \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{kern}    \NC number \NC fixed horizontal or vertical advance \NC \NR
-\stoptabulate
-
-\subsubsection{penalty nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{subtype} \NC number \NC \showsubtypes{penalty} \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{penalty} \NC number \NC the penalty value \NC \NR
-\stoptabulate
-
-The subtypes are just informative and \TEX\ itself doesn't use them. When you
-run into an \type {linebreakpenalty} you need to keep in mind that it's a
-accumulation of \type {club}, \type{widow} and other relevant penalties.
-
-\subsubsection[glyphnodes]{glyph nodes}
-
-\starttabulate[|l|l|p|]
-\BC field                   \BC type    \BC explanation \NC \NR
-\NC \type{subtype}          \NC number  \NC bitfield \NC \NR
-\NC \type{attr}             \NC node    \NC list of attributes \NC \NR
-\NC \type{char}             \NC number  \NC the chatacter index in the font \NC \NR
-\NC \type{font}             \NC number  \NC the font identifier \NC \NR
-\NC \type{lang}             \NC number  \NC the language identifier \NC \NR
-\NC \type{left}             \NC number  \NC the frozen \type {\lefthyphenmnin} value \NC \NR
-\NC \type{right}            \NC number  \NC the frozen \type {\righthyphenmnin} value \NC \NR
-\NC \type{uchyph}           \NC boolean \NC the frozen \type {\uchyph} value \NC \NR
-\NC \type{components}       \NC node    \NC pointer to ligature components \NC \NR
-\NC \type{xoffset}          \NC number  \NC a virtual displacement in horizontal direction \NC \NR
-\NC \type{yoffset}          \NC number  \NC a virtual displacement in vertical direction \NC \NR
-%NC \type{xadvance}         \NC number  \NC an additional advance after the glyph (experimental) \NC \NR
-\NC \type{width}            \NC number  \NC the (original) width of the character \NC \NR
-\NC \type{height}           \NC number  \NC the (original) height of the character\NC \NR
-\NC \type{depth}            \NC number  \NC the (original) depth of the character\NC \NR
-\NC \type{expansion_factor} \NC number  \NC the to be applied expansion_factor \NC \NR
-\stoptabulate
-
-The \type {width}, \type {height} and \type {depth} values are read|-|only. The
-\type {expansion_factor} is assigned in the parbuilder and used in the backend.
-
-A warning: never assign a node list to the components field unless you are sure
-its internal link structure is correct, otherwise an error may be result. Valid
-bits for the \type {subtype} field are:
-
-\starttabulate[|c|l|]
-\BC bit \BC meaning   \NC \NR
-\NC 0   \NC character \NC \NR
-\NC 1   \NC ligature  \NC \NR
-\NC 2   \NC ghost     \NC \NR
-\NC 3   \NC left      \NC \NR
-\NC 4   \NC right     \NC \NR
-\stoptabulate
-
-See \in {section} [charsandglyphs] for a detailed description of the \type
-{subtype} field.
-
-The \type {expansion_factor} has been introduced as part of the separation
-between font- and backend. It is the result of extensive experiments with a more
-efficient implementation of expansion. Early versions of \LUATEX\ already
-replaced multiple instances of fonts in the backend by scaling but contrary to
-\PDFTEX\ in \LUATEX\ we now also got rid of font copies in the frontend and
-replaced them by expansion factors that travel with glyph nodes. Apart from a
-cleaner approach this is also a step towards a better separation between front-
-and backend.
-
-The \type {is_char} function checks if a node is a glyph node with a subtype still
-less than 256. This function can be used to determine if applying font logic to a
-glyph node makes sense. The value \type {nil} gets returned when the node is not
-a glyph, a character number is returned if the node is still tagged as character
-and \type {false} gets returned otherwise. When nil is returned, the id is also
-returned. The \type {is_glyph} variant doesn't check for a subtype being less
-than 256, so it returns either the character value or nil plus the id. These
-helpers are not always faster than separate calls but they sometimes permit
-making more readable tests. The \type {uses_font} helpers takes a node
-and font id and returns true when a glyph or disc node references that font.
-
-\subsubsection{boundary nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{subtype} \NC number \NC \showsubtypes{boundary} \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{value}   \NC number \NC values 0--255 are reserved \NC \NR
-\stoptabulate
-
-This node relates to the \type {\noboundary}, \type {\boundary}, \type
-{\protrusionboundary} and \type {\wordboundary} primitives.
-
-\subsubsection{local_par nodes}
-
-\starttabulate[|l|l|p|]
-\BC field                  \BC type   \BC explanation \NC \NR
-\NC \type{attr}            \NC node   \NC list of attributes \NC \NR
-\NC \type{pen_inter}       \NC number \NC local interline penalty (from \type {\localinterlinepenalty}) \NC \NR
-\NC \type{pen_broken}      \NC number \NC local broken penalty (from \type {\localbrokenpenalty}) \NC \NR
-\NC \type{dir}             \NC string \NC the direction of this par. see~\in [dirnodes] \NC \NR
-\NC \type{box_left}        \NC node   \NC the \type {\localleftbox} \NC \NR
-\NC \type{box_left_width}  \NC number \NC width of the \type {\localleftbox} \NC \NR
-\NC \type{box_right}       \NC node   \NC the \type {\localrightbox} \NC \NR
-\NC \type{box_right_width} \NC number \NC width of the \type {\localrightbox} \NC \NR
-\stoptabulate
-
-A warning: never assign a node list to the \type {box_left} or \type {box_right}
-field unless you are sure its internal link structure is correct, otherwise an
-error may be result.
-
-\subsubsection[dirnodes]{dir nodes}
-
-\starttabulate[|l|l|p|]
-\BC field        \BC type   \BC explanation \NC \NR
-\NC \type{attr}  \NC node   \NC list of attributes \NC \NR
-\NC \type{dir}   \NC string \NC the direction (but see below) \NC \NR
-\NC \type{level} \NC number \NC nesting level of this direction whatsit \NC \NR
-\stoptabulate
-
-A note on \type {dir} strings. Direction specifiers are three|-|letter
-combinations of \type {T}, \type {B}, \type {R}, and \type {L}.
-
-These are built up out of three separate items:
-
-\startitemize[packed]
-\startitem
-    the first  is the direction of the \quote{top}   of paragraphs.
-\stopitem
-\startitem
-    the second is the direction of the \quote{start} of lines.
-\stopitem
-\startitem
-    the third  is the direction of the \quote{top}   of glyphs.
-\stopitem
-\stopitemize
-
-However, only four combinations are accepted: \type {TLT}, \type {TRT}, \type
-{RTT}, and \type {LTL}.
-
-Inside actual \type {dir} whatsit nodes, the representation of \type {dir} is not
-a three-letter but a four|-|letter combination. The first character in this case
-is always either \type {+} or \type {-}, indicating whether the value is pushed
-or popped from the direction stack.
-
-\subsubsection{margin_kern nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{subtype} \NC number \NC \showsubtypes{margin_kern} \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{width}   \NC number \NC the advance of the kern \NC \NR
-\NC \type{glyph}   \NC node   \NC the glyph to be used \NC \NR
-\stoptabulate
-
-\subsection{Math nodes}
-
-These are the so||called \quote {noad}s and the nodes that are specifically
-associated with math processing. Most of these nodes contain subnodes so that the
-list of possible fields is actually quite small. First, the subnodes:
-
-\subsubsection{Math kernel subnodes}
-
-Many object fields in math mode are either simple characters in a specific family
-or math lists or node lists. There are four associated subnodes that represent
-these cases (in the following node descriptions these are indicated by the word
-\type {<kernel>}).
-
-The \type {next} and \type {prev} fields for these subnodes are unused.
-
-\subsubsubsection{math_char and math_text_char subnodes}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type   \BC explanation \NC \NR
-\NC \type{attr} \NC node   \NC list of attributes \NC \NR
-\NC \type{char} \NC number \NC the character index \NC \NR
-\NC \type{fam}  \NC number \NC the family number \NC \NR
-\stoptabulate
-
-The \type {math_char} is the simplest subnode field, it contains the character
-and family for a single glyph object. The \type {math_text_char} is a special
-case that you will not normally encounter, it arises temporarily during math list
-conversion (its sole function is to suppress a following italic correction).
-
-\subsubsubsection{sub_box and sub_mlist subnodes}
-
-\starttabulate[|l|l|p|]
-\BC field            \BC type \BC explanation \NC \NR
-\NC \type{attr}      \NC node \NC list of attributes \NC \NR
-\NC \type{head/list} \NC node \NC list of nodes \NC \NR
-\stoptabulate
-
-These two subnode types are used for subsidiary list items. For \type {sub_box},
-the \type {head} points to a \quote {normal} vbox or hbox. For \type {sub_mlist},
-the \type {head} points to a math list that is yet to be converted.
-
-A warning: never assign a node list to the \type {head} field unless you are sure
-its internal link structure is correct, otherwise an error may be result.
-
-\subsubsection{Math delimiter subnode}
-
-There is a fifth subnode type that is used exclusively for delimiter fields. As
-before, the \type {next} and \type {prev} fields are unused.
-
-\subsubsubsection{delim subnodes}
-
-\starttabulate[|l|l|p|]
-\BC field             \BC type   \BC explanation \NC \NR
-\NC \type{attr}       \NC node   \NC list of attributes \NC \NR
-\NC \type{small_char} \NC number \NC character index of base character \NC \NR
-\NC \type{small_fam}  \NC number \NC family number of base character \NC \NR
-\NC \type{large_char} \NC number \NC character index of next larger character \NC \NR
-\NC \type{large_fam}  \NC number \NC family number of next larger character \NC \NR
-\stoptabulate
-
-The fields \type {large_char} and \type {large_fam} can be zero, in that case the
-font that is sed for the \type {small_fam} is expected to provide the large
-version as an extension to the \type {small_char}.
-
-\subsubsection{Math core nodes}
-
-First, there are the objects (the \TEX book calls then \quote {atoms}) that are
-associated with the simple math objects: ord, op, bin, rel, open, close, punct,
-inner, over, under, vcent. These all have the same fields, and they are combined
-into a single node type with separate subtypes for differentiation.
-
-Some noads have an option field. The values in this bitset are common:
-
-\starttabulate[|l|r|]
-\NC set             \NC               \type{0x08} \NC \NR
-\NC internal        \NC \type{0x00} + \type{0x08} \NC \NR
-\NC internal        \NC \type{0x01} + \type{0x08} \NC \NR
-\NC axis            \NC \type{0x02} + \type{0x08} \NC \NR
-\NC no axis         \NC \type{0x04} + \type{0x08} \NC \NR
-\NC exact           \NC \type{0x10} + \type{0x08} \NC \NR
-\NC left            \NC \type{0x11} + \type{0x08} \NC \NR
-\NC middle          \NC \type{0x12} + \type{0x08} \NC \NR
-\NC right           \NC \type{0x14} + \type{0x08} \NC \NR
-\NC no sub script   \NC \type{0x21} + \type{0x08} \NC \NR
-\NC no super script \NC \type{0x22} + \type{0x08} \NC \NR
-\NC no script       \NC \type{0x23} + \type{0x08} \NC \NR
-\stoptabulate
-
-\subsubsubsection{simple nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type        \BC explanation \NC \NR
-\NC \type{subtype} \NC number      \NC \showsubtypes{noad} \NC \NR
-\NC \type{attr}    \NC node        \NC list of attributes \NC \NR
-\NC \type{nucleus} \NC kernel node \NC base \NC \NR
-\NC \type{sub}     \NC kernel node \NC subscript \NC \NR
-\NC \type{sup}     \NC kernel node \NC superscript \NC \NR
-\NC \type{options} \NC number      \NC bitset of rendering options \NC \NR
-\stoptabulate
-
-\subsubsubsection{accent nodes}
-
-\starttabulate[|l|l|p|]
-\BC field             \BC type        \BC explanation \NC \NR
-\NC \type{subtype}    \NC number      \NC \showsubtypes{accent} \NC \NR
-\NC \type{nucleus}    \NC kernel node \NC base \NC \NR
-\NC \type{sub}        \NC kernel node \NC subscript \NC \NR
-\NC \type{sup}        \NC kernel node \NC superscript \NC \NR
-\NC \type{accent}     \NC kernel node \NC top accent \NC \NR
-\NC \type{bot_accent} \NC kernel node \NC bottom accent \NC \NR
-\NC \type{fraction}   \NC number      \NC larger step criterium (divided by 1000) \NC \NR
-\stoptabulate
-
-\subsubsubsection{style nodes}
-
-\starttabulate[|l|l|p|]
-\BC field        \BC type   \BC explanation    \NC \NR
-\NC \type{style} \NC string \NC contains the style \NC \NR
-\stoptabulate
-
-There are eight possibilities for the string value: one of \type {display},
-\type {text}, \type {script}, or \type {scriptscript}. Each of these can have
-be prefixed by \type {cramped}.
-
-\subsubsubsection{choice nodes}
-
-\starttabulate[|l|l|p|]
-\BC field               \BC type \BC explanation \NC \NR
-\NC \type{attr}         \NC node \NC list of attributes \NC \NR
-\NC \type{display}      \NC node \NC list of display size alternatives \NC \NR
-\NC \type{text}         \NC node \NC list of text size alternatives \NC \NR
-\NC \type{script}       \NC node \NC list of scriptsize alternatives \NC \NR
-\NC \type{scriptscript} \NC node \NC list of scriptscriptsize alternatives \NC \NR
-\stoptabulate
-
-Warning: never assign a node list to the \type {display}, \type {text}, \type
-{script}, or \type {scriptscript} field unless you are sure its internal link
-structure is correct, otherwise an error may be result.
-
-\subsubsubsection{radical nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type           \BC explanation \NC \NR
-\NC \type{subtype} \NC number         \NC \showsubtypes{radical} \NC \NR
-\NC \type{attr}    \NC node           \NC list of attributes \NC \NR
-\NC \type{nucleus} \NC kernel node    \NC base \NC \NR
-\NC \type{sub}     \NC kernel node    \NC subscript \NC \NR
-\NC \type{sup}     \NC kernel node    \NC superscript \NC \NR
-\NC \type{left}    \NC delimiter node \NC \NC \NR
-\NC \type{degree}  \NC kernel node    \NC only set by \type {\Uroot} \NC \NR
-\NC \type{width}   \NC number         \NC required width \NC \NR
-\NC \type{options} \NC number         \NC bitset of rendering options \NC \NR
-\stoptabulate
-
-Warning: never assign a node list to the \type {nucleus}, \type {sub}, \type
-{sup}, \type {left}, or \type {degree} field unless you are sure its internal
-link structure is correct, otherwise an error may be result.
-
-\subsubsubsection{fraction nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type           \BC explanation \NC \NR
-\NC \type{attr}    \NC node           \NC list of attributes \NC \NR
-\NC \type{width}   \NC number         \NC (optional) width of the fraction \NC \NR
-\NC \type{num}     \NC kernel node    \NC numerator \NC \NR
-\NC \type{denom}   \NC kernel node    \NC denominator \NC \NR
-\NC \type{left}    \NC delimiter node \NC left side symbol \NC \NR
-\NC \type{right}   \NC delimiter node \NC right side symbol \NC \NR
-\NC \type{middle}  \NC delimiter node \NC middle symbol \NC \NR
-\NC \type{options} \NC number         \NC bitset of rendering options \NC \NR
-\stoptabulate
-
-Warning: never assign a node list to the \type {num}, or \type {denom} field
-unless you are sure its internal link structure is correct, otherwise an error
-may be result.
-
-\subsubsubsection{fence nodes}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type           \BC explanation \NC \NR
-\NC \type{subtype} \NC number         \NC \showsubtypes{fence} \NC \NR
-\NC \type{attr}    \NC node           \NC list of attributes \NC \NR
-\NC \type{delim}   \NC delimiter node \NC delimiter specification \NC \NR
-\NC \type{italic}  \NC number         \NC italic correction \NC \NR
-\NC \type{height}  \NC number         \NC required height \NC \NR
-\NC \type{depth}   \NC number         \NC required depth \NC \NR
-\NC \type{options} \NC number         \NC bitset of rendering options \NC \NR
-\NC \type{class}   \NC number         \NC spacing related class \NC \NR
-\stoptabulate
-
-Warning: some of these fields are used by the renderer and might get adapted in
-the process.
-
-\subsection{whatsit nodes}
-
-Whatsit nodes come in many subtypes that you can ask for by running
-\type {node.whatsits()}:
-\startluacode
-    for id, name in table.sortedpairs(node.whatsits()) do
-        context.type(name)
-        context(" (%s), ",id)
-    end
-    context.removeunwantedspaces()
-    context.removepunctuation()
-\stopluacode
-. % period
-
-\subsubsection{front|-|end whatsits}
-
-\subsubsubsection{open whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field         \BC type   \BC explanation \NC \NR
-\NC \type{attr}   \NC node   \NC list of attributes \NC \NR
-\NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR
-\NC \type{name}   \NC string \NC file name \NC \NR
-\NC \type{ext}    \NC string \NC file extension \NC \NR
-\NC \type{area}   \NC string \NC file area (this may become obsolete) \NC \NR
-\stoptabulate
-
-\subsubsubsection{write whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field         \BC type   \BC explanation \NC \NR
-\NC \type{attr}   \NC node   \NC list of attributes \NC \NR
-\NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR
-\NC \type{data}   \NC table  \NC a table representing the token list to be written \NC \NR
-\stoptabulate
-
-\subsubsubsection{close whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field         \BC type   \BC explanation \NC \NR
-\NC \type{attr}   \NC node   \NC list of attributes \NC \NR
-\NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR
-\stoptabulate
-
-\subsubsubsection{user_defined whatsits}
-
-User|-|defined whatsit nodes can only be created and handled from \LUA\ code. In
-effect, they are an extension to the extension mechanism. The \LUATEX\ engine
-will simply step over such whatsits without ever looking at the contents.
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{user_id} \NC number \NC id number \NC \NR
-\NC \type{type}    \NC number \NC type of the value \NC \NR
-\NC \type{value}   \NC number \NC a \LUA\ number \NC \NR
-\NC                \NC node   \NC a node list \NC \NR
-\NC                \NC string \NC a \LUA\ string \NC \NR
-\NC                \NC table  \NC a \LUA\ table \NC \NR
-\stoptabulate
-
-The \type {type} can have one of six distinct values. The number is the \ASCII\
-value if the first character if the type name (so you can use string.byte("l")
-instead of \type {108}).
-
-\starttabulate[|r|c|p|]
-\BC value \BC meaning \BC explanation \NC \NR
-\NC   97  \NC a       \NC list of attributes (a node list) \NC \NR
-\NC  100  \NC d       \NC a \LUA\ number \NC \NR
-\NC  108  \NC l       \NC a \LUA\ value (table, number, boolean, etc) \NC \NR
-\NC  110  \NC n       \NC a node list \NC \NR
-\NC  115  \NC s       \NC a \LUA\ string \NC \NR
-\NC  116  \NC t       \NC a \LUA\ token list in \LUA\ table form (a list of triplets) \NC \NR
-\stoptabulate
-
-\subsubsubsection{save_pos whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type \BC explanation \NC \NR
-\NC \type{attr} \NC node \NC list of attributes \NC \NR
-\stoptabulate
-
-\subsubsubsection{late_lua whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field         \BC type     \BC explanation \NC \NR
-\NC \type{attr}   \NC node     \NC list of attributes \NC \NR
-\NC \type{data}   \NC string   \NC data to execute \NC \NR
-\NC \type{string} \NC string   \NC data to execute \NC \NR
-\NC \type{name}   \NC string   \NC the name to use for \LUA\ error reporting \NC \NR
-\stoptabulate
-
-The difference between \type {data} and \type {string} is that on assignment, the
-\type {data} field is converted to a token list, cf. use as \type {\latelua}. The
-\type {string} version is treated as a literal string.
-
-\subsubsection{\DVI\ backend whatsits}
-
-\subsubsection{special whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type   \BC explanation \NC \NR
-\NC \type{attr} \NC node   \NC list of attributes \NC \NR
-\NC \type{data} \NC string \NC the \type {\special} information \NC \NR
-\stoptabulate
-
-\subsubsection{\PDF\ backend whatsits}
-
-\subsubsubsection{pdf_literal whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type   \BC explanation \NC \NR
-\NC \type{attr} \NC node   \NC list of attributes \NC \NR
-\NC \type{mode} \NC number \NC the \quote {mode} setting of this literal \NC \NR
-\NC \type{data} \NC string \NC the \type {\pdfliteral} information \NC \NR
-\stoptabulate
-
-Possible mode values are:
-
-\starttabulate[|l|p|]
-\BC value \BC \PDFTEX\ keyword \NC \NR
-\NC 0     \NC \type{setorigin} \NC \NR
-\NC 1     \NC \type{page}      \NC \NR
-\NC 2     \NC \type{direct}    \NC \NR
-\NC 3     \NC \type{raw}       \NC \NR
-\stoptabulate
-
-The higher the number, the less checking and the more you can run into troubles.
-Especially the \type {raw} variant can produce bad \PDF\ so you can best check
-what you generate.
-
-\subsubsubsection{pdf_refobj whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field         \BC type   \BC explanation \NC \NR
-\NC \type{attr}   \NC node   \NC list of attributes \NC \NR
-\NC \type{objnum} \NC number \NC the referenced \PDF\ object number \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_annot whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field         \BC type   \BC explanation \NC \NR
-\NC \type{attr}   \NC node   \NC list of attributes \NC \NR
-\NC \type{width}  \NC number \NC the width (not used in calculations) \NC \NR
-\NC \type{height} \NC number \NC the height (not used in calculations) \NC \NR
-\NC \type{depth}  \NC number \NC the depth (not used in calculations) \NC \NR
-\NC \type{objnum} \NC number \NC the referenced \PDF\ object number \NC \NR
-\NC \type{data}   \NC string \NC the annotation data \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_start_link whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field            \BC type   \BC explanation \NC \NR
-\NC \type{attr}      \NC node   \NC list of attributes \NC \NR
-\NC \type{width}     \NC number \NC the width (not used in calculations) \NC \NR
-\NC \type{height}    \NC number \NC the height (not used in calculations) \NC \NR
-\NC \type{depth}     \NC number \NC the depth (not used in calculations) \NC \NR
-\NC \type{objnum}    \NC number \NC the referenced \PDF\ object number \NC \NR
-\NC \type{link_attr} \NC table  \NC the link attribute token list \NC \NR
-\NC \type{action}    \NC node   \NC the action to perform \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_end_link whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type \BC explanation \NC \NR
-\NC \type{attr} \NC node \NC \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_dest whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field              \BC type     \BC explanation \NC \NR
-\NC \type{attr}        \NC node     \NC list of attributes \NC \NR
-\NC \type{width}       \NC number   \NC the width (not used in calculations) \NC \NR
-\NC \type{height}      \NC number   \NC the height (not used in calculations) \NC \NR
-\NC \type{depth}       \NC number   \NC the depth (not used in calculations) \NC \NR
-\NC \type{named_id}    \NC number   \NC is the \type {dest_id} a string value? \NC \NR
-\NC \type{dest_id}     \NC number   \NC the destination id \NC \NR
-\NC                    \NC string   \NC the destination name \NC \NR
-\NC \type{dest_type}   \NC number   \NC type of destination \NC \NR
-\NC \type{xyz_zoom}    \NC number   \NC the zoom factor (times 1000) \NC \NR
-\NC \type{objnum}      \NC number   \NC the \PDF\ object number \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_action whatsits}
-
-These are a special kind of item that only appears inside \PDF\ start link
-objects.
-
-\starttabulate[|l|l|p|]
-\BC field              \BC type             \BC explanation \NC \NR
-\BC field              \BC type             \BC explanation \NC \NR
-\NC \type{action_type} \NC number           \NC the kind of action involved \NC \NR
-\NC \type{action_id}   \NC number or string \NC token list reference or string \NC \NR
-\NC \type{named_id}    \NC number           \NC the index of the destination \NC \NR
-\NC \type{file}        \NC string           \NC the target filename \NC \NR
-\NC \type{new_window}  \NC number           \NC the window state of the target \NC \NR
-\NC \type{data}        \NC string           \NC the name of the destination \NC \NR
-\stoptabulate
-
-Valid action types are:
-
-\starttabulate[|l|l|]
-\NC 0 \NC \type{page}   \NC \NR
-\NC 1 \NC \type{goto}   \NC \NR
-\NC 2 \NC \type{thread} \NC \NR
-\NC 3 \NC \type{user}   \NC \NR
-\stoptabulate
-
-Valid window types are:
-
-\starttabulate[|l|l|]
-\NC 0 \NC \type{notset} \NC \NR
-\NC 1 \NC \type{new}    \NC \NR
-\NC 2 \NC \type{nonew}  \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_thread whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field              \BC type   \BC explanation \NC \NR
-\NC \type{attr}        \NC node   \NC list of attributes \NC \NR
-\NC \type{width}       \NC number \NC the width (not used in calculations) \NC \NR
-\NC \type{height}      \NC number \NC the height (not used in calculations) \NC \NR
-\NC \type{depth}       \NC number \NC the depth (not used in calculations) \NC \NR
-\NC \type{named_id}    \NC number \NC is \type {tread_id} a string value? \NC \NR
-\NC \type{tread_id}    \NC number \NC the thread id \NC \NR
-\NC                    \NC string \NC the thread name \NC \NR
-\NC \type{thread_attr} \NC number \NC extra thread information \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_start_thread whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field              \BC type   \BC explanation \NC \NR
-\NC \type{attr}        \NC node   \NC list of attributes \NC \NR
-\NC \type{width}       \NC number \NC the width (not used in calculations) \NC \NR
-\NC \type{height}      \NC number \NC the height (not used in calculations) \NC \NR
-\NC \type{depth}       \NC number \NC the depth (not used in calculations) \NC \NR
-\NC \type{named_id}    \NC number \NC is \type {tread_id} a string value? \NC \NR
-\NC \type{tread_id}    \NC number \NC the thread id \NC \NR
-\NC                    \NC string \NC the thread name \NC \NR
-\NC \type{thread_attr} \NC number \NC extra thread information \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_end_thread whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type \BC explanation \NC \NR
-\NC \type{attr} \NC node \NC \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_colorstack whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field          \BC type   \BC explanation \NC \NR
-\NC \type{attr}    \NC node   \NC list of attributes \NC \NR
-\NC \type{stack}   \NC number \NC colorstack id number \NC \NR
-\NC \type{command} \NC number \NC command to execute \NC \NR
-\NC \type{data}    \NC string \NC data \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_setmatrix whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type   \BC explanation \NC \NR
-\NC \type{attr} \NC node   \NC list of attributes \NC \NR
-\NC \type{data} \NC string \NC data \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_save whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type \BC explanation \NC \NR
-\NC \type{attr} \NC node \NC list of attributes \NC \NR
-\stoptabulate
-
-\subsubsubsection{pdf_restore whatsits}
-
-\starttabulate[|l|l|p|]
-\BC field       \BC type \BC explanation \NC \NR
-\NC \type{attr} \NC node \NC list of attributes \NC \NR
-\stoptabulate
-
-\section{The \type {node} library}
-
-The \type {node} library contains functions that facilitate dealing with (lists
-of) nodes and their values. They allow you to create, alter, copy, delete, and
-insert \LUATEX\ node objects, the core objects within the typesetter.
-
-\LUATEX\ nodes are represented in \LUA\ as userdata with the metadata type
-\type {luatex.node}. The various parts within a node can be accessed using
-named fields.
-
-Each node has at least the three fields \type {next}, \type {id}, and \type
-{subtype}:
-
-\startitemize[intro]
-
-\startitem
-    The \type {next} field returns the userdata object for the next node in a
-    linked list of nodes, or \type {nil}, if there is no next node.
-\stopitem
-
-\startitem
-    The \type {id} indicates \TEX's \quote{node type}. The field \type {id} has a
-    numeric value for efficiency reasons, but some of the library functions also
-    accept a string value instead of \type {id}.
-\stopitem
-
-\startitem
-    The \type {subtype} is another number. It often gives further information
-    about a node of a particular \type {id}, but it is most important when
-    dealing with \quote {whatsits}, because they are differentiated solely based
-    on their \type {subtype}.
-\stopitem
-
-\stopitemize
-
-The other available fields depend on the \type {id} (and for \quote {whatsits},
-the \type {subtype}) of the node. Further details on the various fields and their
-meanings are given in~\in{chapter}[nodes].
-
-Support for \type {unset} (alignment) nodes is partial: they can be queried and
-modified from \LUA\ code, but not created.
-
-Nodes can be compared to each other, but: you are actually comparing indices into
-the node memory. This means that equality tests can only be trusted under very
-limited conditions. It will not work correctly in any situation where one of the
-two nodes has been freed and|/|or reallocated: in that case, there will be false
-positives.
-
-At the moment, memory management of nodes should still be done explicitly by the
-user. Nodes are not \quote {seen} by the \LUA\ garbage collector, so you have to
-call the node freeing functions yourself when you are no longer in need of a node
-(list). Nodes form linked lists without reference counting, so you have to be
-careful that when control returns back to \LUATEX\ itself, you have not deleted
-nodes that are still referenced from a \type {next} pointer elsewhere, and that
-you did not create nodes that are referenced more than once.
-
-There are statistics available with regards to the allocated node memory, which
-can be handy for tracing.
-
-\subsection{Node handling functions}
-
-\subsubsection{\type {node.is_node}}
-
-\startfunctioncall
-<boolean> t =
-    node.is_node(<any> item)
-\stopfunctioncall
-
-This function returns true if the argument is a userdata object of
-type \type {<node>}.
-
-\subsubsection{\type {node.types}}
-
-\startfunctioncall
-<table> t =
-    node.types()
-\stopfunctioncall
-
-This function returns an array that maps node id numbers to node type strings,
-providing an overview of the possible top|-|level \type {id} types.
-
-\subsubsection{\type {node.whatsits}}
-
-\startfunctioncall
-<table> t =
-    node.whatsits()
-\stopfunctioncall
-
-\TEX's \quote{whatsits} all have the same \type {id}. The various subtypes are
-defined by their \type {subtype} fields. The function is much like \type
-{node.types}, except that it provides an array of \type {subtype} mappings.
-
-\subsubsection{\type {node.id}}
-
-\startfunctioncall
-<number> id =
-    node.id(<string> type)
-\stopfunctioncall
-
-This converts a single type name to its internal numeric representation.
-
-\subsubsection{\type {node.subtype}}
-
-\startfunctioncall
-<number> subtype =
-    node.subtype(<string> type)
-\stopfunctioncall
-
-This converts a single whatsit name to its internal numeric representation (\type
-{subtype}).
-
-\subsubsection{\type {node.type}}
-
-\startfunctioncall
-<string> type =
-    node.type(<any> n)
-\stopfunctioncall
-
-In the argument is a number, then this function converts an internal numeric
-representation to an external string representation. Otherwise, it will return
-the string \type {node} if the object represents a node, and \type {nil}
-otherwise.
-
-\subsubsection{\type {node.fields}}
-
-\startfunctioncall
-<table> t =
-    node.fields(<number> id)
-<table> t =
-    node.fields(<number> id, <number> subtype)
-\stopfunctioncall
-
-This function returns an array of valid field names for a particular type of
-node. If you want to get the valid fields for a \quote {whatsit}, you have to
-supply the second argument also. In other cases, any given second argument will
-be silently ignored.
-
-This function accepts string \type {id} and \type {subtype} values as well.
-
-\subsubsection{\type {node.has_field}}
-
-\startfunctioncall
-<boolean> t =
-    node.has_field(<node> n, <string> field)
-\stopfunctioncall
-
-This function returns a boolean that is only true if \type {n} is
-actually a node, and it has the field.
-
-\subsubsection{\type {node.new}}
-
-\startfunctioncall
-<node> n =
-    node.new(<number> id)
-<node> n =
-    node.new(<number> id, <number> subtype)
-\stopfunctioncall
-
-Creates a new node. All of the new node's fields are initialized to either zero
-or \type {nil} except for \type {id} and \type {subtype} (if supplied). If you
-want to create a new whatsit, then the second argument is required, otherwise it
-need not be present. As with all node functions, this function creates a node on
-the \TEX\ level.
-
-This function accepts string \type {id} and \type {subtype} values as well.
-
-\subsubsection{\type {node.free} and \type {node.flush_node}}
-
-\startfunctioncall
-<node> next =
-    node.free(<node> n)
-flush_node(<node> n)
-\stopfunctioncall
-
-Removes the node \type {n} from \TEX's memory. Be careful: no checks are done on
-whether this node is still pointed to from a register or some \type {next} field:
-it is up to you to make sure that the internal data structures remain correct.
-
-The \type {free} function returns the next field of the freed node, while the
-\type {flush_node} alternative returns nothing.
-
-\subsubsection{\type {node.flush_list}}
-
-\startfunctioncall
-node.flush_list(<node> n)
-\stopfunctioncall
-
-Removes the node list \type {n} and the complete node list following \type {n}
-from \TEX's memory. Be careful: no checks are done on whether any of these nodes
-is still pointed to from a register or some \type {next} field: it is up to you
-to make sure that the internal data structures remain correct.
-
-\subsubsection{\type {node.copy}}
-
-\startfunctioncall
-<node> m =
-    node.copy(<node> n)
-\stopfunctioncall
-
-Creates a deep copy of node \type {n}, including all nested lists as in the case
-of a hlist or vlist node. Only the \type {next} field is not copied.
-
-\subsubsection{\type {node.copy_list}}
-
-\startfunctioncall
-<node> m =
-    node.copy_list(<node> n)
-<node> m =
-    node.copy_list(<node> n, <node> m)
-\stopfunctioncall
-
-Creates a deep copy of the node list that starts at \type {n}. If \type {m} is
-also given, the copy stops just before node \type {m}.
-
-Note that you cannot copy attribute lists this way, specialized functions for
-dealing with attribute lists will be provided later but are not there yet.
-However, there is normally no need to copy attribute lists as when you do
-assignments to the \type {attr} field or make changes to specific attributes, the
-needed copying and freeing takes place automatically.
-
-\subsubsection{\type {node.next}}
-
-\startfunctioncall
-<node> m =
-    node.next(<node> n)
-\stopfunctioncall
-
-Returns the node following this node, or \type {nil} if there is no such node.
-
-\subsubsection{\type {node.prev}}
-
-\startfunctioncall
-<node> m =
-    node.prev(<node> n)
-\stopfunctioncall
-
-Returns the node preceding this node, or \type {nil} if there is no such node.
-
-\subsubsection{\type {node.current_attr}}
-
-\startfunctioncall
-<node> m =
-    node.current_attr()
-\stopfunctioncall
-
-Returns the currently active list of attributes, if there is one.
-
-The intended usage of \type {current_attr} is as follows:
-
-\starttyping
-local x1 = node.new("glyph")
-x1.attr = node.current_attr()
-local x2 = node.new("glyph")
-x2.attr = node.current_attr()
-\stoptyping
-
-or:
-
-\starttyping
-local x1 = node.new("glyph")
-local x2 = node.new("glyph")
-local ca = node.current_attr()
-x1.attr = ca
-x2.attr = ca
-\stoptyping
-
-The attribute lists are ref counted and the assignment takes care of incrementing
-the refcount. You cannot expect the value \type {ca} to be valid any more when
-you assign attributes (using \type {tex.setattribute}) or when control has been
-passed back to \TEX.
-
-Note: this function is somewhat experimental, and it returns the {\it actual}
-attribute list, not a copy thereof. Therefore, changing any of the attributes in
-the list will change these values for all nodes that have the current attribute
-list assigned to them.
-
-\subsubsection{\type {node.hpack}}
-
-\startfunctioncall
-<node> h, <number> b =
-    node.hpack(<node> n)
-<node> h, <number> b =
-    node.hpack(<node> n, <number> w, <string> info)
-<node> h, <number> b =
-    node.hpack(<node> n, <number> w, <string> info, <string> dir)
-\stopfunctioncall
-
-This function creates a new hlist by packaging the list that begins at node \type
-{n} into a horizontal box. With only a single argument, this box is created using
-the natural width of its components. In the three argument form, \type {info}
-must be either \type {additional} or \type {exactly}, and \type {w} is the
-additional (\type {\hbox spread}) or exact (\type {\hbox to}) width to be used. The
-second return value is the badness of the generated box.
-
-Caveat: at this moment, there can be unexpected side|-|effects to this function,
-like updating some of the \type {\marks} and \type {\inserts}. Also note that the
-content of \type {h} is the original node list \type {n}: if you call \type
-{node.free(h)} you will also free the node list itself, unless you explicitly set
-the \type {list} field to \type {nil} beforehand. And in a similar way, calling
-\type {node.free(n)} will invalidate \type {h} as well!
-
-\subsubsection{\type {node.vpack}}
-
-\startfunctioncall
-<node> h, <number> b =
-    node.vpack(<node> n)
-<node> h, <number> b =
-    node.vpack(<node> n, <number> w, <string> info)
-<node> h, <number> b =
-    node.vpack(<node> n, <number> w, <string> info, <string> dir)
-\stopfunctioncall
-
-This function creates a new vlist by packaging the list that begins at node \type
-{n} into a vertical box. With only a single argument, this box is created using
-the natural height of its components. In the three argument form, \type {info}
-must be either \type {additional} or \type {exactly}, and \type {w} is the
-additional (\type {\vbox spread}) or exact (\type {\vbox to}) height to be used.
-
-The second return value is the badness of the generated box.
-
-See the description of \type {node.hpack()} for a few memory allocation caveats.
-
-\subsubsection{\type {node.dimensions}, \type {node.rangedimensions}}
-
-\startfunctioncall
-<number> w, <number> h, <number> d  =
-    node.dimensions(<node> n)
-<number> w, <number> h, <number> d  =
-    node.dimensions(<node> n, <string> dir)
-<number> w, <number> h, <number> d  =
-    node.dimensions(<node> n, <node> t)
-<number> w, <number> h, <number> d  =
-    node.dimensions(<node> n, <node> t, <string> dir)
-\stopfunctioncall
-
-This function calculates the natural in|-|line dimensions of the node list starting
-at node \type {n} and terminating just before node \type {t} (or the end of the
-list, if there is no second argument). The return values are scaled points. An
-alternative format that starts with glue parameters as the first three arguments
-is also possible:
-
-\startfunctioncall
-<number> w, <number> h, <number> d  =
-    node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,
-        <node> n)
-<number> w, <number> h, <number> d  =
-    node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,
-        <node> n, <string> dir)
-<number> w, <number> h, <number> d  =
-    node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,
-        <node> n, <node> t)
-<number> w, <number> h, <number> d  =
-    node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,
-        <node> n, <node> t, <string> dir)
-\stopfunctioncall
-
-This calling method takes glue settings into account and is especially useful for
-finding the actual width of a sublist of nodes that are already boxed, for
-example in code like this, which prints the width of the space in between the
-\type {a} and \type {b} as it would be if \type {\box0} was used as-is:
-
-\starttyping
-\setbox0 = \hbox to 20pt {a b}
-
-\directlua{print (node.dimensions(
-    tex.box[0].glue_set,
-    tex.box[0].glue_sign,
-    tex.box[0].glue_order,
-    tex.box[0].head.next,
-    node.tail(tex.box[0].head)
-)) }
-\stoptyping
-
-You need to keep in mind that this is one of the few places in \TEX\ where floats
-are used, which means that you can get small differences in rounding when you
-compare the width reported by \type {hpack} with \type {dimensions}.
-
-The second alternative saves a few lookups and can be more convenient in some
-cases:
-
-\startfunctioncall
-<number> w, <number> h, <number> d  =
-    node.rangedimensions(<node> parent, <node> first)
-<number> w, <number> h, <number> d  =
-    node.rangedimensions(<node> parent, <node> first, <node> last)
-\stopfunctioncall
-
-\subsubsection{\type {node.mlist_to_hlist}}
-
-\startfunctioncall
-<node> h =
-    node.mlist_to_hlist(<node> n, <string> display_type, <boolean> penalties)
-\stopfunctioncall
-
-This runs the internal mlist to hlist conversion, converting the math list in
-\type {n} into the horizontal list \type {h}. The interface is exactly the same
-as for the callback \type {mlist_to_hlist}.
-
-\subsubsection{\type {node.slide}}
-
-\startfunctioncall
-<node> m =
-    node.slide(<node> n)
-\stopfunctioncall
-
-Returns the last node of the node list that starts at \type {n}. As a
-side|-|effect, it also creates a reverse chain of \type {prev} pointers between
-nodes.
-
-\subsubsection{\type {node.tail}}
-
-\startfunctioncall
-<node> m =
-    node.tail(<node> n)
-\stopfunctioncall
-
-Returns the last node of the node list that starts at \type {n}.
-
-\subsubsection{\type {node.length}}
-
-\startfunctioncall
-<number> i =
-    node.length(<node> n)
-<number> i =
-    node.length(<node> n, <node> m)
-\stopfunctioncall
-
-Returns the number of nodes contained in the node list that starts at \type {n}.
-If \type {m} is also supplied it stops at \type {m} instead of at the end of the
-list. The node \type {m} is not counted.
-
-\subsubsection{\type {node.count}}
-
-\startfunctioncall
-<number> i =
-    node.count(<number> id, <node> n)
-<number> i =
-    node.count(<number> id, <node> n, <node> m)
-\stopfunctioncall
-
-Returns the number of nodes contained in the node list that starts at \type {n}
-that have a matching \type {id} field. If \type {m} is also supplied, counting
-stops at \type {m} instead of at the end of the list. The node \type {m} is not
-counted.
-
-This function also accept string \type {id}'s.
-
-\subsubsection{\type {node.traverse}}
-
-\startfunctioncall
-<node> t =
-    node.traverse(<node> n)
-\stopfunctioncall
-
-This is a \LUA\ iterator that loops over the node list that starts at \type {n}.
-Typically code looks like this:
-
-\starttyping
-for n in node.traverse(head) do
-   ...
-end
-\stoptyping
-
-is functionally equivalent to:
-
-\starttyping
-do
-  local n
-  local function f (head,var)
-    local t
-    if var == nil then
-       t = head
-    else
-       t = var.next
-    end
-    return t
-  end
-  while true do
-    n = f (head, n)
-    if n == nil then break end
-    ...
-  end
-end
-\stoptyping
-
-It should be clear from the definition of the function \type {f} that even though
-it is possible to add or remove nodes from the node list while traversing, you
-have to take great care to make sure all the \type {next} (and \type {prev})
-pointers remain valid.
-
-If the above is unclear to you, see the section \quote {For Statement} in the
-\LUA\ Reference Manual.
-
-\subsubsection{\type {node.traverse_id}}
-
-\startfunctioncall
-<node> t =
-    node.traverse_id(<number> id, <node> n)
-\stopfunctioncall
-
-This is an iterator that loops over all the nodes in the list that starts at
-\type {n} that have a matching \type {id} field.
-
-See the previous section for details. The change is in the local function \type
-{f}, which now does an extra while loop checking against the upvalue \type {id}:
-
-\starttyping
- local function f(head,var)
-   local t
-   if var == nil then
-      t = head
-   else
-      t = var.next
-   end
-   while not t.id == id do
-      t = t.next
-   end
-   return t
- end
-\stoptyping
-
-\subsubsection{\type {node.traverse_char}}
-
-This iterators loops over the glyph nodes in a list. Only nodes with a subtype
-less than 256 are seen.
-
-\startfunctioncall
-<node> n =
-    node.traverse_char(<node> n)
-\stopfunctioncall
-
-\subsubsection{\type {node.has_glyph}}
-
-This function returns the first glyph or disc node in the given list:
-
-\startfunctioncall
-<node> n =
-    node.has_glyph(<node> n)
-\stopfunctioncall
-
-\subsubsection{\type {node.end_of_math}}
-
-\startfunctioncall
-<node> t =
-    node.end_of_math(<node> start)
-\stopfunctioncall
-
-Looks for and returns the next \type {math_node} following the \type {start}. If
-the given node is a math endnode this helper return that node, else it follows
-the list and return the next math endnote. If no such node is found nil is
-returned.
-
-\subsubsection{\type {node.remove}}
-
-\startfunctioncall
-<node> head, current =
-    node.remove(<node> head, <node> current)
-\stopfunctioncall
-
-This function removes the node \type {current} from the list following \type
-{head}. It is your responsibility to make sure it is really part of that list.
-The return values are the new \type {head} and \type {current} nodes. The
-returned \type {current} is the node following the \type {current} in the calling
-argument, and is only passed back as a convenience (or \type {nil}, if there is
-no such node). The returned \type {head} is more important, because if the
-function is called with \type {current} equal to \type {head}, it will be
-changed.
-
-\subsubsection{\type {node.insert_before}}
-
-\startfunctioncall
-<node> head, new =
-    node.insert_before(<node> head, <node> current, <node> new)
-\stopfunctioncall
-
-This function inserts the node \type {new} before \type {current} into the list
-following \type {head}. It is your responsibility to make sure that \type
-{current} is really part of that list. The return values are the (potentially
-mutated) \type {head} and the node \type {new}, set up to be part of the list
-(with correct \type {next} field). If \type {head} is initially \type {nil}, it
-will become \type {new}.
-
-\subsubsection{\type {node.insert_after}}
-
-\startfunctioncall
-<node> head, new =
-    node.insert_after(<node> head, <node> current, <node> new)
-\stopfunctioncall
-
-This function inserts the node \type {new} after \type {current} into the list
-following \type {head}. It is your responsibility to make sure that \type
-{current} is really part of that list. The return values are the \type {head} and
-the node \type {new}, set up to be part of the list (with correct \type {next}
-field). If \type {head} is initially \type {nil}, it will become \type {new}.
-
-\subsubsection{\type {node.first_glyph}}
-
-\startfunctioncall
-<node> n =
-    node.first_glyph(<node> n)
-<node> n =
-    node.first_glyph(<node> n, <node> m)
-\stopfunctioncall
-
-Returns the first node in the list starting at \type {n} that is a glyph node
-with a subtype indicating it is a glyph, or \type {nil}. If \type {m} is given,
-processing stops at (but including) that node, otherwise processing stops at the
-end of the list.
-
-\subsubsection{\type {node.ligaturing}}
-
-\startfunctioncall
-<node> h, <node> t, <boolean> success =
-    node.ligaturing(<node> n)
-<node> h, <node> t, <boolean> success =
-    node.ligaturing(<node> n, <node> m)
-\stopfunctioncall
-
-Apply \TEX-style ligaturing to the specified nodelist. The tail node \type {m} is
-optional. The two returned nodes \type {h} and \type {t} are the new head and
-tail (both \type {n} and \type {m} can change into a new ligature).
-
-\subsubsection{\type {node.kerning}}
-
-\startfunctioncall
-<node> h, <node> t, <boolean> success =
-    node.kerning(<node> n)
-<node> h, <node> t, <boolean> success =
-    node.kerning(<node> n, <node> m)
-\stopfunctioncall
-
-Apply \TEX|-|style kerning to the specified node list. The tail node \type {m} is
-optional. The two returned nodes \type {h} and \type {t} are the head and tail
-(either one of these can be an inserted kern node, because special kernings with
-word boundaries are possible).
-
-\subsubsection{\type {node.unprotect_glyphs} and \type {node.unprotect_glyph}}
-
-\startfunctioncall
-node.unprotect_glyph(<node> n)
-node.unprotect_glyphs(<node> n,[<node> n])
-\stopfunctioncall
-
-Subtracts 256 from all glyph node subtypes. This and the next function are
-helpers to convert from \type {characters} to \type {glyphs} during node
-processing. The second argument is option and indicates the end of a range.
-
-\subsubsection{\type {node.protect_glyphs} and \type {node.protect_glyph}}
-
-\startfunctioncall
-node.protect_glyph(<node> n)
-node.protect_glyphs(<node> n,[<node> n])
-\stopfunctioncall
-
-Adds 256 to all glyph node subtypes in the node list starting at \type {n},
-except that if the value is 1, it adds only 255. The special handling of 1 means
-that \type {characters} will become \type {glyphs} after subtraction of 256. A
-single character can be marked by the singular call. The second argument is
-option and indicates the end of a range.
-
-\subsubsection{\type {node.last_node}}
-
-\startfunctioncall
-<node> n =
-    node.last_node()
-\stopfunctioncall
-
-This function pops the last node from \TEX's \quote{current list}. It returns
-that node, or \type {nil} if the current list is empty.
-
-\subsubsection{\type {node.write}}
-
-\startfunctioncall
-node.write(<node> n)
-\stopfunctioncall
-
-This is an experimental function that will append a node list to \TEX's \quote
-{current list} The node list is not deep|-|copied! There is no error checking
-either!
-
-\subsubsection{\type {node.protrusion_skippable}}
-
-\startfunctioncall
-<boolean> skippable =
-    node.protrusion_skippable(<node> n)
-\stopfunctioncall
-
-Returns \type {true} if, for the purpose of line boundary discovery when
-character protrusion is active, this node can be skipped.
-
-\subsection{Glue handling}
-
-\subsubsection{\type {node.setglue}}
-
-You can set the properties of a glue in one go. If you pass no values, the glue
-will become a zero glue.
-
-\startfunctioncall
-node.setglue(<node> n)
-node.setglue(<node> n,width,stretch,shrink,stretch_order,shrink_order)
-\stopfunctioncall
-
-When you pass values, only arguments that are numbers are assigned so
-
-\starttyping
-node.setglue(n,655360,false,65536)
-\stoptyping
-
-will only adapt the width and shrink.
-
-When a list node is passed, you set the glue, order and sign instead.
-
-\subsubsection{\type {node.getglue}}
-
-The next call will return 5 values (or northing when no glue is passed).
-
-\startfunctioncall
-<integer> width, <integer> stretch, <integer> shrink, <integer> stretch_order,
-    <integer> shrink_order = node.getglue(<node> n)
-\stopfunctioncall
-
-When the second argument is false, only the width is returned (this is consistent
-with \type {tex.get}).
-
-When a list node is passed, you get back the glue that is set, the order of that
-glue and the sign.
-
-\subsubsection{\type {node.is_zero_glue}}
-
-This function returns \type {true} when the width, stretch and shrink properties
-are zero.
-
-\startfunctioncall
-<boolean> isglue =
-    node.is_zero_glue(<node> n)
-\stopfunctioncall
-
-\subsection{Attribute handling}
-
-Attributes appear as linked list of userdata objects in the \type {attr} field of
-individual nodes. They can be handled individually, but it is much safer and more
-efficient to use the dedicated functions associated with them.
-
-\subsubsection{\type {node.has_attribute}}
-
-\startfunctioncall
-<number> v =
-    node.has_attribute(<node> n, <number> id)
-<number> v =
-    node.has_attribute(<node> n, <number> id, <number> val)
-\stopfunctioncall
-
-Tests if a node has the attribute with number \type {id} set. If \type {val} is
-also supplied, also tests if the value matches \type {val}. It returns the value,
-or, if no match is found, \type {nil}.
-
-\subsubsection{\type {node.get_attribute}}
-
-\startfunctioncall
-<number> v =
-    node.get_attribute(<node> n, <number> id)
-\stopfunctioncall
-
-Tests if a node has an attribute with number \type {id} set. It returns the
-value, or, if no match is found, \type {nil}.
-
-\subsubsection{\type {node.find_attribute}}
-
-\startfunctioncall
-<number> v, <node> n =
-    node.find_attribute(<node> n, <number> id)
-\stopfunctioncall
-
-Finds the first node that has attribute with number \type {id} set. It returns
-the value and the node if there is a match and otherwise nothing.
-
-\subsubsection{\type {node.set_attribute}}
-
-\startfunctioncall
-node.set_attribute(<node> n, <number> id, <number> val)
-\stopfunctioncall
-
-Sets the attribute with number \type {id} to the value \type {val}. Duplicate
-assignments are ignored.
-
-\subsubsection{\type {node.unset_attribute}}
-
-\startfunctioncall
-<number> v =
-    node.unset_attribute(<node> n, <number> id)
-<number> v =
-    node.unset_attribute(<node> n, <number> id, <number> val)
-\stopfunctioncall
-
-Unsets the attribute with number \type {id}. If \type {val} is also supplied, it
-will only perform this operation if the value matches \type {val}. Missing
-attributes or attribute|-|value pairs are ignored.
-
-If the attribute was actually deleted, returns its old value. Otherwise, returns
-\type {nil}.
-
-\subsubsection{\type {node.slide}}
-
-This helper makes sure that the node lists is double linked and returns the found
-tail node.
-
-\startfunctioncall
-<node> tail =
-    node.slide(<node> n)
-\stopfunctioncall
-
-After some callbacks automatic sliding takes place. This feature can be turned
-off with \type {node.fix_node_lists(false)} but you better make sure then that
-you don't mess up lists. In most cases \TEX\ itself only uses \type {next}
-pointers but your other callbacks might expect proper \type {prev} pointers too.
-Future versions of \LUATEX\ can add more checking but this will not influence
-usage.
-
-\subsubsection{\type {node.check_discretionary} and \type {node.check_discretionaries}}
-
-When you fool around with disc nodes you need to be aware of the fact that they
-have a special internal data structure. As long as you reassign the fields when
-you have extended the lists it's ok because then the tail pointers get updated,
-but when you add to list without reassigning you might end up in troubles when
-the linebreak routien kicks in. You can call this function to check the list for
-issues with disc nodes.
-
-\startfunctioncall
-node.check_discretionary(<node> n)
-node.check_discretionaries(<node> head)
-\stopfunctioncall
-
-The plural variant runs over all disc nodes in a list, the singular variant
-checks one node only (it also checks if the node is a disc node).
-
-\subsubsection{\type {node.flatten_discretionaries}}
-
-This function will remove the discretionaries in the list and inject the replace
-field when set.
-
-\startfunctioncall
-<node> head, count = node.flatten_discretionaries(<node> n)
-\stopfunctioncall
-
-\subsubsection{\type {node.family_font}}
-
-When you pass it a proper family identifier the next helper will return the font
-currently associated with it. You can normally also access the font with the normal
-font field or getter because it will resolve the family automatically for noads.
-
-\startfunctioncall
-<integer> id =
-    node.family_font(<integer> fam)
-\stopfunctioncall
-
-\subsubsection{\type {node.set_synctex_fields} and \type {node.get_synctex_fields}}
-
-You can set and query the synctex fields, a file number aka tag and a line
-number, for a glue, kern, hlist, vlist, rule and math nodes as well as glyph
-nodes (although this last one are not used in native synctex).
-
-\startfunctioncall
-node.set_synctex_fields(<integer> f, <integer> l)
-<integer> f, <integer> l =
-    node.get_synctex_fields(<node> n)
-\stopfunctioncall
-
-Of course you need to know what you're doing as no checking on sane values takes
-place. Also, the synctex interpreter used in editors is rather peculiar and has
-some assumptions (heuristics).
-
-\section{Two access models}
-
-Deep down in \TEX\ a node has a number which is an numeric entry in a memory
-table. In fact, this model, where \TEX\ manages memory is real fast and one of
-the reasons why plugging in callbacks that operate on nodes is quite fast too.
-Each node gets a number that is in fact an index in the memory table and that
-number often gets reported when you print node related information.
-
-There are two access models, a robust one using a so called user data object that
-provides a virtual interface to the internal nodes, and a more direct access which
-uses the node numbers directly. The first model provide key based access while
-the second always accesses fields via functions:
-
-\starttyping
-nodeobject.char
-getfield(nodenumber,"char")
-\stoptyping
-
-If you use the direct model, even if you know that you deal with numbers, you
-should not depend on that property but treat it an abstraction just like
-traditional nodes. In fact, the fact that we use a simple basic datatype has the
-penalty that less checking can be done, but less checking is also the reason why
-it's somewhat faster. An important aspect is that one cannot mix both methods,
-but you can cast both models. So, multiplying a node number makes no sense.
-
-So our advice is: use the indexed (table) approach when possible and investigate
-the direct one when speed might be an real issue. For that reason we also provide
-the \type {get*} and \type {set*} functions in the top level node namespace.
-There is a limited set of getters. When implementing this direct approach the
-regular index by key variant was also optimized, so direct access only makes
-sense when we're accessing nodes millions of times (which happens in some font
-processing for instance).
-
-We're talking mostly of getters because setters are less important. Documents
-have not that many content related nodes and setting many thousands of properties
-is hardly a burden contrary to millions of consultations.
-
-Normally you will access nodes like this:
-
-\starttyping
-local next = current.next
-if next then
-    -- do something
-end
-\stoptyping
-
-Here \type {next} is not a real field, but a virtual one. Accessing it results in
-a metatable method being called. In practice it boils down to looking up the node
-type and based on the node type checking for the field name. In a worst case you
-have a node type that sits at the end of the lookup list and a field that is last
-in the lookup chain. However, in successive versions of \LUATEX\ these lookups
-have been optimized and the most frequently accessed nodes and fields have a
-higher priority.
-
-Because in practice the \type {next} accessor results in a function call, there
-is some overhead involved. The next code does the same and performs a tiny bit
-faster (but not that much because it is still a function call but one that knows
-what to look up).
-
-\starttyping
-local next = node.next(current)
-if next then
-    -- do something
-end
-\stoptyping
-
-Some accessors are used frequently and for these we provide more efficient helpers:
-
-\starttabulate[|t|p|]
-\NC \type{getnext}    \NC parsing nodelist always involves this one \NC \NR
-\NC \type{getprev}    \NC used less but is logical companion to \type {getnext} \NC \NR
-\NC \type{getboth}    \NC returns the next and prev pointer of a node \NC \NR
-\NC \type{getid}      \NC consulted a lot \NC \NR
-\NC \type{getsubtype} \NC consulted less but also a topper \NC \NR
-\NC \type{getfont}    \NC used a lot in \OPENTYPE\ handling (glyph nodes are consulted a lot) \NC \NR
-\NC \type{getchar}    \NC idem and also in other places \NC \NR
-\NC \type{getwhd}     \NC returns the \type {width}, \type {height} and \type {depth} of a list, rule or
-                          (unexpanded) glyph as well as glue (its spec is looked at) and unset nodes\NC \NR
-\NC \type{getdisc}    \NC returns the \type {pre}, \type {post} and \type {replace} fields and
-                          optionally when true is passed also the tail fields. \NC \NR
-\NC \type{getlist}    \NC we often parse nested lists so this is a convenient one too \NC \NR
-\NC \type{getleader}  \NC comparable to list, seldom used in \TEX\ (but needs frequent consulting
-                          like lists; leaders could have been made a dedicated node type) \NC \NR
-\NC \type{getfield}   \NC generic getter, sufficient for the rest (other field names are
-                          often shared so a specific getter makes no sense then) \NC \NR
-\NC \type{getbox}     \NC gets the given box (a list node) \NC \NR
-\stoptabulate
-
-In the direct namespace there are more such helpers and most of them are
-accompanied by setters. The getters and setters are clever enough to see what
-node is meant. We don't deal with whatsit nodes: their fields are always accessed
-by name. It doesn't make sense to add getters for all fields, we just identifier
-the most likely candidates. In complex documents, many node and fields types
-never get seen, or seen only a few times, but for instance glyphs are candidates
-for such optimization. The \type {node.direct} interface has some more helpers.
-\footnote {We can define the helpers in the node namespace with \type {getfield}
-which is about as efficient, so at some point we might provide that as module.}
-
-The \type {setdisc} helper takes three (optional) arguments plus an optional
-fourth indicating the subtype. Its \type {getdisc} takes an optional boolean;
-when its value is \type {true} the tail nodes will also be returned. The \type
-{setfont} helper takes an optional second argument, it being the character. The
-directmode setter \type {setlink} takes a list of nodes and will link them,
-thereby ignoring \type {nil} entries. The first valid node is returned (beware:
-for good reason it assumes single nodes). For rarely used fields no helpers are
-provided and there are a few that probably are used seldom too but were added for
-consistency. You can of course always define additional accessor using \type
-{getfield} and \type {setfield} with little overhead.
-
-% \startcolumns[balance=yes]
-
-\def\yes{$+$} \def\nop{$-$}
-
-\starttabulate[|l|c|c|]
-\HL
-\BC function                 \BC node \BC direct \NC \NR
-\HL
-%NC \type {do_ligature_n}        \NC \yes \NC \yes  \NC \NR % was never documented and experimental
-\NC \type {check_discretionaries}\NC \yes \NC \yes  \NC \NR
-\NC \type {copy_list}            \NC \yes \NC \yes  \NC \NR
-\NC \type {copy}                 \NC \yes \NC \yes  \NC \NR
-\NC \type {count}                \NC \yes \NC \yes  \NC \NR
-\NC \type {current_attr}         \NC \yes \NC \yes  \NC \NR
-\NC \type {dimensions}           \NC \yes \NC \yes  \NC \NR
-\NC \type {effective_glue}       \NC \yes \NC \yes  \NC \NR
-\NC \type {end_of_math}          \NC \yes \NC \yes  \NC \NR
-\NC \type {family_font}          \NC \yes \NC \nop  \NC \NR
-\NC \type {fields}               \NC \yes \NC \nop  \NC \NR
-\NC \type {find_attribute}       \NC \yes \NC \yes  \NC \NR
-\NC \type {first_glyph}          \NC \yes \NC \yes  \NC \NR
-\NC \type {flush_list}           \NC \yes \NC \yes  \NC \NR
-\NC \type {flush_node}           \NC \yes \NC \yes  \NC \NR
-\NC \type {free}                 \NC \yes \NC \yes  \NC \NR
-\NC \type {get_attribute}        \NC \yes \NC \yes  \NC \NR
-\NC \type {getattributelist}     \NC \nop \NC \yes  \NC \NR
-\NC \type {getboth}              \NC \yes \NC \yes  \NC \NR
-\NC \type {getbox}               \NC \nop \NC \yes  \NC \NR
-\NC \type {getchar}              \NC \yes \NC \yes  \NC \NR
-\NC \type {getcomponents}        \NC \nop \NC \yes  \NC \NR
-\NC \type {getdepth}             \NC \nop \NC \yes  \NC \NR
-\NC \type {getdir}               \NC \nop \NC \yes  \NC \NR
-\NC \type {getdisc}              \NC \yes \NC \yes  \NC \NR
-\NC \type {getfam}               \NC \nop \NC \yes  \NC \NR
-\NC \type {getfield}             \NC \yes \NC \yes  \NC \NR
-\NC \type {getfont}              \NC \yes \NC \yes  \NC \NR
-\NC \type {getglue}              \NC \yes \NC \yes  \NC \NR
-\NC \type {getheight}            \NC \nop \NC \yes  \NC \NR
-\NC \type {getid}                \NC \yes \NC \yes  \NC \NR
-\NC \type {getkern}              \NC \nop \NC \yes  \NC \NR
-\NC \type {getlang}              \NC \nop \NC \yes  \NC \NR
-\NC \type {getleader}            \NC \yes \NC \yes  \NC \NR
-\NC \type {getlist}              \NC \yes \NC \yes  \NC \NR
-\NC \type {getnext}              \NC \yes \NC \yes  \NC \NR
-\NC \type {getnucleus}           \NC \nop \NC \yes  \NC \NR
-\NC \type {getoffsets}           \NC \nop \NC \yes  \NC \NR
-\NC \type {getpenalty}           \NC \nop \NC \yes  \NC \NR
-\NC \type {getprev}              \NC \yes \NC \yes  \NC \NR
-\NC \type {getproperty}          \NC \yes \NC \yes  \NC \NR
-\NC \type {getshift}             \NC \nop \NC \yes  \NC \NR
-\NC \type {getwidth}             \NC \nop \NC \yes  \NC \NR
-\NC \type {getwhd}               \NC \nop \NC \yes  \NC \NR
-\NC \type {getsub}               \NC \nop \NC \yes  \NC \NR
-\NC \type {getsubtype}           \NC \yes \NC \yes  \NC \NR
-\NC \type {getsup}               \NC \nop \NC \yes  \NC \NR
-\NC \type {has_attribute}        \NC \yes \NC \yes  \NC \NR
-\NC \type {has_field}            \NC \yes \NC \yes  \NC \NR
-\NC \type {has_glyph}            \NC \yes \NC \yes  \NC \NR
-\NC \type {hpack}                \NC \yes \NC \yes  \NC \NR
-\NC \type {id}                   \NC \yes \NC \nop  \NC \NR
-\NC \type {insert_after}         \NC \yes \NC \yes  \NC \NR
-\NC \type {insert_before}        \NC \yes \NC \yes  \NC \NR
-\NC \type {is_char}              \NC \yes \NC \yes  \NC \NR
-\NC \type {is_direct}            \NC \nop \NC \yes  \NC \NR
-\NC \type {is_glue_zero}         \NC \yes \NC \yes  \NC \NR
-\NC \type {is_glyph}             \NC \yes \NC \yes  \NC \NR
-\NC \type {is_node}              \NC \yes \NC \yes  \NC \NR
-\NC \type {kerning}              \NC \yes \NC \yes  \NC \NR
-\NC \type {last_node}            \NC \yes \NC \yes  \NC \NR
-\NC \type {length}               \NC \yes \NC \yes  \NC \NR
-\NC \type {ligaturing}           \NC \yes \NC \yes  \NC \NR
-\NC \type {mlist_to_hlist}       \NC \yes \NC \nop  \NC \NR
-\NC \type {new}                  \NC \yes \NC \yes  \NC \NR
-\NC \type {next}                 \NC \yes \NC \nop  \NC \NR
-\NC \type {prev}                 \NC \yes \NC \nop  \NC \NR
-\NC \type {protect_glyphs}       \NC \yes \NC \yes  \NC \NR
-\NC \type {protect_glyph}        \NC \yes \NC \yes  \NC \NR
-\NC \type {protrusion_skippable} \NC \yes \NC \yes  \NC \NR
-\NC \type {rangedimensions}      \NC \yes \NC \yes  \NC \NR
-\NC \type {remove}               \NC \yes \NC \yes  \NC \NR
-\NC \type {set_attribute}        \NC \nop \NC \yes  \NC \NR
-\NC \type {setattributelist}     \NC \nop \NC \yes  \NC \NR
-\NC \type {setboth}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setbox}               \NC \nop \NC \yes  \NC \NR
-\NC \type {setchar}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setcomponents}        \NC \nop \NC \yes  \NC \NR
-\NC \type {setdepth}             \NC \nop \NC \yes  \NC \NR
-\NC \type {setdir}               \NC \nop \NC \yes  \NC \NR
-\NC \type {setdisc}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setfield}             \NC \yes \NC \yes  \NC \NR
-\NC \type {setfont}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setglue}              \NC \yes \NC \yes  \NC \NR
-\NC \type {setheight}            \NC \nop \NC \yes  \NC \NR
-\NC \type {setid}                \NC \nop \NC \yes  \NC \NR
-\NC \type {setkern}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setlang}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setleader}            \NC \nop \NC \yes  \NC \NR
-\NC \type {setlist}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setnext}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setnucleus}           \NC \nop \NC \yes  \NC \NR
-\NC \type {setoffsets}           \NC \nop \NC \yes  \NC \NR
-\NC \type {setpenalty}           \NC \nop \NC \yes  \NC \NR
-\NC \type {setprev}              \NC \nop \NC \yes  \NC \NR
-\NC \type {setproperty}          \NC \nop \NC \yes  \NC \NR
-\NC \type {setshift}             \NC \nop \NC \yes  \NC \NR
-\NC \type {setwidth}             \NC \nop \NC \yes  \NC \NR
-\NC \type {setwhd}               \NC \nop \NC \yes  \NC \NR
-\NC \type {setsub}               \NC \nop \NC \yes  \NC \NR
-\NC \type {setsubtype}           \NC \nop \NC \yes  \NC \NR
-\NC \type {setsup}               \NC \nop \NC \yes  \NC \NR
-\NC \type {slide}                \NC \yes \NC \yes  \NC \NR
-\NC \type {subtypes}             \NC \yes \NC \nop  \NC \NR
-\NC \type {subtype}              \NC \yes \NC \nop  \NC \NR
-\NC \type {tail}                 \NC \yes \NC \yes  \NC \NR
-\NC \type {todirect}             \NC \yes \NC \yes  \NC \NR
-\NC \type {tonode}               \NC \yes \NC \yes  \NC \NR
-\NC \type {tostring}             \NC \yes \NC \yes  \NC \NR
-\NC \type {traverse_char}        \NC \yes \NC \yes  \NC \NR
-\NC \type {traverse_id}          \NC \yes \NC \yes  \NC \NR
-\NC \type {traverse}             \NC \yes \NC \yes  \NC \NR
-\NC \type {types}                \NC \yes \NC \nop  \NC \NR
-\NC \type {type}                 \NC \yes \NC \nop  \NC \NR
-\NC \type {unprotect_glyphs}     \NC \yes \NC \yes  \NC \NR
-\NC \type {unset_attribute}      \NC \yes \NC \yes  \NC \NR
-\NC \type {usedlist}             \NC \yes \NC \yes  \NC \NR
-\NC \type {uses_font}            \NC \yes \NC \yes  \NC \NR
-\NC \type {vpack}                \NC \yes \NC \yes  \NC \NR
-\NC \type {whatsitsubtypes}      \NC \yes \NC \nop  \NC \NR
-\NC \type {whatsits}             \NC \yes \NC \nop  \NC \NR
-\NC \type {write}                \NC \yes \NC \yes  \NC \NR
-\NC \type {set_synctex_fields}   \NC \yes \NC \yes  \NC \NR
-\NC \type {get_synctex_fields}   \NC \yes \NC \yes  \NC \NR
-\stoptabulate
-
-% \stopcolumns
-
-The \type {node.next} and \type {node.prev} functions will stay but for
-consistency there are variants called \type {getnext} and \type {getprev}. We had
-to use \type {get} because \type {node.id} and \type {node.subtype} are already
-taken for providing meta information about nodes. Note: The getters do only basic
-checking for valid keys. You should just stick to the keys mentioned in the
-sections that describe node properties.
-
-Some nodes have indirect references. For instance a math character refers to a
-family instead of a font. In that case we provide a virtual font field as
-accessor. So, \type {getfont} and \type {.font} can be used on them. The same is
-true for the \type {width}, \type {height} and \type {depth} of glue nodes. These
-actually access the spec node properties, and here we can set as well as get the
-values.
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-nodes

+

+\startchapter[reference=nodes,title={Nodes}]

+

+\startsection[title={\LUA\ node representation}][library=node]

+

+\topicindex {nodes}

+

+\libindex {fields}

+\libindex {subtypes}

+\libindex {values}

+

+\TEX's nodes are represented in \LUA\ as userdata objects with a variable set of

+fields. In the following syntax tables, such as the type of such a userdata object

+is represented as \syntax {<node>}.

+

+The current return value of \type {node.types()} is:

+\startluacode

+    for id, name in table.sortedhash(node.types()) do

+        context.type(name)

+        context(" (%s), ",id)

+    end

+    context.removeunwantedspaces()

+    context.removepunctuation()

+\stopluacode

+. % period

+

+The \prm {lastnodetype} primitive is \ETEX\ compliant. The valid range is still

+$[-1,15]$ and glyph nodes (formerly known as char nodes) have number~0 while

+ligature nodes are mapped to~7. That way macro packages can use the same symbolic

+names as in traditional \ETEX. Keep in mind that these \ETEX\ node numbers are

+different from the real internal ones and that there are more \ETEX\ node types

+than~15.

+

+You can ask for a list of fields with \type {node.fields} and for valid subtypes

+with \type {node.subtypes}. The \type {node.values} function reports some used

+values. Valid arguments are \nod {dir}, \type {direction}, \nod {glue}, \whs

+{pdf_literal}, \whs {pdf_action}, \whs {pdf_window} and \whs {color_stack}. Keep

+in mind that the setters normally expect a number, but this helper gives you a

+list of what numbers matter. For practical reason the \type {pagestate} values

+are also reported with this helper.

+

+\stopsection

+

+\startsection[title={Main text nodes}]

+

+\topicindex {nodes+text}

+

+These are the nodes that comprise actual typesetting commands. A few fields are

+present in all nodes regardless of their type, these are:

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{next}    \NC node   \NC the next node in a list, or nil \NC \NR

+\NC \type{id}      \NC number \NC the node's type (\type {id}) number \NC \NR

+\NC \type{subtype} \NC number \NC the node \type {subtype} identifier \NC \NR

+\LL

+\stoptabulate

+

+The \type {subtype} is sometimes just a dummy entry because not all nodes

+actually use the \type {subtype}, but this way you can be sure that all nodes

+accept it as a valid field name, and that is often handy in node list traversal.

+In the following tables \type {next} and \type {id} are not explicitly mentioned.

+

+Besides these three fields, almost all nodes also have an \type {attr} field, and

+there is a also a field called \type {prev}. That last field is always present,

+but only initialized on explicit request: when the function \type {node.slide()}

+is called, it will set up the \type {prev} fields to be a backwards pointer in

+the argument node list. By now most of \TEX's node processing makes sure that the

+\type {prev} nodes are valid but there can be exceptions, especially when the

+internal magic uses a leading \nod {temp} nodes to temporarily store a state.

+

+\subsection{\nod {hlist} nodes}

+

+\starttabulate[|l|l|p|]

+\DB field             \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype}    \NC number \NC \showsubtypes{list} \NC \NR

+\NC \type{attr}       \NC node   \NC list of attributes \NC \NR

+\NC \type{width}      \NC number \NC the width of the box \NC \NR

+\NC \type{height}     \NC number \NC the height of the box \NC \NR

+\NC \type{depth}      \NC number \NC the depth of the box \NC \NR

+\NC \type{shift}      \NC number \NC a displacement perpendicular to the character

+                                     progression direction \NC \NR

+\NC \type{glue_order} \NC number \NC a number in the range $[0,4]$, indicating the

+                                     glue order \NC \NR

+\NC \type{glue_set}   \NC number \NC the calculated glue ratio \NC \NR

+\NC \type{glue_sign}  \NC number \NC 0 = \type {normal}, 1 = \type {stretching}, 2 =

+                                     \type {shrinking} \NC \NR

+\NC \type{head/list}  \NC node   \NC the first node of the body of this list \NC \NR

+\NC \type{dir}        \NC string \NC the direction of this box, see~\in [dirnodes] \NC \NR

+\LL

+\stoptabulate

+

+\topicindex {nodes+lists}

+\topicindex {lists}

+

+A warning: never assign a node list to the \type {head} field unless you are sure

+its internal link structure is correct, otherwise an error may result.

+

+Note: the field name \type {head} and \type {list} are both valid. Sometimes it

+makes more sense to refer to a list by \type {head}, sometimes \type {list} makes

+more sense.

+

+\subsection{\nod {vlist} nodes}

+

+\topicindex {nodes+lists}

+\topicindex {lists}

+

+This node is similar to \nod {hlist}, except that \quote {shift} is a displacement

+perpendicular to the line progression direction, and \quote {subtype} only has

+the values 0, 4, and~5.

+

+\subsection{\nod {rule} nodes}

+

+\topicindex {nodes+rules}

+\topicindex {rules}

+

+Contrary to traditional \TEX, \LUATEX\ has more \prm {rule} subtypes because we

+also use rules to store reuseable objects and images. User nodes are invisible

+and can be intercepted by a callback.

+

+\starttabulate[|l|l|p|]

+\DB field            \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype}   \NC number \NC \showsubtypes {rule} \NC \NR

+\NC \type{attr}      \NC node   \NC list of attributes \NC \NR

+\NC \type{width}     \NC number \NC the width of the rule where the special value

+                                    $-1073741824$ is used for \quote {running} glue dimensions \NC \NR

+\NC \type{height}    \NC number \NC the height of the rule (can be negative) \NC \NR

+\NC \type{depth}     \NC number \NC the depth of the rule (can be negative) \NC \NR

+\NC \type{left}      \NC number \NC shift at the left end (also subtracted from width) \NC \NR

+\NC \type{right}     \NC number \NC (subtracted from width) \NC \NR

+\NC \type{dir}       \NC string \NC the direction of this rule, see~\in[dirnodes] \NC \NR

+\NC \type{index}     \NC number \NC an optional index that can be referred to \NC \NR

+\NC \type{transform} \NC number \NC an private variable (also used to specify outline width) \NC \NR

+\LL

+\stoptabulate

+

+The \type {left} and type {right} keys are somewhat special (and experimental).

+When rules are auto adapting to the surrounding box width you can enforce a shift

+to the right by setting \type {left}. The value is also subtracted from the width

+which can be a value set by the engine itself and is not entirely under user

+control. The \type {right} is also subtracted from the width. It all happens in

+the backend so these are not affecting the calculations in the frontend (actually

+the auto settings also happen in the backend). For a vertical rule \type {left}

+affects the height and \type {right} affects the depth. There is no matching

+interface at the \TEX\ end (although we can have more keywords for rules it would

+complicate matters and introduce a speed penalty.) However, you can just

+construct a rule node with \LUA\ and write it to the \TEX\ input. The \type

+{outline} subtype is just a convenient variant and the \type {transform} field

+specifies the width of the outline.

+

+\subsection{\nod {ins} nodes}

+

+\topicindex {nodes+insertions}

+\topicindex {insertions}

+

+This node relates to the \prm {insert} primitive.

+

+\starttabulate[|l|l|p|]

+\DB field            \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype}   \NC number \NC the insertion class \NC \NR

+\NC \type{attr}      \NC node   \NC list of attributes \NC \NR

+\NC \type{cost}      \NC number \NC the penalty associated with this insert \NC \NR

+\NC \type{height}    \NC number \NC height of the insert \NC \NR

+\NC \type{depth}     \NC number \NC depth of the insert \NC \NR

+\NC \type{head/list} \NC node   \NC the first node of the body of this insert \NC \NR

+\LL

+\stoptabulate

+

+There is a set of extra fields that concern the associated glue: \type {width},

+\type {stretch}, \type {stretch_order}, \type {shrink} and \type {shrink_order}.

+These are all numbers.

+

+A warning: never assign a node list to the \type {head} field unless you are sure

+its internal link structure is correct, otherwise an error may result. You can use

+\type {list} instead (often in functions you want to use local variable with similar

+names and both names are equally sensible).

+

+\subsection{\nod {mark} nodes}

+

+\topicindex {nodes+marks}

+\topicindex {marks}

+

+This one relates to the \prm {mark} primitive.

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number \NC unused \NC \NR

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{class}   \NC number \NC the mark class \NC \NR

+\NC \type{mark}    \NC table  \NC a table representing a token list \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\nod {adjust} nodes}

+

+\topicindex {nodes+adjust}

+\topicindex {adjust}

+

+This node comes from \prm {vadjust} primitive.

+

+\starttabulate[|l|l|p|]

+\DB field            \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype}   \NC number \NC \showsubtypes{adjust} \NC \NR

+\NC \type{attr}      \NC node   \NC list of attributes \NC \NR

+\NC \type{head/list} \NC node   \NC adjusted material \NC \NR

+\LL

+\stoptabulate

+

+A warning: never assign a node list to the \type {head} field unless you are sure

+its internal link structure is correct, otherwise an error may be the result.

+

+\subsection{\nod {disc} nodes}

+

+\topicindex {nodes+discretionaries}

+\topicindex {discretionaries}

+

+The \prm {discretionary} and \prm {-}, the \type {-} character but also the

+hyphenation mechanism produces these nodes.

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number \NC \showsubtypes{disc} \NC \NR

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{pre}     \NC node   \NC pointer to the pre|-|break text \NC \NR

+\NC \type{post}    \NC node   \NC pointer to the post|-|break text \NC \NR

+\NC \type{replace} \NC node   \NC pointer to the no|-|break text \NC \NR

+\NC \type{penalty} \NC number \NC the penalty associated with the break, normally

+                                  \prm {hyphenpenalty} or \prm {exhyphenpenalty} \NC \NR

+\LL

+\stoptabulate

+

+The subtype numbers~4 and~5 belong to the \quote {of-f-ice} explanation given

+elsewhere. These disc nodes are kind of special as at some point they also keep

+information about breakpoints and nested ligatures.

+

+The \type {pre}, \type {post} and \type {replace} fields at the \LUA\ end are in

+fact indirectly accessed and have a \type {prev} pointer that is not \type {nil}.

+This means that when you mess around with the head of these (three) lists, you

+also need to reassign them because that will restore the proper \type {prev}

+pointer, so:

+

+\starttyping

+pre = d.pre

+-- change the list starting with pre

+d.pre = pre

+\stoptyping

+

+Otherwise you can end up with an invalid internal perception of reality and

+\LUATEX\ might even decide to crash on you. It also means that running forward

+over for instance \type {pre} is ok but backward you need to stop at \type {pre}.

+And you definitely must not mess with the node that \type {prev} points to, if

+only because it is not really a node but part of the disc data structure (so

+freeing it again might crash \LUATEX).

+

+\subsection{\nod {math} nodes}

+

+\topicindex {nodes+math}

+\topicindex {math+nodes}

+

+Math nodes represent the boundaries of a math formula, normally wrapped into

+\type {$} signs.

+

+\starttabulate[|l|l|p|]

+\DB field           \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype}  \NC number \NC \showsubtypes{math} \NC \NR

+\NC \type{attr}     \NC node   \NC list of attributes \NC \NR

+\NC \type{surround} \NC number \NC width of the \prm {mathsurround} kern \NC \NR

+\LL

+\stoptabulate

+

+There is a set of extra fields that concern the associated glue: \type {width},

+\type {stretch}, \type {stretch_order}, \type {shrink} and \type {shrink_order}.

+These are all numbers.

+

+\subsection{\nod {glue} nodes}

+

+\topicindex {nodes+glue}

+\topicindex {glue}

+

+Skips are about the only type of data objects in traditional \TEX\ that are not a

+simple value. They are inserted when \TEX\ sees a space in the text flow but also

+by \prm {hskip} and \prm {vskip}. The structure that represents the glue

+components of a skip is called a \nod {glue_spec}, and it has the following

+accessible fields:

+

+\starttabulate[|l|l|p|]

+\DB field                \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{width}         \NC number \NC the horizontal or vertical displacement \NC \NR

+\NC \type{stretch}       \NC number \NC extra (positive) displacement or stretch amount \NC \NR

+\NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR

+\NC \type{shrink}        \NC number \NC extra (negative) displacement or shrink amount\NC \NR

+\NC \type{shrink_order}  \NC number \NC factor applied to shrink amount \NC \NR

+\LL

+\stoptabulate

+

+The effective width of some glue subtypes depends on the stretch or shrink needed

+to make the encapsulating box fit its dimensions. For instance, in a paragraph

+lines normally have glue representing spaces and these stretch or shrink to make

+the content fit in the available space. The \type {effective_glue} function that

+takes a glue node and a parent (hlist or vlist) returns the effective width of

+that glue item. When you pass \type {true} as third argument the value will be

+rounded.

+

+A \nod {glue_spec} node is a special kind of node that is used for storing a set

+of glue values in registers. Originally they were also used to store properties

+of glue nodes (using a system of reference counts) but we now keep these

+properties in the glue nodes themselves, which gives a cleaner interface to \LUA.

+

+The indirect spec approach was in fact an optimization in the original \TEX\

+code. First of all it can save quite some memory because all these spaces that

+become glue now share the same specification (only the reference count is

+incremented), and zero testing is also a bit faster because only the pointer has

+to be checked (this is no longer true for engines that implement for instance

+protrusion where we really need to ensure that zero is zero when we test for

+bounds). Another side effect is that glue specifications are read|-|only, so in

+the end copies need to be made when they are used from \LUA\ (each assignment to

+a field can result in a new copy). So in the end the advantages of sharing are

+not that high (and nowadays memory is less an issue, also given that a glue node

+is only a few memory words larger than a spec).

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number \NC \showsubtypes{glue} \NC \NR

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{leader}  \NC node   \NC pointer to a box or rule for leaders \NC \NR

+\LL

+\stoptabulate

+

+In addition there are the \type {width}, \type {stretch} \type {stretch_order},

+\type {shrink}, and \type {shrink_order} fields. Note that we use the key \type

+{width} in both horizontal and vertical glue. This suits the \TEX\ internals well

+so we decided to stick to that naming.

+

+A regular word space also results in a \type {spaceskip} subtype (this used to be

+a \type {userskip} with subtype zero).

+

+\subsection{\nod {kern} nodes}

+

+\topicindex {nodes+kerns}

+\topicindex {kerns}

+

+The \prm {kern} command creates such nodes but for instance the font and math

+machinery can also add them.

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number \NC \showsubtypes{kern} \NC \NR

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{kern}    \NC number \NC fixed horizontal or vertical advance \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\nod {penalty} nodes}

+

+\topicindex {nodes+penalty}

+\topicindex {penalty}

+

+The \prm {penalty} command is one that generates these nodes.

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number \NC \showsubtypes{penalty} \NC \NR

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{penalty} \NC number \NC the penalty value \NC \NR

+\LL

+\stoptabulate

+

+The subtypes are just informative and \TEX\ itself doesn't use them. When you

+run into an \type {linebreakpenalty} you need to keep in mind that it's a

+accumulation of \type {club}, \type{widow} and other relevant penalties.

+

+\subsection[glyphnodes]{\nod {glyph} nodes}

+

+\topicindex {nodes+glyph}

+\topicindex {glyphs}

+

+These are probably the mostly used nodes and although you can push them in the

+current list with for instance \prm {char} \TEX\ will normally do it for you when

+it considers some input to be text.

+

+\starttabulate[|l|l|p|]

+\DB field                   \BC type    \BC explanation \NC \NR

+\TB

+\NC \type{subtype}          \NC number  \NC bit field \NC \NR

+\NC \type{attr}             \NC node    \NC list of attributes \NC \NR

+\NC \type{char}             \NC number  \NC the character index in the font \NC \NR

+\NC \type{font}             \NC number  \NC the font identifier \NC \NR

+\NC \type{lang}             \NC number  \NC the language identifier \NC \NR

+\NC \type{left}             \NC number  \NC the frozen \type {\lefthyphenmnin} value \NC \NR

+\NC \type{right}            \NC number  \NC the frozen \type {\righthyphenmnin} value \NC \NR

+\NC \type{uchyph}           \NC boolean \NC the frozen \prm {uchyph} value \NC \NR

+\NC \type{components}       \NC node    \NC pointer to ligature components \NC \NR

+\NC \type{xoffset}          \NC number  \NC a virtual displacement in horizontal direction \NC \NR

+\NC \type{yoffset}          \NC number  \NC a virtual displacement in vertical direction \NC \NR

+\NC \type{width}            \NC number  \NC the (original) width of the character \NC \NR

+\NC \type{height}           \NC number  \NC the (original) height of the character\NC \NR

+\NC \type{depth}            \NC number  \NC the (original) depth of the character\NC \NR

+\NC \type{expansion_factor} \NC number  \NC the to be applied expansion_factor \NC \NR

+\NC \type{data}             \NC number  \NC a general purpose field for users (we had room for it) \NC \NR

+\LL

+\stoptabulate

+

+The \type {width}, \type {height} and \type {depth} values are read|-|only. The

+\type {expansion_factor} is assigned in the par builder and used in the backend.

+

+A warning: never assign a node list to the components field unless you are sure

+its internal link structure is correct, otherwise an error may be result. Valid

+bits for the \type {subtype} field are:

+

+\starttabulate[|c|l|]

+\DB bit \BC meaning   \NC \NR

+\TB

+\NC 0   \NC character \NC \NR

+\NC 1   \NC ligature  \NC \NR

+\NC 2   \NC ghost     \NC \NR

+\NC 3   \NC left      \NC \NR

+\NC 4   \NC right     \NC \NR

+\LL

+\stoptabulate

+

+See \in {section} [charsandglyphs] for a detailed description of the \type

+{subtype} field.

+

+The \type {expansion_factor} has been introduced as part of the separation

+between font- and backend. It is the result of extensive experiments with a more

+efficient implementation of expansion. Early versions of \LUATEX\ already

+replaced multiple instances of fonts in the backend by scaling but contrary to

+\PDFTEX\ in \LUATEX\ we now also got rid of font copies in the frontend and

+replaced them by expansion factors that travel with glyph nodes. Apart from a

+cleaner approach this is also a step towards a better separation between front-

+and backend.

+

+The \type {is_char} function checks if a node is a glyph node with a subtype still

+less than 256. This function can be used to determine if applying font logic to a

+glyph node makes sense. The value \type {nil} gets returned when the node is not

+a glyph, a character number is returned if the node is still tagged as character

+and \type {false} gets returned otherwise. When nil is returned, the id is also

+returned. The \type {is_glyph} variant doesn't check for a subtype being less

+than 256, so it returns either the character value or nil plus the id. These

+helpers are not always faster than separate calls but they sometimes permit

+making more readable tests. The \type {uses_font} helpers takes a node

+and font id and returns true when a glyph or disc node references that font.

+

+\subsection{\nod {boundary} nodes}

+

+\topicindex {nodes+boundary}

+\topicindex {boundary}

+

+This node relates to the \prm {noboundary}, \prm {boundary}, \prm

+{protrusionboundary} and \prm {wordboundary} primitives.

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number \NC \showsubtypes{boundary} \NC \NR

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{value}   \NC number \NC values 0--255 are reserved \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\nod {local_par} nodes}

+

+\topicindex {nodes+paragraphs}

+\topicindex {paragraphs}

+

+This node is inserted at the start of a paragraph. You should not mess

+too much with this one.

+

+\starttabulate[|l|l|p|]

+\DB field                  \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}            \NC node   \NC list of attributes \NC \NR

+\NC \type{pen_inter}       \NC number \NC local interline penalty (from \lpr {localinterlinepenalty}) \NC \NR

+\NC \type{pen_broken}      \NC number \NC local broken penalty (from \lpr {localbrokenpenalty}) \NC \NR

+\NC \type{dir}             \NC string \NC the direction of this par. see~\in [dirnodes] \NC \NR

+\NC \type{box_left}        \NC node   \NC the \lpr {localleftbox} \NC \NR

+\NC \type{box_left_width}  \NC number \NC width of the \lpr {localleftbox} \NC \NR

+\NC \type{box_right}       \NC node   \NC the \lpr {localrightbox} \NC \NR

+\NC \type{box_right_width} \NC number \NC width of the \lpr {localrightbox} \NC \NR

+\LL

+\stoptabulate

+

+A warning: never assign a node list to the \type {box_left} or \type {box_right}

+field unless you are sure its internal link structure is correct, otherwise an

+error may result.

+

+\subsection[dirnodes]{\nod {dir} nodes}

+

+\topicindex {nodes+direction}

+\topicindex {directions}

+

+Direction nodes mark parts of the running text that need a change of direction and \

+the \prm {textdir} command generates them.

+

+\starttabulate[|l|l|p|]

+\DB field        \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}  \NC node   \NC list of attributes \NC \NR

+\NC \type{dir}   \NC string \NC the direction (but see below) \NC \NR

+\NC \type{level} \NC number \NC nesting level of this direction whatsit \NC \NR

+\LL

+\stoptabulate

+

+Direction specifiers are three|-|letter combinations of \type {T}, \type {B},

+\type {R}, and \type {L}. These are built up out of three separate items:

+

+\startitemize[packed]

+\startitem

+    the first  is the direction of the \quote{top}   of paragraphs

+\stopitem

+\startitem

+    the second is the direction of the \quote{start} of lines

+\stopitem

+\startitem

+    the third  is the direction of the \quote{top}   of glyphs

+\stopitem

+\stopitemize

+

+However, only four combinations are accepted: \type {TLT}, \type {TRT}, \type

+{RTT}, and \type {LTL}. Inside actual \nod {dir} nodes, the representation of

+\nod {dir} is not a three|-|letter but a combination of numbers. When printed the

+direction is indicated by a \type {+} or \type {-}, indicating whether the value

+is pushed or popped from the direction stack.

+

+\subsection{\nod {marginkern} nodes}

+

+\topicindex {nodes+paragraphs}

+\topicindex {paragraphs}

+\topicindex {protrusion}

+

+Margin kerns result from protrusion.

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number \NC \showsubtypes{marginkern} \NC \NR

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{width}   \NC number \NC the advance of the kern \NC \NR

+\NC \type{glyph}   \NC node   \NC the glyph to be used \NC \NR

+\LL

+\stoptabulate

+

+\stopsection

+

+\startsection[title={Math noads}]

+

+\topicindex {nodes+math}

+\topicindex {math+nodes}

+

+These are the so||called \quote {noad}s and the nodes that are specifically

+associated with math processing. Most of these nodes contain subnodes so that the

+list of possible fields is actually quite small. First, the subnodes:

+

+\subsection{Math kernel subnodes}

+

+Many object fields in math mode are either simple characters in a specific family

+or math lists or node lists. There are four associated subnodes that represent

+these cases (in the following node descriptions these are indicated by the word

+\type {<kernel>}).

+

+The \type {next} and \type {prev} fields for these subnodes are unused.

+

+\subsection{\nod {math_char} and \nod {math_text_char} subnodes}

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr} \NC node   \NC list of attributes \NC \NR

+\NC \type{char} \NC number \NC the character index \NC \NR

+\NC \type{fam}  \NC number \NC the family number \NC \NR

+\LL

+\stoptabulate

+

+The \nod {math_char} is the simplest subnode field, it contains the character

+and family for a single glyph object. The \nod {math_text_char} is a special

+case that you will not normally encounter, it arises temporarily during math list

+conversion (its sole function is to suppress a following italic correction).

+

+\subsection{\nod {sub_box} and \nod {sub_mlist} subnodes}

+

+\starttabulate[|l|l|p|]

+\DB field            \BC type \BC explanation \NC \NR

+\TB

+\NC \type{attr}      \NC node \NC list of attributes \NC \NR

+\NC \type{head/list} \NC node \NC list of nodes \NC \NR

+\LL

+\stoptabulate

+

+These two subnode types are used for subsidiary list items. For \nod {sub_box},

+the \type {head} points to a \quote {normal} vbox or hbox. For \nod {sub_mlist},

+the \type {head} points to a math list that is yet to be converted.

+

+A warning: never assign a node list to the \type {head} field unless you are sure

+its internal link structure is correct, otherwise an error is triggered.

+

+\subsection{\nod {delim} subnodes}

+

+There is a fifth subnode type that is used exclusively for delimiter fields. As

+before, the \type {next} and \type {prev} fields are unused.

+

+\starttabulate[|l|l|p|]

+\DB field             \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}       \NC node   \NC list of attributes \NC \NR

+\NC \type{small_char} \NC number \NC character index of base character \NC \NR

+\NC \type{small_fam}  \NC number \NC family number of base character \NC \NR

+\NC \type{large_char} \NC number \NC character index of next larger character \NC \NR

+\NC \type{large_fam}  \NC number \NC family number of next larger character \NC \NR

+\LL

+\stoptabulate

+

+The fields \type {large_char} and \type {large_fam} can be zero, in that case the

+font that is set for the \type {small_fam} is expected to provide the large

+version as an extension to the \type {small_char}.

+

+\subsection{Math core nodes}

+

+First, there are the objects (the \TEX book calls them \quote {atoms}) that are

+associated with the simple math objects: ord, op, bin, rel, open, close, punct,

+inner, over, under, vcent. These all have the same fields, and they are combined

+into a single node type with separate subtypes for differentiation.

+

+Some noads have an option field. The values in this bitset are common:

+

+\starttabulate[|l|r|]

+\DB meaning         \BC bits                      \NC \NR

+\TB

+\NC set             \NC               \type{0x08} \NC \NR

+\NC internal        \NC \type{0x00} + \type{0x08} \NC \NR

+\NC internal        \NC \type{0x01} + \type{0x08} \NC \NR

+\NC axis            \NC \type{0x02} + \type{0x08} \NC \NR

+\NC no axis         \NC \type{0x04} + \type{0x08} \NC \NR

+\NC exact           \NC \type{0x10} + \type{0x08} \NC \NR

+\NC left            \NC \type{0x11} + \type{0x08} \NC \NR

+\NC middle          \NC \type{0x12} + \type{0x08} \NC \NR

+\NC right           \NC \type{0x14} + \type{0x08} \NC \NR

+\NC no sub script   \NC \type{0x21} + \type{0x08} \NC \NR

+\NC no super script \NC \type{0x22} + \type{0x08} \NC \NR

+\NC no script       \NC \type{0x23} + \type{0x08} \NC \NR

+\LL

+\stoptabulate

+

+\subsection{simple \nod {noad} nodes}

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type        \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number      \NC \showsubtypes{noad} \NC \NR

+\NC \type{attr}    \NC node        \NC list of attributes \NC \NR

+\NC \type{nucleus} \NC kernel node \NC base \NC \NR

+\NC \type{sub}     \NC kernel node \NC subscript \NC \NR

+\NC \type{sup}     \NC kernel node \NC superscript \NC \NR

+\NC \type{options} \NC number      \NC bitset of rendering options \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\nod {accent} nodes}

+

+\starttabulate[|l|l|p|]

+\DB field             \BC type        \BC explanation \NC \NR

+\TB

+\NC \type{subtype}    \NC number      \NC \showsubtypes{accent} \NC \NR

+\NC \type{nucleus}    \NC kernel node \NC base \NC \NR

+\NC \type{sub}        \NC kernel node \NC subscript \NC \NR

+\NC \type{sup}        \NC kernel node \NC superscript \NC \NR

+\NC \type{accent}     \NC kernel node \NC top accent \NC \NR

+\NC \type{bot_accent} \NC kernel node \NC bottom accent \NC \NR

+\NC \type{fraction}   \NC number      \NC larger step criterium (divided by 1000) \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\nod {style} nodes}

+

+\starttabulate[|l|l|p|]

+\DB field        \BC type   \BC explanation    \NC \NR

+\TB

+\NC \type{style} \NC string \NC contains the style \NC \NR

+\LL

+\stoptabulate

+

+There are eight possibilities for the string value: one of \type {display},

+\type {text}, \type {script}, or \type {scriptscript}. Each of these can have

+be prefixed by \type {cramped}.

+

+\subsection{\nod {choice} nodes}

+

+\starttabulate[|l|l|p|]

+\DB field               \BC type \BC explanation \NC \NR

+\TB

+\NC \type{attr}         \NC node \NC list of attributes \NC \NR

+\NC \type{display}      \NC node \NC list of display size alternatives \NC \NR

+\NC \type{text}         \NC node \NC list of text size alternatives \NC \NR

+\NC \type{script}       \NC node \NC list of scriptsize alternatives \NC \NR

+\NC \type{scriptscript} \NC node \NC list of scriptscriptsize alternatives \NC \NR

+\LL

+\stoptabulate

+

+Warning: never assign a node list to the \type {display}, \type {text}, \type

+{script}, or \type {scriptscript} field unless you are sure its internal link

+structure is correct, otherwise an error can occur.

+

+\subsection{\nod {radical} nodes}

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type           \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number         \NC \showsubtypes{radical} \NC \NR

+\NC \type{attr}    \NC node           \NC list of attributes \NC \NR

+\NC \type{nucleus} \NC kernel node    \NC base \NC \NR

+\NC \type{sub}     \NC kernel node    \NC subscript \NC \NR

+\NC \type{sup}     \NC kernel node    \NC superscript \NC \NR

+\NC \type{left}    \NC delimiter node \NC \NC \NR

+\NC \type{degree}  \NC kernel node    \NC only set by \lpr {Uroot} \NC \NR

+\NC \type{width}   \NC number         \NC required width \NC \NR

+\NC \type{options} \NC number         \NC bitset of rendering options \NC \NR

+\LL

+\stoptabulate

+

+Warning: never assign a node list to the \type {nucleus}, \type {sub}, \type

+{sup}, \type {left}, or \type {degree} field unless you are sure its internal

+link structure is correct, otherwise an error can be triggered.

+

+\subsection{\nod {fraction} nodes}

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type           \BC explanation \NC \NR

+\TB

+\NC \type{attr}    \NC node           \NC list of attributes \NC \NR

+\NC \type{width}   \NC number         \NC (optional) width of the fraction \NC \NR

+\NC \type{num}     \NC kernel node    \NC numerator \NC \NR

+\NC \type{denom}   \NC kernel node    \NC denominator \NC \NR

+\NC \type{left}    \NC delimiter node \NC left side symbol \NC \NR

+\NC \type{right}   \NC delimiter node \NC right side symbol \NC \NR

+\NC \type{middle}  \NC delimiter node \NC middle symbol \NC \NR

+\NC \type{options} \NC number         \NC bitset of rendering options \NC \NR

+\LL

+\stoptabulate

+

+Warning: never assign a node list to the \type {num}, or \type {denom} field

+unless you are sure its internal link structure is correct, otherwise an error

+can result.

+

+\subsection{\nod {fence} nodes}

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type           \BC explanation \NC \NR

+\TB

+\NC \type{subtype} \NC number         \NC \showsubtypes{fence} \NC \NR

+\NC \type{attr}    \NC node           \NC list of attributes \NC \NR

+\NC \type{delim}   \NC delimiter node \NC delimiter specification \NC \NR

+\NC \type{italic}  \NC number         \NC italic correction \NC \NR

+\NC \type{height}  \NC number         \NC required height \NC \NR

+\NC \type{depth}   \NC number         \NC required depth \NC \NR

+\NC \type{options} \NC number         \NC bitset of rendering options \NC \NR

+\NC \type{class}   \NC number         \NC spacing related class \NC \NR

+\LL

+\stoptabulate

+

+Warning: some of these fields are used by the renderer and might get adapted in

+the process.

+

+\stopsection

+

+\startsection[title={Front|-|end whatsits}]

+

+Whatsit nodes come in many subtypes that you can ask for them by running

+\type {node.whatsits}:

+\startluacode

+    for id, name in table.sortedpairs(node.whatsits()) do

+        context.type(name)

+        context(" (%s), ",id)

+    end

+    context.removeunwantedspaces()

+    context.removepunctuation()

+\stopluacode

+. % period

+

+Some of them are generic and independent of the output mode and others are

+specific to the chosen backend: \DVI\ or \PDF. Here we discuss the generic

+font|-|end nodes nodes.

+

+\subsection{\whs {open}}

+

+\starttabulate[|l|l|p|]

+\DB field         \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}   \NC node   \NC list of attributes \NC \NR

+\NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR

+\NC \type{name}   \NC string \NC file name \NC \NR

+\NC \type{ext}    \NC string \NC file extension \NC \NR

+\NC \type{area}   \NC string \NC file area (this may become obsolete) \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {write}}

+

+\starttabulate[|l|l|p|]

+\DB field         \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}   \NC node   \NC list of attributes \NC \NR

+\NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR

+\NC \type{data}   \NC table  \NC a table representing the token list to be written \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {close}}

+

+\starttabulate[|l|l|p|]

+\DB field         \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}   \NC node   \NC list of attributes \NC \NR

+\NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {user_defined}}

+

+User|-|defined whatsit nodes can only be created and handled from \LUA\ code. In

+effect, they are an extension to the extension mechanism. The \LUATEX\ engine

+will simply step over such whatsits without ever looking at the contents.

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{user_id} \NC number \NC id number \NC \NR

+\NC \type{type}    \NC number \NC type of the value \NC \NR

+\NC \type{value}   \NC number \NC a \LUA\ number \NC \NR

+\NC                \NC node   \NC a node list \NC \NR

+\NC                \NC string \NC a \LUA\ string \NC \NR

+\NC                \NC table  \NC a \LUA\ table \NC \NR

+\LL

+\stoptabulate

+

+The \type {type} can have one of six distinct values. The number is the \ASCII\

+value if the first character of the type name (so you can use string.byte("l")

+instead of \type {108}).

+

+\starttabulate[|r|c|p|]

+\DB value \BC meaning \BC explanation \NC \NR

+\TB

+\NC   97  \NC a       \NC list of attributes (a node list) \NC \NR

+\NC  100  \NC d       \NC a \LUA\ number \NC \NR

+\NC  108  \NC l       \NC a \LUA\ value (table, number, boolean, etc) \NC \NR

+\NC  110  \NC n       \NC a node list \NC \NR

+\NC  115  \NC s       \NC a \LUA\ string \NC \NR

+\NC  116  \NC t       \NC a \LUA\ token list in \LUA\ table form (a list of triplets) \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {save_pos}}

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type \BC explanation \NC \NR

+\TB

+\NC \type{attr} \NC node \NC list of attributes \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {late_lua}}

+

+\starttabulate[|l|l|p|]

+\DB field        \BC type               \BC explanation \NC \NR

+\TB

+\NC \type{attr}  \NC node               \NC list of attributes \NC \NR

+\NC \type{data}  \NC string or function \NC the to be written information stored as \LUA\ value \NC \NR

+\NC \type{token} \NC string             \NC the to be written information stored as token list \NC \NR

+\NC \type{name}  \NC string             \NC the name to use for \LUA\ error reporting \NC \NR

+\LL

+\stoptabulate

+

+The difference between \type {data} and \type {string} is that on assignment, the

+\type {data} field is converted to a token list, cf.\ use as \lpr {latelua}. The

+\type {string} version is treated as a literal string.

+

+\stopsection

+

+\startsection[title={\DVI\ backend whatsits}]

+

+\subsection{\whs {special}}

+

+There is only one \DVI\ backend whatsit, and it just flushes its content to the

+output file.

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr} \NC node   \NC list of attributes \NC \NR

+\NC \type{data} \NC string \NC the \prm {special} information \NC \NR

+\LL

+\stoptabulate

+

+\stopsection

+

+\startsection[title={\PDF\ backend whatsits}]

+

+\subsection{\whs {pdf_literal}}

+

+\starttabulate[|l|l|p|]

+\DB field        \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}  \NC node   \NC list of attributes \NC \NR

+\NC \type{mode}  \NC number \NC the \quote {mode} setting of this literal \NC \NR

+\NC \type{data}  \NC string \NC the to be written information stored as \LUA\ string \NC \NR

+\NC \type{token} \NC string \NC the to be written information stored as token list \NC \NR

+\LL

+\stoptabulate

+

+Possible mode values are:

+

+\starttabulate[|c|p|]

+\DB value \BC keyword        \NC \NR

+\TB

+\NC 0     \NC \type{origin}  \NC \NR

+\NC 1     \NC \type{page}    \NC \NR

+\NC 2     \NC \type{direct}  \NC \NR

+\NC 3     \NC \type{raw}     \NC \NR

+\NC 4     \NC \type{text}    \NC \NR

+\LL

+\stoptabulate

+

+The higher the number, the less checking and the more you can run into trouble.

+Especially the \type {raw} variant can produce bad \PDF\ so you can best check

+what you generate.

+

+\subsection{\whs {pdf_refobj}}

+

+\starttabulate[|l|l|p|]

+\DB field         \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}   \NC node   \NC list of attributes \NC \NR

+\NC \type{objnum} \NC number \NC the referenced \PDF\ object number \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_annot}}

+

+\starttabulate[|l|l|p|]

+\DB field         \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}   \NC node   \NC list of attributes \NC \NR

+\NC \type{width}  \NC number \NC the width (not used in calculations) \NC \NR

+\NC \type{height} \NC number \NC the height (not used in calculations) \NC \NR

+\NC \type{depth}  \NC number \NC the depth (not used in calculations) \NC \NR

+\NC \type{objnum} \NC number \NC the referenced \PDF\ object number \NC \NR

+\NC \type{data}   \NC string \NC the annotation data \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_start_link}}

+

+\starttabulate[|l|l|p|]

+\DB field            \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}      \NC node   \NC list of attributes \NC \NR

+\NC \type{width}     \NC number \NC the width (not used in calculations) \NC \NR

+\NC \type{height}    \NC number \NC the height (not used in calculations) \NC \NR

+\NC \type{depth}     \NC number \NC the depth (not used in calculations) \NC \NR

+\NC \type{objnum}    \NC number \NC the referenced \PDF\ object number \NC \NR

+\NC \type{link_attr} \NC table  \NC the link attribute token list \NC \NR

+\NC \type{action}    \NC node   \NC the action to perform \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_end_link}}

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type \BC explanation \NC \NR

+\TB

+\NC \type{attr} \NC node \NC \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_dest}}

+

+\starttabulate[|l|l|p|]

+\DB field              \BC type     \BC explanation \NC \NR

+\TB

+\NC \type{attr}        \NC node     \NC list of attributes \NC \NR

+\NC \type{width}       \NC number   \NC the width (not used in calculations) \NC \NR

+\NC \type{height}      \NC number   \NC the height (not used in calculations) \NC \NR

+\NC \type{depth}       \NC number   \NC the depth (not used in calculations) \NC \NR

+\NC \type{named_id}    \NC number   \NC is the \type {dest_id} a string value? \NC \NR

+\NC \type{dest_id}     \NC number   \NC the destination id \NC \NR

+\NC                    \NC string   \NC the destination name \NC \NR

+\NC \type{dest_type}   \NC number   \NC type of destination \NC \NR

+\NC \type{xyz_zoom}    \NC number   \NC the zoom factor (times 1000) \NC \NR

+\NC \type{objnum}      \NC number   \NC the \PDF\ object number \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_action}}

+

+These are a special kind of items that only appear inside \PDF\ start link

+objects.

+

+\starttabulate[|l|l|p|]

+\DB field              \BC type             \BC explanation \NC \NR

+\TB

+\NC \type{action_type} \NC number           \NC the kind of action involved \NC \NR

+\NC \type{action_id}   \NC number or string \NC token list reference or string \NC \NR

+\NC \type{named_id}    \NC number           \NC the index of the destination \NC \NR

+\NC \type{file}        \NC string           \NC the target filename \NC \NR

+\NC \type{new_window}  \NC number           \NC the window state of the target \NC \NR

+\NC \type{data}        \NC string           \NC the name of the destination \NC \NR

+\LL

+\stoptabulate

+

+Valid action types are:

+

+\starttabulate[|l|l|]

+\DB value \BC meaning       \NC \NR

+\TB

+\NC 0     \NC \type{page}   \NC \NR

+\NC 1     \NC \type{goto}   \NC \NR

+\NC 2     \NC \type{thread} \NC \NR

+\NC 3     \NC \type{user}   \NC \NR

+\LL

+\stoptabulate

+

+Valid window types are:

+

+\starttabulate[|l|l|]

+\DB value \BC meaning       \NC \NR

+\TB

+\NC 0     \NC \type{notset} \NC \NR

+\NC 1     \NC \type{new}    \NC \NR

+\NC 2     \NC \type{nonew}  \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_thread}}

+

+\starttabulate[|l|l|p|]

+\DB field              \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}        \NC node   \NC list of attributes \NC \NR

+\NC \type{width}       \NC number \NC the width (not used in calculations) \NC \NR

+\NC \type{height}      \NC number \NC the height (not used in calculations) \NC \NR

+\NC \type{depth}       \NC number \NC the depth (not used in calculations) \NC \NR

+\NC \type{named_id}    \NC number \NC is \type {tread_id} a string value? \NC \NR

+\NC \type{tread_id}    \NC number \NC the thread id \NC \NR

+\NC                    \NC string \NC the thread name \NC \NR

+\NC \type{thread_attr} \NC number \NC extra thread information \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_start_thread}}

+

+\starttabulate[|l|l|p|]

+\DB field              \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}        \NC node   \NC list of attributes \NC \NR

+\NC \type{width}       \NC number \NC the width (not used in calculations) \NC \NR

+\NC \type{height}      \NC number \NC the height (not used in calculations) \NC \NR

+\NC \type{depth}       \NC number \NC the depth (not used in calculations) \NC \NR

+\NC \type{named_id}    \NC number \NC is \type {tread_id} a string value? \NC \NR

+\NC \type{tread_id}    \NC number \NC the thread id \NC \NR

+\NC                    \NC string \NC the thread name \NC \NR

+\NC \type{thread_attr} \NC number \NC extra thread information \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_end_thread}}

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type \BC explanation \NC \NR

+\TB

+\NC \type{attr} \NC node \NC \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_colorstack}}

+

+\starttabulate[|l|l|p|]

+\DB field          \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr}    \NC node   \NC list of attributes \NC \NR

+\NC \type{stack}   \NC number \NC colorstack id number \NC \NR

+\NC \type{command} \NC number \NC command to execute \NC \NR

+\NC \type{data}    \NC string \NC data \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_setmatrix}}

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{attr} \NC node   \NC list of attributes \NC \NR

+\NC \type{data} \NC string \NC data \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_save}}

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type \BC explanation \NC \NR

+\TB

+\NC \type{attr} \NC node \NC list of attributes \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\whs {pdf_restore}}

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type \BC explanation \NC \NR

+\TB

+\NC \type{attr} \NC node \NC list of attributes \NC \NR

+\LL

+\stoptabulate

+

+\stopsection

+

+\startsection[title={The \type {node} library}][library=node]

+

+\subsection {Introduction}

+

+The \type {node} library contains functions that facilitate dealing with (lists

+of) nodes and their values. They allow you to create, alter, copy, delete, and

+insert \LUATEX\ node objects, the core objects within the typesetter.

+

+\LUATEX\ nodes are represented in \LUA\ as userdata with the metadata type

+\type {luatex.node}. The various parts within a node can be accessed using

+named fields.

+

+Each node has at least the three fields \type {next}, \type {id}, and \type {subtype}:

+

+\startitemize[intro]

+

+\startitem

+    The \type {next} field returns the userdata object for the next node in a

+    linked list of nodes, or \type {nil}, if there is no next node.

+\stopitem

+

+\startitem

+    The \type {id} indicates \TEX's \quote{node type}. The field \type {id} has a

+    numeric value for efficiency reasons, but some of the library functions also

+    accept a string value instead of \type {id}.

+\stopitem

+

+\startitem

+    The \type {subtype} is another number. It often gives further information

+    about a node of a particular \type {id}, but it is most important when

+    dealing with \quote {whatsits}, because they are differentiated solely based

+    on their \type {subtype}.

+\stopitem

+

+\stopitemize

+

+The other available fields depend on the \type {id} (and for \quote {whatsits},

+the \type {subtype}) of the node.

+

+Support for \nod {unset} (alignment) nodes is partial: they can be queried and

+modified from \LUA\ code, but not created.

+

+Nodes can be compared to each other, but: you are actually comparing indices into

+the node memory. This means that equality tests can only be trusted under very

+limited conditions. It will not work correctly in any situation where one of the

+two nodes has been freed and|/|or reallocated: in that case, there will be false

+positives.

+

+At the moment, memory management of nodes should still be done explicitly by the

+user. Nodes are not \quote {seen} by the \LUA\ garbage collector, so you have to

+call the node freeing functions yourself when you are no longer in need of a node

+(list). Nodes form linked lists without reference counting, so you have to be

+careful that when control returns back to \LUATEX\ itself, you have not deleted

+nodes that are still referenced from a \type {next} pointer elsewhere, and that

+you did not create nodes that are referenced more than once. Normally the setters

+and getters handle this for you.

+

+There are statistics available with regards to the allocated node memory, which

+can be handy for tracing.

+

+\subsection{\type {is_node}}

+

+\topicindex {nodes+functions}

+

+\libindex {is_node}

+

+\startfunctioncall

+<boolean|integer> t =

+    node.is_node(<any> item)

+\stopfunctioncall

+

+This function returns a number (the internal index of the node) if the argument

+is a userdata object of type \type {<node>} and false when no node is passed.

+

+\subsection{\type {types} and \type {whatsits}}

+

+\libindex {types}

+\libindex {whatsits}

+

+This function returns an array that maps node id numbers to node type strings,

+providing an overview of the possible top|-|level \type {id} types.

+

+\startfunctioncall

+<table> t =

+    node.types()

+\stopfunctioncall

+

+\TEX's \quote {whatsits} all have the same \type {id}. The various subtypes are

+defined by their \type {subtype} fields. The function is much like \type {types},

+except that it provides an array of \type {subtype} mappings.

+

+\startfunctioncall

+<table> t =

+    node.whatsits()

+\stopfunctioncall

+

+\subsection{\type {id}}

+

+\libindex{id}

+

+This converts a single type name to its internal numeric representation.

+

+\startfunctioncall

+<number> id =

+    node.id(<string> type)

+\stopfunctioncall

+

+\subsection{\type {type} and \type {subtype}}

+

+\libindex {type}

+\libindex {subtype}

+

+In the argument is a number, then the next function converts an internal numeric

+representation to an external string representation. Otherwise, it will return

+the string \type {node} if the object represents a node, and \type {nil}

+otherwise.

+

+\startfunctioncall

+<string> type =

+    node.type(<any> n)

+\stopfunctioncall

+

+This next one converts a single whatsit name to its internal numeric

+representation (\type {subtype}).

+

+\startfunctioncall

+<number> subtype =

+    node.subtype(<string> type)

+\stopfunctioncall

+

+\subsection{\type {fields}}

+

+\libindex {fields}

+

+This function returns an array of valid field names for a particular type of

+node. If you want to get the valid fields for a \quote {whatsit}, you have to

+supply the second argument also. In other cases, any given second argument will

+be silently ignored.

+

+\startfunctioncall

+<table> t =

+    node.fields(<number> id)

+<table> t =

+    node.fields(<number> id, <number> subtype)

+\stopfunctioncall

+

+The function accepts string \type {id} and \type {subtype} values as well.

+

+\subsection{\type {has_field}}

+

+\libindex {has_field}

+

+This function returns a boolean that is only true if \type {n} is

+actually a node, and it has the field.

+

+\startfunctioncall

+<boolean> t =

+    node.has_field(<node> n, <string> field)

+\stopfunctioncall

+

+\subsection{\type {new}}

+

+\libindex{new}

+

+The \type {new} function creates a new node. All its fields are initialized to

+either zero or \type {nil} except for \type {id} and \type {subtype}. Instead of

+numbers you can also use strings (names). If you create a new \nod {whatsit} node

+the second argument is required. As with all node functions, this function

+creates a node at the \TEX\ level.

+

+\startfunctioncall

+<node> n =

+    node.new(<number> id)

+<node> n =

+    node.new(<number> id, <number> subtype)

+\stopfunctioncall

+

+\subsection{\type {free}, \type {flush_node} and \type {flush_list}}

+

+\libindex{free}

+\libindex{flush_node}

+\libindex{flush_list}

+

+The next one the node \type {n} from \TEX's memory. Be careful: no checks are

+done on whether this node is still pointed to from a register or some \type

+{next} field: it is up to you to make sure that the internal data structures

+remain correct.

+

+\startfunctioncall

+<node> next =

+    node.free(<node> n)

+flush_node(<node> n)

+\stopfunctioncall

+

+The \type {free} function returns the next field of the freed node, while the

+\type {flush_node} alternative returns nothing.

+

+A list starting with node \type {n} can be flushed from \TEX's memory too. Be

+careful: no checks are done on whether any of these nodes is still pointed to

+from a register or some \type {next} field: it is up to you to make sure that the

+internal data structures remain correct.

+

+\startfunctioncall

+node.flush_list(<node> n)

+\stopfunctioncall

+

+\subsection{\type {copy} and \type {copy_list}}

+

+\libindex{copy}

+\libindex{copy_list}

+

+This creates a deep copy of node \type {n}, including all nested lists as in the case

+of a hlist or vlist node. Only the \type {next} field is not copied.

+

+\startfunctioncall

+<node> m =

+    node.copy(<node> n)

+\stopfunctioncall

+

+A deep copy of the node list that starts at \type {n} can be created too. If

+\type {m} is also given, the copy stops just before node \type {m}.

+

+\startfunctioncall

+<node> m =

+    node.copy_list(<node> n)

+<node> m =

+    node.copy_list(<node> n, <node> m)

+\stopfunctioncall

+

+Note that you cannot copy attribute lists this way. However, there is normally no

+need to copy attribute lists as when you do assignments to the \type {attr} field

+or make changes to specific attributes, the needed copying and freeing takes

+place automatically.

+

+\subsection{\type {prev} and \type{next}}

+

+\libindex{prev}

+\libindex{next}

+

+These returns the node preceding or following the given node, or \type {nil} if

+there is no such node.

+

+\startfunctioncall

+<node> m =

+    node.next(<node> n)

+<node> m =

+    node.prev(<node> n)

+\stopfunctioncall

+

+\subsection{\type {current_attr}}

+

+\libindex{current_attr}

+

+This returns the currently active list of attributes, if there is one.

+

+\startfunctioncall

+<node> m =

+    node.current_attr()

+\stopfunctioncall

+

+The intended usage of \type {current_attr} is as follows:

+

+\starttyping

+local x1 = node.new("glyph")

+x1.attr = node.current_attr()

+local x2 = node.new("glyph")

+x2.attr = node.current_attr()

+\stoptyping

+

+or:

+

+\starttyping

+local x1 = node.new("glyph")

+local x2 = node.new("glyph")

+local ca = node.current_attr()

+x1.attr = ca

+x2.attr = ca

+\stoptyping

+

+The attribute lists are ref counted and the assignment takes care of incrementing

+the refcount. You cannot expect the value \type {ca} to be valid any more when

+you assign attributes (using \type {tex.setattribute}) or when control has been

+passed back to \TEX.

+

+Note: this function is somewhat experimental, and it returns the {\it actual}

+attribute list, not a copy thereof. Therefore, changing any of the attributes in

+the list will change these values for all nodes that have the current attribute

+list assigned to them.

+

+\subsection{\type {hpack}}

+

+\libindex {hpack}

+

+This function creates a new hlist by packaging the list that begins at node \type

+{n} into a horizontal box. With only a single argument, this box is created using

+the natural width of its components. In the three argument form, \type {info}

+must be either \type {additional} or \type {exactly}, and \type {w} is the

+additional (\type {\hbox spread}) or exact (\type {\hbox to}) width to be used.

+The second return value is the badness of the generated box.

+

+\startfunctioncall

+<node> h, <number> b =

+    node.hpack(<node> n)

+<node> h, <number> b =

+    node.hpack(<node> n, <number> w, <string> info)

+<node> h, <number> b =

+    node.hpack(<node> n, <number> w, <string> info, <string> dir)

+\stopfunctioncall

+

+Caveat: there can be unexpected side|-|effects to this function, like updating

+some of the \prm {marks} and \type {\inserts}. Also note that the content of

+\type {h} is the original node list \type {n}: if you call \type {node.free(h)}

+you will also free the node list itself, unless you explicitly set the \type

+{list} field to \type {nil} beforehand. And in a similar way, calling \type

+{node.free(n)} will invalidate \type {h} as well!

+

+\subsection{\type {vpack}}

+

+\libindex {vpack}

+

+This function creates a new vlist by packaging the list that begins at node \type

+{n} into a vertical box. With only a single argument, this box is created using

+the natural height of its components. In the three argument form, \type {info}

+must be either \type {additional} or \type {exactly}, and \type {w} is the

+additional (\type {\vbox spread}) or exact (\type {\vbox to}) height to be used.

+

+\startfunctioncall

+<node> h, <number> b =

+    node.vpack(<node> n)

+<node> h, <number> b =

+    node.vpack(<node> n, <number> w, <string> info)

+<node> h, <number> b =

+    node.vpack(<node> n, <number> w, <string> info, <string> dir)

+\stopfunctioncall

+

+The second return value is the badness of the generated box. See the description

+of \type {hpack} for a few memory allocation caveats.

+

+\subsection{\type {prepend_prevdepth}}

+

+\libindex {prepend_prevdepth}

+

+This function is somewhat special in the sense that it is an experimental helper

+that adds the interlinespace to a line keeping the baselineskip and lineskip into

+account.

+

+\startfunctioncall

+<node> n, <number> delta =

+    node.prepend_prevdepth(<node> n,<number> prevdepth)

+\stopfunctioncall

+

+\subsection{\type {dimensions} and \type {rangedimensions}}

+

+\libindex{dimensions}

+\libindex{rangedimensions}

+

+\startfunctioncall

+<number> w, <number> h, <number> d  =

+    node.dimensions(<node> n)

+<number> w, <number> h, <number> d  =

+    node.dimensions(<node> n, <string> dir)

+<number> w, <number> h, <number> d  =

+    node.dimensions(<node> n, <node> t)

+<number> w, <number> h, <number> d  =

+    node.dimensions(<node> n, <node> t, <string> dir)

+\stopfunctioncall

+

+This function calculates the natural in|-|line dimensions of the node list starting

+at node \type {n} and terminating just before node \type {t} (or the end of the

+list, if there is no second argument). The return values are scaled points. An

+alternative format that starts with glue parameters as the first three arguments

+is also possible:

+

+\startfunctioncall

+<number> w, <number> h, <number> d  =

+    node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,

+        <node> n)

+<number> w, <number> h, <number> d  =

+    node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,

+        <node> n, <string> dir)

+<number> w, <number> h, <number> d  =

+    node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,

+        <node> n, <node> t)

+<number> w, <number> h, <number> d  =

+    node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,

+        <node> n, <node> t, <string> dir)

+\stopfunctioncall

+

+This calling method takes glue settings into account and is especially useful for

+finding the actual width of a sublist of nodes that are already boxed, for

+example in code like this, which prints the width of the space in between the

+\type {a} and \type {b} as it would be if \type {\box0} was used as-is:

+

+\starttyping

+\setbox0 = \hbox to 20pt {a b}

+

+\directlua{print (node.dimensions(

+    tex.box[0].glue_set,

+    tex.box[0].glue_sign,

+    tex.box[0].glue_order,

+    tex.box[0].head.next,

+    node.tail(tex.box[0].head)

+)) }

+\stoptyping

+

+You need to keep in mind that this is one of the few places in \TEX\ where floats

+are used, which means that you can get small differences in rounding when you

+compare the width reported by \type {hpack} with \type {dimensions}.

+

+The second alternative saves a few lookups and can be more convenient in some

+cases:

+

+\startfunctioncall

+<number> w, <number> h, <number> d  =

+    node.rangedimensions(<node> parent, <node> first)

+<number> w, <number> h, <number> d  =

+    node.rangedimensions(<node> parent, <node> first, <node> last)

+\stopfunctioncall

+

+\subsection{\type {mlist_to_hlist}}

+

+\libindex {mlist_to_hlist}

+

+\startfunctioncall

+<node> h =

+    node.mlist_to_hlist(<node> n, <string> display_type, <boolean> penalties)

+\stopfunctioncall

+

+This runs the internal mlist to hlist conversion, converting the math list in

+\type {n} into the horizontal list \type {h}. The interface is exactly the same

+as for the callback \cbk {mlist_to_hlist}.

+

+\subsection{\type {slide}}

+

+\startfunctioncall

+<node> m =

+    node.slide(<node> n)

+\stopfunctioncall

+

+Returns the last node of the node list that starts at \type {n}. As a

+side|-|effect, it also creates a reverse chain of \type {prev} pointers between

+nodes.

+

+\subsection{\type {tail}}

+

+\libindex {tail}

+

+\startfunctioncall

+<node> m =

+    node.tail(<node> n)

+\stopfunctioncall

+

+Returns the last node of the node list that starts at \type {n}.

+

+\subsection{\type {length} and type {count}}

+

+\libindex {length}

+\libindex {count}

+

+\startfunctioncall

+<number> i =

+    node.length(<node> n)

+<number> i =

+    node.length(<node> n, <node> m)

+\stopfunctioncall

+

+Returns the number of nodes contained in the node list that starts at \type {n}.

+If \type {m} is also supplied it stops at \type {m} instead of at the end of the

+list. The node \type {m} is not counted.

+

+\startfunctioncall

+<number> i =

+    node.count(<number> id, <node> n)

+<number> i =

+    node.count(<number> id, <node> n, <node> m)

+\stopfunctioncall

+

+Returns the number of nodes contained in the node list that starts at \type {n}

+that have a matching \type {id} field. If \type {m} is also supplied, counting

+stops at \type {m} instead of at the end of the list. The node \type {m} is not

+counted. This function also accept string \type {id}'s.

+

+\subsection{\type {is_char} and \type {is_glyph}}

+

+\libindex {is_char}

+\libindex {is_glyph}

+

+The subtype of a glyph node signals if the glyph is already turned into a character reference

+or not.

+

+\startfunctioncall

+<boolean> b =

+    node.is_char(<node> n)

+<boolean> b =

+    node.is_glyph(<node> n)

+\stopfunctioncall

+

+\subsection{\type {traverse}}

+

+\libindex {traverse}

+

+\startfunctioncall

+<node> t, id, subtype =

+    node.traverse(<node> n)

+\stopfunctioncall

+

+This is a \LUA\ iterator that loops over the node list that starts at \type {n}.

+Typically code looks like this:

+

+\starttyping

+for n in node.traverse(head) do

+   ...

+end

+\stoptyping

+

+is functionally equivalent to:

+

+\starttyping

+do

+  local n

+  local function f (head,var)

+    local t

+    if var == nil then

+       t = head

+    else

+       t = var.next

+    end

+    return t

+  end

+  while true do

+    n = f (head, n)

+    if n == nil then break end

+    ...

+  end

+end

+\stoptyping

+

+It should be clear from the definition of the function \type {f} that even though

+it is possible to add or remove nodes from the node list while traversing, you

+have to take great care to make sure all the \type {next} (and \type {prev})

+pointers remain valid.

+

+If the above is unclear to you, see the section \quote {For Statement} in the

+\LUA\ Reference Manual.

+

+\subsection{\type {traverse_id}}

+

+\libindex {traverse_id}

+

+\startfunctioncall

+<node> t, subtype =

+    node.traverse_id(<number> id, <node> n)

+\stopfunctioncall

+

+This is an iterator that loops over all the nodes in the list that starts at

+\type {n} that have a matching \type {id} field.

+

+See the previous section for details. The change is in the local function \type

+{f}, which now does an extra while loop checking against the upvalue \type {id}:

+

+\starttyping

+ local function f(head,var)

+   local t

+   if var == nil then

+      t = head

+   else

+      t = var.next

+   end

+   while not t.id == id do

+      t = t.next

+   end

+   return t

+ end

+\stoptyping

+

+\subsection{\type {traverse_char} and \type {traverse_glyph}}

+

+\libindex {traverse_char}

+\libindex {traverse_glyph}

+

+The \type{traverse_char} iterator loops over the \nod {glyph} nodes in a list.

+Only nodes with a subtype less than 256 are seen.

+

+\startfunctioncall

+<node> n, font, char =

+    node.traverse_char(<node> n)

+\stopfunctioncall

+

+The \type{traverse_glyph} iterator loops over a list and returns the list and

+filters all glyphs:

+

+\startfunctioncall

+<node> n, font, char =

+    node.traverse_glyph(<node> n)

+\stopfunctioncall

+

+\subsection{\type {traverse_list}}

+

+\libindex {traverse_list}

+

+This iterator loops over the \nod {hlist} and \nod {vlist} nodes in a list.

+

+\startfunctioncall

+<node> n, id, subtype, list =

+    node.traverse_list(<node> n)

+\stopfunctioncall

+

+The four return values can save some time compared to fetching these fields but

+in practice you seldom need them all. So consider it a (side effect of

+experimental) convenience.

+

+\subsection{\type {has_glyph}}

+

+\libindex {has_glyph}

+

+This function returns the first glyph or disc node in the given list:

+

+\startfunctioncall

+<node> n =

+    node.has_glyph(<node> n)

+\stopfunctioncall

+

+\subsection{\type {end_of_math}}

+

+\libindex {end_of_math}

+

+\startfunctioncall

+<node> t =

+    node.end_of_math(<node> start)

+\stopfunctioncall

+

+Looks for and returns the next \type {math_node} following the \type {start}. If

+the given node is a math end node this helper returns that node, else it follows

+the list and returns the next math endnote. If no such node is found nil is

+returned.

+

+\subsection{\type {remove}}

+

+\libindex {remove}

+

+\startfunctioncall

+<node> head, current =

+    node.remove(<node> head, <node> current)

+\stopfunctioncall

+

+This function removes the node \type {current} from the list following \type

+{head}. It is your responsibility to make sure it is really part of that list.

+The return values are the new \type {head} and \type {current} nodes. The

+returned \type {current} is the node following the \type {current} in the calling

+argument, and is only passed back as a convenience (or \type {nil}, if there is

+no such node). The returned \type {head} is more important, because if the

+function is called with \type {current} equal to \type {head}, it will be

+changed.

+

+\subsection{\type {insert_before}}

+

+\libindex {insert_before}

+

+\startfunctioncall

+<node> head, new =

+    node.insert_before(<node> head, <node> current, <node> new)

+\stopfunctioncall

+

+This function inserts the node \type {new} before \type {current} into the list

+following \type {head}. It is your responsibility to make sure that \type

+{current} is really part of that list. The return values are the (potentially

+mutated) \type {head} and the node \type {new}, set up to be part of the list

+(with correct \type {next} field). If \type {head} is initially \type {nil}, it

+will become \type {new}.

+

+\subsection{\type {insert_after}}

+

+\libindex {insert_after}

+

+\startfunctioncall

+<node> head, new =

+    node.insert_after(<node> head, <node> current, <node> new)

+\stopfunctioncall

+

+This function inserts the node \type {new} after \type {current} into the list

+following \type {head}. It is your responsibility to make sure that \type

+{current} is really part of that list. The return values are the \type {head} and

+the node \type {new}, set up to be part of the list (with correct \type {next}

+field). If \type {head} is initially \type {nil}, it will become \type {new}.

+

+\subsection{\type {first_glyph}}

+

+\libindex {first_glyph}

+

+\startfunctioncall

+<node> n =

+    node.first_glyph(<node> n)

+<node> n =

+    node.first_glyph(<node> n, <node> m)

+\stopfunctioncall

+

+Returns the first node in the list starting at \type {n} that is a glyph node

+with a subtype indicating it is a glyph, or \type {nil}. If \type {m} is given,

+processing stops at (but including) that node, otherwise processing stops at the

+end of the list.

+

+\subsection{\type {ligaturing}}

+

+\libindex {ligaturing}

+

+\startfunctioncall

+<node> h, <node> t, <boolean> success =

+    node.ligaturing(<node> n)

+<node> h, <node> t, <boolean> success =

+    node.ligaturing(<node> n, <node> m)

+\stopfunctioncall

+

+Apply \TEX-style ligaturing to the specified nodelist. The tail node \type {m} is

+optional. The two returned nodes \type {h} and \type {t} are the new head and

+tail (both \type {n} and \type {m} can change into a new ligature).

+

+\subsection{\type {kerning}}

+

+\libindex {kerning}

+

+\startfunctioncall

+<node> h, <node> t, <boolean> success =

+    node.kerning(<node> n)

+<node> h, <node> t, <boolean> success =

+    node.kerning(<node> n, <node> m)

+\stopfunctioncall

+

+Apply \TEX|-|style kerning to the specified node list. The tail node \type {m} is

+optional. The two returned nodes \type {h} and \type {t} are the head and tail

+(either one of these can be an inserted kern node, because special kernings with

+word boundaries are possible).

+

+\subsection{\type {unprotect_glyph[s]}}

+

+\libindex {unprotect_glyphs}

+\libindex {unprotect_glyph}

+

+\startfunctioncall

+node.unprotect_glyph(<node> n)

+node.unprotect_glyphs(<node> n,[<node> n])

+\stopfunctioncall

+

+Subtracts 256 from all glyph node subtypes. This and the next function are

+helpers to convert from \type {characters} to \type {glyphs} during node

+processing. The second argument is optional and indicates the end of a range.

+

+\subsection{\type {protect_glyph[s]}}

+

+\libindex {protect_glyphs}

+\libindex {protect_glyph}

+

+\startfunctioncall

+node.protect_glyph(<node> n)

+node.protect_glyphs(<node> n,[<node> n])

+\stopfunctioncall

+

+Adds 256 to all glyph node subtypes in the node list starting at \type {n},

+except that if the value is 1, it adds only 255. The special handling of 1 means

+that \type {characters} will become \type {glyphs} after subtraction of 256. A

+single character can be marked by the singular call. The second argument is

+optional and indicates the end of a range.

+

+\subsection{\type {last_node}}

+

+\libindex {last_node}

+

+\startfunctioncall

+<node> n =

+    node.last_node()

+\stopfunctioncall

+

+This function pops the last node from \TEX's \quote{current list}. It returns

+that node, or \type {nil} if the current list is empty.

+

+\subsection{\type {write}}

+

+\libindex {write}

+

+\startfunctioncall

+node.write(<node> n)

+\stopfunctioncall

+

+This function that will append a node list to \TEX's \quote {current list}. The

+node list is not deep|-|copied! There is no error checking either! You mignt need

+to enforce horizontal mode in order for this to work as expected.

+

+\subsection{\type {protrusion_skippable}}

+

+\libindex {protrusion_skippable}

+

+\startfunctioncall

+<boolean> skippable =

+    node.protrusion_skippable(<node> n)

+\stopfunctioncall

+

+Returns \type {true} if, for the purpose of line boundary discovery when

+character protrusion is active, this node can be skipped.

+

+\stopsection

+

+\startsection[title={Glue handling}][library=node]

+

+\subsection{\type {setglue}}

+

+\libindex {setglue}

+

+You can set the properties of a glue in one go. If you pass no values, the glue

+will become a zero glue.

+

+\startfunctioncall

+node.setglue(<node> n)

+node.setglue(<node> n,width,stretch,shrink,stretch_order,shrink_order)

+\stopfunctioncall

+

+When you pass values, only arguments that are numbers are assigned so

+

+\starttyping

+node.setglue(n,655360,false,65536)

+\stoptyping

+

+will only adapt the width and shrink.

+

+When a list node is passed, you set the glue, order and sign instead.

+

+\subsection{\type {getglue}}

+

+\libindex {getglue}

+

+The next call will return 5 values or nothing when no glue is passed.

+

+\startfunctioncall

+<integer> width, <integer> stretch, <integer> shrink, <integer> stretch_order,

+    <integer> shrink_order = node.getglue(<node> n)

+\stopfunctioncall

+

+When the second argument is false, only the width is returned (this is consistent

+with \type {tex.get}).

+

+When a list node is passed, you get back the glue that is set, the order of that

+glue and the sign.

+

+\subsection{\type {is_zero_glue}}

+

+\libindex {is_zero_glue}

+

+This function returns \type {true} when the width, stretch and shrink properties

+are zero.

+

+\startfunctioncall

+<boolean> isglue =

+    node.is_zero_glue(<node> n)

+\stopfunctioncall

+

+\stopsection

+

+\startsection[title={Attribute handling}][library=node]

+

+\subsection{Attributes}

+

+\topicindex {attributes}

+

+The newly introduced attribute registers are non|-|trivial, because the value

+that is attached to a node is essentially a sparse array of key|-|value pairs. It

+is generally easiest to deal with attribute lists and attributes by using the

+dedicated functions in the \type {node} library, but for completeness, here is

+the low|-|level interface.

+

+Attributes appear as linked list of userdata objects in the \type {attr} field of

+individual nodes. They can be handled individually, but it is much safer and more

+efficient to use the dedicated functions associated with them.

+

+\subsection{\nod {attribute_list} nodes}

+

+\topicindex {nodes+attributes}

+

+An \nod {attribute_list} item is used as a head pointer for a list of attribute

+items. It has only one user-visible field:

+

+\starttabulate[|l|l|p|]

+\DB field       \BC type \BC explanation \NC \NR

+\TB

+\NC \type{next} \NC node \NC pointer to the first attribute \NC \NR

+\LL

+\stoptabulate

+

+\subsection{\nod {attr} nodes}

+

+A normal node's attribute field will point to an item of type \nod

+{attribute_list}, and the \type {next} field in that item will point to the first

+defined \quote {attribute} item, whose \type {next} will point to the second

+\quote {attribute} item, etc.

+

+\starttabulate[|l|l|p|]

+\DB field         \BC type   \BC explanation \NC \NR

+\TB

+\NC \type{next}   \NC node   \NC pointer to the next attribute \NC \NR

+\NC \type{number} \NC number \NC the attribute type id \NC \NR

+\NC \type{value}  \NC number \NC the attribute value \NC \NR

+\LL

+\stoptabulate

+

+As mentioned it's better to use the official helpers rather than edit these

+fields directly. For instance the \type {prev} field is used for other purposes

+and there is no double linked list.

+

+\subsection{\type {has_attribute}}

+

+\libindex {has_attribute}

+

+\startfunctioncall

+<number> v =

+    node.has_attribute(<node> n, <number> id)

+<number> v =

+    node.has_attribute(<node> n, <number> id, <number> val)

+\stopfunctioncall

+

+Tests if a node has the attribute with number \type {id} set. If \type {val} is

+also supplied, also tests if the value matches \type {val}. It returns the value,

+or, if no match is found, \type {nil}.

+

+\subsection{\type {get_attribute}}

+

+\libindex {get_attribute}

+

+\startfunctioncall

+<number> v =

+    node.get_attribute(<node> n, <number> id)

+\stopfunctioncall

+

+Tests if a node has an attribute with number \type {id} set. It returns the

+value, or, if no match is found, \type {nil}. If no \type {id} is given then the

+zero attributes is assumed.

+

+\subsection{\type {find_attribute}}

+

+\libindex {find_attribute}

+

+\startfunctioncall

+<number> v, <node> n =

+    node.find_attribute(<node> n, <number> id)

+\stopfunctioncall

+

+Finds the first node that has attribute with number \type {id} set. It returns

+the value and the node if there is a match and otherwise nothing.

+

+\subsection{\type {set_attribute}}

+

+\libindex {set_attribute}

+

+\startfunctioncall

+node.set_attribute(<node> n, <number> id, <number> val)

+\stopfunctioncall

+

+Sets the attribute with number \type {id} to the value \type {val}. Duplicate

+assignments are ignored.

+

+\subsection{\type {unset_attribute}}

+

+\libindex {unset_attribute}

+

+\startfunctioncall

+<number> v =

+    node.unset_attribute(<node> n, <number> id)

+<number> v =

+    node.unset_attribute(<node> n, <number> id, <number> val)

+\stopfunctioncall

+

+Unsets the attribute with number \type {id}. If \type {val} is also supplied, it

+will only perform this operation if the value matches \type {val}. Missing

+attributes or attribute|-|value pairs are ignored.

+

+If the attribute was actually deleted, returns its old value. Otherwise, returns

+\type {nil}.

+

+\subsection{\type {slide}}

+

+\libindex {slide}

+

+This helper makes sure that the node lists is double linked and returns the found

+tail node.

+

+\startfunctioncall

+<node> tail =

+    node.slide(<node> n)

+\stopfunctioncall

+

+After some callbacks automatic sliding takes place. This feature can be turned

+off with \type {node.fix_node_lists(false)} but you better make sure then that

+you don't mess up lists. In most cases \TEX\ itself only uses \type {next}

+pointers but your other callbacks might expect proper \type {prev} pointers too.

+Future versions of \LUATEX\ can add more checking but this will not influence

+usage.

+

+\subsection{\type {check_discretionary}, \type {check_discretionaries}}

+

+\libindex{check_discretionary}

+\libindex{check_discretionaries}

+

+When you fool around with disc nodes you need to be aware of the fact that they

+have a special internal data structure. As long as you reassign the fields when

+you have extended the lists it's ok because then the tail pointers get updated,

+but when you add to list without reassigning you might end up in trouble when

+the linebreak routine kicks in. You can call this function to check the list for

+issues with disc nodes.

+

+\startfunctioncall

+node.check_discretionary(<node> n)

+node.check_discretionaries(<node> head)

+\stopfunctioncall

+

+The plural variant runs over all disc nodes in a list, the singular variant

+checks one node only (it also checks if the node is a disc node).

+

+\subsection{\type {flatten_discretionaries}}

+

+\libindex {flatten_discretionaries}

+

+This function will remove the discretionaries in the list and inject the replace

+field when set.

+

+\startfunctioncall

+<node> head, count = node.flatten_discretionaries(<node> n)

+\stopfunctioncall

+

+\subsection{\type {family_font}}

+

+\libindex {family_font}

+

+When you pass a proper family identifier the next helper will return the font

+currently associated with it. You can normally also access the font with the

+normal font field or getter because it will resolve the family automatically for

+noads.

+

+\startfunctioncall

+<integer> id =

+    node.family_font(<integer> fam)

+\stopfunctioncall

+

+\stopsection

+

+\startsection[title={Two access models}][library=node]

+

+\topicindex{nodes+direct}

+\topicindex{direct nodes}

+

+\libindex {todirect}

+\libindex {tonode}

+\libindex {tostring}

+

+Deep down in \TEX\ a node has a number which is a numeric entry in a memory

+table. In fact, this model, where \TEX\ manages memory is real fast and one of

+the reasons why plugging in callbacks that operate on nodes is quite fast too.

+Each node gets a number that is in fact an index in the memory table and that

+number often is reported when you print node related information. You go from

+userdata nodes and there numeric references and back with:

+

+\startfunctioncall

+<integer> d = node.todirect(<node> n))

+<node> n = node.tonode(<integer> d))

+\stopfunctioncall

+

+The userdata model is rather robust as it is a virtual interface with some

+additional checking while the more direct access which uses the node numbers

+directly. However, even with userdata you can get into troubles when you free

+nodes that are no longer allocated or mess up lists. if you apply \type

+{tostring} to a node you see its internal (direct) number and id.

+

+The first model provides key based access while the second always accesses fields

+via functions:

+

+\starttyping

+nodeobject.char

+getfield(nodenumber,"char")

+\stoptyping

+

+If you use the direct model, even if you know that you deal with numbers, you

+should not depend on that property but treat it as an abstraction just like

+traditional nodes. In fact, the fact that we use a simple basic datatype has the

+penalty that less checking can be done, but less checking is also the reason why

+it's somewhat faster. An important aspect is that one cannot mix both methods,

+but you can cast both models. So, multiplying a node number makes no sense.

+

+So our advice is: use the indexed (table) approach when possible and investigate

+the direct one when speed might be a real issue. For that reason \LUATEX\ also

+provide the \type {get*} and \type {set*} functions in the top level node

+namespace. There is a limited set of getters. When implementing this direct

+approach the regular index by key variant was also optimized, so direct access

+only makes sense when nodes are accessed millions of times (which happens in some

+font processing for instance).

+

+We're talking mostly of getters because setters are less important. Documents

+have not that many content related nodes and setting many thousands of properties

+is hardly a burden contrary to millions of consultations.

+

+Normally you will access nodes like this:

+

+\starttyping

+local next = current.next

+if next then

+    -- do something

+end

+\stoptyping

+

+Here \type {next} is not a real field, but a virtual one. Accessing it results in

+a metatable method being called. In practice it boils down to looking up the node

+type and based on the node type checking for the field name. In a worst case you

+have a node type that sits at the end of the lookup list and a field that is last

+in the lookup chain. However, in successive versions of \LUATEX\ these lookups

+have been optimized and the most frequently accessed nodes and fields have a

+higher priority.

+

+Because in practice the \type {next} accessor results in a function call, there

+is some overhead involved. The next code does the same and performs a tiny bit

+faster (but not that much because it is still a function call but one that knows

+what to look up).

+

+\starttyping

+local next = node.next(current)

+if next then

+    -- do something

+end

+\stoptyping

+

+Some accessors are used frequently and for these we provide more efficient helpers:

+

+\starttabulate[|l|p|]

+\DB function          \BC explanation \NC \NR

+\TB

+\NC \type{getnext}    \NC parsing nodelist always involves this one \NC \NR

+\NC \type{getprev}    \NC used less but a logical companion to \type {getnext} \NC \NR

+\NC \type{getboth}    \NC returns the next and prev pointer of a node \NC \NR

+\NC \type{getid}      \NC consulted a lot \NC \NR

+\NC \type{getsubtype} \NC consulted less but also a topper \NC \NR

+\NC \type{getfont}    \NC used a lot in \OPENTYPE\ handling (glyph nodes are consulted a lot) \NC \NR

+\NC \type{getchar}    \NC idem and also in other places \NC \NR

+\NC \type{getwhd}     \NC returns the \type {width}, \type {height} and \type {depth} of a list, rule or

+                          (unexpanded) glyph as well as glue (its spec is looked at) and unset nodes\NC \NR

+\NC \type{getdisc}    \NC returns the \type {pre}, \type {post} and \type {replace} fields and

+                          optionally when true is passed also the tail fields \NC \NR

+\NC \type{getlist}    \NC we often parse nested lists so this is a convenient one too \NC \NR

+\NC \type{getleader}  \NC comparable to list, seldom used in \TEX\ (but needs frequent consulting

+                          like lists; leaders could have been made a dedicated node type) \NC \NR

+\NC \type{getfield}   \NC generic getter, sufficient for the rest (other field names are

+                          often shared so a specific getter makes no sense then) \NC \NR

+\NC \type{getbox}     \NC gets the given box (a list node) \NC \NR

+\NC \type{getoffsets} \NC gets the \type {xoffset} and \type {yoffset} of a glyph or

+                          \type {left} and \type {right} values of a rule \NC \NR

+\LL

+\stoptabulate

+

+In the direct namespace there are more such helpers and most of them are

+accompanied by setters. The getters and setters are clever enough to see what

+node is meant. We don't deal with whatsit nodes: their fields are always accessed

+by name. It doesn't make sense to add getters for all fields, we just identifier

+the most likely candidates. In complex documents, many node and fields types

+never get seen, or seen only a few times, but for instance glyphs are candidates

+for such optimization. The \type {node.direct} interface has some more helpers.

+\footnote {We can define the helpers in the node namespace with \type {getfield}

+which is about as efficient, so at some point we might provide that as module.}

+

+The \type {setdisc} helper takes three (optional) arguments plus an optional

+fourth indicating the subtype. Its \type {getdisc} takes an optional boolean;

+when its value is \type {true} the tail nodes will also be returned. The \type

+{setfont} helper takes an optional second argument, it being the character. The

+directmode setter \type {setlink} takes a list of nodes and will link them,

+thereby ignoring \type {nil} entries. The first valid node is returned (beware:

+for good reason it assumes single nodes). For rarely used fields no helpers are

+provided and there are a few that probably are used seldom too but were added for

+consistency. You can of course always define additional accessors using \type

+{getfield} and \type {setfield} with little overhead. When the second argument of

+\type {setattributelist} is \type {true} the current attribute list is assumed.

+

+\def\yes{$+$} \def\nop{$-$}

+

+\def\supported#1#2#3%

+ {\NC \type{#1}

+  \NC \ifx#2\yes\libidx{node}       {#1}\fi #2

+  \NC \ifx#3\yes\libidx{node.direct}{#1}\fi #3 \NC

+  \NR}

+

+\starttabulate[|l|c|c|]

+\DB function \BC node \BC direct \NC \NR

+\TB

+\supported {check_discretionaries}   \yes \yes

+\supported {check_discretionary}     \yes \yes

+\supported {copy_list}               \yes \yes

+\supported {copy}                    \yes \yes

+\supported {count}                   \yes \yes

+\supported {current_attr}            \yes \yes

+\supported {dimensions}              \yes \yes

+\supported {effective_glue}          \yes \yes

+\supported {end_of_math}             \yes \yes

+\supported {family_font}             \yes \nop

+\supported {fields}                  \yes \nop

+\supported {find_attribute}          \yes \yes

+\supported {first_glyph}             \yes \yes

+\supported {flatten_discretionaries} \yes \yes

+\supported {flush_list}              \yes \yes

+\supported {flush_node}              \yes \yes

+\supported {free}                    \yes \yes

+\supported {get_attribute}           \yes \yes

+\supported {get_synctex_fields}      \nop \yes

+\supported {getattributelist}        \nop \yes

+\supported {getboth}                 \yes \yes

+\supported {getbox}                  \nop \yes

+\supported {getchar}                 \yes \yes

+\supported {getcomponents}           \nop \yes

+\supported {getdepth}                \nop \yes

+\supported {getdirection}            \nop \yes

+\supported {getdir}                  \nop \yes

+\supported {getdisc}                 \yes \yes

+\supported {getfam}                  \nop \yes

+\supported {getfield}                \yes \yes

+\supported {getfont}                 \yes \yes

+\supported {getglue}                 \yes \yes

+\supported {getheight}               \nop \yes

+\supported {getid}                   \yes \yes

+\supported {getkern}                 \nop \yes

+\supported {getlang}                 \nop \yes

+\supported {getleader}               \yes \yes

+\supported {getlist}                 \yes \yes

+\supported {getnext}                 \yes \yes

+\supported {getnucleus}              \nop \yes

+\supported {getoffsets}              \nop \yes

+\supported {getpenalty}              \nop \yes

+\supported {getprev}                 \yes \yes

+\supported {getproperty}             \yes \yes

+\supported {getshift}                \nop \yes

+\supported {getsubtype}              \yes \yes

+\supported {getsub}                  \nop \yes

+\supported {getsup}                  \nop \yes

+\supported {getdata}                 \nop \yes

+\supported {getwhd}                  \yes \yes

+\supported {getwidth}                \nop \yes

+\supported {has_attribute}           \yes \yes

+\supported {has_field}               \yes \yes

+\supported {has_glyph}               \yes \yes

+\supported {hpack}                   \yes \yes

+\supported {id}                      \yes \nop

+\supported {insert_after}            \yes \yes

+\supported {insert_before}           \yes \yes

+\supported {is_char}                 \yes \yes

+\supported {is_direct}               \nop \yes

+\supported {is_glyph}                \yes \yes

+\supported {is_node}                 \yes \yes

+\supported {is_zero_glue}            \yes \yes

+\supported {kerning}                 \yes \yes

+\supported {last_node}               \yes \yes

+\supported {length}                  \yes \yes

+\supported {ligaturing}              \yes \yes

+\supported {mlist_to_hlist}          \yes \nop

+\supported {new}                     \yes \yes

+\supported {next}                    \yes \nop

+\supported {prepend_prevdepth}       \nop \yes

+\supported {prev}                    \yes \nop

+\supported {protect_glyphs}          \yes \yes

+\supported {protect_glyph}           \yes \yes

+\supported {protrusion_skippable}    \yes \yes

+\supported {rangedimensions}         \yes \yes

+\supported {remove}                  \yes \yes

+\supported {set_attribute}           \yes \yes

+\supported {set_synctex_fields}      \nop \yes

+\supported {setattributelist}        \nop \yes

+\supported {setboth}                 \nop \yes

+\supported {setbox}                  \nop \yes

+\supported {setchar}                 \nop \yes

+\supported {setcomponents}           \nop \yes

+\supported {setdepth}                \nop \yes

+\supported {setdirection}            \nop \yes

+\supported {setdir}                  \nop \yes

+\supported {setdisc}                 \nop \yes

+\supported {setfam}                  \nop \yes

+\supported {setfield}                \yes \yes

+\supported {setfont}                 \nop \yes

+\supported {setexpansion}            \nop \yes

+\supported {setglue}                 \yes \yes

+\supported {setheight}               \nop \yes

+\supported {setkern}                 \nop \yes

+\supported {setlang}                 \nop \yes

+\supported {setleader}               \nop \yes

+\supported {setlink}                 \nop \yes

+\supported {setlist}                 \nop \yes

+\supported {setnext}                 \nop \yes

+\supported {setnucleus}              \nop \yes

+\supported {setoffsets}              \nop \yes

+\supported {setpenalty}              \nop \yes

+\supported {setprev}                 \nop \yes

+\supported {setproperty}             \yes \yes

+\supported {setshift}                \nop \yes

+\supported {setsplit}                \nop \yes

+\supported {setsubtype}              \nop \yes

+\supported {setsub}                  \nop \yes

+\supported {setsup}                  \nop \yes

+\supported {setwhd}                  \nop \yes

+\supported {setwidth}                \nop \yes

+\supported {slide}                   \yes \yes

+\supported {subtypes}                \yes \nop

+\supported {subtype}                 \yes \nop

+\supported {tail}                    \yes \yes

+\supported {todirect}                \yes \yes

+\supported {tonode}                  \yes \yes

+\supported {tostring}                \yes \yes

+\supported {traverse_char}           \yes \yes

+\supported {traverse_glyph}          \yes \yes

+\supported {traverse_id}             \yes \yes

+\supported {traverse}                \yes \yes

+\supported {types}                   \yes \nop

+\supported {type}                    \yes \nop

+\supported {unprotect_glyphs}        \yes \yes

+\supported {unprotect_glyph}         \yes \yes

+\supported {unset_attribute}         \yes \yes

+\supported {usedlist}                \yes \yes

+\supported {uses_font}               \yes \yes

+\supported {vpack}                   \yes \yes

+\supported {whatsits}                \yes \nop

+\supported {write}                   \yes \yes

+\LL

+\stoptabulate

+

+The \type {node.next} and \type {node.prev} functions will stay but for

+consistency there are variants called \type {getnext} and \type {getprev}. We had

+to use \type {get} because \type {node.id} and \type {node.subtype} are already

+taken for providing meta information about nodes. Note: The getters do only basic

+checking for valid keys. You should just stick to the keys mentioned in the

+sections that describe node properties.

+

+Some of the getters and setters handle multiple node types, given that the field

+is relevant. In that case, some field names are considered similar (like \type

+{kern} and \type {width}, or \type {data} and \type {value}. In retrospect we

+could have normalized field names better but we decided to stick to the original

+(internal) names as much as possible. After all, at the \LUA\ end one can easily

+create synonyms.

+

+Some nodes have indirect references. For instance a math character refers to a

+family instead of a font. In that case we provide a virtual font field as

+accessor. So, \type {getfont} and \type {.font} can be used on them. The same is

+true for the \type {width}, \type {height} and \type {depth} of glue nodes. These

+actually access the spec node properties, and here we can set as well as get the

+values.

+

+In some places \LUATEX\ can do a bit of extra checking for valid node lists and

+you can enable that with:

+

+\startfunctioncall

+node.fix_node_lists(<boolean> b)

+\stopfunctioncall

+

+You can set and query the \SYNCTEX\ fields, a file number aka tag and a line

+number, for a glue, kern, hlist, vlist, rule and math nodes as well as glyph

+nodes (although this last one is not used in native \SYNCTEX).

+

+\startfunctioncall

+node.set_synctex_fields(<integer> f, <integer> l)

+<integer> f, <integer> l =

+    node.get_synctex_fields(<node> n)

+\stopfunctioncall

+

+Of course you need to know what you're doing as no checking on sane values takes

+place. Also, the synctex interpreter used in editors is rather peculiar and has

+some assumptions (heuristics).

+

+\stopsection

+

+\startsection[title={Properties}][library=node]

+

+\topicindex {nodes+properties}

+\topicindex {properties}

+

+\libindex{flush_properties_table}

+\libindex{get_properties_table}

+\libindex{set_properties_mode}

+

+Attributes are a convenient way to relate extra information to a node. You can

+assign them at the \TEX\ end as well as at the \LUA\ end and and consult them at

+the \LUA\ end. One big advantage is that they obey grouping. They are linked

+lists and normally checking for them is pretty efficient, even if you use a lot

+of them. A macro package has to provide some way to manage these attributes at the

+\TEX\ end because otherwise clashes in their usage can occur.

+

+Each node also can have a properties table and you can assign values to this

+table using the \type {setproperty} function and get properties using the \type

+{getproperty} function. Managing properties is way more demanding than managing

+attributes.

+

+Take the following example:

+

+\starttyping

+\directlua {

+    local n = node.new("glyph")

+

+    node.setproperty(n,"foo")

+    print(node.getproperty(n))

+

+    node.setproperty(n,"bar")

+    print(node.getproperty(n))

+

+    node.free(n)

+}

+\stoptyping

+

+This will print \type {foo} and \type {bar} which in itself is not that useful

+when multiple mechanisms want to use this feature. A variant is:

+

+\starttyping

+\directlua {

+    local n = node.new("glyph")

+

+    node.setproperty(n,{ one = "foo", two = "bar" })

+    print(node.getproperty(n).one)

+    print(node.getproperty(n).two)

+

+    node.free(n)

+}

+\stoptyping

+

+This time we store two properties with the node. It really makes sense to have a

+table as property because that way we can store more. But in order for that to

+work well you need to do it this way:

+

+\starttyping

+\directlua {

+    local n = node.new("glyph")

+

+    local t = node.getproperty(n)

+

+    if not t then

+        t = { }

+        node.setproperty(n,t)

+    end

+    t.one = "foo"

+    t.two = "bar"

+

+    print(node.getproperty(n).one)

+    print(node.getproperty(n).two)

+

+    node.free(n)

+}

+\stoptyping

+

+Here our own properties will not overwrite other users properties unless of

+course they use the same keys. So, eventually you will end up with something:

+

+\starttyping

+\directlua {

+    local n = node.new("glyph")

+

+    local t = node.getproperty(n)

+

+    if not t then

+        t = { }

+        node.setproperty(n,t)

+    end

+    t.myself = { one = "foo", two = "bar" }

+

+    print(node.getproperty(n).myself.one)

+    print(node.getproperty(n).myself.two)

+

+    node.free(n)

+}

+\stoptyping

+

+This assumes that only you use \type {myself} as subtable. The possibilities are

+endless but care is needed. For instance, the generic font handler that ships

+with \CONTEXT\ uses the \type {injections} subtable and you should not mess with

+that one!

+

+There are a few helper functions that you normally should not touch as user: \typ

+{flush_properties_table} will wipe the table (normally a bad idea), \typ

+{get_properties_table} and will give the table that stores properties (using

+direct entries) and you can best not mess too much with that one either because

+\LUATEX\ itself will make sure that entries related to nodes will get wiped when

+nodes get freed, so that the \LUA\ garbage collector can do its job. In fact, the

+main reason why we have this mechanism is that it saves the user (or macro

+package) some work. One can easily write a property mechanism in \LUA\ where

+after a shipout properties gets cleaned up but it's not entirely trivial to make

+sure that with each freed node also its properties get freed, due to the fact

+that there can be nodes left over for a next page. And having a callback bound to

+the node deallocator would add way to much overhead.

+

+Managing properties in the node (de)allocator functions is disabled by default

+and is enabled by:

+

+\starttyping

+node.set_properties_mode(true)

+\stoptyping

+

+When we copy a node list that has a table as property, there are several possibilities: we do the same as

+a new node, we copy the entry to the table in properties (a reference), we do

+a deep copy of a table in the properties, we create a new table and give it

+the original one as a metatable. After some experiments (that also included

+timing) with these scenarios we decided that a deep copy made no sense, nor

+did nilling. In the end both the shallow copy and the metatable variant were

+both ok, although the second one is slower. The most important aspect to keep

+in mind is that references to other nodes in properties no longer can be

+valid for that copy. We could use two tables (one unique and one shared) or

+metatables but that only complicates matters.

+

+When defining a new node, we could already allocate a table but it is rather easy

+to do that at the lua end e.g.\ using a metatable \type {__index} method. That

+way it is under macro package control. When deleting a node, we could keep the

+slot (e.g. setting it to false) but it could make memory consumption raise

+unneeded when we have temporary large node lists and after that only small lists.

+Both are not done.

+

+So in the end this is what happens now: when a node is copied, and it has a table

+as property, the new node will share that table. If the second argument of \typ

+{set_properties_mode} is \type {true} then a metatable approach is chosen: the

+copy gets its own table with the original table as metatable. If you use the

+generic font loader the mode is enabled that way.

+

+A few more xperiments were done. For instance: copy attributes to the properties

+so that we have fast access at the \LUA\ end. In the end the overhead is not

+compensated by speed and convenience, in fact, attributes are not that slow when

+it comes to accessing them. So this was rejected.

+

+Another experiment concerned a bitset in the node but again the gain compared to

+attributes was neglectable and given the small amount of available bits it also

+demands a pretty strong agreement over what bit represents what, and this is

+unlikely to succeed in the \TEX\ community. It doesn't pay off.

+

+Just in case one wonders why properties make sense: it is not so much speed that

+we gain, but more convenience: storing all kind of (temporary) data in attributes

+is no fun and this mechanism makes sure that properties are cleaned up when a

+node is freed. Also, the advantage of a more or less global properties table is

+that we stay at the \LUA\ end. An alternative is to store a reference in the node

+itself but that is complicated by the fact that the register has some limitations

+(no numeric keys) and we also don't want to mess with it too much.

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-preamble.tex b/Master/texmf-dist/doc/luatex/base/luatex-preamble.tex
new file mode 100644
index 00000000000..635ff4134ba
--- /dev/null
+++ b/Master/texmf-dist/doc/luatex/base/luatex-preamble.tex
@@ -0,0 +1,110 @@
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-preamble

+

+\startchapter[reference=preamble,title={Preamble}]

+

+\topicindex{nodes}

+\topicindex{boxes}

+\topicindex{\LUA}

+

+This is a reference manual, not a tutorial. This means that we discuss changes

+relative to traditonal \TEX\ and also present new functionality. As a consequence

+we will refer to concepts that we assume to be known or that might be explained

+later.

+

+The average user doesn't need to know much about what is in this manual. For

+instance fonts and languages are normally dealt with in the macro package that

+you use. Messing around with node lists is also often not really needed at the

+user level. If you do mess around, you'd better know what you're dealing with.

+Reading \quotation {The \TEX\ Book} by Donald Knuth is a good investment of time

+then also because it's good to know where it all started. A more summarizing

+overview is given by \quotation {\TEX\ by Topic} by Victor Eijkhout. You might

+want to peek in \quotation {The \ETEX\ manual} and documentation about \PDFTEX.

+

+But \unknown\ if you're here because of \LUA, then all you need to know is that

+you can call it from within a run. The macro package that you use probably will

+provide a few wrapper mechanisms but the basic \lpr {directlua} command that

+does the job is:

+

+\starttyping

+\directlua{tex.print("Hi there")}

+\stoptyping

+

+You can put code between curly braces but if it's a lot you can also put it in a

+file and load that file with the usual \LUA\ commands.

+

+If you still decide to read on, then it's good to know what nodes are, so we do a

+quick introduction here. If you input this text:

+

+\starttyping

+Hi There

+\stoptyping

+

+eventually we will get a linked lists of nodes, which in \ASCII\ art looks like:

+

+\starttyping

+H <=> i <=> [glue] <=> T <=> h <=> e <=> r <=> e

+\stoptyping

+

+When we have a paragraph, we actually get something:

+

+\starttyping

+[localpar] <=> H <=> i <=> [glue] <=> T <=> h <=> e <=> r <=> e <=> [glue]

+\stoptyping

+

+Each character becomes a so called glyph node, a record with properties like the

+current font, the character code and the current language. Spaces become glue

+nodes. There are many node types that we will discuss later. Each node points

+back to a previous node or next node, given that these exist.

+

+It's also good to know beforehand that \TEX\ is basically centered around

+creating paragraphs and pages. The par builder takes a list and breaks it into

+lines. We turn horizontal material into vertical. Lines are so called boxes and

+can be separated by glue, penalties and more. The page builder accumulates lines

+and when feasible triggers an output routine that will take the list so far.

+Constructing the actual page is not part of \TEX\ but done using primitives that

+permit manipulation of boxes. The result is handled back to \TEX\ and flushed to

+a (often \PDF) file.

+

+The \LUATEX\ engine provides hooks for \LUA\ code at nearly every reasonable

+point in the process: collecting content, hyphenating, applying font features,

+breaking into lines, etc. This means that you can overload \TEX's natural

+behaviour, which still is the benchmark. When we refer to \quote {callbacks} we

+means these hooks.

+

+Where plain \TEX\ is basically a basic framework for writing a specific style,

+macro packages like \CONTEXT\ and \LATEX\ provide the user a whole lot of

+additional tools to make documents look good. They hide the dirty details of font

+management, language demands, turning structure into typeset results, wrapping

+pages, including images, and so on. You should be aware of the fact that when you

+hook in your own code to manipulate lists, this can interfere with the macro

+package that you use.

+

+When you read about nodes in the following chapters it's good to keep in mind their

+commands that relate to then. Here are a few:

+

+\starttabulate[|l|l|p|]

+\DB command                \BC node          \BC explanation \NC \NR

+\TB

+\NC \prm {hbox}            \NC \nod {hlist} \NC horizontal box \NC \NR

+\NC \prm {vbox}            \NC \nod {vlist} \NC vertical box with the baseline at the bottom \NC \NR

+\NC \prm {vtop}            \NC \nod {vlist} \NC vertical box with the baseline at the top \NC \NR

+\NC \prm {hskip}           \NC \nod {glue}  \NC horizontal skip with optional stretch and shrink \NC \NR

+\NC \prm {vskip}           \NC \nod {glue}  \NC vertical skip with optional stretch and shrink \NC \NR

+\NC \prm {kern}            \NC \nod {kern}  \NC horizontal or vertical fixed skip \NC \NR

+\NC \prm {discretionary}   \NC \nod {disc}  \NC hyphenation point (pre, post, replace) \NC \NR

+\NC \prm {char}            \NC \nod {glyph} \NC a character \NC \NR

+\NC \prm {hrule}           \NC \nod {rule}  \NC a horizontal rule \NC \NR

+\NC \prm {vrule}           \NC \nod {rule}  \NC a vertical rule \NC \NR

+\NC \prm {textdir(ection)} \NC \nod {dir}   \NC a change in text direction \NC \NR

+\LL

+\stoptabulate

+

+For now this should be enough to enable you to understand the next chapters.

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-registers.tex b/Master/texmf-dist/doc/luatex/base/luatex-registers.tex
new file mode 100644
index 00000000000..29302300704
--- /dev/null
+++ b/Master/texmf-dist/doc/luatex/base/luatex-registers.tex
@@ -0,0 +1,47 @@
+\environment luatex-style

+

+\startcomponent luatex-registers

+

+\startchapter[title=Topics]

+

+    \placeregister[topicindex]

+

+\stopchapter

+

+\startchapter[title=Primitives]

+

+    This register contains the primitives that are mentioned in the manual. There

+    are of course many more primitives. The \LUATEX\ primitives are typeset in

+    bold. The primitives from \PDFTEX\ are not supported that way but mentioned

+    anyway.

+

+    \placeregister[primitiveindex][indicator=no]

+

+\stopchapter

+

+\startchapter[title=Callbacks]

+

+    \placeregister[callbackindex]

+

+\stopchapter

+

+\startchapter[title=Nodes]

+

+    This register contains the nodes that are known to \LUATEX. The primary nodes

+    are in bold, whatsits that are determined by their subtype are normal. The

+    names prefixed by \type {pdf_} are backend specific.

+

+    \placeregister[nodeindex]

+

+\stopchapter

+

+\startchapter[title=Libraries]

+

+    This register contains the functions available in libraries. Not all functions

+    are documented, for instance because they can be experimental or obsolete.

+

+    \placeregister[libraryindex]

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-statistics.tex b/Master/texmf-dist/doc/luatex/base/luatex-statistics.tex
new file mode 100644
index 00000000000..2428a2929d6
--- /dev/null
+++ b/Master/texmf-dist/doc/luatex/base/luatex-statistics.tex
@@ -0,0 +1,17 @@
+% language=uk

+

+\environment luatex-style

+

+\startcomponent luatex-statistics

+

+\startchapter[title={Statistics}]

+

+    \topicindex{fonts+used}

+

+    The following fonts are used in this document:

+

+    \showfontusage

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-style.tex b/Master/texmf-dist/doc/luatex/base/luatex-style.tex
index a277a117836..f176afab7d2 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-style.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-style.tex
@@ -1,376 +1,442 @@
-\startenvironment luatex-style
-
-% todo: use \useMPlibrary[lua]
-
-\usemodule[abr-02]
-
-\setuplayout
-  [height=middle,
-   width=middle,
-   backspace=2cm,
-   topspace=10mm,
-   bottomspace=10mm,
-   header=10mm,
-   footer=10mm,
-   footerdistance=10mm,
-   headerdistance=10mm]
-
-\setuppagenumbering
-  [alternative=doublesided]
-
-\setuptolerance
-  [stretch,tolerant]
-
-\setuptype
-  [lines=hyphenated]
-
-\setuptyping
-  [lines=hyphenated]
-
-\setupitemize
-  [each]
-  [packed]
-
-\setupwhitespace
-  [medium]
-
-\startluacode
-    local skipped = table.tohash { 'id', 'subtype', 'next', 'prev' }
-
-    function document.functions.showfields(s)
-        local t = string.split(s,',')
-        local f = node.fields(t[1],t[2])
-        if f then
-            local d = false
-            for i=1,#f do
-                local fi = f[i]
-                if skipped[fi] then
-                    -- okay
-                elseif d then
-                    context(', {\tttf %s}', fi)
-                else
-                    context('{\tttf %s}', fi)
-                    d = true
-                end
-            end
-        end
-    end
-
-    function document.functions.showid(s)
-        local t = string.split(s,',')
-        context('{tttf %s}',node.id(t[1]))
-        if t[2] then
-            context(', {tttf %s}',node.subtype(t[2]))
-        end
-    end
-
-    function document.functions.showsubtypes(s)
-        local s = node.subtypes(s)
-        local d = false
-        for k, v in table.sortedhash(s) do
-            if d then
-                context(', %s = {\\tttf %s}',k,v)
-            else
-                context('%s = {\\tttf %s}',k,v)
-                d = true
-            end
-        end
-    end
-\stopluacode
-
-\unexpanded\def\showfields  #1{\ctxlua{document.functions.showfields("#1")}}
-\unexpanded\def\showid      #1{\ctxlua{document.functions.showid("#1")}}
-\unexpanded\def\showsubtypes#1{\ctxlua{document.functions.showsubtypes("#1")}}
-
-\definecolor[blue]      [b=.5]
-\definecolor[red]       [r=.5]
-\definecolor[green]     [g=.5]
-\definecolor[maincolor] [b=.5]
-\definecolor[keptcolor] [b=.5]
-\definecolor[othercolor][r=.5,g=.5]
-
-\usebodyfont[lucidaot]
-\usebodyfont[pagella]
-\usebodyfont[cambria]
-%usebodyfont[dejavu]
-\usebodyfont[modern] % we need this in examples so we predefine
-
-% \doifmodeelse {atpragma} {
-%
-%   %  \setupbodyfont
-%   %    [lucidaot,10pt]
-%
-%     \setupbodyfont
-%       [dejavu,10pt]
-%
-%     \setuphead [chapter]      [style=\bfd]
-%     \setuphead [section]      [style=\bfb]
-%     \setuphead [subsection]   [style=\bfa]
-%     \setuphead [subsubsection][style=\bf]
-%
-% } {
-%
-%     \definetypeface[mainfacenormal]  [ss][sans] [iwona]       [default]
-%     \definetypeface[mainfacenormal]  [rm][serif][palatino]    [default]
-%     \definetypeface[mainfacenormal]  [tt][mono] [modern]      [default][rscale=1.1]
-%     \definetypeface[mainfacenormal]  [mm][math] [iwona]       [default]
-%
-%     \definetypeface[mainfacemedium]  [ss][sans] [iwona-medium][default]
-%     \definetypeface[mainfacemedium]  [rm][serif][palatino]    [default]
-%     \definetypeface[mainfacemedium]  [tt][mono] [modern]      [default][rscale=1.1]
-%     \definetypeface[mainfacemedium]  [mm][math] [iwona-medium][default]
-%
-%     \setupbodyfont
-%       [mainfacenormal,10pt]
-%
-%     \setuphead [chapter]      [style=\mainfacemedium\bfd]
-%     \setuphead [section]      [style=\mainfacemedium\bfb]
-%     \setuphead [subsection]   [style=\mainfacemedium\bfa]
-%     \setuphead [subsubsection][style=\mainfacemedium\bf]
-%
-% }
-
-\writestatus{luatex manual}{we assume that dejavu math is available}
-
-\setupbodyfont % assumes dejavu-math
-  [dejavu,10pt]
-
-\setuphead [chapter]      [align={flushleft,broad},style=\bfd]
-\setuphead [section]      [align={flushleft,broad},style=\bfb]
-\setuphead [subsection]   [align={flushleft,broad},style=\bfa]
-\setuphead [subsubsection][align={flushleft,broad},style=\bf]
-
-\setuphead [chapter]      [color=maincolor]
-\setuphead [section]      [color=maincolor]
-\setuphead [subsection]   [color=maincolor]
-\setuphead [subsubsection][color=maincolor]
-
-\definehead
-  [remark]
-  [subsubsubject]
-
-\setupheadertexts
-  []
-
-\definemixedcolumns
-  [twocolumns]
-  [n=2,
-   balance=yes,
-   before=\blank,
-   after=\blank]
-
-\definemixedcolumns
-  [threecolumns]
-  [twocolumns]
-  [n=3]
-
-\definemixedcolumns
-  [fourcolumns]
-  [threecolumns]
-  [n=4]
-
-% if we do this we also need to do it in table cells
-%
-% \setuptyping
-%   [color=maincolor]
-%
-% \setuptype
-%   [color=maincolor]
-
-\definetyping
-  [functioncall]
-
-\startMPdefinitions
-
-    color   luaplanetcolor ; luaplanetcolor := \MPcolor{maincolor} ;
-    color   luaholecolor   ; luaholecolor   := white ;
-    numeric luaextraangle  ; luaextraangle  := 0 ;
-    numeric luaorbitfactor ; luaorbitfactor := .25 ;
-
-    vardef lualogo = image (
-
-        % Graphic design by A. Nakonechnyj. Copyright (c) 1998, All rights reserved.
-
-        save d, r, p ; numeric d, r, p ;
-
-        d := sqrt(2)/4 ; r := 1/4 ; p := r/8 ;
-
-        fill fullcircle scaled 1
-            withcolor luaplanetcolor ;
-        draw fullcircle rotated 40.5 scaled (1+r)
-            dashed evenly scaled p
-            withpen pencircle scaled (p/2)
-            withcolor (luaorbitfactor * luaholecolor) ;
-        fill fullcircle scaled r shifted (d+1/8,d+1/8)
-            rotated - luaextraangle
-            withcolor luaplanetcolor ;
-        fill fullcircle scaled r shifted (d-1/8,d-1/8)
-            withcolor luaholecolor   ;
-        luaorbitfactor := .25 ;
-    )  enddef ;
-
-\stopMPdefinitions
-
-\startuseMPgraphic{luapage}
-    StartPage ;
-
-        fill Page withcolor \MPcolor{othercolor} ;
-
-        luaorbitfactor := 1 ;
-        picture p ; p := lualogo xsized (3PaperWidth/5) ;
-        draw p shifted center Page shifted (0,-.85ypart center ulcorner p) ;
-
-    StopPage ;
-\stopuseMPgraphic
-
-% \starttexdefinition luaextraangle
-%     % we can also just access the last page and so in mp directly
-%     \ctxlua {
-%         context(\lastpage == 0 and 0 or \realfolio*360/\lastpage)
-%     }
-% \stoptexdefinition
-
-\startuseMPgraphic{luanumber}
-  % luaextraangle  := \luaextraangle;
-    luaextraangle  := if (LastPageNumber == 0) : 0 else : (RealPageNumber / LastPageNumber) * 360  fi;
-    luaorbitfactor := 0.25 ;
-    picture p ; p := lualogo ;
-    setbounds p to boundingbox fullcircle ;
-    draw p ysized 1cm ;
-\stopuseMPgraphic
-
-\definelayer
-  [page]
-  [width=\paperwidth,
-   height=\paperheight]
-
-\setupbackgrounds
-  [leftpage]
-  [background=page]
-
-\setupbackgrounds
-  [rightpage]
-  [background=page]
-
-\startsetups pagenumber:right
-  \setlayerframed
-    [page]
-    [preset=rightbottom,offset=1cm]
-    [frame=off,height=1cm,offset=overlay]
-    {\useMPgraphic{luanumber}}
-  \setlayerframed
-    [page]
-    [preset=rightbottom,offset=1cm,x=1.5cm]
-    [frame=off,height=1cm,width=1cm,offset=overlay]
-    {\pagenumber}
-  \setlayerframed
-    [page]
-    [preset=rightbottom,offset=1cm,x=2.5cm]
-    [frame=off,height=1cm,offset=overlay]
-    {\getmarking[chapter]}
-\stopsetups
-
-\startsetups pagenumber:left
-  \setlayerframed
-    [page]
-    [preset=leftbottom,offset=1cm,x=2.5cm]
-    [frame=off,height=1cm,offset=overlay]
-    {\getmarking[chapter]}
-  \setlayerframed
-    [page]
-    [preset=leftbottom,offset=1cm,x=1.5cm]
-    [frame=off,height=1cm,width=1cm,offset=overlay]
-    {\pagenumber}
-  \setlayerframed
-    [page]
-    [preset=leftbottom,offset=1cm]
-    [frame=off,height=1cm,offset=overlay]
-    {\useMPgraphic{luanumber}}
-\stopsetups
-
-\unexpanded\def\nonterminal#1>{\mathematics{\langle\hbox{\rm #1}\rangle}}
-
-% taco's brainwave -) .. todo: create a typing variant so that we can avoid the !crlf
-
-\newcatcodetable\syntaxcodetable
-
-\unexpanded\def\makesyntaxcodetable
-  {\begingroup
-   \catcode`\<=13 \catcode`\|=12
-   \catcode`\!= 0 \catcode`\\=12
-   \savecatcodetable\syntaxcodetable
-   \endgroup}
-
-\makesyntaxcodetable
-
-\unexpanded\def\startsyntax {\begingroup\catcodetable\syntaxcodetable  \dostartsyntax}
-\unexpanded\def\syntax      {\begingroup\catcodetable\syntaxcodetable  \dosyntax}
-           \let\stopsyntax   \relax
-
-\unexpanded\def\syntaxenvbody#1%
-  {\par
-   \tt
-   \startnarrower
-   \maincolor #1
-   \stopnarrower
-   \par}
-
-\unexpanded\def\syntaxbody#1%
-  {\begingroup
-   \maincolor \tt #1%
-   \endgroup}
-
-\bgroup \catcodetable\syntaxcodetable
-
-!gdef!dostartsyntax#1\stopsyntax{!let<!nonterminal!syntaxenvbody{#1}!endgroup}
-!gdef!dosyntax     #1{!let<!nonterminal!syntaxbody{#1}!endgroup}
-
-!egroup
-
-\definetyping
-  [texsyntax]
-  [color=maincolor]
-
-% end of wave
-
-\setupinteraction
-  [state=start,
-   focus=standard,
-   style=,
-   color=,
-   contrastcolor=]
-
-\placebookmarks
-  [chapter,section,subsection]
-
-\setuplist
-  [chapter,section,subsection,subsubsection]
-  [interaction=all,
-   width=3em]
-
-\setuplist
-  [chapter]
-  [style=bold,
-   color=keptcolor]
-
-\setuplist
-  [subsection,subsubsection]
-  [margin=3em,
-   width=5em]
-
-\definestartstop
-  [notabene]
-  [style=slanted]
-
-\definestartstop
-  [preamble]
-  [style=normal,
-   before=\blank,
-   after=\blank]
-
-% Hans doesn't like the bookmarks opening by default so we comment this:
-%
-% \setupinteractionscreen
-%   [option=bookmark]
-
-\stopenvironment
+\startenvironment luatex-style

+

+% todo: use \useMPlibrary[lua]

+

+\enabletrackers[fonts.usage]

+

+\usemodule[fonts-statistics]

+

+\setuplayout

+  [height=middle,

+   width=middle,

+   backspace=2cm,

+   topspace=10mm,

+   bottomspace=10mm,

+   header=10mm,

+   footer=10mm,

+   footerdistance=10mm,

+   headerdistance=10mm]

+

+\setuppagenumbering

+  [alternative=doublesided]

+

+\setuptolerance

+  [stretch,tolerant]

+

+\setuptype

+  [lines=hyphenated]

+

+\setuptyping

+  [lines=hyphenated]

+

+\setupitemize

+  [each]

+  [packed]

+

+\definesymbol[1][\Uchar"2023]

+\definesymbol[2][\endash]

+\definesymbol[3][\wait]      % we want to catch it

+

+\setupitemize

+  [each]

+  [headcolor=maincolor,

+   symbolcolor=maincolor,

+   color=maincolor]

+

+\setupwhitespace

+  [medium]

+

+\setuptabulate

+  [blank={small,samepage},

+   headstyle=bold,

+   rulecolor=maincolor,

+   rulethickness=1pt,

+   foregroundcolor=white,

+   foregroundstyle=\ss\bfx\WORD,

+   backgroundcolor=maincolor]

+

+\setupcaptions

+  [headcolor=darkblue]

+

+\startluacode

+    local skipped = table.tohash { 'id', 'subtype', 'next', 'prev' }

+

+    function document.functions.showfields(s)

+        local t = string.split(s,',')

+        local f = node.fields(t[1],t[2])

+        if f then

+            local d = false

+            for i=1,#f do

+                local fi = f[i]

+                if skipped[fi] then

+                    -- okay

+                elseif d then

+                    context(', {\tttf %s}', fi)

+                else

+                    context('{\tttf %s}', fi)

+                    d = true

+                end

+            end

+        end

+    end

+

+    function document.functions.showid(s)

+        local t = string.split(s,',')

+        context('{tttf %s}',node.id(t[1]))

+        if t[2] then

+            context(', {tttf %s}',node.subtype(t[2]))

+        end

+    end

+

+    function document.functions.showsubtypes(s)

+        local s = node.subtypes(s)

+        local d = false

+        for k, v in table.sortedhash(s) do

+            if d then

+                context(', %s = {\\tttf %s}',k,v)

+            else

+                context('%s = {\\tttf %s}',k,v)

+                d = true

+            end

+        end

+    end

+\stopluacode

+

+\unexpanded\def\showfields  #1{\ctxlua{document.functions.showfields("#1")}}

+\unexpanded\def\showid      #1{\ctxlua{document.functions.showid("#1")}}

+\unexpanded\def\showsubtypes#1{\ctxlua{document.functions.showsubtypes("#1")}}

+

+\definecolor[blue]      [b=.5]

+\definecolor[red]       [r=.5]

+\definecolor[green]     [g=.5]

+\definecolor[maincolor] [b=.5]

+\definecolor[keptcolor] [b=.5]

+\definecolor[othercolor][r=.5,g=.5]

+

+\writestatus{luatex manual}{}

+\writestatus{luatex manual}{defining lucodaot} \usebodyfont  [lucidaot]

+\writestatus{luatex manual}{defining pagella}  \usebodyfont  [pagella]

+\writestatus{luatex manual}{defining cambria}  \usebodyfont  [cambria]

+\writestatus{luatex manual}{defining modern}   \usebodyfont  [modern]

+\writestatus{luatex manual}{defining dejavu}   \setupbodyfont[dejavu,10pt]

+\writestatus{luatex manual}{}

+

+\setuphead [chapter]      [align={flushleft,broad},style=\bfd]

+\setuphead [section]      [align={flushleft,broad},style=\bfb]

+\setuphead [subsection]   [align={flushleft,broad},style=\bfa]

+\setuphead [subsubsection][align={flushleft,broad},style=\bf]

+

+\setuphead [chapter]      [color=maincolor]

+\setuphead [section]      [color=maincolor]

+\setuphead [subsection]   [color=maincolor]

+\setuphead [subsubsection][color=maincolor]

+

+\setupfloats

+  [ntop=4]

+

+\definehead

+  [remark]

+  [subsubsubject]

+

+\setupheadertexts

+  []

+

+% \setuplayout

+%   [style=bold,

+%    color=maincolor]

+

+\definemixedcolumns

+  [twocolumns]

+  [n=2,

+   balance=yes,

+   before=\blank,

+   after=\blank]

+

+\definemixedcolumns

+  [threecolumns]

+  [twocolumns]

+  [n=3]

+

+\definemixedcolumns

+  [fourcolumns]

+  [threecolumns]

+  [n=4]

+

+% if we do this we also need to do it in table cells

+%

+% \setuptyping

+%   [color=maincolor]

+%

+% \setuptype

+%   [color=maincolor]

+

+\definetyping

+  [functioncall]

+

+\startMPdefinitions

+

+    color   luaplanetcolor ; luaplanetcolor := \MPcolor{maincolor} ;

+    color   luaholecolor   ; luaholecolor   := white ;

+    numeric luaextraangle  ; luaextraangle  := 0 ;

+    numeric luaorbitfactor ; luaorbitfactor := .25 ;

+

+    vardef lualogo = image (

+

+        % Graphic design by A. Nakonechnyj. Copyright (c) 1998, All rights reserved.

+

+        save d, r, p ; numeric d, r, p ;

+

+        d := sqrt(2)/4 ; r := 1/4 ; p := r/8 ;

+

+        fill fullcircle scaled 1

+            withcolor luaplanetcolor ;

+        draw fullcircle rotated 40.5 scaled (1+r)

+            dashed evenly scaled p

+            withpen pencircle scaled (p/2)

+            withcolor (luaorbitfactor * luaholecolor) ;

+        fill fullcircle scaled r shifted (d+1/8,d+1/8)

+            rotated - luaextraangle

+            withcolor luaplanetcolor ;

+        fill fullcircle scaled r shifted (d-1/8,d-1/8)

+            withcolor luaholecolor   ;

+        luaorbitfactor := .25 ;

+    ) enddef ;

+

+\stopMPdefinitions

+

+\startuseMPgraphic{luapage}

+    StartPage ;

+

+        fill Page withcolor \MPcolor{othercolor} ;

+

+        luaorbitfactor := 1 ;

+

+        picture p ; p := lualogo ysized (5*\measure{paperheight}/10) ;

+        draw p

+            shifted - center p

+            shifted (

+                \measure{spreadwidth} - .5*\measure{paperwidth} + \measure{spinewidth},

+                .375*\measure{paperheight}

+            )

+        ;

+

+    StopPage ;

+\stopuseMPgraphic

+

+% \starttexdefinition luaextraangle

+%     % we can also just access the last page and so in mp directly

+%     \ctxlua {

+%         context(\lastpage == 0 and 0 or \realfolio*360/\lastpage)

+%     }

+% \stoptexdefinition

+

+\startuseMPgraphic{luanumber}

+  % luaextraangle  := \luaextraangle;

+    luaextraangle  := if (LastPageNumber < 10) : 10 else : (RealPageNumber / LastPageNumber) * 360  fi;

+    luaorbitfactor := 0.25 ;

+    picture p ; p := lualogo ;

+    setbounds p to boundingbox fullcircle ;

+    draw p ysized 1cm ;

+\stopuseMPgraphic

+

+\definelayer

+  [page]

+  [width=\paperwidth,

+   height=\paperheight]

+

+\setupbackgrounds

+  [leftpage]

+  [background=page]

+

+\setupbackgrounds

+  [rightpage]

+  [background=page]

+

+\definemeasure[banneroffset][\bottomspace-\footerheight-\footerdistance+2cm]

+

+\startsetups pagenumber:right

+  \setlayerframed

+    [page]

+    [preset=rightbottom,x=1.0cm,y=\measure{banneroffset}]

+    [frame=off,height=1cm,offset=overlay]

+    {\strut\useMPgraphic{luanumber}}

+  \setlayerframed

+    [page]

+    [preset=rightbottom,x=2.5cm,y=\measure{banneroffset}]

+    [frame=off,height=1cm,width=1cm,offset=overlay,

+     foregroundstyle=bold,foregroundcolor=maincolor]

+    {\strut\pagenumber}

+  \setlayerframed

+    [page]

+    [preset=rightbottom,x=3.5cm,y=\measure{banneroffset}]

+    [frame=off,height=1cm,offset=overlay,

+     foregroundstyle=bold,foregroundcolor=maincolor]

+    {\strut\getmarking[chapter]}

+\stopsetups

+

+\startsetups pagenumber:left

+  \setlayerframed

+    [page]

+    [preset=leftbottom,x=3.5cm,y=\measure{banneroffset}]

+    [frame=off,height=1cm,offset=overlay,

+     foregroundstyle=bold,foregroundcolor=maincolor]

+    {\strut\getmarking[chapter]}

+  \setlayerframed

+    [page]

+    [preset=leftbottom,x=2.5cm,y=\measure{banneroffset}]

+    [frame=off,height=1cm,width=1cm,offset=overlay,

+     foregroundstyle=bold,foregroundcolor=maincolor]

+    {\strut\pagenumber}

+  \setlayerframed

+    [page]

+    [preset=leftbottom,x=1.0cm,y=\measure{banneroffset}]

+    [frame=off,height=1cm,offset=overlay]

+    {\strut\useMPgraphic{luanumber}}

+\stopsetups

+

+\unexpanded\def\nonterminal#1>{\mathematics{\langle\hbox{\rm #1}\rangle}}

+

+% taco's brainwave -) .. todo: create a typing variant so that we can avoid the !crlf

+

+\newcatcodetable\syntaxcodetable

+

+\unexpanded\def\makesyntaxcodetable

+  {\begingroup

+   \catcode`\<=13 \catcode`\|=12

+   \catcode`\!= 0 \catcode`\\=12

+   \savecatcodetable\syntaxcodetable

+   \endgroup}

+

+\makesyntaxcodetable

+

+\unexpanded\def\startsyntax {\begingroup\catcodetable\syntaxcodetable  \dostartsyntax}

+\unexpanded\def\syntax      {\begingroup\catcodetable\syntaxcodetable  \dosyntax}

+           \let\stopsyntax   \relax

+

+\unexpanded\def\syntaxenvbody#1%

+  {\par

+   \tt

+   \startnarrower

+ % \maincolor

+   #1

+   \stopnarrower

+   \par}

+

+\unexpanded\def\syntaxbody#1%

+  {\begingroup

+ % \maincolor

+   \tt #1%

+   \endgroup}

+

+\bgroup \catcodetable\syntaxcodetable

+

+!gdef!dostartsyntax#1\stopsyntax{!let<!nonterminal!syntaxenvbody{#1}!endgroup}

+!gdef!dosyntax     #1{!let<!nonterminal!syntaxbody{#1}!endgroup}

+

+!egroup

+

+\definetyping

+  [texsyntax]

+% [color=maincolor]

+

+% end of wave

+

+\setupinteraction

+  [state=start,

+   focus=standard,

+   style=,

+   color=,

+   contrastcolor=]

+

+\placebookmarks

+  [chapter,section,subsection]

+

+\setuplist

+  [chapter,section,subsection,subsubsection]

+  [interaction=all,

+   width=3em]

+

+\setuplist

+  [chapter]

+  [style=bold,

+   before={\testpage[4]\blank},

+   color=keptcolor]

+

+\setuplist

+  [section]

+  [before={\testpage[3]}]

+

+\setuplist

+  [subsection,subsubsection]

+  [margin=3em,

+   width=5em]

+

+\definestartstop

+  [notabene]

+  [style=slanted]

+

+\definestartstop

+  [preamble]

+  [style=normal,

+   before=\blank,

+   after=\blank]

+

+% Hans doesn't like the bookmarks opening by default so we comment this:

+%

+% \setupinteractionscreen

+%   [option=bookmark]

+

+\startbuffer[stylecalculations]

+

+    \normalexpanded{\definemeasure[spinewidth] [0pt]}

+    \normalexpanded{\definemeasure[paperwidth] [\the\paperwidth ]}

+    \normalexpanded{\definemeasure[paperheight][\the\paperheight]}

+    \normalexpanded{\definemeasure[spreadwidth][\measure{paperwidth}]}

+

+\stopbuffer

+

+\getbuffer[stylecalculations]

+

+\dontcomplain

+

+\environment luatex-logos

+

+\defineregister[topicindex]

+\defineregister[primitiveindex]

+\defineregister[callbackindex]

+\defineregister[nodeindex]

+\defineregister[libraryindex]

+

+\unexpanded\def\lpr#1{\doifmode{*bodypart}{\primitiveindex[#1]{\bf\tex {#1}}}\tex {#1}}

+\unexpanded\def\prm#1{\doifmode{*bodypart}{\primitiveindex[#1]{\tex    {#1}}}\tex {#1}}

+\unexpanded\def\orm#1{\doifmode{*bodypart}{\primitiveindex[#1]{\tex    {#1}}}\tex {#1}}

+\unexpanded\def\cbk#1{\doifmode{*bodypart}{\callbackindex [#1]{\type   {#1}}}\type{#1}}

+\unexpanded\def\nod#1{\doifmode{*bodypart}{\nodeindex     [#1]{\bf\type{#1}}}\type{#1}}

+\unexpanded\def\whs#1{\doifmode{*bodypart}{\nodeindex     [#1]{\type   {#1}}}\type{#1}}

+\unexpanded\def\noa#1{\doifmode{*bodypart}{\nodeindex     [#1]{\type   {#1}}}\type{#1}}

+

+\hyphenation{sub-nodes}

+

+\def\currentlibraryindex{\namedstructureuservariable{section}{library}}

+

+\setupregister

+  [libraryindex]

+  [indicator=no,before=]

+

+\setupregister

+  [libraryindex]

+  [1]

+  [textstyle=\ttbf]

+

+\setupregister

+  [libraryindex]

+  [2]

+  [textstyle=\tttf]

+

+\unexpanded\def\lib     #1{\doifmode{*bodypart}{\expanded{\libraryindex{\currentlibraryindex+#1}}\type{\currentlibraryindex.#1}}}

+\unexpanded\def\libindex#1{\doifmode{*bodypart}{\expanded{\libraryindex{\currentlibraryindex+#1}}}}

+\unexpanded\def\libidx#1#2{\doifmode{*bodypart}{\expanded{\libraryindex{#1+#2}}\type{#1.#2}}}

+

+% \setstructurepageregister[][keys:1=,entries:1=]

+

+\stopenvironment

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-tex.tex b/Master/texmf-dist/doc/luatex/base/luatex-tex.tex
index 2dd9521c4dd..6b1ba1689b0 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-tex.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-tex.tex
@@ -1,2070 +1,2755 @@
-% language=uk
-
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-tex
-
-\startchapter[reference=tex,title={The \TEX\ related libraries}]
-
-\section{The \type {lua} library}
-
-\subsection{\LUA\ version}
-
-This library contains one read|-|only item:
-
-\starttyping
-<string> s = lua.version
-\stoptyping
-
-This returns the \LUA\ version identifier string. The value is currently
-\directlua {tex.print(lua.version)}.
-
-\subsection{\LUA\ bytecode registers}
-
-\LUA\ registers can be used to communicate \LUA\ functions across \LUA\ chunks.
-The accepted values for assignments are functions and \type {nil}. Likewise, the
-retrieved value is either a function or \type {nil}.
-
-\starttyping
-lua.bytecode[<number> n] = <function> f
-lua.bytecode[<number> n]()
-\stoptyping
-
-The contents of the \type {lua.bytecode} array is stored inside the format file
-as actual \LUA\ bytecode, so it can also be used to preload \LUA\ code.
-
-Note: The function must not contain any upvalues. Currently, functions containing
-upvalues can be stored (and their upvalues are set to \type {nil}), but this is
-an artifact of the current \LUA\ implementation and thus subject to change.
-
-The associated function calls are
-
-\startfunctioncall
-<function> f = lua.getbytecode(<number> n)
-lua.setbytecode(<number> n, <function> f)
-\stopfunctioncall
-
-Note: Since a \LUA\ file loaded using \type {loadfile(filename)} is essentially
-an anonymous function, a complete file can be stored in a bytecode register like
-this:
-
-\startfunctioncall
-lua.bytecode[n] = loadfile(filename)
-\stopfunctioncall
-
-Now all definitions (functions, variables) contained in the file can be
-created by executing this bytecode register:
-
-\startfunctioncall
-lua.bytecode[n]()
-\stopfunctioncall
-
-Note that the path of the file is stored in the \LUA\ bytecode to be used in
-stack backtraces and therefore dumped into the format file if the above code is
-used in \INITEX. If it contains private information, i.e. the user name, this
-information is then contained in the format file as well. This should be kept in
-mind when preloading files into a bytecode register in \INITEX.
-
-\subsection{\LUA\ chunk name registers}
-
-There is an array of 65536 (0--65535) potential chunk names for use with the
-\type {\directlua} and \type {\latelua} primitives.
-
-\startfunctioncall
-lua.name[<number> n] = <string> s
-<string> s = lua.name[<number> n]
-\stopfunctioncall
-
-If you want to unset a \LUA\ name, you can assign \type {nil} to it.
-
-\section{The \type {status} library}
-
-This contains a number of run|-|time configuration items that you may find useful
-in message reporting, as well as an iterator function that gets all of the names
-and values as a table.
-
-\startfunctioncall
-<table> info = status.list()
-\stopfunctioncall
-
-The keys in the table are the known items, the value is the current value. Almost
-all of the values in \type {status} are fetched through a metatable at run|-|time
-whenever they are accessed, so you cannot use \type {pairs} on \type {status},
-but you {\it can\/} use \type {pairs} on \type {info}, of course. If you do not
-need the full list, you can also ask for a single item by using its name as an
-index into \type {status}.
-
-The current list is:
-
-\starttabulate[|l|p|]
-\BC key                       \BC explanation \NC \NR
-\NC \type{banner}             \NC terminal display banner \NC \NR
-\NC \type{best_page_break}    \NC the current best break (a node) \NR \NR
-\NC \type{buf_size}           \NC current allocated size of the line buffer \NC \NR
-\NC \type{callbacks}          \NC total number of executed callbacks so far \NC \NR
-\NC \type{cs_count}           \NC number of control sequences \NC \NR
-\NC \type{dest_names_size}    \NC \PDF\ destination table size \NC \NR
-\NC \type{dvi_gone}           \NC written \DVI\ bytes \NC \NR
-\NC \type{dvi_ptr}            \NC not yet written \DVI\ bytes \NC \NR
-\NC \type{dyn_used}           \NC token (multi|-|word) memory in use  \NC \NR
-\NC \type{filename}           \NC name of the current input file \NC \NR
-\NC \type{fix_mem_end}        \NC maximum number of used tokens \NC \NR
-\NC \type{fix_mem_min}        \NC minimum number of allocated words for tokens \NC \NR
-\NC \type{fix_mem_max}        \NC maximum number of allocated words for tokens \NC \NR
-\NC \type{font_ptr}           \NC number of active fonts \NC \NR
-\NC \type{hash_extra}         \NC extra allowed hash \NC \NR
-\NC \type{hash_size}          \NC size of hash \NC \NR
-\NC \type{indirect_callbacks} \NC number of those that were themselves a result of other callbacks (e.g. file readers) \NC \NR
-\NC \type{ini_version}        \NC \type {true} if this is an \INITEX\ run \NC \NR
-\NC \type{init_pool_ptr}      \NC \INITEX\ string pool index \NC \NR
-\NC \type{init_str_ptr}       \NC number of \INITEX\ strings \NC \NR
-\NC \type{input_ptr}          \NC th elevel of input we're at \NC \NR
-\NC \type{inputid}            \NC numeric id of the current input \NC \NR
-\NC \type{largest_used_mark}  \NC max referenced marks class \NC \NR
-\NC \type{lasterrorcontext}   \NC last error context string (with newlines) \NC \NR
-\NC \type{lasterrorstring}    \NC last \TEX\ error string \NC \NR
-\NC \type{lastluaerrorstring} \NC last \LUA\ error string \NC \NR
-\NC \type{lastwarningstring}  \NC last warning tag, normally an indication of in what part\NC \NR
-\NC \type{lastwarningtag}     \NC last warning string\NC \NR
-\NC \type{linenumber}         \NC location in the current input file \NC \NR
-\NC \type{log_name}           \NC name of the log file \NC \NR
-\NC \type{luabytecode_bytes}  \NC number of bytes in \LUA\ bytecode registers \NC \NR
-\NC \type{luabytecodes}       \NC number of active \LUA\ bytecode registers \NC \NR
-\NC \type{luastate_bytes}     \NC number of bytes in use by \LUA\ interpreters \NC \NR
-\NC \type{luatex_engine}      \NC the \LUATEX\ engine identifier \NC \NR
-\NC \type{luatex_hashchars}   \NC length to which \LUA\ hashes strings ($2^n$) \NC \NR
-\NC \type{luatex_hashtype}    \NC the hash method used (in \LUAJITTEX) \NC \NR
-\NC \type{luatex_revision}    \NC the \LUATEX\ revision string \NC \NR
-\NC \type{luatex_revision}    \NC the \LUATEX\ revision string \NC \NR
-\NC \type{luatex_version}     \NC the \LUATEX\ version number \NC \NR
-\NC \type{max_buf_stack}      \NC max used buffer position \NC \NR
-\NC \type{max_in_stack}       \NC max used input stack entries \NC \NR
-\NC \type{max_nest_stack}     \NC max used nesting stack entries \NC \NR
-\NC \type{max_param_stack}    \NC max used parameter stack entries \NC \NR
-\NC \type{max_save_stack}     \NC max used save stack entries \NC \NR
-\NC \type{max_strings}        \NC maximum allowed strings \NC \NR
-\NC \type{nest_size}          \NC nesting stack size \NC \NR
-\NC \type{node_mem_usage}     \NC a string giving insight into currently used nodes \NC \NR
-\NC \type{obj_ptr}            \NC max \PDF\ object pointer \NC \NR
-\NC \type{obj_tab_size}       \NC \PDF\ object table size \NC \NR
-\NC \type{output_active}      \NC \type {true} if the \type {\output} routine is active \NC \NR
-\NC \type{output_file_name}   \NC name of the \PDF\ or \DVI\ file \NC \NR
-\NC \type{param_size}         \NC parameter stack size \NC \NR
-\NC \type{pdf_dest_names_ptr} \NC max \PDF\ destination pointer \NC \NR
-\NC \type{pdf_gone}           \NC written \PDF\ bytes \NC \NR
-\NC \type{pdf_mem_ptr}        \NC max \PDF\ memory used \NC \NR
-\NC \type{pdf_mem_size}       \NC \PDF\ memory size \NC \NR
-\NC \type{pdf_os_cntr}        \NC max \PDF\ object stream pointer \NC \NR
-\NC \type{pdf_os_objidx}      \NC \PDF\ object stream index \NC \NR
-\NC \type{pdf_ptr}            \NC not yet written \PDF\ bytes \NC \NR
-\NC \type{pool_ptr}           \NC string pool index \NC \NR
-\NC \type{pool_size}          \NC current size allocated for string characters \NC \NR
-\NC \type{save_size}          \NC save stack size \NC \NR
-\NC \type{shell_escape}       \NC \type {0} means disabled, \type {1} means anything is permitted, and \type {2} is restricted \NC \NR
-\NC \type{safer_option}       \NC \type {1} means safer is enforced \NC \NR
-\NC \type{kpse_used}          \NC \type {1} means that kpse is used \NC \NR
-\NC \type{stack_size}         \NC input stack size \NC \NR
-\NC \type{str_ptr}            \NC number of strings \NC \NR
-\NC \type{total_pages}        \NC number of written pages \NC \NR
-\NC \type{var_mem_max}        \NC number of allocated words for nodes \NC \NR
-\NC \type{var_used}           \NC variable (one|-|word) memory in use \NC \NR
-\NC \type{lc_collate}         \NC the value of \type {LC_COLLATE}  at startup time (becomes \type {C} at startup) \NC \NR
-\NC \type{lc_ctype}           \NC the value of \type {LC_CTYPE}    at startup time (becomes \type {C} at startup) \NC \NR
-%NC \type{lc_monetary}        \NC the value of \type {LC_MONETARY} at startup time \NC \NR
-\NC \type{lc_numeric}         \NC the value of \type {LC_NUMERIC}  at startup time \NC \NR
-%NC \type{lc_time}            \NC the value of \type {LC_TIME}     at startup time (becomes \type {C} at startup) \NC \NR
-\stoptabulate
-
-The error and warning messages can be wiped with the \type {resetmessages}
-function. A return value can be set with \type {setexitcode}.
-
-\section{The \type {tex} library}
-
-The \type {tex} table contains a large list of virtual internal \TEX\
-parameters that are partially writable.
-
-The designation \quote {virtual} means that these items are not properly defined
-in \LUA, but are only front\-ends that are handled by a metatable that operates
-on the actual \TEX\ values. As a result, most of the \LUA\ table operators (like
-\type {pairs} and \type {#}) do not work on such items.
-
-At the moment, it is possible to access almost every parameter that has these
-characteristics:
-
-\startitemize[packed]
-\item You can use it after \type {\the}
-\item It is a single token.
-\item Some special others, see the list below
-\stopitemize
-
-This excludes parameters that need extra arguments, like \type {\the\scriptfont}.
-
-The subset comprising simple integer and dimension registers are
-writable as well as readable (stuff like \type {\tracingcommands} and
-\type {\parindent}).
-
-\subsection{Internal parameter values}
-
-For all the parameters in this section, it is possible to access them directly
-using their names as index in the \type {tex} table, or by using one of the
-functions \type {tex.get} and \type {tex.set}.
-
-The exact parameters and return values differ depending on the actual parameter,
-and so does whether \type {tex.set} has any effect. For the parameters that {\it
-can\/} be set, it is possible to use \type {global} as the first argument to
-\type {tex.set}; this makes the assignment global instead of local.
-
-\startfunctioncall
-tex.set (["global",] <string> n, ...)
-... = tex.get (<string> n)
-\stopfunctioncall
-
-Glue is kind of special because there are five values involved. The return value
-is a \type {glue_spec} node btu when you pass \type {false} as last argument to
-\type {tex.get} you get the width of the glue and when you pass \type {true} you
-get all five values. Otherwise you get a node which is a copy of the internal
-value so you are responsible for its freeing at the \LUA\ end. When you set a
-glue quantity you can either pass a \type {glue_spec} or upto five numbers. If
-you pass \type {true} to \type {get} you get 5 values returned for a glue and
-when you pass \type {false} you only get the width returned.
-
-For the registers you can use \type {getskip} (node), \type {getglue} (numbers)
-\type {setskip} (node) and \type {setglue} (numbers). If you pass \type {false}
-as second argument to \type {getglue} you only get the width returned.
-
-There are also dedicated setters, getters and checkers:
-
-\startfunctioncall
-local d = tex.getdimen("foo")
-if tex.isdimen("bar") then
-    tex.setdimen("bar",d)
-end
-\stopfunctioncall
-
-There are such helpers for \type {dimen}, \type {count}, \type {skip}, \type
-{box} and \type {attribute} registers.
-
-\subsubsection{Integer parameters}
-
-The integer parameters accept and return \LUA\ numbers.
-
-Read|-|write:
-
-\starttwocolumns
-\starttyping
-tex.adjdemerits
-tex.binoppenalty
-tex.brokenpenalty
-tex.catcodetable
-tex.clubpenalty
-tex.day
-tex.defaulthyphenchar
-tex.defaultskewchar
-tex.delimiterfactor
-tex.displaywidowpenalty
-tex.doublehyphendemerits
-tex.endlinechar
-tex.errorcontextlines
-tex.escapechar
-tex.exhyphenpenalty
-tex.fam
-tex.finalhyphendemerits
-tex.floatingpenalty
-tex.globaldefs
-tex.hangafter
-tex.hbadness
-tex.holdinginserts
-tex.hyphenpenalty
-tex.interlinepenalty
-tex.language
-tex.lastlinefit
-tex.lefthyphenmin
-tex.linepenalty
-tex.localbrokenpenalty
-tex.localinterlinepenalty
-tex.looseness
-tex.mag
-tex.maxdeadcycles
-tex.month
-tex.newlinechar
-tex.outputpenalty
-tex.pausing
-tex.postdisplaypenalty
-tex.predisplaydirection
-tex.predisplaypenalty
-tex.pretolerance
-tex.relpenalty
-tex.righthyphenmin
-tex.savinghyphcodes
-tex.savingvdiscards
-tex.showboxbreadth
-tex.showboxdepth
-tex.time
-tex.tolerance
-tex.tracingassigns
-tex.tracingcommands
-tex.tracinggroups
-tex.tracingifs
-tex.tracinglostchars
-tex.tracingmacros
-tex.tracingnesting
-tex.tracingonline
-tex.tracingoutput
-tex.tracingpages
-tex.tracingparagraphs
-tex.tracingrestores
-tex.tracingscantokens
-tex.tracingstats
-tex.uchyph
-tex.vbadness
-tex.widowpenalty
-tex.year
-\stoptyping
-\stoptwocolumns
-
-Read|-|only:
-
-\startthreecolumns
-\starttyping
-tex.deadcycles
-tex.insertpenalties
-tex.parshape
-tex.prevgraf
-tex.spacefactor
-\stoptyping
-\stopthreecolumns
-
-\subsubsection{Dimension parameters}
-
-The dimension parameters accept \LUA\ numbers (signifying scaled points) or
-strings (with included dimension). The result is always a number in scaled
-points.
-
-Read|-|write:
-
-\startthreecolumns
-\starttyping
-tex.boxmaxdepth
-tex.delimitershortfall
-tex.displayindent
-tex.displaywidth
-tex.emergencystretch
-tex.hangindent
-tex.hfuzz
-tex.hoffset
-tex.hsize
-tex.lineskiplimit
-tex.mathsurround
-tex.maxdepth
-tex.nulldelimiterspace
-tex.overfullrule
-tex.pagebottomoffset
-tex.pageheight
-tex.pageleftoffset
-tex.pagerightoffset
-tex.pagetopoffset
-tex.pagewidth
-tex.parindent
-tex.predisplaysize
-tex.scriptspace
-tex.splitmaxdepth
-tex.vfuzz
-tex.voffset
-tex.vsize
-tex.prevdepth
-tex.prevgraf
-tex.spacefactor
-\stoptyping
-\stopthreecolumns
-
-Read|-|only:
-
-\startthreecolumns
-\starttyping
-tex.pagedepth
-tex.pagefilllstretch
-tex.pagefillstretch
-tex.pagefilstretch
-tex.pagegoal
-tex.pageshrink
-tex.pagestretch
-tex.pagetotal
-\stoptyping
-\stopthreecolumns
-
-Beware: as with all \LUA\ tables you can add values to them. So, the following is valid:
-
-\starttyping
-tex.foo = 123
-\stoptyping
-
-When you access a \TEX\ parameter a look up takes place. For read||only variables
-that means that you will get something back, but when you set them you create a
-new entry in the table thereby making the original invisible.
-
-There are a few special cases that we make an exception for: \type {prevdepth},
-\type {prevgraf} and \type {spacefactor}. These normally are accessed via the
-\type {tex.nest} table:
-
-\starttyping
-tex.nest[tex.nest.ptr].prevdepth   = p
-tex.nest[tex.nest.ptr].spacefactor = s
-\stoptyping
-
-However, the following also works:
-
-\starttyping
-tex.prevdepth   = p
-tex.spacefactor = s
-\stoptyping
-
-Keep in mind that when you mess with node lists directly at the \LUA\ end you
-might need to update the top of the nesting stack's \type {prevdepth} explicitly
-as there is no way \LUATEX\ can guess your intentions. By using the accessor in
-the \type {tex} tables, you get and set the values atthe top of the nest stack.
-
-\subsubsection{Direction parameters}
-
-The direction parameters are read|-|only and return a \LUA\ string.
-
-\startthreecolumns
-\starttyping
-tex.bodydir
-tex.mathdir
-tex.pagedir
-tex.pardir
-tex.textdir
-\stoptyping
-\stopthreecolumns
-
-\subsubsection{Glue parameters}
-
-The glue parameters accept and return a userdata object that represents a \type
-{glue_spec} node.
-
-\startthreecolumns
-\starttyping
-tex.abovedisplayshortskip
-tex.abovedisplayskip
-tex.baselineskip
-tex.belowdisplayshortskip
-tex.belowdisplayskip
-tex.leftskip
-tex.lineskip
-tex.parfillskip
-tex.parskip
-tex.rightskip
-tex.spaceskip
-tex.splittopskip
-tex.tabskip
-tex.topskip
-tex.xspaceskip
-\stoptyping
-\stopthreecolumns
-
-\subsubsection{Muglue parameters}
-
-All muglue parameters are to be used read|-|only and return a \LUA\ string.
-
-\startthreecolumns
-\starttyping
-tex.medmuskip
-tex.thickmuskip
-tex.thinmuskip
-\stoptyping
-\stopthreecolumns
-
-\subsubsection{Tokenlist parameters}
-
-The tokenlist parameters accept and return \LUA\ strings. \LUA\ strings are
-converted to and from token lists using \type {\the} \type {\toks} style expansion:
-all category codes are either space (10) or other (12). It follows that assigning
-to some of these, like \quote {tex.output}, is actually useless, but it feels bad
-to make exceptions in view of a coming extension that will accept full|-|blown
-token strings.
-
-\startthreecolumns
-\starttyping
-tex.errhelp
-tex.everycr
-tex.everydisplay
-tex.everyeof
-tex.everyhbox
-tex.everyjob
-tex.everymath
-tex.everypar
-tex.everyvbox
-tex.output
-\stoptyping
-\stopthreecolumns
-
-\subsection{Convert commands}
-
-All \quote {convert} commands are read|-|only and return a \LUA\ string. The
-supported commands at this moment are:
-
-\starttwocolumns
-\starttyping
-tex.eTeXVersion
-tex.eTeXrevision
-tex.formatname
-tex.jobname
-tex.luatexbanner
-tex.luatexrevision
-tex.fontname(number)
-tex.uniformdeviate(number)
-tex.number(number)
-tex.romannumeral(number)
-tex.fontidentifier(number)
-\stoptyping
-\stoptwocolumns
-
-If you are wondering why this list looks haphazard; these are all the cases of
-the \quote {convert} internal command that do not require an argument, as well as
-the ones that require only a simple numeric value.
-
-The special (lua-only) case of \type {tex.fontidentifier} returns the \type
-{csname} string that matches a font id number (if there is one).
-
-if these are really needed in a macro package.
-
-\subsection{Last item commands}
-
-All \quote {last item} commands are read|-|only and return a number.
-
-The supported commands at this moment are:
-
-\startthreecolumns
-\starttyping
-tex.lastpenalty
-tex.lastkern
-tex.lastskip
-tex.lastnodetype
-tex.inputlineno
-tex.lastxpos
-tex.lastypos
-tex.randomseed
-tex.luatexversion
-tex.eTeXminorversion
-tex.eTeXversion
-tex.currentgrouplevel
-tex.currentgrouptype
-tex.currentiflevel
-tex.currentiftype
-tex.currentifbranch
-\stoptyping
-\stopthreecolumns
-
-\subsection{Attribute, count, dimension, skip and token registers}
-
-\TEX's attributes (\type {\attribute}), counters (\type {\count}), dimensions (\type
-{\dimen}), skips (\type {\skip}) and token (\type {\toks}) registers can be accessed
-and written to using two times five virtual sub|-|tables of the \type {tex}
-table:
-
-\startthreecolumns
-\starttyping
-tex.attribute
-tex.count
-tex.dimen
-tex.skip
-tex.toks
-\stoptyping
-\stopthreecolumns
-
-It is possible to use the names of relevant \type {\attributedef}, \type {\countdef},
-\type {\dimendef}, \type {\skipdef}, or \type {\toksdef} control sequences as indices
-to these tables:
-
-\starttyping
-tex.count.scratchcounter = 0
-enormous = tex.dimen['maxdimen']
-\stoptyping
-
-In this case, \LUATEX\ looks up the value for you on the fly. You have to use a
-valid \type {\countdef} (or \type {\attributedef}, or \type {\dimendef}, or \type
-{\skipdef}, or \type {\toksdef}), anything else will generate an error (the intent
-is to eventually also allow \type {<chardef tokens>} and even macros that expand
-into a number).
-
-The attribute and count registers accept and return \LUA\ numbers.
-
-The dimension registers accept \LUA\ numbers (in scaled points) or strings (with
-an included absolute dimension; \type {em} and \type {ex} and \type {px} are
-forbidden). The result is always a number in scaled points.
-
-The token registers accept and return \LUA\ strings. \LUA\ strings are converted
-to and from token lists using \type {\the} \type {\toks} style expansion: all
-category codes are either space (10) or other (12).
-
-The skip registers accept and return \type {glue_spec} userdata node objects (see
-the description of the node interface elsewhere in this manual).
-
-As an alternative to array addressing, there are also accessor functions defined
-for all cases, for example, here is the set of possibilities for \type {\skip}
-registers:
-
-\startfunctioncall
-tex.setskip (["global",] <number> n, <node> s)
-tex.setskip (["global",] <string> s, <node> s)
-<node> s = tex.getskip (<number> n)
-<node> s = tex.getskip (<string> s)
-\stopfunctioncall
-
-We have similar setters for \type {count}, \type {dimen}, \type {muskip}, and
-\type {toks}. Counters and dimen are represented by numbers, skips and muskips by
-nodes, and toks by strings. For tokens registers we have an alternative where a
-catcode table is specified:
-
-\startfunctioncall
-tex.scantoks(0,3,"$e=mc^2$")
-tex.scantoks("global",0,"$\int\limits^1_2$")
-\stopfunctioncall
-
-In the function-based interface, it is possible to define values globally by
-using the string \type {global} as the first function argument.
-
-There are four extra skip related helpers:
-
-\startfunctioncall
-tex.setglue (["global"], <number> n,
-    width, stretch, shrink, stretch_order, shrink_order)
-tex.setglue (["global"], <string> s,
-    width, stretch, shrink, stretch_order, shrink_order)
-width, stretch, shrink, stretch_order, shrink_order =
-    tex.getglue (<number> n)
-width, stretch, shrink, stretch_order, shrink_order =
-    tex.getglue (<string> s)
-\stopfunctioncall
-
-The other two are \type {tex.setmuglue} and \type {tex.getmuglue}.
-
-\subsection{Character code registers}
-
-\TEX's character code tables (\type {\lccode}, \type {\uccode}, \type {\sfcode}, \type
-{\catcode}, \type {\mathcode}, \type {\delcode}) can be accessed and written to using
-six virtual subtables of the \type {tex} table
-
-\startthreecolumns
-\starttyping
-tex.lccode
-tex.uccode
-tex.sfcode
-tex.catcode
-tex.mathcode
-tex.delcode
-\stoptyping
-\stopthreecolumns
-
-The function call interfaces are roughly as above, but there are a few twists.
-\type {sfcode}s are the simple ones:
-
-\startfunctioncall
-tex.setsfcode (["global",] <number> n, <number> s)
-<number> s = tex.getsfcode (<number> n)
-\stopfunctioncall
-
-The function call interface for \type {lccode} and \type {uccode} additionally
-allows you to set the associated sibling at the same time:
-
-\startfunctioncall
-tex.setlccode (["global"], <number> n, <number> lc)
-tex.setlccode (["global"], <number> n, <number> lc, <number> uc)
-<number> lc = tex.getlccode (<number> n)
-tex.setuccode (["global"], <number> n, <number> uc)
-tex.setuccode (["global"], <number> n, <number> uc, <number> lc)
-<number> uc = tex.getuccode (<number> n)
-\stopfunctioncall
-
-The function call interface for \type {catcode} also allows you to specify a
-category table to use on assignment or on query (default in both cases is the
-current one):
-
-\startfunctioncall
-tex.setcatcode (["global"], <number> n, <number> c)
-tex.setcatcode (["global"], <number> cattable, <number> n, <number> c)
-<number> lc = tex.getcatcode (<number> n)
-<number> lc = tex.getcatcode (<number> cattable, <number> n)
-\stopfunctioncall
-
-The interfaces for \type {delcode} and \type {mathcode} use small array tables to
-set and retrieve values:
-
-\startfunctioncall
-tex.setmathcode (["global"], <number> n, <table> mval )
-<table> mval = tex.getmathcode (<number> n)
-tex.setdelcode (["global"], <number> n, <table> dval )
-<table> dval = tex.getdelcode (<number> n)
-\stopfunctioncall
-
-Where the table for \type {mathcode} is an array of 3 numbers, like this:
-
-\starttyping
-{
-    <number> class,
-    <number> family,
-    <number> character
-}
-\stoptyping
-
-And the table for \type {delcode} is an array with 4 numbers, like this:
-
-\starttyping
-{
-    <number> small_fam,
-    <number> small_char,
-    <number> large_fam,
-    <number> large_char
-}
-\stoptyping
-
-You can also avoid the table:
-
-\startfunctioncall
-tex.setmathcode (["global"], <number> n, <number> class,
-    <number> family, <number> character)
-class, family, char =
-    tex.getmathcodes (<number> n)
-tex.setdelcode (["global"], <number> n, <number> smallfam,
-    <number> smallchar, <number> largefam, <number> largechar)
-smallfam, smallchar, largefam, largechar =
-    tex.getdelcodes (<number> n)
-\stopfunctioncall
-
-Normally, the third and fourth values in a delimiter code assignment will be zero
-according to \type {\Udelcode} usage, but the returned table can have values there
-(if the delimiter code was set using \type {\delcode}, for example). Unset \type
-{delcode}'s can be recognized because \type {dval[1]} is $-1$.
-
-\subsection{Box registers}
-
-It is possible to set and query actual boxes, using the node interface as defined
-in the \type {node} library:
-
-\starttyping
-tex.box
-\stoptyping
-
-for array access, or
-
-\starttyping
-tex.setbox(["global",] <number> n, <node> s)
-tex.setbox(["global",] <string> cs, <node> s)
-<node> n = tex.getbox(<number> n)
-<node> n = tex.getbox(<string> cs)
-\stoptyping
-
-for function|-|based access. In the function-based interface, it is possible to
-define values globally by using the string \type {global} as the first function
-argument.
-
-Be warned that an assignment like
-
-\starttyping
-tex.box[0] = tex.box[2]
-\stoptyping
-
-does not copy the node list, it just duplicates a node pointer. If \type {\box2}
-will be cleared by \TEX\ commands later on, the contents of \type {\box0} becomes
-invalid as well. To prevent this from happening, always use \type
-{node.copy_list()} unless you are assigning to a temporary variable:
-
-\starttyping
-tex.box[0] = node.copy_list(tex.box[2])
-\stoptyping
-
-The following function will register a box for reuse (this is modelled after so
-called xforms in \PDF). You can (re)use the box with \type {\useboxresource} or
-by creating a rule node with subtype~2.
-
-\starttyping
-local index = tex.saveboxresource(n,attributes,resources,immediate,type,margin)
-\stoptyping
-
-The optional second and third arguments are strings, the fourth is a boolean. The
-fifth argument is a type. When set to non|-|zero the \type {/Type} entry is
-omitted. A value of 1 or 3 still writes a \type {/BBox}, while 2 or 3 will write
-a \type {/Matrix}. The sixth argument is the (virtual) margin that extends beyond
-the effective boundingbox as seen by \TEX.
-
-You can generate the reference (a rule type) with:
-
-\starttyping
-local reused = tex.useboxresource(n,wd,ht,dp)
-\stoptyping
-
-The dimensions are optional and the final ones are returned as extra values. The
-following is just a bonus (no dimensions returned means that the resource is
-unknown):
-
-\starttyping
-local w, h, d, m = tex.getboxresourcedimensions(n)
-\stoptyping
-
-This returns the width, height, depth and margin of the resource.
-
-You can split a box:
-
-\starttyping
-local vlist = tex.splitbox(n,height,mode)
-\stoptyping
-
-The remainder is kept in the original box and a packaged vlist is returned. This
-operation is comparable to the \type {\vsplit} operation. The mode can be \type
-{additional} or \type {exactly} and concerns the split off box.
-
-\subsection{Math parameters}
-
-It is possible to set and query the internal math parameters using:
-
-\startfunctioncall
-tex.setmath(["global",] <string> n, <string> t, <number> n)
-<number> n = tex.getmath(<string> n, <string> t)
-\stopfunctioncall
-
-As before an optional first parameter \type {global} indicates a global
-assignment.
-
-The first string is the parameter name minus the leading \quote {Umath}, and the
-second string is the style name minus the trailing \quote {style}. Just to be
-complete, the values for the math parameter name are:
-
-\starttyping
-quad                axis                operatorsize
-overbarkern         overbarrule         overbarvgap
-underbarkern        underbarrule        underbarvgap
-radicalkern         radicalrule         radicalvgap
-radicaldegreebefore radicaldegreeafter  radicaldegreeraise
-stackvgap           stacknumup          stackdenomdown
-fractionrule        fractionnumvgap     fractionnumup
-fractiondenomvgap   fractiondenomdown   fractiondelsize
-limitabovevgap      limitabovebgap      limitabovekern
-limitbelowvgap      limitbelowbgap      limitbelowkern
-underdelimitervgap  underdelimiterbgap
-overdelimitervgap   overdelimiterbgap
-subshiftdrop        supshiftdrop        subshiftdown
-subsupshiftdown     subtopmax           supshiftup
-supbottommin        supsubbottommax     subsupvgap
-spaceafterscript    connectoroverlapmin
-ordordspacing       ordopspacing        ordbinspacing     ordrelspacing
-ordopenspacing      ordclosespacing     ordpunctspacing   ordinnerspacing
-opordspacing        opopspacing         opbinspacing      oprelspacing
-opopenspacing       opclosespacing      oppunctspacing    opinnerspacing
-binordspacing       binopspacing        binbinspacing     binrelspacing
-binopenspacing      binclosespacing     binpunctspacing   bininnerspacing
-relordspacing       relopspacing        relbinspacing     relrelspacing
-relopenspacing      relclosespacing     relpunctspacing   relinnerspacing
-openordspacing      openopspacing       openbinspacing    openrelspacing
-openopenspacing     openclosespacing    openpunctspacing  openinnerspacing
-closeordspacing     closeopspacing      closebinspacing   closerelspacing
-closeopenspacing    closeclosespacing   closepunctspacing closeinnerspacing
-punctordspacing     punctopspacing      punctbinspacing   punctrelspacing
-punctopenspacing    punctclosespacing   punctpunctspacing punctinnerspacing
-innerordspacing     inneropspacing      innerbinspacing   innerrelspacing
-inneropenspacing    innerclosespacing   innerpunctspacing innerinnerspacing
-\stoptyping
-
-The values for the style parameter name are:
-
-\starttyping
-display       crampeddisplay
-text          crampedtext
-script        crampedscript
-scriptscript  crampedscriptscript
-\stoptyping
-
-The value is either a number (representing a dimension or number) or a glue spec
-node representing a muskip for \type {ordordspacing} and similar spacing
-parameters.
-
-\subsection{Special list heads}
-
-The virtual table \type {tex.lists} contains the set of internal registers that
-keep track of building page lists.
-
-\starttabulate[|l|p|]
-\BC field                      \BC description \NC \NR
-\NC \type{page_ins_head}       \NC circular list of pending insertions \NC \NR
-\NC \type{contrib_head}        \NC the recent contributions \NC \NR
-\NC \type{page_head}           \NC the current page content \NC \NR
-%NC \type{temp_head}           \NC \NC \NR
-\NC \type{hold_head}           \NC used for held-over items for next page \NC \NR
-\NC \type{adjust_head}         \NC head of the current \type {\vadjust} list \NC \NR
-\NC \type{pre_adjust_head}     \NC head of the current \type {\vadjust pre} list \NC \NR
-%NC \type{align_head}          \NC \NC \NR
-\NC \type{page_discards_head}  \NC head of the discarded items of a page break \NC \NR
-\NC \type{split_discards_head} \NC head of the discarded items in a vsplit \NC \NR
-\stoptabulate
-
-\subsection{Semantic nest levels}
-
-The virtual table \type {tex.nest} contains the currently active
-semantic nesting state. It has two main parts: a zero-based array of userdata for
-the semantic nest itself, and the numerical value \type {tex.nest.ptr}, which
-gives the highest available index. Neither the array items in \type {tex.nest[]}
-nor \type {tex.nest.ptr} can be assigned to (as this would confuse the
-typesetting engine beyond repair), but you can assign to the individual values
-inside the array items, e.g.\ \type {tex.nest[tex.nest.ptr].prevdepth}.
-
-\type {tex.nest[tex.nest.ptr]} is the current nest state, \type {tex.nest[0]} the
-outermost (main vertical list) level.
-
-The known fields are:
-
-\starttabulate[|l|l|l|p|]
-\BC key                \BC type    \BC modes \BC explanation \NC \NR
-\NC \type{mode}        \NC number  \NC all   \NC a number representing the main mode at this level:
-                                                 \type {0}    = no mode (this happens during \type {\write}),
-                                                 \type {1}    = vertical,
-                                                 \type {127}  = horizontal,
-                                                 \type {253}  = display math,
-                                                 \type {-1}   = internal vertical,
-                                                 \type {-127} = restricted horizontal,
-                                                 \type {-253} = inline math \NC \NR
-\NC \type{modeline}    \NC number  \NC all   \NC source input line where this mode was entered in,
-                                                 negative inside the output routine \NC \NR
-\NC \type{head}        \NC node    \NC all   \NC the head of the current list \NC \NR
-\NC \type{tail}        \NC node    \NC all   \NC the tail of the current list \NC \NR
-\NC \type{prevgraf}    \NC number  \NC vmode \NC number of lines in the previous paragraph \NC \NR
-\NC \type{prevdepth}   \NC number  \NC vmode \NC depth of the previous paragraph \NC \NR
-\NC \type{spacefactor} \NC number  \NC hmode \NC the current space factor \NC \NR
-\NC \type{dirs}        \NC node    \NC hmode \NC used for temporary storage by the line break algorithm\NC \NR
-\NC \type{noad}        \NC node    \NC mmode \NC used for temporary storage of a pending fraction numerator,
-                                                 for \type {\over} etc. \NC \NR
-\NC \type{delimptr}    \NC node    \NC mmode \NC used for temporary storage of the previous math delimiter,
-                                                 for \type {\middle} \NC \NR
-\NC \type{mathdir}     \NC boolean \NC mmode \NC true when during math processing the \type {\mathdir} is not
-                                                 the same as the surrounding \type {\textdir} \NC \NR
-\NC \type{mathstyle}   \NC number  \NC mmode \NC the current \type {\mathstyle} \NC \NR
-\stoptabulate
-
-\subsection[sec:luaprint]{Print functions}
-
-The \type {tex} table also contains the three print functions that are the
-major interface from \LUA\ scripting to \TEX.
-
-The arguments to these three functions are all stored in an in|-|memory virtual
-file that is fed to the \TEX\ scanner as the result of the expansion of
-\type {\directlua}.
-
-The total amount of returnable text from a \type {\directlua} command is only
-limited by available system \RAM. However, each separate printed string has to
-fit completely in \TEX's input buffer.
-
-The result of using these functions from inside callbacks is undefined
-at the moment.
-
-\subsubsection{\type {tex.print}}
-
-\startfunctioncall
-tex.print(<string> s, ...)
-tex.print(<number> n, <string> s, ...)
-tex.print(<table> t)
-tex.print(<number> n, <table> t)
-\stopfunctioncall
-
-Each string argument is treated by \TEX\ as a separate input line. If there is a
-table argument instead of a list of strings, this has to be a consecutive array
-of strings to print (the first non-string value will stop the printing process).
-
-The optional parameter can be used to print the strings using the catcode regime
-defined by \type {\catcodetable}~\type {n}. If \type {n} is $-1$, the currently
-active catcode regime is used. If \type {n} is $-2$, the resulting catcodes are
-the result of \type {\the} \type {\toks}: all category codes are 12 (other) except for
-the space character, that has category code 10 (space). Otherwise, if \type {n}
-is not a valid catcode table, then it is ignored, and the currently active
-catcode regime is used instead.
-
-The very last string of the very last \type {tex.print()} command in a \type
-{\directlua} will not have the \type {\endlinechar} appended, all others do.
-
-\subsubsection{\type {tex.sprint}}
-
-\startfunctioncall
-tex.sprint(<string> s, ...)
-tex.sprint(<number> n, <string> s, ...)
-tex.sprint(<table> t)
-tex.sprint(<number> n, <table> t)
-\stopfunctioncall
-
-Each string argument is treated by \TEX\ as a special kind of input line that
-makes it suitable for use as a partial line input mechanism:
-
-\startitemize[packed]
-\startitem
-    \TEX\ does not switch to the \quote {new line} state, so that leading spaces
-    are not ignored.
-\stopitem
-\startitem
-    No \type {\endlinechar} is inserted.
-\stopitem
-\startitem
-    Trailing spaces are not removed.
-
-    Note that this does not prevent \TEX\ itself from eating spaces as result of
-    interpreting the line. For example, in
-
-\starttyping
-before\directlua{tex.sprint("\\relax")tex.sprint(" inbetween")}after
-\stoptyping
-    the space before \type {in between} will be gobbled as a result of the \quote
-    {normal} scanning of \type {\relax}.
-\stopitem
-\stopitemize
-
-If there is a table argument instead of a list of strings, this has to
-be a consecutive array of strings to print (the first non-string value
-will stop the printing process).
-
-The optional argument sets the catcode regime, as with \type {tex.print()}.
-
-\subsubsection{\type {tex.tprint}}
-
-\startfunctioncall
-tex.tprint({<number> n, <string> s, ...}, {...})
-\stopfunctioncall
-
-This function is basically a shortcut for repeated calls to \type
-{tex.sprint(<number> n, <string> s, ...)}, once for each of the supplied argument
-tables.
-
-\subsubsection{\type {tex.cprint}}
-
-This function takes a number indicating the to be used catcode, plus either a
-table of strings or an argument list of strings that will be pushed into the
-input stream.
-
-\startfunctioncall
-tex.cprint( 1," 1: $&{\\foo}") tex.print("\\par") -- a lot of \bgroup s
-tex.cprint( 2," 2: $&{\\foo}") tex.print("\\par") -- matching \egroup s
-tex.cprint( 9," 9: $&{\\foo}") tex.print("\\par") -- all get ignored
-tex.cprint(10,"10: $&{\\foo}") tex.print("\\par") -- all become spaces
-tex.cprint(11,"11: $&{\\foo}") tex.print("\\par") -- letters
-tex.cprint(12,"12: $&{\\foo}") tex.print("\\par") -- other characters
-tex.cprint(14,"12: $&{\\foo}") tex.print("\\par") -- comment triggers
-\stopfunctioncall
-
-\subsubsection{\type {tex.write}}
-
-\startfunctioncall
-tex.write(<string> s, ...)
-tex.write(<table> t)
-\stopfunctioncall
-
-Each string argument is treated by \TEX\ as a special kind of input line that
-makes it suitable for use as a quick way to dump information:
-
-\startitemize
-\item All catcodes on that line are either \quote{space} (for '~') or
-     \quote{character} (for all others).
-\item There is no \type {\endlinechar} appended.
-\stopitemize
-
-If there is a table argument instead of a list of strings, this has to be a
-consecutive array of strings to print (the first non-string value will stop the
-printing process).
-
-\subsection{Helper functions}
-
-\subsubsection{\type {tex.round}}
-
-\startfunctioncall
-<number> n = tex.round(<number> o)
-\stopfunctioncall
-
-Rounds \LUA\ number \type {o}, and returns a number that is in the range of a
-valid \TEX\ register value. If the number starts out of range, it generates a
-\quote {number to big} error as well.
-
-\subsubsection{\type {tex.scale}}
-
-\startfunctioncall
-<number> n = tex.scale(<number> o, <number> delta)
-<table> n = tex.scale(table o, <number> delta)
-\stopfunctioncall
-
-Multiplies the \LUA\ numbers \type {o} and \type {delta}, and returns a rounded
-number that is in the range of a valid \TEX\ register value. In the table
-version, it creates a copy of the table with all numeric top||level values scaled
-in that manner. If the multiplied number(s) are of range, it generates
-\quote{number to big} error(s) as well.
-
-Note: the precision of the output of this function will depend on your computer's
-architecture and operating system, so use with care! An interface to \LUATEX's
-internal, 100\% portable scale function will be added at a later date.
-
-\subsubsection{\type {tex.sp}}
-
-\startfunctioncall
-<number> n = tex.sp(<number> o)
-<number> n = tex.sp(<string> s)
-\stopfunctioncall
-
-Converts the number \type {o} or a string \type {s} that represents an explicit
-dimension into an integer number of scaled points.
-
-For parsing the string, the same scanning and conversion rules are used that
-\LUATEX\ would use if it was scanning a dimension specifier in its \TEX|-|like
-input language (this includes generating errors for bad values), expect for the
-following:
-
-\startitemize[n]
-\startitem
-    only explicit values are allowed, control sequences are not handled
-\stopitem
-\startitem
-    infinite dimension units (\type {fil...}) are forbidden
-\stopitem
-\startitem
-    \type {mu} units do not generate an error (but may not be useful either)
-\stopitem
-\stopitemize
-
-\subsubsection{\type {tex.definefont}}
-
-\startfunctioncall
-tex.definefont(<string> csname, <number> fontid)
-tex.definefont(<boolean> global, <string> csname, <number> fontid)
-\stopfunctioncall
-
-Associates \type {csname} with the internal font number \type {fontid}. The
-definition is global if (and only if) \type {global} is specified and true (the
-setting of \type {globaldefs} is not taken into account).
-
-\subsubsection{\type {tex.getlinenumber} and \type {tex.setlinenumber}}
-
-You can mess with the current line number:
-
-\startfunctioncall
-local n = tex.getlinenumber()
-tex.setlinenumber(n+10)
-\stopfunctioncall
-
-which can be shortcut to:
-
-\startfunctioncall
-tex.setlinenumber(10,true)
-\stopfunctioncall
-
-This might be handy when you have a callback that read numbers from a file and
-combines them in one line (in which case an error message probably has to refer
-to the original line). Interference with \TEX's internal handling of numbers is
-of course possible.
-
-\subsubsection{\type {tex.error}}
-
-\startfunctioncall
-tex.error(<string> s)
-tex.error(<string> s, <table> help)
-\stopfunctioncall
-
-This creates an error somewhat like the combination of \type {\errhelp} and \type
-{\errmessage} would. During this error, deletions are disabled.
-
-The array part of the \type {help} table has to contain strings, one for each
-line of error help.
-
-\subsubsection{\type {tex.hashtokens}}
-
-\startfunctioncall
-for i,v in pairs (tex.hashtokens()) do ... end
-\stopfunctioncall
-
-Returns a list of names. This can be useful for debugging, but note that this
-also reports control sequences that may be unreachable at this moment due to
-local redefinitions: it is strictly a dump of the hash table. You can use \type
-{token.create} to inspect properties, for instance when the \type {command} key
-in a created table equals \type {123}, you have the \type {cmdname} value \type
-{undefined_cs}.
-
-\subsection[luaprimitives]{Functions for dealing with primitives}
-
-\subsubsection{\type {tex.enableprimitives}}
-
-\startfunctioncall
-tex.enableprimitives(<string> prefix, <table> primitive names)
-\stopfunctioncall
-
-This function accepts a prefix string and an array of primitive names.
-
-For each combination of \quote {prefix} and \quote {name}, the \type
-{tex.enableprimitives} first verifies that \quote {name} is an actual primitive
-(it must be returned by one of the \type {tex.extraprimitives()} calls explained
-below, or part of \TEX82, or \type {\directlua}). If it is not, \type
-{tex.enableprimitives} does nothing and skips to the next pair.
-
-But if it is, then it will construct a csname variable by concatenating the
-\quote {prefix} and \quote {name}, unless the \quote {prefix} is already the
-actual prefix of \quote {name}. In the latter case, it will discard the \quote
-{prefix}, and just use \quote {name}.
-
-Then it will check for the existence of the constructed csname. If the csname is
-currently undefined (note: that is not the same as \type {\relax}), it will
-globally define the csname to have the meaning: run code belonging to the
-primitive \quote {name}. If for some reason the csname is already defined, it
-does nothing and tries the next pair.
-
-An example:
-
-\starttyping
-  tex.enableprimitives('LuaTeX', {'formatname'})
-\stoptyping
-
-will define \type {\LuaTeXformatname} with the same intrinsic meaning as the
-documented primitive \type {\formatname}, provided that the control sequences \type
-{\LuaTeXformatname} is currently undefined.
-
-When \LUATEX\ is run with \type {--ini} only the \TEX82 primitives and \type
-{\directlua} are available, so no extra primitives {\bf at all}.
-
-If you want to have all the new functionality available using their default
-names, as it is now, you will have to add
-
-\starttyping
-  \ifx\directlua\undefined \else
-     \directlua {tex.enableprimitives('',tex.extraprimitives ())}
-  \fi
-\stoptyping
-
-near the beginning of your format generation file. Or you can choose different
-prefixes for different subsets, as you see fit.
-
-Calling some form of \type {tex.enableprimitives()} is highly important though,
-because if you do not, you will end up with a \TEX82-lookalike that can run \LUA\
-code but not do much else. The defined csnames are (of course) saved in the
-format and will be available at runtime.
-
-\subsubsection{\type {tex.extraprimitives}}
-
-\startfunctioncall
-<table> t = tex.extraprimitives(<string> s, ...)
-\stopfunctioncall
-
-This function returns a list of the primitives that originate from the engine(s)
-given by the requested string value(s). The possible values and their (current)
-return values are:
-
-\startluacode
-function document.showprimitives(tag)
-    for k, v in table.sortedpairs(tex.extraprimitives(tag)) do
-        if v == ' ' then
-            v = '\\normalcontrolspace'
-        end
-        context.type(v)
-        context.space()
-    end
-end
-\stopluacode
-
-\starttabulate[|l|pl|]
-\BC name    \BC values \NC \NR
-\NC tex     \NC \ctxlua{document.showprimitives('tex')    } \NC \NR
-\NC core    \NC \ctxlua{document.showprimitives('core')   } \NC \NR
-\NC etex    \NC \ctxlua{document.showprimitives('etex')   } \NC \NR
-\NC luatex  \NC \ctxlua{document.showprimitives('luatex') } \NC \NR
-\stoptabulate
-
-Note that \type {luatex} does not contain \type {directlua}, as that is
-considered to be a core primitive, along with all the \TEX82 primitives, so it is
-part of the list that is returned from \type {'core'}.
-
-% \type {'umath'} is a subset of \type {'luatex'} that covers the Unicode math
-% primitives as it might be desired to handle the prefixing of that subset
-% differently.
-
-Running \type {tex.extraprimitives()} will give you the complete list of
-primitives \type {-ini} startup. It is exactly equivalent to \type
-{tex.extraprimitives("etex","luatex")}.
-
-\subsubsection{\type {tex.primitives}}
-
-\startfunctioncall
-<table> t = tex.primitives()
-\stopfunctioncall
-
-This function returns a list of all primitives that \LUATEX\ knows about.
-
-\subsection{Core functionality interfaces}
-
-\subsubsection{\type {tex.badness}}
-
-\startfunctioncall
-<number> b = tex.badness(<number> t, <number> s)
-\stopfunctioncall
-
-This helper function is useful during linebreak calculations. \type {t} and \type
-{s} are scaled values; the function returns the badness for when total \type {t}
-is supposed to be made from amounts that sum to \type {s}. The returned number is
-a reasonable approximation of $100(t/s)^3$;
-
-\subsubsection{\type {tex.resetparagraph}}
-
-This function resets the parameters that \TEX\ normally resets when a new paragraph
-is seen.
-
-\subsubsection{\type {tex.linebreak}}
-
-\startfunctioncall
-local <node> nodelist, <table> info =
-       tex.linebreak(<node> listhead, <table> parameters)
-\stopfunctioncall
-
-The understood parameters are as follows:
-
-\starttabulate[|l|l|p|]
-\BC name                        \BC type            \BC description \NC \NR
-\NC \type{pardir}               \NC string          \NC \NC \NR
-\NC \type{pretolerance}         \NC number          \NC \NC \NR
-\NC \type{tracingparagraphs}    \NC number          \NC \NC \NR
-\NC \type{tolerance}            \NC number          \NC \NC \NR
-\NC \type{looseness}            \NC number          \NC \NC \NR
-\NC \type{hyphenpenalty}        \NC number          \NC \NC \NR
-\NC \type{exhyphenpenalty}      \NC number          \NC \NC \NR
-\NC \type{pdfadjustspacing}     \NC number          \NC \NC \NR
-\NC \type{adjdemerits}          \NC number          \NC \NC \NR
-\NC \type{pdfprotrudechars}     \NC number          \NC \NC \NR
-\NC \type{linepenalty}          \NC number          \NC \NC \NR
-\NC \type{lastlinefit}          \NC number          \NC \NC \NR
-\NC \type{doublehyphendemerits} \NC number          \NC \NC \NR
-\NC \type{finalhyphendemerits}  \NC number          \NC \NC \NR
-\NC \type{hangafter}            \NC number          \NC \NC \NR
-\NC \type{interlinepenalty}     \NC number or table \NC if a table, then it is an array like \type {\interlinepenalties} \NC \NR
-\NC \type{clubpenalty}          \NC number or table \NC if a table, then it is an array like \type {\clubpenalties} \NC \NR
-\NC \type{widowpenalty}         \NC number or table \NC if a table, then it is an array like \type {\widowpenalties} \NC \NR
-\NC \type{brokenpenalty}        \NC number          \NC \NC \NR
-\NC \type{emergencystretch}     \NC number          \NC in scaled points \NC \NR
-\NC \type{hangindent}           \NC number          \NC in scaled points \NC \NR
-\NC \type{hsize}                \NC number          \NC in scaled points \NC \NR
-\NC \type{leftskip}             \NC glue_spec node  \NC \NC \NR
-\NC \type{rightskip}            \NC glue_spec node  \NC \NC \NR
-\NC \type{parshape}             \NC table           \NC \NC \NR
-\stoptabulate
-
-Note that there is no interface for \type {\displaywidowpenalties}, you have to
-pass the right choice for \type {widowpenalties} yourself.
-
-It is your own job to make sure that \type {listhead} is a proper paragraph list:
-this function does not add any nodes to it. To be exact, if you want to replace
-the core line breaking, you may have to do the following (when you are not
-actually working in the \type {pre_linebreak_filter} or \type {linebreak_filter}
-callbacks, or when the original list starting at listhead was generated in
-horizontal mode):
-
-\startitemize
-\startitem
-    add an \quote {indent box} and perhaps a \type {local_par} node at the start
-    (only if you need them)
-\stopitem
-\startitem
-    replace any found final glue by an infinite penalty (or add such a penalty,
-    if the last node is not a glue)
-\stopitem
-\startitem
-    add a glue node for the \type {\parfillskip} after that penalty node
-\stopitem
-\startitem
-    make sure all the \type {prev} pointers are OK
-\stopitem
-\stopitemize
-
-The result is a node list, it still needs to be vpacked if you want to assign it
-to a \type {\vbox}.
-
-The returned \type {info} table contains four values that are all numbers:
-
-\starttabulate[|l|p|]
-\NC prevdepth \NC depth of the last line in the broken paragraph \NC \NR
-\NC prevgraf  \NC number of lines in the broken paragraph \NC \NR
-\NC looseness \NC the actual looseness value in the broken paragraph \NC \NR
-\NC demerits  \NC the total demerits of the chosen solution  \NC \NR
-\stoptabulate
-
-Note there are a few things you cannot interface using this function: You cannot
-influence font expansion other than via \type {pdfadjustspacing}, because the
-settings for that take place elsewhere. The same is true for hbadness and hfuzz
-etc. All these are in the \type {hpack()} routine, and that fetches its own
-variables via globals.
-
-\subsubsection{\type {tex.shipout}}
-
-\startfunctioncall
-tex.shipout(<number> n)
-\stopfunctioncall
-
-Ships out box number \type {n} to the output file, and clears the box register.
-
-\subsection[synctex]{Functions related to synctex}
-
-The next helpers only make sense when you implement your own synctex logic. Keep in
-mind that the library used in editors assumes a certain logic and is geared for
-plain and \LATEX, so after a decade users expect a certain behaviour.
-
-\starttabulate[|l|p|]
-\NC \type{set_synctex_mode}     \NC \type {0} is the default and used normal synctex
-                                    logic, \type {1} uses the values set by the next
-                                    helpers while \type {2} also sets these for glyph
-                                    nodes; \type{3} sets glyphs and glue and \type {4}
-                                    sets only glyphs \NC \NR
-\NC \type{set_synctex_tag}      \NC set the current tag (file) value (obeys save stack) \NC \NR
-\NC \type{set_synctex_line}     \NC set the current line value (obeys save stack) \NC \NR
-\NC \type{force_synctex_tag}    \NC overload the tag (file) value (\type {0} resets) \NC \NR
-\NC \type{force_synctex_line}   \NC overload the line value  (\type {0} resets) \NC \NR
-\NC \type{get_synctex_tag}      \NC get the currently set value of tag (file) \NC \NR
-\NC \type{get_synctex_line}     \NC get the currently set value of line \NC \NR
-\NC \type{set_synctex_no_files} \NC disable synctex file logging \NC \NR
-\stoptabulate
-
-The last one is somewhat special. Due to the way files are registered in \SYNCTEX\ we need
-to explicitly disable that feature if we provide our own alternative if we want to avoid
-that overhead. Passing a value of 1 disables registering.
-
-This mechanism is somewhat experimental.
-
-\section[texconfig]{The \type {texconfig} table}
-
-This is a table that is created empty. A startup \LUA\ script could
-fill this table with a number of settings that are read out by
-the executable after loading and executing the startup file.
-
-\starttabulate[|l|l|l|p|]
-\BC key                   \BC type     \BC default \BC explanation \NC \NR
-\NC \type{kpse_init}      \NC boolean  \NC true
-\NC
-    \type {false} totally disables \KPATHSEA\ initialisation, and enables
-    interpretation of the following numeric key--value pairs. (only ever unset
-    this if you implement {\it all\/} file find callbacks!)
-\NC \NR
-\NC
-    \type{shell_escape}   \NC string   \NC \type {'f'} \NC
-    Use \type {'y'} or \type {'t'} or \type {'1'} to enable \type {\write18}
-    unconditionally, \type {'p'} to enable the commands that are listed in \type
-    {shell_escape_commands}
-\NC \NR
-\NC
-    shell_escape_commands \NC string \NC \NC Comma-separated list of command
-    names that may be executed by \type {\write18} even if \type {shell_escape}
-    is set to \type {'p'}. Do {\it not\/} use spaces around commas, separate any
-    required command arguments by using a space, and use the \ASCII\ double quote
-    (\type {"}) for any needed argument or path quoting
-\NC \NR
-\NC \type{string_vacancies} \NC number   \NC  75000  \NC cf.\ web2c docs \NC \NR
-\NC \type{pool_free}        \NC number   \NC   5000  \NC cf.\ web2c docs \NC \NR
-\NC \type{max_strings}      \NC number   \NC  15000  \NC cf.\ web2c docs \NC \NR
-\NC \type{strings_free}     \NC number   \NC    100  \NC cf.\ web2c docs \NC \NR
-\NC \type{nest_size}        \NC number   \NC     50  \NC cf.\ web2c docs \NC \NR
-\NC \type{max_in_open}      \NC number   \NC     15  \NC cf.\ web2c docs \NC \NR
-\NC \type{param_size}       \NC number   \NC     60  \NC cf.\ web2c docs \NC \NR
-\NC \type{save_size}        \NC number   \NC   4000  \NC cf.\ web2c docs \NC \NR
-\NC \type{stack_size}       \NC number   \NC    300  \NC cf.\ web2c docs \NC \NR
-\NC \type{dvi_buf_size}     \NC number   \NC  16384  \NC cf.\ web2c docs \NC \NR
-\NC \type{error_line}       \NC number   \NC     79  \NC cf.\ web2c docs \NC \NR
-\NC \type{half_error_line}  \NC number   \NC     50  \NC cf.\ web2c docs \NC \NR
-\NC \type{max_print_line}   \NC number   \NC     79  \NC cf.\ web2c docs \NC \NR
-\NC \type{hash_extra}       \NC number   \NC      0  \NC cf.\ web2c docs \NC \NR
-\NC \type{pk_dpi}           \NC number   \NC     72  \NC cf.\ web2c docs \NC \NR
-\NC \type{trace_file_names} \NC boolean  \NC true
-\NC
-    \type {false} disables \TEX's normal file open|-|close feedback (the
-    assumption is that callbacks will take care of that)
-\NC \NR
-\NC \type{file_line_error}  \NC boolean  \NC false
-\NC
-    do \type {file:line} style error messages
-\NC \NR
-\NC \type{halt_on_error}    \NC boolean  \NC false
-\NC
-    abort run on the first encountered error
-\NC \NR
-\NC \type{formatname}       \NC string   \NC
-\NC
-    if no format name was given on the command line, this key will be tested first
-    instead of simply quitting
-\NC \NR
-\NC \type{jobname}        \NC string   \NC
-\NC
-    if no input file name was given on the command line, this key will be tested
-    first instead of simply giving up
-\NC \NR
-\stoptabulate
-
-Note: the numeric values that match web2c parameters are only used if \type
-{kpse_init} is explicitly set to \type {false}. In all other cases, the normal
-values from \type {texmf.cnf} are used.
-
-\section{The \type {texio} library}
-
-This library takes care of the low|-|level I/O interface: writing to the log file
-and|/|or console.
-
-\subsection{\type {texio.write}}
-
-\startfunctioncall
-texio.write(<string> target, <string> s, ...)
-texio.write(<string> s, ...)
-\stopfunctioncall
-
-Without the \type {target} argument, writes all given strings to the same
-location(s) \TEX\ writes messages to at this moment. If \type {\batchmode} is in
-effect, it writes only to the log, otherwise it writes to the log and the
-terminal. The optional \type {target} can be one of three possibilities: \type
-{term}, \type {log} or \type {term and log}.
-
-Note: If several strings are given, and if the first of these strings is or might
-be one of the targets above, the \type {target} must be specified explicitly to
-prevent \LUA\ from interpreting the first string as the target.
-
-\subsection{\type {texio.write_nl}}
-
-\startfunctioncall
-texio.write_nl(<string> target, <string> s, ...)
-texio.write_nl(<string> s, ...)
-\stopfunctioncall
-
-This function behaves like \type {texio.write}, but make sure that the given
-strings will appear at the beginning of a new line. You can pass a single empty
-string if you only want to move to the next line.
-
-\subsection{\type {texio.setescape}}
-
-You can disable \type {^^} escaping of control characters by passing a value of
-zero.
-
-\section{The \type {token} library}
-
-\subsection{The scanner}
-
-The token library provides means to intercept the input and deal with it at the
-\LUA\ level. The library provides a basic scanner infrastructure that can be used
-to write macros that accept a wide range of arguments. This interface is on
-purpose kept general and as performance is quite ok one can build additional
-parsers without too much overhead. It's up to macro package writers to see how
-they can benefit from this as the main principle behind \LUATEX\ is to provide a
-minimal set of tools and no solutions. The functions provided in the \type
-{token} namespace are given in the next table:
-
-\starttabulate[|l|l|p|]
-\BC function            \BC argument           \BC result \NC \NR
-\HL
-\NC \type{is_token}     \NC token              \NC checks if the given argument is a token userdatum \NC \NR
-\NC \type{get_next}     \NC                    \NC returns the next token in the input \NC \NR
-\NC \type{scan_keyword} \NC string             \NC returns true if the given keyword is gobbled \NC \NR
-\NC \type{scan_int}     \NC                    \NC returns a number \NC \NR
-\NC \type{scan_dimen}   \NC infinity, mu-units \NC returns a number representing a dimension and or two numbers being the filler and order \NC \NR
-\NC \type{scan_glue}    \NC mu-units           \NC returns a glue spec node \NC \NR
-\NC \type{scan_toks}    \NC definer, expand    \NC returns a table of tokens token list (this can become a linked list in later releases) \NC \NR
-\NC \type{scan_code}    \NC bitset             \NC returns a character if its category is in the given bitset (representing catcodes) \NC \NR
-\NC \type{scan_string}  \NC                    \NC returns a string given between \type {{}}, as \type {\macro} or as sequence of characters with catcode 11 or 12 \NC \NR
-\NC \type{scan_word}    \NC                    \NC returns a sequence of characters with catcode 11 or 12 as string \NC \NR
-\NC \type{scan_csname}  \NC                    \NC returns \type {foo} after scanning \type {\foo} \NC \NR
-\NC \type{set_macro}    \NC see below          \NC assign a macro \NC \NR
-\NC \type{create}       \NC                    \NC returns a userdata token object of the given control sequence name (or character); this interface can change  \NC \NR
-\stoptabulate
-
-The scanners can be considered stable apart from the one scanning for a token.
-This is because futures releases can return a linked list instead of a table (as
-with nodes). The \type {scan_code} function takes an optional number, the \type
-{keyword} function a normal \LUA\ string. The \type {infinity} boolean signals
-that we also permit \type {fill} as dimension and the \type {mu-units} flags the
-scanner that we expect math units. When scanning tokens we can indicate that we
-are defining a macro, in which case the result will also provide information
-about what arguments are expected and in the result this is separated from the
-meaning by a separator token. The \type {expand} flag determines if the list will
-be expanded.
-
-The string scanner scans for something between curly braces and expands on the
-way, or when it sees a control sequence it will return its meaning. Otherwise it
-will scan characters with catcode \type {letter} or \type {other}. So, given the
-following definition:
-
-\startbuffer
-\def\bar{bar}
-\def\foo{foo-\bar}
-\stopbuffer
-
-\typebuffer \getbuffer
-
-we get:
-
-\starttabulate[|l|Tl|l|]
-\NC \type {\directlua{token.scan_string()}{foo}} \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} {foo} \NC full expansion \NC \NR
-\NC \type {\directlua{token.scan_string()}foo}   \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} foo   \NC letters and others \NC \NR
-\NC \type {\directlua{token.scan_string()}\foo}  \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")}\foo   \NC meaning \NC \NR
-\stoptabulate
-
-The \type {\foo} case only gives the meaning, but one can pass an already
-expanded definition (\type {\edef}'d). In the case of the braced variant one can of
-course use the \type {\detokenize} and \type {\unexpanded} primitives as there we
-do expand.
-
-The \type {scan_word} scanner can be used to implement for instance a number scanner:
-
-\starttyping
-function token.scan_number(base)
-    return tonumber(token.scan_word(),base)
-end
-\stoptyping
-
-This scanner accepts any valid \LUA\ number so it is a way to pick up floats
-in the input.
-
-The creator function can be used as follows:
-
-\starttyping
-local t = token.create("relax")
-\stoptyping
-
-This gives back a token object that has the properties of the \type {\relax}
-primitive. The possible properties of tokens are:
-
-\starttabulate[|l|p|]
-\NC \type {command}   \NC a number representing the internal command number \NC \NR
-\NC \type {cmdname}   \NC the type of the command (for instance the catcode in case of a
-                          character or the classifier that determines the internal
-                          treatment \NC \NR
-\NC \type{csname}     \NC the associated control sequence (if applicable) \NC \NR
-\NC \type{id}         \NC the unique id of the token \NC \NR
-%NC \type{tok}        \NC \NC \NR % might change
-\NC \type{active}     \NC a boolean indicating the active state of the token \NC \NR
-\NC \type{expandable} \NC a boolean indicating if the token (macro) is expandable \NC \NR
-\NC \type{protected}  \NC a boolean indicating if the token (macro) is protected \NC \NR
-\stoptabulate
-
-The numbers that represent a catcode are the same as in \TEX\ itself, so using
-this information assumes that you know a bit about \TEX's internals. The other
-numbers and names are used consistently but are not frozen. So, when you use them
-for comparing you can best query a known primitive or character first to see the
-values.
-
-More interesting are the scanners. You can use the \LUA\ interface as follows:
-
-\starttyping
-\directlua {
-    function mymacro(n)
-        ...
-    end
-}
-
-\def\mymacro#1{%
-    \directlua {
-        mymacro(\number\dimexpr#1)
-    }%
-}
-
-\mymacro{12pt}
-\mymacro{\dimen0}
-\stoptyping
-
-You can also do this:
-
-\starttyping
-\directlua {
-    function mymacro()
-        local d = token.scan_dimen()
-        ...
-    end
-}
-
-\def\mymacro{%
-    \directlua {
-        mymacro()
-    }%
-}
-
-\mymacro 12pt
-\mymacro \dimen0
-\stoptyping
-
-It is quite clear from looking at the code what the first method needs as
-argument(s). For the second method you need to look at the \LUA\ code to see what
-gets picked up. Instead of passing from \TEX\ to \LUA\ we let \LUA\ fetch from
-the input stream.
-
-In the first case the input is tokenized and then turned into a string when it's
-passed to \LUA\ where it gets interpreted. In the second case only a function
-call gets interpreted but then the input is picked up by explicitly calling the
-scanner functions. These return proper \LUA\ variables so no further conversion
-has to be done. This is more efficient but in practice (given what \TEX\ has to
-do) this effect should not be overestimated. For numbers and dimensions it saves a
-bit but for passing strings conversion to and from tokens has to be done anyway
-(although we can probably speed up the process in later versions if needed).
-
-\subsection{Macros}
-
-The \type {set_macro} function can get upto 4 arguments:
-
-\starttyping
-setmacro("csname","content")
-setmacro("csname","content","global")
-setmacro("csname")
-\stoptyping
-
-You can pass a catcodetable identifier as first argument:
-
-\starttyping
-setmacro(catcodetable,"csname","content")
-setmacro(catcodetable,"csname","content","global")
-setmacro(catcodetable,"csname")
-\stoptyping
-
-The results are like:
-
-\starttyping
- \def\csname{content}
-\gdef\csname{content}
- \def\csname{}
-\stoptyping
-
-\subsection{Pushing back}
-
-There is a (for now) experimental putter:
-
-\starttyping
-local t1 = token.get_next()
-local t2 = token.get_next()
-local t3 = token.get_next()
-local t4 = token.get_next()
--- watch out, we flush in sequence
-token.put_next { t1, t2 }
--- but this one gets pushed in front
-token.put_next ( t3, t4 )
-\stoptyping
-
-When we scan \type {wxyz!} we get \type {yzwx!} back. The argument is either a table
-with tokens or a list of tokens.
-
-\subsection{Nota bene}
-
-When scanning for the next token you need to keep in mind that we're not scanning
-like \TEX\ does: expanding, changing modes and doing things as it goes. When we
-scan with \LUA\ we just pick up tokens. Say that we have:
-
-\starttyping
-\bar
-\stoptyping
-
-but \type {\bar} is undefined. Normally \TEX\ will then issue an error message.
-However, when we have:
-
-\starttyping
-\def\foo{\bar}
-\stoptyping
-
-We get no error, unless we expand \type {\foo} while \type {\bar} is still
-undefined. What happens is that as soon as \TEX\ sees an undefined macro it will
-create a hash entry and when later it gets defined that entry will be reused. So,
-\type {\bar} really exists but can be in an undefined state.
-
-\startbuffer[demo]
-bar        : \directlua{tex.print(token.scan_csname())}\bar
-foo        : \directlua{tex.print(token.scan_csname())}\foo
-myfirstbar : \directlua{tex.print(token.scan_csname())}\myfirstbar
-\stopbuffer
-
-\startlines
-\getbuffer[demo]
-\stoplines
-
-This was entered as:
-
-\typebuffer[demo]
-
-The reason that you see \type {bar} reported and not \type {myfirstbar} is that
-\type {\bar} was already used in a previous paragraph.
-
-If we now say:
-
-\startbuffer
-\def\foo{}
-\stopbuffer
-
-\typebuffer \getbuffer
-
-we get:
-
-\startlines
-\getbuffer[demo]
-\stoplines
-
-And if we say
-
-\startbuffer
-\def\foo{\bar}
-\stopbuffer
-
-\typebuffer \getbuffer
-
-we get:
-
-\startlines
-\getbuffer[demo]
-\stoplines
-
-When scanning from \LUA\ we are not in a mode that defines (undefined) macros at
-all. There we just get the real primitive undefined macro token.
-
-\startbuffer
-\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\myfirstbar
-\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\mysecondbar
-\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\mythirdbar
-\stopbuffer
-
-\startlines
-\getbuffer
-\stoplines
-
-This was generated with:
-
-\typebuffer
-
-So, we do get a unique token because after all we need some kind of \LUA\ object
-that can be used and garbage collected, but it is basically the same one,
-representing an undefined control sequence.
-
-\section{The \type {kpse} library}
-
-This library provides two separate, but nearly identical interfaces to the
-\KPATHSEA\ file search functionality: there is a \quote {normal} procedural
-interface that shares its kpathsea instance with \LUATEX\ itself, and an object
-oriented interface that is completely on its own.
-
-\subsection{\type {kpse.set_program_name} and \type {kpse.new}}
-
-Before the search library can be used at all, its database has to be initialized.
-There are three possibilities, two of which belong to the procedural interface.
-
-First, when \LUATEX\ is used to typeset documents, this initialization happens
-automatically and the \KPATHSEA\ executable and program names are set to \type
-{luatex} (that is, unless explicitly prohibited by the user's startup script.
-See~\in {section} [init] for more details).
-
-Second, in \TEXLUA\ mode, the initialization has to be done explicitly via the
-\type {kpse.set_program_name} function, which sets the \KPATHSEA\ executable
-(and optionally program) name.
-
-\startfunctioncall
-kpse.set_program_name(<string> name)
-kpse.set_program_name(<string> name, <string> progname)
-\stopfunctioncall
-
-The second argument controls the use of the \quote {dotted} values in the \type
-{texmf.cnf} configuration file, and defaults to the first argument.
-
-Third, if you prefer the object oriented interface, you have to call a different
-function. It has the same arguments, but it returns a userdata variable.
-
-\startfunctioncall
-local kpathsea = kpse.new(<string> name)
-local kpathsea = kpse.new(<string> name, <string> progname)
-\stopfunctioncall
-
-Apart from these two functions, the calling conventions of the interfaces are
-identical. Depending on the chosen interface, you either call \type
-{kpse.find_file()} or \type {kpathsea:find_file()}, with identical arguments and
-return vales.
-
-\subsection{\type {find_file}}
-
-The most often used function in the library is \type {find_file}:
-
-\startfunctioncall
-<string> f = kpse.find_file(<string> filename)
-<string> f = kpse.find_file(<string> filename, <string> ftype)
-<string> f = kpse.find_file(<string> filename, <boolean> mustexist)
-<string> f = kpse.find_file(<string> filename, <string> ftype, <boolean> mustexist)
-<string> f = kpse.find_file(<string> filename, <string> ftype, <number> dpi)
-\stopfunctioncall
-
-Arguments:
-
-\startitemize[intro]
-
-\sym{filename}
-
-the name of the file you want to find, with or without extension.
-
-\sym{ftype}
-
-maps to the \type {-format} argument of \KPSEWHICH. The supported \type {ftype}
-values are the same as the ones supported by the standalone \type {kpsewhich}
-program: \startluacode
-    local list = {
-        "afm",
-        "base",
-        "bib",
-        "bitmap font",
-        "bst",
-        "cid maps",
-        "clua",
-        "cmap files",
-        "cnf",
-        "cweb",
-        "dvips config",
-        "enc files",
-        "fmt",
-        "font feature files",
-        "gf",
-        "graphic|/|figure",
-        "ist",
-        "lig files",
-        "ls-R",
-        "lua",
-        "map",
-        "mem",
-        "MetaPost support",
-        "mf",
-        "mfpool",
-        "mft",
-        "misc fonts",
-        "mlbib",
-        "mlbst",
-        "mp",
-        "mppool",
-        "ocp",
-        "ofm",
-        "opentype fonts",
-        "opl",
-        "other binary files",
-        "other text files",
-        "otp",
-        "ovf",
-        "ovp",
-        "pdftex config",
-        "pk",
-        "PostScript header",
-        "subfont definition files",
-        "tex",
-        "TeX system documentation",
-        "TeX system sources",
-        "texmfscripts",
-        "texpool",
-        "tfm",
-        "Troff fonts",
-        "truetype fonts",
-        "type1 fonts",
-        "type42 fonts",
-        "vf",
-        "web",
-        "web2c files",
-    }
-    table.sort(list)
-    context("{\\tttf \letterpercent, t}",list)
-\stopluacode
-
-The default type is \type {tex}. Note: this is different from \KPSEWHICH, which
-tries to deduce the file type itself from looking at the supplied extension.
-
-\sym{mustexist}
-
-is similar to \KPSEWHICH's \type {-must-exist}, and the default is \type {false}.
-If you specify \type {true} (or a non|-|zero integer), then the \KPSE\ library
-will search the disk as well as the \type {ls-R} databases.
-
-\sym{dpi}
-
-This is used for the size argument of the formats \type {pk}, \type {gf}, and
-\type {bitmap font}. \stopitemize
-
-
-\subsection{\type {lookup}}
-
-A more powerful (but slower) generic method for finding files is also available.
-It returns a string for each found file.
-
-\startfunctioncall
-<string> f, ... = kpse.lookup(<string> filename, <table> options)
-\stopfunctioncall
-
-The options match commandline arguments from \type {kpsewhich}:
-
-\starttabulate[|l|l|p|]
-\BC key              \BC type     \BC description \NC \NR
-\NC \type{debug}     \NC number   \NC set debugging flags for this lookup\NC     \NR
-\NC \type{format}    \NC string   \NC use specific file type (see list above)\NC \NR
-\NC \type{dpi}       \NC number   \NC use this resolution for this lookup; default 600\NC \NR
-\NC \type{path}      \NC string   \NC search in the given path\NC \NR
-\NC \type{all}       \NC boolean  \NC output all matches, not just the first\NC \NR
-\NC \type{mustexist} \NC boolean  \NC search the disk as well as ls-R if necessary\NC \NR
-\NC \type{mktexpk}   \NC boolean  \NC disable/enable mktexpk generation for this lookup\NC \NR
-\NC \type{mktextex}  \NC boolean  \NC disable/enable mktextex generation for this lookup\NC \NR
-\NC \type{mktexmf}   \NC boolean  \NC disable/enable mktexmf generation for this lookup\NC \NR
-\NC \type{mktextfm}  \NC boolean  \NC disable/enable mktextfm generation for this lookup\NC \NR
-\NC \type{subdir}    \NC string
-                         or table \NC only output matches whose directory part
-                                      ends with the given string(s) \NC \NR
-\stoptabulate
-
-\subsection{\type {init_prog}}
-
-Extra initialization for programs that need to generate bitmap fonts.
-
-\startfunctioncall
-kpse.init_prog(<string> prefix, <number> base_dpi, <string> mfmode)
-kpse.init_prog(<string> prefix, <number> base_dpi, <string> mfmode, <string> fallback)
-\stopfunctioncall
-
-\subsection{\type {readable_file}}
-
-Test if an (absolute) file name is a readable file.
-
-\startfunctioncall
-<string> f = kpse.readable_file(<string> name)
-\stopfunctioncall
-
-The return value is the actual absolute filename you should use, because the disk
-name is not always the same as the requested name, due to aliases and
-system|-|specific handling under e.g.\ \MSDOS. Returns \type {nil} if the file
-does not exist or is not readable.
-
-\subsection{\type {expand_path}}
-
-Like kpsewhich's \type {-expand-path}:
-
-\startfunctioncall
-<string> r = kpse.expand_path(<string> s)
-\stopfunctioncall
-
-\subsection{\type {expand_var}}
-
-Like kpsewhich's  \type {-expand-var}:
-
-\startfunctioncall
-<string> r = kpse.expand_var(<string> s)
-\stopfunctioncall
-
-\subsection{\type {expand_braces}}
-
-Like kpsewhich's \type {-expand-braces}:
-
-\startfunctioncall
-<string> r = kpse.expand_braces(<string> s)
-\stopfunctioncall
-
-\subsection{\type {show_path}}
-
-Like kpsewhich's \type {-show-path}:
-
-\startfunctioncall
-<string> r = kpse.show_path(<string> ftype)
-\stopfunctioncall
-
-
-\subsection{\type {var_value}}
-
-Like kpsewhich's \type {-var-value}:
-
-\startfunctioncall
-<string> r = kpse.var_value(<string> s)
-\stopfunctioncall
-
-\subsection{\type {version}}
-
-Returns the kpathsea version string.
-
-\startfunctioncall
-<string> r = kpse.version()
-\stopfunctioncall
-
-\stopchapter
-
-\stopcomponent
+% language=uk

+

+% lua.newtable

+

+\environment luatex-style

+

+\startcomponent luatex-tex

+

+\startchapter[reference=tex,title={The \TEX\ related libraries}]

+

+\startsection[title={The \type {lua} library}][library=lua]

+

+\startsubsection[title={Version information}]

+

+\topicindex{libraries+\type{lua}}

+\topicindex{version}

+

+\libindex{version}

+

+This library contains one read|-|only item:

+

+\starttyping

+<string> s = lua.version

+\stoptyping

+

+This returns the \LUA\ version identifier string. The value is currently

+\directlua {tex.print(lua.version)}.

+

+\stopsubsection

+

+\startsubsection[title={Bytecode registers}]

+

+\topicindex{bytecodes}

+\topicindex{registers+bytecodes}

+

+\libindex{bytecode}

+\libindex{setbytecode}

+\libindex{getbytecode}

+

+\LUA\ registers can be used to store \LUA\ code chunks. The accepted values for

+assignments are functions and \type {nil}. Likewise, the retrieved value is

+either a function or \type {nil}.

+

+\starttyping

+lua.bytecode[<number> n] = <function> f

+lua.bytecode[<number> n]()

+\stoptyping

+

+The contents of the \type {lua.bytecode} array is stored inside the format file

+as actual \LUA\ bytecode, so it can also be used to preload \LUA\ code. The

+function must not contain any upvalues. The associated function calls are:

+

+\startfunctioncall

+<function> f = lua.getbytecode(<number> n)

+lua.setbytecode(<number> n, <function> f)

+\stopfunctioncall

+

+Note: Since a \LUA\ file loaded using \type {loadfile(filename)} is essentially

+an anonymous function, a complete file can be stored in a bytecode register like

+this:

+

+\startfunctioncall

+lua.bytecode[n] = loadfile(filename)

+\stopfunctioncall

+

+Now all definitions (functions, variables) contained in the file can be

+created by executing this bytecode register:

+

+\startfunctioncall

+lua.bytecode[n]()

+\stopfunctioncall

+

+Note that the path of the file is stored in the \LUA\ bytecode to be used in

+stack backtraces and therefore dumped into the format file if the above code is

+used in \INITEX. If it contains private information, i.e. the user name, this

+information is then contained in the format file as well. This should be kept in

+mind when preloading files into a bytecode register in \INITEX.

+

+\stopsubsection

+

+\startsubsection[title={Chunk name registers}]

+

+\libindex{name}

+\libindex{setluaname}

+\libindex{getluaname}

+

+There is an array of 65536 (0--65535) potential chunk names for use with the

+\prm {directlua} and \lpr {latelua} primitives.

+

+\startfunctioncall

+lua.name[<number> n] = <string> s

+<string> s = lua.name[<number> n]

+\stopfunctioncall

+

+If you want to unset a \LUA\ name, you can assign \type {nil} to it. The function

+accessors are:

+

+\startfunctioncall

+lua.setluaname(<string> s,<number> n])

+<string> s = lua.getluaname(<number> n)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={Introspection}]

+

+\libindex{getstacktop}

+\libindex{getcalllevel}

+

+The \type {getstacktop} and\type {getcalllevel} functions return numbers

+indicating how much nesting is going on. They are only of use as breakpoints when

+checking some mechanism going haywire.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={The \type {status} library}][library=status]

+

+\topicindex{libraries+\type{status}}

+

+\libindex{list}

+\libindex{resetmessages}

+\libindex{setexitcode}

+

+This contains a number of run|-|time configuration items that you may find useful

+in message reporting, as well as an iterator function that gets all of the names

+and values as a table.

+

+\startfunctioncall

+<table> info = status.list()

+\stopfunctioncall

+

+The keys in the table are the known items, the value is the current value. Almost

+all of the values in \type {status} are fetched through a metatable at run|-|time

+whenever they are accessed, so you cannot use \type {pairs} on \type {status},

+but you {\it can\/} use \type {pairs} on \type {info}, of course. If you do not

+need the full list, you can also ask for a single item by using its name as an

+index into \type {status}. The current list is:

+

+\starttabulate[|l|p|]

+\DB key                       \BC explanation \NC \NR

+\TB

+\NC \type{banner}             \NC terminal display banner \NC \NR

+\NC \type{best_page_break}    \NC the current best break (a node) \NC \NR

+\NC \type{buf_size}           \NC current allocated size of the line buffer \NC \NR

+\NC \type{callbacks}          \NC total number of executed callbacks so far \NC \NR

+\NC \type{cs_count}           \NC number of control sequences \NC \NR

+\NC \type{dest_names_size}    \NC \PDF\ destination table size \NC \NR

+\NC \type{dvi_gone}           \NC written \DVI\ bytes \NC \NR

+\NC \type{dvi_ptr}            \NC not yet written \DVI\ bytes \NC \NR

+\NC \type{dyn_used}           \NC token (multi|-|word) memory in use  \NC \NR

+\NC \type{filename}           \NC name of the current input file \NC \NR

+\NC \type{fix_mem_end}        \NC maximum number of used tokens \NC \NR

+\NC \type{fix_mem_min}        \NC minimum number of allocated words for tokens \NC \NR

+\NC \type{fix_mem_max}        \NC maximum number of allocated words for tokens \NC \NR

+\NC \type{font_ptr}           \NC number of active fonts \NC \NR

+\NC \type{hash_extra}         \NC extra allowed hash \NC \NR

+\NC \type{hash_size}          \NC size of hash \NC \NR

+\NC \type{indirect_callbacks} \NC number of those that were themselves a result of other callbacks (e.g. file readers) \NC \NR

+\NC \type{ini_version}        \NC \type {true} if this is an \INITEX\ run \NC \NR

+\NC \type{init_pool_ptr}      \NC \INITEX\ string pool index \NC \NR

+\NC \type{init_str_ptr}       \NC number of \INITEX\ strings \NC \NR

+\NC \type{input_ptr}          \NC the level of input we're at \NC \NR

+\NC \type{inputid}            \NC numeric id of the current input \NC \NR

+\NC \type{largest_used_mark}  \NC max referenced marks class \NC \NR

+\NC \type{lasterrorcontext}   \NC last error context string (with newlines) \NC \NR

+\NC \type{lasterrorstring}    \NC last \TEX\ error string \NC \NR

+\NC \type{lastluaerrorstring} \NC last \LUA\ error string \NC \NR

+\NC \type{lastwarningstring}  \NC last warning tag, normally an indication of in what part\NC \NR

+\NC \type{lastwarningtag}     \NC last warning string\NC \NR

+\NC \type{linenumber}         \NC location in the current input file \NC \NR

+\NC \type{log_name}           \NC name of the log file \NC \NR

+\NC \type{luabytecode_bytes}  \NC number of bytes in \LUA\ bytecode registers \NC \NR

+\NC \type{luabytecodes}       \NC number of active \LUA\ bytecode registers \NC \NR

+\NC \type{luastate_bytes}     \NC number of bytes in use by \LUA\ interpreters \NC \NR

+\NC \type{luatex_engine}      \NC the \LUATEX\ engine identifier \NC \NR

+\NC \type{luatex_hashchars}   \NC length to which \LUA\ hashes strings ($2^n$) \NC \NR

+\NC \type{luatex_hashtype}    \NC the hash method used (in \LUAJITTEX) \NC \NR

+\NC \type{luatex_version}     \NC the \LUATEX\ version number \NC \NR

+\NC \type{luatex_revision}    \NC the \LUATEX\ revision string \NC \NR

+\NC \type{max_buf_stack}      \NC max used buffer position \NC \NR

+\NC \type{max_in_stack}       \NC max used input stack entries \NC \NR

+\NC \type{max_nest_stack}     \NC max used nesting stack entries \NC \NR

+\NC \type{max_param_stack}    \NC max used parameter stack entries \NC \NR

+\NC \type{max_save_stack}     \NC max used save stack entries \NC \NR

+\NC \type{max_strings}        \NC maximum allowed strings \NC \NR

+\NC \type{nest_size}          \NC nesting stack size \NC \NR

+\NC \type{node_mem_usage}     \NC a string giving insight into currently used nodes \NC \NR

+\NC \type{obj_ptr}            \NC max \PDF\ object pointer \NC \NR

+\NC \type{obj_tab_size}       \NC \PDF\ object table size \NC \NR

+\NC \type{output_active}      \NC \type {true} if the \prm {output} routine is active \NC \NR

+\NC \type{output_file_name}   \NC name of the \PDF\ or \DVI\ file \NC \NR

+\NC \type{param_size}         \NC parameter stack size \NC \NR

+\NC \type{pdf_dest_names_ptr} \NC max \PDF\ destination pointer \NC \NR

+\NC \type{pdf_gone}           \NC written \PDF\ bytes \NC \NR

+\NC \type{pdf_mem_ptr}        \NC max \PDF\ memory used \NC \NR

+\NC \type{pdf_mem_size}       \NC \PDF\ memory size \NC \NR

+\NC \type{pdf_os_cntr}        \NC max \PDF\ object stream pointer \NC \NR

+\NC \type{pdf_os_objidx}      \NC \PDF\ object stream index \NC \NR

+\NC \type{pdf_ptr}            \NC not yet written \PDF\ bytes \NC \NR

+\NC \type{pool_ptr}           \NC string pool index \NC \NR

+\NC \type{pool_size}          \NC current size allocated for string characters \NC \NR

+\NC \type{save_size}          \NC save stack size \NC \NR

+\NC \type{shell_escape}       \NC \type {0} means disabled, \type {1} means anything is permitted, and \type {2} is restricted \NC \NR

+\NC \type{safer_option}       \NC \type {1} means safer is enforced \NC \NR

+\NC \type{kpse_used}          \NC \type {1} means that kpse is used \NC \NR

+\NC \type{stack_size}         \NC input stack size \NC \NR

+\NC \type{str_ptr}            \NC number of strings \NC \NR

+\NC \type{total_pages}        \NC number of written pages \NC \NR

+\NC \type{var_mem_max}        \NC number of allocated words for nodes \NC \NR

+\NC \type{var_used}           \NC variable (one|-|word) memory in use \NC \NR

+\NC \type{lc_collate}         \NC the value of \type {LC_COLLATE}  at startup time (becomes \type {C} at startup) \NC \NR

+\NC \type{lc_ctype}           \NC the value of \type {LC_CTYPE}    at startup time (becomes \type {C} at startup) \NC \NR

+%NC \type{lc_monetary}        \NC the value of \type {LC_MONETARY} at startup time \NC \NR

+\NC \type{lc_numeric}         \NC the value of \type {LC_NUMERIC}  at startup time \NC \NR

+%NC \type{lc_time}            \NC the value of \type {LC_TIME}     at startup time (becomes \type {C} at startup) \NC \NR

+\LL

+\stoptabulate

+

+The error and warning messages can be wiped with the \type {resetmessages}

+function. A return value can be set with \type {setexitcode}.

+

+\stopsection

+

+\startsection[title={The \type {tex} library}][library=tex]

+

+\startsubsection[title={Introduction}]

+

+\topicindex{libraries+\type{tex}}

+

+The \type {tex} table contains a large list of virtual internal \TEX\

+parameters that are partially writable.

+

+The designation \quote {virtual} means that these items are not properly defined

+in \LUA, but are only front\-ends that are handled by a metatable that operates

+on the actual \TEX\ values. As a result, most of the \LUA\ table operators (like

+\type {pairs} and \type {#}) do not work on such items.

+

+At the moment, it is possible to access almost every parameter that you can use

+after \prm {the}, is a single tokens or is sort of special in \TEX. This excludes

+parameters that need extra arguments, like \type {\the\scriptfont}. The subset

+comprising simple integer and dimension registers are writable as well as

+readable (like \prm {tracingcommands} and \prm {parindent}).

+

+\stopsubsection

+

+\startsubsection[title={Internal parameter values, \type {set} and \type {get}}]

+

+\topicindex{parameters+internal}

+

+\libindex{set}           \libindex{get}

+

+For all the parameters in this section, it is possible to access them directly

+using their names as index in the \type {tex} table, or by using one of the

+functions \type {tex.get} and \type {tex.set}.

+

+The exact parameters and return values differ depending on the actual parameter,

+and so does whether \type {tex.set} has any effect. For the parameters that {\em

+can} be set, it is possible to use \type {global} as the first argument to \type

+{tex.set}; this makes the assignment global instead of local.

+

+\startfunctioncall

+tex.set (["global",] <string> n, ...)

+... = tex.get (<string> n)

+\stopfunctioncall

+

+Glue is kind of special because there are five values involved. The return value

+is a \nod {glue_spec} node but when you pass \type {false} as last argument to

+\type {tex.get} you get the width of the glue and when you pass \type {true} you

+get all five values. Otherwise you get a node which is a copy of the internal

+value so you are responsible for its freeing at the \LUA\ end. When you set a

+glue quantity you can either pass a \nod {glue_spec} or upto five numbers. If

+you pass \type {true} to \type {get} you get 5 values returned for a glue and

+when you pass \type {false} you only get the width returned.

+

+\subsubsection{Integer parameters}

+

+The integer parameters accept and return \LUA\ numbers. These are read|-|write:

+

+\starttwocolumns

+\starttyping

+tex.adjdemerits

+tex.binoppenalty

+tex.brokenpenalty

+tex.catcodetable

+tex.clubpenalty

+tex.day

+tex.defaulthyphenchar

+tex.defaultskewchar

+tex.delimiterfactor

+tex.displaywidowpenalty

+tex.doublehyphendemerits

+tex.endlinechar

+tex.errorcontextlines

+tex.escapechar

+tex.exhyphenpenalty

+tex.fam

+tex.finalhyphendemerits

+tex.floatingpenalty

+tex.globaldefs

+tex.hangafter

+tex.hbadness

+tex.holdinginserts

+tex.hyphenpenalty

+tex.interlinepenalty

+tex.language

+tex.lastlinefit

+tex.lefthyphenmin

+tex.linepenalty

+tex.localbrokenpenalty

+tex.localinterlinepenalty

+tex.looseness

+tex.mag

+tex.maxdeadcycles

+tex.month

+tex.newlinechar

+tex.outputpenalty

+tex.pausing

+tex.postdisplaypenalty

+tex.predisplaydirection

+tex.predisplaypenalty

+tex.pretolerance

+tex.relpenalty

+tex.righthyphenmin

+tex.savinghyphcodes

+tex.savingvdiscards

+tex.showboxbreadth

+tex.showboxdepth

+tex.time

+tex.tolerance

+tex.tracingassigns

+tex.tracingcommands

+tex.tracinggroups

+tex.tracingifs

+tex.tracinglostchars

+tex.tracingmacros

+tex.tracingnesting

+tex.tracingonline

+tex.tracingoutput

+tex.tracingpages

+tex.tracingparagraphs

+tex.tracingrestores

+tex.tracingscantokens

+tex.tracingstats

+tex.uchyph

+tex.vbadness

+tex.widowpenalty

+tex.year

+\stoptyping

+\stoptwocolumns

+

+These are read|-|only:

+

+\startthreecolumns

+\starttyping

+tex.deadcycles

+tex.insertpenalties

+tex.parshape

+tex.prevgraf

+tex.spacefactor

+\stoptyping

+\stopthreecolumns

+

+\subsubsection{Dimension parameters}

+

+The dimension parameters accept \LUA\ numbers (signifying scaled points) or

+strings (with included dimension). The result is always a number in scaled

+points. These are read|-|write:

+

+\startthreecolumns

+\starttyping

+tex.boxmaxdepth

+tex.delimitershortfall

+tex.displayindent

+tex.displaywidth

+tex.emergencystretch

+tex.hangindent

+tex.hfuzz

+tex.hoffset

+tex.hsize

+tex.lineskiplimit

+tex.mathsurround

+tex.maxdepth

+tex.nulldelimiterspace

+tex.overfullrule

+tex.pagebottomoffset

+tex.pageheight

+tex.pageleftoffset

+tex.pagerightoffset

+tex.pagetopoffset

+tex.pagewidth

+tex.parindent

+tex.predisplaysize

+tex.scriptspace

+tex.splitmaxdepth

+tex.vfuzz

+tex.voffset

+tex.vsize

+tex.prevdepth

+tex.prevgraf

+tex.spacefactor

+\stoptyping

+\stopthreecolumns

+

+These are read|-|only:

+

+\startthreecolumns

+\starttyping

+tex.pagedepth

+tex.pagefilllstretch

+tex.pagefillstretch

+tex.pagefilstretch

+tex.pagegoal

+tex.pageshrink

+tex.pagestretch

+tex.pagetotal

+\stoptyping

+\stopthreecolumns

+

+Beware: as with all \LUA\ tables you can add values to them. So, the following is

+valid:

+

+\starttyping

+tex.foo = 123

+\stoptyping

+

+When you access a \TEX\ parameter a look up takes place. For read||only variables

+that means that you will get something back, but when you set them you create a

+new entry in the table thereby making the original invisible.

+

+There are a few special cases that we make an exception for: \type {prevdepth},

+\type {prevgraf} and \type {spacefactor}. These normally are accessed via the

+\type {tex.nest} table:

+

+\starttyping

+tex.nest[tex.nest.ptr].prevdepth   = p

+tex.nest[tex.nest.ptr].spacefactor = s

+\stoptyping

+

+However, the following also works:

+

+\starttyping

+tex.prevdepth   = p

+tex.spacefactor = s

+\stoptyping

+

+Keep in mind that when you mess with node lists directly at the \LUA\ end you

+might need to update the top of the nesting stack's \type {prevdepth} explicitly

+as there is no way \LUATEX\ can guess your intentions. By using the accessor in

+the \type {tex} tables, you get and set the values at the top of the nesting

+stack.

+

+\subsubsection{Direction parameters}

+

+The direction parameters are read|-|only and return a \LUA\ string.

+

+\startthreecolumns

+\starttyping

+tex.bodydir

+tex.mathdir

+tex.pagedir

+tex.pardir

+tex.textdir

+\stoptyping

+\stopthreecolumns

+

+\subsubsection{Glue parameters}

+

+The glue parameters accept and return a userdata object that represents a \nod {glue_spec} node.

+

+\startthreecolumns

+\starttyping

+tex.abovedisplayshortskip

+tex.abovedisplayskip

+tex.baselineskip

+tex.belowdisplayshortskip

+tex.belowdisplayskip

+tex.leftskip

+tex.lineskip

+tex.parfillskip

+tex.parskip

+tex.rightskip

+tex.spaceskip

+tex.splittopskip

+tex.tabskip

+tex.topskip

+tex.xspaceskip

+\stoptyping

+\stopthreecolumns

+

+\subsubsection{Muglue parameters}

+

+All muglue parameters are to be used read|-|only and return a \LUA\ string.

+

+\startthreecolumns

+\starttyping

+tex.medmuskip

+tex.thickmuskip

+tex.thinmuskip

+\stoptyping

+\stopthreecolumns

+

+\subsubsection{Tokenlist parameters}

+

+The tokenlist parameters accept and return \LUA\ strings. \LUA\ strings are

+converted to and from token lists using \prm {the} \prm {toks} style expansion:

+all category codes are either space (10) or other (12). It follows that assigning

+to some of these, like \quote {tex.output}, is actually useless, but it feels bad

+to make exceptions in view of a coming extension that will accept full|-|blown

+token strings.

+

+\startthreecolumns

+\starttyping

+tex.errhelp

+tex.everycr

+tex.everydisplay

+tex.everyeof

+tex.everyhbox

+tex.everyjob

+tex.everymath

+tex.everypar

+tex.everyvbox

+tex.output

+\stoptyping

+\stopthreecolumns

+

+\stopsubsection

+

+\startsubsection[title={Convert commands}]

+

+\topicindex{convert commands}

+

+All \quote {convert} commands are read|-|only and return a \LUA\ string. The

+supported commands at this moment are:

+

+\starttwocolumns

+\starttyping

+tex.eTeXVersion

+tex.eTeXrevision

+tex.formatname

+tex.jobname

+tex.luatexbanner

+tex.luatexrevision

+tex.fontname(number)

+tex.uniformdeviate(number)

+tex.number(number)

+tex.romannumeral(number)

+tex.fontidentifier(number)

+\stoptyping

+\stoptwocolumns

+

+If you are wondering why this list looks haphazard; these are all the cases of

+the \quote {convert} internal command that do not require an argument, as well as

+the ones that require only a simple numeric value. The special (\LUA|-|only) case

+of \type {tex.fontidentifier} returns the \type {csname} string that matches a

+font id number (if there is one).

+

+\stopsubsection

+

+\startsubsection[title={Last item commands}]

+

+\topicindex{last items}

+

+All \quote {last item} commands are read|-|only and return a number. The

+supported commands at this moment are:

+

+\startthreecolumns

+\starttyping

+tex.lastpenalty

+tex.lastkern

+tex.lastskip

+tex.lastnodetype

+tex.inputlineno

+tex.lastxpos

+tex.lastypos

+tex.randomseed

+tex.luatexversion

+tex.eTeXminorversion

+tex.eTeXversion

+tex.currentgrouplevel

+tex.currentgrouptype

+tex.currentiflevel

+tex.currentiftype

+tex.currentifbranch

+\stoptyping

+\stopthreecolumns

+

+\stopsubsection

+

+\startsubsection[title={Accessing registers: \type {set*}, \type {get*} and \type {is*}}]

+

+\topicindex{attributes}

+\topicindex{registers}

+

+\libindex{attribute}  \libindex{setattribute}  \libindex{getattribute}  \libindex{isattribute}

+\libindex{count}      \libindex{setcount}      \libindex{getcount}      \libindex{iscount}

+\libindex{dimen}      \libindex{setdimen}      \libindex{getdimen}      \libindex{isdimen}

+\libindex{skip}       \libindex{setskip}       \libindex{getskip}       \libindex{isskip}

+\libindex{muskip}     \libindex{setmuskip}     \libindex{getmuskip}     \libindex{ismuskip}

+\libindex{glue}       \libindex{setglue}       \libindex{getglue}       \libindex{isglue}

+\libindex{muglue}     \libindex{setmuglue}     \libindex{getmuglue}     \libindex{ismuglue}

+\libindex{toks}       \libindex{settoks}       \libindex{gettoks}       \libindex{istoks}

+\libindex{box}        \libindex{setbox}        \libindex{getbox}        \libindex{isbox}

+

+\libindex{scantoks}

+

+\TEX's attributes (\lpr {attribute}), counters (\prm {count}), dimensions (\prm

+{dimen}), skips (\prm {skip}, \prm {muskip}) and token (\prm {toks}) registers

+can be accessed and written to using two times five virtual sub|-|tables of the

+\type {tex} table:

+

+\startthreecolumns

+\starttyping

+tex.attribute

+tex.count

+tex.dimen

+tex.skip

+tex.glue

+tex.muskip

+tex.muglue

+tex.toks

+\stoptyping

+\stopthreecolumns

+

+It is possible to use the names of relevant \lpr {attributedef}, \prm {countdef},

+\prm {dimendef}, \prm {skipdef}, or \prm {toksdef} control sequences as indices

+to these tables:

+

+\starttyping

+tex.count.scratchcounter = 0

+enormous = tex.dimen['maxdimen']

+\stoptyping

+

+In this case, \LUATEX\ looks up the value for you on the fly. You have to use a

+valid \prm {countdef} (or \lpr {attributedef}, or \prm {dimendef}, or \prm

+{skipdef}, or \prm {toksdef}), anything else will generate an error (the intent

+is to eventually also allow \type {<chardef tokens>} and even macros that expand

+into a number).

+

+\startitemize

+

+    \startitem

+        The count registers accept and return \LUA\ numbers.

+    \stopitem

+

+    \startitem

+        The dimension registers accept \LUA\ numbers (in scaled points) or

+        strings (with an included absolute dimension; \type {em} and \type {ex}

+        and \type {px} are forbidden). The result is always a number in scaled

+        points.

+    \stopitem

+

+    \startitem

+        The token registers accept and return \LUA\ strings. \LUA\ strings are

+        converted to and from token lists using \prm {the} \prm {toks} style

+        expansion: all category codes are either space (10) or other (12).

+    \stopitem

+

+    \startitem

+        The skip registers accept and return \nod {glue_spec} userdata node

+        objects (see the description of the node interface elsewhere in this

+        manual).

+    \stopitem

+

+    \startitem

+        The glue registers are just skip registers but instead of userdata

+        are verbose.

+    \stopitem

+

+    \startitem

+        Like the counts, the attribute registers accept and return \LUA\ numbers.

+    \stopitem

+

+\stopitemize

+

+As an alternative to array addressing, there are also accessor functions defined

+for all cases, for example, here is the set of possibilities for \prm {skip}

+registers:

+

+\startfunctioncall

+tex.setskip (["global",] <number> n, <node> s)

+tex.setskip (["global",] <string> s, <node> s)

+<node> s = tex.getskip (<number> n)

+<node> s = tex.getskip (<string> s)

+\stopfunctioncall

+

+We have similar setters for \type {count}, \type {dimen}, \type {muskip}, and

+\type {toks}. Counters and dimen are represented by numbers, skips and muskips by

+nodes, and toks by strings.

+

+Again the glue variants are not using the \nod {glue-spec} userdata nodes. The

+\type {setglue} function accepts upto 5 arguments: width, stretch, shrink,

+stretch order and shrink order and the \type {getglue} function reports them,

+unless the second argument is \type {false} in which care only the width is

+returned.

+

+Here is an example usign a threesome:

+

+\startfunctioncall

+local d = tex.getdimen("foo")

+if tex.isdimen("bar") then

+    tex.setdimen("bar",d)

+end

+\stopfunctioncall

+

+There are six extra skip (glue) related helpers:

+

+\startfunctioncall

+tex.setglue (["global"], <number> n,

+    width, stretch, shrink, stretch_order, shrink_order)

+tex.setglue (["global"], <string> s,

+    width, stretch, shrink, stretch_order, shrink_order)

+width, stretch, shrink, stretch_order, shrink_order =

+    tex.getglue (<number> n)

+width, stretch, shrink, stretch_order, shrink_order =

+    tex.getglue (<string> s)

+\stopfunctioncall

+

+The other two are \type {tex.setmuglue} and \type {tex.getmuglue}.

+

+There are such helpers for \type {dimen}, \type {count}, \type {skip}, \type

+{muskip}, \type {box} and \type {attribute} registers but the glue ones

+are special because they have to deal with more properties.

+

+As with the general \type {get} and \type {set} function discussed before, for

+the skip registers \type {getskip} returns a node and \type {getglue} returns

+numbers, while \type {setskip} accepts a node and \type {setglue} expects upto 5

+numbers. Again, when you pass \type {false} as second argument to \type {getglue}

+you only get the width returned. The same is true for the \type {mu} variants

+\type {getmuskip}, \type {setmuskip}, \type {getmuskip} and\type {setmuskip}.

+

+For tokens registers we have an alternative where a catcode table is specified:

+

+\startfunctioncall

+tex.scantoks(0,3,"$e=mc^2$")

+tex.scantoks("global",0,"$\int\limits^1_2$")

+\stopfunctioncall

+

+In the function-based interface, it is possible to define values globally by

+using the string \type {global} as the first function argument.

+

+\stopsubsection

+

+\startsubsection[title={Character code registers: \type {[get|set]*code[s]}}]

+

+\topicindex{characters+codes}

+

+\libindex{lccode}    \libindex{setlccode}    \libindex{getlccode}

+\libindex{uccode}    \libindex{setuccode}    \libindex{getuccode}

+\libindex{sfcode}    \libindex{setsfcode}    \libindex{getsfcode}

+\libindex{catcode}   \libindex{setcatcode}   \libindex{getcatcode}

+\libindex{mathcode}  \libindex{setmathcode}  \libindex{getmathcode}

+\libindex{delcode}   \libindex{setdelcode}   \libindex{getdelcode}

+

+\libindex{setdelcodes}   \libindex{getdelcodes}

+\libindex{setmathcodes}  \libindex{getmathcodes}

+

+\TEX's character code tables (\prm {lccode}, \prm {uccode}, \prm {sfcode}, \prm

+{catcode}, \prm {mathcode}, \prm {delcode}) can be accessed and written to using

+six virtual subtables of the \type {tex} table

+

+\startthreecolumns

+\starttyping

+tex.lccode

+tex.uccode

+tex.sfcode

+tex.catcode

+tex.mathcode

+tex.delcode

+\stoptyping

+\stopthreecolumns

+

+The function call interfaces are roughly as above, but there are a few twists.

+\type {sfcode}s are the simple ones:

+

+\startfunctioncall

+tex.setsfcode (["global",] <number> n, <number> s)

+<number> s = tex.getsfcode (<number> n)

+\stopfunctioncall

+

+The function call interface for \type {lccode} and \type {uccode} additionally

+allows you to set the associated sibling at the same time:

+

+\startfunctioncall

+tex.setlccode (["global"], <number> n, <number> lc)

+tex.setlccode (["global"], <number> n, <number> lc, <number> uc)

+<number> lc = tex.getlccode (<number> n)

+tex.setuccode (["global"], <number> n, <number> uc)

+tex.setuccode (["global"], <number> n, <number> uc, <number> lc)

+<number> uc = tex.getuccode (<number> n)

+\stopfunctioncall

+

+The function call interface for \type {catcode} also allows you to specify a

+category table to use on assignment or on query (default in both cases is the

+current one):

+

+\startfunctioncall

+tex.setcatcode (["global"], <number> n, <number> c)

+tex.setcatcode (["global"], <number> cattable, <number> n, <number> c)

+<number> lc = tex.getcatcode (<number> n)

+<number> lc = tex.getcatcode (<number> cattable, <number> n)

+\stopfunctioncall

+

+The interfaces for \type {delcode} and \type {mathcode} use small array tables to

+set and retrieve values:

+

+\startfunctioncall

+tex.setmathcode (["global"], <number> n, <table> mval )

+<table> mval = tex.getmathcode (<number> n)

+tex.setdelcode (["global"], <number> n, <table> dval )

+<table> dval = tex.getdelcode (<number> n)

+\stopfunctioncall

+

+Where the table for \type {mathcode} is an array of 3 numbers, like this:

+

+\starttyping

+{

+    <number> class,

+    <number> family,

+    <number> character

+}

+\stoptyping

+

+And the table for \type {delcode} is an array with 4 numbers, like this:

+

+\starttyping

+{

+    <number> small_fam,

+    <number> small_char,

+    <number> large_fam,

+    <number> large_char

+}

+\stoptyping

+

+You can also avoid the table:

+

+\startfunctioncall

+tex.setmathcode (["global"], <number> n, <number> class,

+    <number> family, <number> character)

+class, family, char =

+    tex.getmathcodes (<number> n)

+tex.setdelcode (["global"], <number> n, <number> smallfam,

+    <number> smallchar, <number> largefam, <number> largechar)

+smallfam, smallchar, largefam, largechar =

+    tex.getdelcodes (<number> n)

+\stopfunctioncall

+

+Normally, the third and fourth values in a delimiter code assignment will be zero

+according to \lpr {Udelcode} usage, but the returned table can have values there

+(if the delimiter code was set using \prm {delcode}, for example). Unset \type

+{delcode}'s can be recognized because \type {dval[1]} is $-1$.

+

+\stopsubsection

+

+\startsubsection[title={Box registers: \type {[get|set]box}}]

+

+\topicindex{registers}

+\topicindex{boxes}

+

+\libindex{box}

+\libindex{setbox}  \libindex{getbox}

+

+It is possible to set and query actual boxes, coming for instance from \prm

+{hbox}, \prm {vbox} or \prm {vtop}, using the node interface as defined in the

+\type {node} library:

+

+\starttyping

+tex.box

+\stoptyping

+

+for array access, or

+

+\starttyping

+tex.setbox(["global",] <number> n, <node> s)

+tex.setbox(["global",] <string> cs, <node> s)

+<node> n = tex.getbox(<number> n)

+<node> n = tex.getbox(<string> cs)

+\stoptyping

+

+for function|-|based access. In the function-based interface, it is possible to

+define values globally by using the string \type {global} as the first function

+argument.

+

+Be warned that an assignment like

+

+\starttyping

+tex.box[0] = tex.box[2]

+\stoptyping

+

+does not copy the node list, it just duplicates a node pointer. If \type {\box2}

+will be cleared by \TEX\ commands later on, the contents of \type {\box0} becomes

+invalid as well. To prevent this from happening, always use \type

+{node.copy_list} unless you are assigning to a temporary variable:

+

+\starttyping

+tex.box[0] = node.copy_list(tex.box[2])

+\stoptyping

+

+\stopsubsection

+

+\startsubsection[title={Reusing boxes: \type {[use|save]boxresource} and \type {getboxresourcedimensions}}]

+

+\topicindex{boxes+reuse}

+

+\libindex{useboxresource}

+\libindex{saveboxresource}

+\libindex{getboxresourcedimensions}

+

+The following function will register a box for reuse (this is modelled after so

+called xforms in \PDF). You can (re)use the box with \lpr {useboxresource} or

+by creating a rule node with subtype~2.

+

+\starttyping

+local index = tex.saveboxresource(n,attributes,resources,immediate,type,margin)

+\stoptyping

+

+The optional second and third arguments are strings, the fourth is a boolean. The

+fifth argument is a type. When set to non|-|zero the \type {/Type} entry is

+omitted. A value of 1 or 3 still writes a \type {/BBox}, while 2 or 3 will write

+a \type {/Matrix}. The sixth argument is the (virtual) margin that extends beyond

+the effective boundingbox as seen by \TEX. Instead of a box number one can also

+pass a \type {[h|v]list} node.

+

+You can generate the reference (a rule type) with:

+

+\starttyping

+local reused = tex.useboxresource(n,wd,ht,dp)

+\stoptyping

+

+The dimensions are optional and the final ones are returned as extra values. The

+following is just a bonus (no dimensions returned means that the resource is

+unknown):

+

+\starttyping

+local w, h, d, m = tex.getboxresourcedimensions(n)

+\stoptyping

+

+This returns the width, height, depth and margin of the resource.

+

+\stopsubsection

+

+\startsubsection[title={\type {triggerbuildpage}}]

+

+\topicindex{pages}

+

+\libindex{triggerbuildpage}

+

+You should not expect to much from the \type {triggerbuildpage} helpers because

+often \TEX\ doesn't do much if it thinks nothing has to be done, but it might be

+useful for some applications. It just does as it says it calls the internal

+function that build a page, given that there is something to build.

+

+\stopsubsection

+

+\startsubsection[title={\type {splitbox}}]

+

+\topicindex{boxes+split}

+

+\libindex{splitbox}

+

+You can split a box:

+

+\starttyping

+local vlist = tex.splitbox(n,height,mode)

+\stoptyping

+

+The remainder is kept in the original box and a packaged vlist is returned. This

+operation is comparable to the \prm {vsplit} operation. The mode can be \type

+{additional} or \type {exactly} and concerns the split off box.

+

+\stopsubsection

+

+\startsubsection[title={Accessing math parameters: \type {[get|set]math}}]

+

+\topicindex{math+parameters}

+\topicindex{parameters+math}

+

+\libindex{setmath}

+\libindex{getmath}

+

+It is possible to set and query the internal math parameters using:

+

+\startfunctioncall

+tex.setmath(["global",] <string> n, <string> t, <number> n)

+<number> n = tex.getmath(<string> n, <string> t)

+\stopfunctioncall

+

+As before an optional first parameter \type {global} indicates a global

+assignment.

+

+The first string is the parameter name minus the leading \quote {Umath}, and the

+second string is the style name minus the trailing \quote {style}. Just to be

+complete, the values for the math parameter name are:

+

+\starttyping

+quad                axis                operatorsize

+overbarkern         overbarrule         overbarvgap

+underbarkern        underbarrule        underbarvgap

+radicalkern         radicalrule         radicalvgap

+radicaldegreebefore radicaldegreeafter  radicaldegreeraise

+stackvgap           stacknumup          stackdenomdown

+fractionrule        fractionnumvgap     fractionnumup

+fractiondenomvgap   fractiondenomdown   fractiondelsize

+limitabovevgap      limitabovebgap      limitabovekern

+limitbelowvgap      limitbelowbgap      limitbelowkern

+underdelimitervgap  underdelimiterbgap

+overdelimitervgap   overdelimiterbgap

+subshiftdrop        supshiftdrop        subshiftdown

+subsupshiftdown     subtopmax           supshiftup

+supbottommin        supsubbottommax     subsupvgap

+spaceafterscript    connectoroverlapmin

+ordordspacing       ordopspacing        ordbinspacing     ordrelspacing

+ordopenspacing      ordclosespacing     ordpunctspacing   ordinnerspacing

+opordspacing        opopspacing         opbinspacing      oprelspacing

+opopenspacing       opclosespacing      oppunctspacing    opinnerspacing

+binordspacing       binopspacing        binbinspacing     binrelspacing

+binopenspacing      binclosespacing     binpunctspacing   bininnerspacing

+relordspacing       relopspacing        relbinspacing     relrelspacing

+relopenspacing      relclosespacing     relpunctspacing   relinnerspacing

+openordspacing      openopspacing       openbinspacing    openrelspacing

+openopenspacing     openclosespacing    openpunctspacing  openinnerspacing

+closeordspacing     closeopspacing      closebinspacing   closerelspacing

+closeopenspacing    closeclosespacing   closepunctspacing closeinnerspacing

+punctordspacing     punctopspacing      punctbinspacing   punctrelspacing

+punctopenspacing    punctclosespacing   punctpunctspacing punctinnerspacing

+innerordspacing     inneropspacing      innerbinspacing   innerrelspacing

+inneropenspacing    innerclosespacing   innerpunctspacing innerinnerspacing

+\stoptyping

+

+The values for the style parameter are:

+

+\starttyping

+display       crampeddisplay

+text          crampedtext

+script        crampedscript

+scriptscript  crampedscriptscript

+\stoptyping

+

+The value is either a number (representing a dimension or number) or a glue spec

+node representing a muskip for \type {ordordspacing} and similar spacing

+parameters.

+

+\stopsubsection

+

+\startsubsection[title={Special list heads: \type {[get|set]list}}]

+

+\topicindex{lists}

+

+\libindex{lists}

+\libindex{setlist}

+\libindex{getlist}

+

+The virtual table \type {tex.lists} contains the set of internal registers that

+keep track of building page lists.

+

+\starttabulate[|l|p|]

+\DB field                      \BC explanation \NC \NR

+\TB

+\NC \type{page_ins_head}       \NC circular list of pending insertions \NC \NR

+\NC \type{contrib_head}        \NC the recent contributions \NC \NR

+\NC \type{page_head}           \NC the current page content \NC \NR

+%NC \type{temp_head}           \NC \NC \NR

+\NC \type{hold_head}           \NC used for held-over items for next page \NC \NR

+\NC \type{adjust_head}         \NC head of the current \prm {vadjust} list \NC \NR

+\NC \type{pre_adjust_head}     \NC head of the current \type {\vadjust pre} list \NC \NR

+%NC \type{align_head}          \NC \NC \NR

+\NC \type{page_discards_head}  \NC head of the discarded items of a page break \NC \NR

+\NC \type{split_discards_head} \NC head of the discarded items in a vsplit \NC \NR

+\LL

+\stoptabulate

+

+The getter and setter functions are \type {getlist} and \type {setlist}. You have

+to be careful with what you set as \TEX\ can have expectations with regards to

+how a list is constructed or in what state it is.

+

+\stopsubsection

+

+\startsubsection[title={Semantic nest levels: \type {getnest} and \type {ptr}}]

+

+\topicindex{nesting}

+

+\libindex{nest}

+\libindex{ptr}

+%libindex{setnest} % only a message

+\libindex{getnest}

+

+The virtual table \type {nest} contains the currently active semantic nesting

+state. It has two main parts: a zero-based array of userdata for the semantic

+nest itself, and the numerical value \type {ptr}, which gives the highest

+available index. Neither the array items in \type {nest[]} nor \type {ptr} can be

+assigned to (as this would confuse the typesetting engine beyond repair), but you

+can assign to the individual values inside the array items, e.g.\ \type

+{tex.nest[tex.nest.ptr].prevdepth}.

+

+\type {tex.nest[tex.nest.ptr]} is the current nest state, \type {nest[0]} the

+outermost (main vertical list) level. The getter function is \type {getnest}. You

+can pass a number (which gives you a list), nothing or \type {top}, which returns

+the topmost list, or the string \type {ptr} which gives you the index of the

+topmost list.

+

+The known fields are:

+

+\starttabulate[|l|l|l|p|]

+\DB key                \BC type    \BC modes \BC explanation \NC \NR

+\TB

+\NC \type{mode}        \NC number  \NC all   \NC a string:

+                                                 \type {none} (this happens during \prm {write}),

+                                                 \type {vmode}, \type {hmode}, \type {displaymath},

+                                                 \type {innervmode}, \type {innerhmode}, \type {inlinemath};

+                                                 these fields cannot be set \NC \NR

+\NC \type{modeline}    \NC number  \NC all   \NC source input line where this mode was entered in,

+                                                 negative inside the output routine \NC \NR

+\NC \type{head}        \NC node    \NC all   \NC the head of the current list \NC \NR

+\NC \type{tail}        \NC node    \NC all   \NC the tail of the current list \NC \NR

+\NC \type{prevgraf}    \NC number  \NC vmode \NC number of lines in the previous paragraph \NC \NR

+\NC \type{prevdepth}   \NC number  \NC vmode \NC depth of the previous paragraph \NC \NR

+\NC \type{spacefactor} \NC number  \NC hmode \NC the current space factor \NC \NR

+\NC \type{dirs}        \NC node    \NC hmode \NC used for temporary storage by the line break algorithm\NC \NR

+\NC \type{noad}        \NC node    \NC mmode \NC used for temporary storage of a pending fraction numerator,

+                                                 for \prm {over} etc. \NC \NR

+\NC \type{delimptr}    \NC node    \NC mmode \NC used for temporary storage of the previous math delimiter,

+                                                 for \prm {middle} \NC \NR

+\NC \type{mathdir}     \NC boolean \NC mmode \NC true when during math processing the \lpr {mathdir} is not

+                                                 the same as the surrounding \lpr {textdir} \NC \NR

+\NC \type{mathstyle}   \NC number  \NC mmode \NC the current \lpr {mathstyle} \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[reference=sec:luaprint,title={Print functions}]

+

+\topicindex{printing}

+

+The \type {tex} table also contains the three print functions that are the major

+interface from \LUA\ scripting to \TEX. The arguments to these three functions

+are all stored in an in|-|memory virtual file that is fed to the \TEX\ scanner as

+the result of the expansion of \prm {directlua}.

+

+The total amount of returnable text from a \prm {directlua} command is only

+limited by available system \RAM. However, each separate printed string has to

+fit completely in \TEX's input buffer. The result of using these functions from

+inside callbacks is undefined at the moment.

+

+\subsubsection{\type {print}}

+

+\libindex{print}

+

+\startfunctioncall

+tex.print(<string> s, ...)

+tex.print(<number> n, <string> s, ...)

+tex.print(<table> t)

+tex.print(<number> n, <table> t)

+\stopfunctioncall

+

+Each string argument is treated by \TEX\ as a separate input line. If there is a

+table argument instead of a list of strings, this has to be a consecutive array

+of strings to print (the first non-string value will stop the printing process).

+

+The optional parameter can be used to print the strings using the catcode regime

+defined by \lpr {catcodetable}~\type {n}. If \type {n} is $-1$, the currently

+active catcode regime is used. If \type {n} is $-2$, the resulting catcodes are

+the result of \prm {the} \prm {toks}: all category codes are 12 (other) except for

+the space character, that has category code 10 (space). Otherwise, if \type {n}

+is not a valid catcode table, then it is ignored, and the currently active

+catcode regime is used instead.

+

+The very last string of the very last \type {tex.print} command in a \prm

+{directlua} will not have the \prm {endlinechar} appended, all others do.

+

+\subsubsection{\type {sprint}}

+

+\libindex{sprint}

+

+\startfunctioncall

+tex.sprint(<string> s, ...)

+tex.sprint(<number> n, <string> s, ...)

+tex.sprint(<table> t)

+tex.sprint(<number> n, <table> t)

+\stopfunctioncall

+

+Each string argument is treated by \TEX\ as a special kind of input line that

+makes it suitable for use as a partial line input mechanism:

+

+\startitemize[packed]

+\startitem

+    \TEX\ does not switch to the \quote {new line} state, so that leading spaces

+    are not ignored.

+\stopitem

+\startitem

+    No \prm {endlinechar} is inserted.

+\stopitem

+\startitem

+    Trailing spaces are not removed. Note that this does not prevent \TEX\ itself

+    from eating spaces as result of interpreting the line. For example, in

+

+    \starttyping

+    before\directlua{tex.sprint("\\relax")tex.sprint(" inbetween")}after

+    \stoptyping

+

+    the space before \type {in between} will be gobbled as a result of the \quote

+    {normal} scanning of \prm {relax}.

+\stopitem

+\stopitemize

+

+If there is a table argument instead of a list of strings, this has to be a

+consecutive array of strings to print (the first non-string value will stop the

+printing process).

+

+The optional argument sets the catcode regime, as with \type {tex.print}. This

+influences the string arguments (or numbers turned into strings).

+

+Although this needs to be used with care, you can also pass token or node

+userdata objects. These get injected into the stream. Tokens had best be valid

+tokens, while nodes need to be around when they get injected. Therefore it is

+important to realize the following:

+

+\startitemize

+\startitem

+    When you inject a token, you need to pass a valid token userdata object. This

+    object will be collected by \LUA\ when it no longer is referenced. When it gets

+    printed to \TEX\ the token itself gets copied so there is no interference with the

+    \LUA\ garbage collection. You manage the object yourself. Because tokens are

+    actually just numbers, there is no real extra overhead at the \TEX\ end.

+\stopitem

+\startitem

+    When you inject a node, you need to pass a valid node userdata object. The

+    node related to the object will not be collected by \LUA\ when it no longer

+    is referenced. It lives on at the \TEX\ end in its own memory space. When it

+    gets printed to \TEX\ the node reference is used assuming that node stays

+    around. There is no \LUA\ garbage collection involved. Again, you manage the

+    object yourself. The node itself is freed when \TEX\ is done with it.

+\stopitem

+\stopitemize

+

+If you consider the last remark you might realize that we have a problem when a

+printed mix of strings, tokens and nodes is reused. Inside \TEX\ the sequence

+becomes a linked list of input buffers. So, \type {"123"} or \type {"\foo{123}"}

+gets read and parsed on the fly, while \typ {<token userdata>} already is

+tokenized and effectively is a token list now. A \typ {<node userdata>} is also

+tokenized into a token list but it has a reference to a real node. Normally this

+goes fine. But now assume that you store the whole lot in a macro: in that case

+the tokenized node can be flushed many times. But, after the first such flush the

+node is used and its memory freed. You can prevent this by using copies which is

+controlled by setting \lpr {luacopyinputnodes} to a non|-|zero value. This is one

+of these fuzzy areas you have to live with if you really mess with these low

+level issues.

+

+\subsubsection{\type {tprint}}

+

+\libindex{tprint}

+

+\startfunctioncall

+tex.tprint({<number> n, <string> s, ...}, {...})

+\stopfunctioncall

+

+This function is basically a shortcut for repeated calls to \type

+{tex.sprint(<number> n, <string> s, ...)}, once for each of the supplied argument

+tables.

+

+\subsubsection{\type {cprint}}

+

+\libindex{cprint}

+

+This function takes a number indicating the to be used catcode, plus either a

+table of strings or an argument list of strings that will be pushed into the

+input stream.

+

+\startfunctioncall

+tex.cprint( 1," 1: $&{\\foo}") tex.print("\\par") -- a lot of \bgroup s

+tex.cprint( 2," 2: $&{\\foo}") tex.print("\\par") -- matching \egroup s

+tex.cprint( 9," 9: $&{\\foo}") tex.print("\\par") -- all get ignored

+tex.cprint(10,"10: $&{\\foo}") tex.print("\\par") -- all become spaces

+tex.cprint(11,"11: $&{\\foo}") tex.print("\\par") -- letters

+tex.cprint(12,"12: $&{\\foo}") tex.print("\\par") -- other characters

+tex.cprint(14,"12: $&{\\foo}") tex.print("\\par") -- comment triggers

+\stopfunctioncall

+

+\subsubsection{\type {write}, \type {twrite}, \type {nwrite}}

+

+\libindex{write}

+\libindex{twrite}

+\libindex{nwrite}

+

+\startfunctioncall

+tex.write(<string> s, ...)

+tex.write(<table> t)

+\stopfunctioncall

+

+Each string argument is treated by \TEX\ as a special kind of input line that

+makes it suitable for use as a quick way to dump information:

+

+\startitemize

+\item All catcodes on that line are either \quote{space} (for '~') or \quote

+      {character} (for all others).

+\item There is no \prm {endlinechar} appended.

+\stopitemize

+

+If there is a table argument instead of a list of strings, this has to be a

+consecutive array of strings to print (the first non-string value will stop the

+printing process).

+

+The functions \type {twrite} and \type {nwrite} can be used to write a token or

+node back to \TEX\, possibly intermixed with regular strings that will be

+tokenized. You have to make sure that you pass the right data because sometimes

+\TEX\ has expectations that need to be met.

+

+\stopsubsection

+

+\startsubsection[title={Helper functions}]

+

+\subsubsection{\type {round}}

+

+\topicindex {helpers}

+

+\libindex{round}

+

+\startfunctioncall

+<number> n = tex.round(<number> o)

+\stopfunctioncall

+

+Rounds \LUA\ number \type {o}, and returns a number that is in the range of a

+valid \TEX\ register value. If the number starts out of range, it generates a

+\quote {number too big} error as well.

+

+\subsubsection{\type {scale}}

+

+\libindex{scale}

+

+\startfunctioncall

+<number> n = tex.scale(<number> o, <number> delta)

+<table> n = tex.scale(table o, <number> delta)

+\stopfunctioncall

+

+Multiplies the \LUA\ numbers \type {o} and \nod {delta}, and returns a rounded

+number that is in the range of a valid \TEX\ register value. In the table

+version, it creates a copy of the table with all numeric top||level values scaled

+in that manner. If the multiplied number(s) are of range, it generates

+\quote{number too big} error(s) as well.

+

+Note: the precision of the output of this function will depend on your computer's

+architecture and operating system, so use with care! An interface to \LUATEX's

+internal, 100\% portable scale function will be added at a later date.

+

+\subsubsection{\type {number} and \type {romannumeral}}

+

+\libindex{number}

+\libindex{romannumeral}

+

+These are the companions to the primitives \prm {number} and \prm

+{romannumeral}. They can be used like:

+

+\startfunctioncall

+tex.print(tex.romannumeral(123))

+\stopfunctioncall

+

+\subsubsection{\type {fontidentifier} and \type {fontname}}

+

+\libindex{fontidentifier}

+\libindex{fontname}

+

+The first one returns the name only, the second one reports the size too.

+

+\startfunctioncall

+tex.print(tex.fontname(tex.fontname))

+tex.print(tex.fontname(tex.fontidentidier))

+\stopfunctioncall

+

+\subsubsection{\type {sp}}

+

+\libindex{sp}

+

+\startfunctioncall

+<number> n = tex.sp(<number> o)

+<number> n = tex.sp(<string> s)

+\stopfunctioncall

+

+Converts the number \type {o} or a string \type {s} that represents an explicit

+dimension into an integer number of scaled points.

+

+For parsing the string, the same scanning and conversion rules are used that

+\LUATEX\ would use if it was scanning a dimension specifier in its \TEX|-|like

+input language (this includes generating errors for bad values), expect for the

+following:

+

+\startitemize[n]

+\startitem

+    only explicit values are allowed, control sequences are not handled

+\stopitem

+\startitem

+    infinite dimension units (\type {fil...}) are forbidden

+\stopitem

+\startitem

+    \type {mu} units do not generate an error (but may not be useful either)

+\stopitem

+\stopitemize

+

+\subsubsection{\type {tex.getlinenumber} and \type {tex.setlinenumber}}

+

+\libindex{getlinenumber}

+\libindex{setlinenumber}

+

+You can mess with the current line number:

+

+\startfunctioncall

+local n = tex.getlinenumber()

+tex.setlinenumber(n+10)

+\stopfunctioncall

+

+which can be shortcut to:

+

+\startfunctioncall

+tex.setlinenumber(10,true)

+\stopfunctioncall

+

+This might be handy when you have a callback that read numbers from a file and

+combines them in one line (in which case an error message probably has to refer

+to the original line). Interference with \TEX's internal handling of numbers is

+of course possible.

+

+\subsubsection{\type {error} and \type {show_context}}

+

+\topicindex{errors}

+

+\libindex{error}

+\libindex{show_context}

+

+\startfunctioncall

+tex.error(<string> s)

+tex.error(<string> s, <table> help)

+\stopfunctioncall

+

+This creates an error somewhat like the combination of \prm {errhelp} and \prm

+{errmessage} would. During this error, deletions are disabled.

+

+The array part of the \type {help} table has to contain strings, one for each

+line of error help.

+

+In case of an error the \type {show_context} function will show the current

+context where we're at (in the expansion).

+

+\subsubsection{\type {run}, \type {runtoks}, \type {finish}}

+

+\libindex{run}

+\libindex{runtoks}

+\libindex{finish}

+

+These two functions start the interpretations and force its end. A runs normally

+boils down to \TEX\ entering the so called main loop. A token is fetched and

+depending on it current meaning some actions takes place. Sometimes that actions

+comes immediately, sometimes more scanning is needed. Quite often tokens get

+pushed back into the input. This all means that the \TEX\ scanner is constantly

+pushing and popping input states, but in the end after all the action is done

+returns to the main loop.

+

+Because of the fact that \TEX\ is in a complex dance of expanding, dealing with

+fonts, typesetting paragraphs, messing around with boxes, building pages, and so

+on, you cannot easily run a nested \TEX\ run (read nested main loop). However,

+there is an option to force a local run with \type {runtoks}. The content of the

+given token list register gets expanded locally after which we return to where we

+triggered this expansion, at the \LUA\ end. Instead a function can get passed

+that does some work. You have to make sure that at the end \TEX\ is in a sane

+state and this is not always trivial. A more complex mechanism would complicate

+\TEX\ itself (and probably also harm performance) so this simple local expansion

+loop has to do.

+

+\startfunctioncall

+tex.runtoks(<token register>)

+tex.runtoks(<lua function>)

+\stopfunctioncall

+

+When the \prm {tracingnesting} parameter is set to a value larger than~2 some

+information is reported about the state of the local loop.

+

+\subsubsection{\type {run}, \type {runtoks}, \type {finish}}

+

+\libindex{forcehmode}

+

+An example of a (possible error triggering) complication is that \TEX\ expects to

+be in some state, say horizontal mode, and you have to make sure it is when you

+start feeding back something from \LUA\ into \TEX. Normally a user will not run

+into issues but when you start writing tokens or nodes or have a nested run there

+can be situations that you need to run \type {forcehmode}. There is no recipe for

+this and intercepting possible cases would weaken \LUATEX's flexibility.

+

+\subsubsection{\type {hashtokens}}

+

+\libindex{hashtokens}

+

+\topicindex{hash}

+

+\startfunctioncall

+for i,v in pairs (tex.hashtokens()) do ... end

+\stopfunctioncall

+

+Returns a list of names. This can be useful for debugging, but note that this

+also reports control sequences that may be unreachable at this moment due to

+local redefinitions: it is strictly a dump of the hash table. You can use \type

+{token.create} to inspect properties, for instance when the \type {command} key

+in a created table equals \type {123}, you have the \type {cmdname} value \type

+{undefined_cs}.

+

+\subsubsection{\type {definefont}}

+

+\topicindex{fonts+defining}

+

+\libindex{definefont}

+

+\startfunctioncall

+tex.definefont(<string> csname, <number> fontid)

+tex.definefont(<boolean> global, <string> csname, <number> fontid)

+\stopfunctioncall

+

+Associates \type {csname} with the internal font number \type {fontid}. The

+definition is global if (and only if) \type {global} is specified and true (the

+setting of \type {globaldefs} is not taken into account).

+

+\stopsubsection

+

+\startsubsection[reference=luaprimitives,title={Functions for dealing with primitives}]

+

+\subsubsection{\type {enableprimitives}}

+

+\libindex{enableprimitives}

+

+\topicindex{initialization}

+\topicindex{primitives}

+

+\startfunctioncall

+tex.enableprimitives(<string> prefix, <table> primitive names)

+\stopfunctioncall

+

+This function accepts a prefix string and an array of primitive names. For each

+combination of \quote {prefix} and \quote {name}, the \type

+{tex.enableprimitives} first verifies that \quote {name} is an actual primitive

+(it must be returned by one of the \type {tex.extraprimitives} calls explained

+below, or part of \TEX82, or \prm {directlua}). If it is not, \type

+{tex.enableprimitives} does nothing and skips to the next pair.

+

+But if it is, then it will construct a csname variable by concatenating the

+\quote {prefix} and \quote {name}, unless the \quote {prefix} is already the

+actual prefix of \quote {name}. In the latter case, it will discard the \quote

+{prefix}, and just use \quote {name}.

+

+Then it will check for the existence of the constructed csname. If the csname is

+currently undefined (note: that is not the same as \prm {relax}), it will

+globally define the csname to have the meaning: run code belonging to the

+primitive \quote {name}. If for some reason the csname is already defined, it

+does nothing and tries the next pair.

+

+An example:

+

+\starttyping

+tex.enableprimitives('LuaTeX', {'formatname'})

+\stoptyping

+

+will define \type {\LuaTeXformatname} with the same intrinsic meaning as the

+documented primitive \lpr {formatname}, provided that the control sequences \type

+{\LuaTeXformatname} is currently undefined.

+

+When \LUATEX\ is run with \type {--ini} only the \TEX82 primitives and \prm

+{directlua} are available, so no extra primitives {\bf at all}.

+

+If you want to have all the new functionality available using their default

+names, as it is now, you will have to add

+

+\starttyping

+\ifx\directlua\undefined \else

+    \directlua {tex.enableprimitives('',tex.extraprimitives ())}

+\fi

+\stoptyping

+

+near the beginning of your format generation file. Or you can choose different

+prefixes for different subsets, as you see fit.

+

+Calling some form of \type {tex.enableprimitives} is highly important though,

+because if you do not, you will end up with a \TEX82-lookalike that can run \LUA\

+code but not do much else. The defined csnames are (of course) saved in the

+format and will be available at runtime.

+

+\subsubsection{\type {extraprimitives}}

+

+\libindex{extraprimitives}

+

+\startfunctioncall

+<table> t = tex.extraprimitives(<string> s, ...)

+\stopfunctioncall

+

+This function returns a list of the primitives that originate from the engine(s)

+given by the requested string value(s). The possible values and their (current)

+return values are given in the following table. In addition the somewhat special

+primitives \quote{\tex{ }}, \quote{\tex {/}} and \quote{\type {-}} are defined.

+

+\startluacode

+function document.showprimitives(tag)

+    for k, v in table.sortedpairs(tex.extraprimitives(tag)) do

+        if v ~= ' ' and v ~= "/" and v ~= "-" then

+            context.type(v)

+            context.space()

+        end

+    end

+end

+\stopluacode

+

+\starttabulate[|l|pl|]

+\DB name    \BC values \NC \NR

+\TB

+\NC tex     \NC \ctxlua{document.showprimitives('tex')    } \NC \NR

+\NC core    \NC \ctxlua{document.showprimitives('core')   } \NC \NR

+\NC etex    \NC \ctxlua{document.showprimitives('etex')   } \NC \NR

+\NC luatex  \NC \ctxlua{document.showprimitives('luatex') } \NC \NR

+\LL

+\stoptabulate

+

+Note that \type {luatex} does not contain \type {directlua}, as that is

+considered to be a core primitive, along with all the \TEX82 primitives, so it is

+part of the list that is returned from \type {'core'}.

+

+Running \type {tex.extraprimitives} will give you the complete list of

+primitives \type {-ini} startup. It is exactly equivalent to \type

+{tex.extraprimitives("etex","luatex")}.

+

+\subsubsection{\type {primitives}}

+

+\libindex{primitives}

+

+\startfunctioncall

+<table> t = tex.primitives()

+\stopfunctioncall

+

+This function returns a list of all primitives that \LUATEX\ knows about.

+

+\stopsubsection

+

+\startsubsection[title={Core functionality interfaces}]

+

+\subsubsection{\type {badness}}

+

+\libindex{badness}

+

+\startfunctioncall

+<number> b = tex.badness(<number> t, <number> s)

+\stopfunctioncall

+

+This helper function is useful during linebreak calculations. \type {t} and \type

+{s} are scaled values; the function returns the badness for when total \type {t}

+is supposed to be made from amounts that sum to \type {s}. The returned number is

+a reasonable approximation of \mathematics {100(t/s)^3};

+

+\subsubsection{\type {tex.resetparagraph}}

+

+\topicindex {paragraphs+reset}

+

+\libindex{resetparagraph}

+

+This function resets the parameters that \TEX\ normally resets when a new paragraph

+is seen.

+

+\subsubsection{\type {linebreak}}

+

+\topicindex {linebreaks}

+

+\libindex{linebreak}

+

+\startfunctioncall

+local <node> nodelist, <table> info =

+    tex.linebreak(<node> listhead, <table> parameters)

+\stopfunctioncall

+

+The understood parameters are as follows:

+

+\starttabulate[|l|l|p|]

+\DB name                        \BC type            \BC explanation \NC \NR

+\TB

+\NC \type{pardir}               \NC string          \NC \NC \NR

+\NC \type{pretolerance}         \NC number          \NC \NC \NR

+\NC \type{tracingparagraphs}    \NC number          \NC \NC \NR

+\NC \type{tolerance}            \NC number          \NC \NC \NR

+\NC \type{looseness}            \NC number          \NC \NC \NR

+\NC \type{hyphenpenalty}        \NC number          \NC \NC \NR

+\NC \type{exhyphenpenalty}      \NC number          \NC \NC \NR

+\NC \type{pdfadjustspacing}     \NC number          \NC \NC \NR

+\NC \type{adjdemerits}          \NC number          \NC \NC \NR

+\NC \type{pdfprotrudechars}     \NC number          \NC \NC \NR

+\NC \type{linepenalty}          \NC number          \NC \NC \NR

+\NC \type{lastlinefit}          \NC number          \NC \NC \NR

+\NC \type{doublehyphendemerits} \NC number          \NC \NC \NR

+\NC \type{finalhyphendemerits}  \NC number          \NC \NC \NR

+\NC \type{hangafter}            \NC number          \NC \NC \NR

+\NC \type{interlinepenalty}     \NC number or table \NC if a table, then it is an array like \prm {interlinepenalties} \NC \NR

+\NC \type{clubpenalty}          \NC number or table \NC if a table, then it is an array like \prm {clubpenalties} \NC \NR

+\NC \type{widowpenalty}         \NC number or table \NC if a table, then it is an array like \prm {widowpenalties} \NC \NR

+\NC \type{brokenpenalty}        \NC number          \NC \NC \NR

+\NC \type{emergencystretch}     \NC number          \NC in scaled points \NC \NR

+\NC \type{hangindent}           \NC number          \NC in scaled points \NC \NR

+\NC \type{hsize}                \NC number          \NC in scaled points \NC \NR

+\NC \type{leftskip}             \NC glue_spec node  \NC \NC \NR

+\NC \type{rightskip}            \NC glue_spec node  \NC \NC \NR

+\NC \type{parshape}             \NC table           \NC \NC \NR

+\LL

+\stoptabulate

+

+Note that there is no interface for \prm {displaywidowpenalties}, you have to

+pass the right choice for \type {widowpenalties} yourself.

+

+It is your own job to make sure that \type {listhead} is a proper paragraph list:

+this function does not add any nodes to it. To be exact, if you want to replace

+the core line breaking, you may have to do the following (when you are not

+actually working in the \cbk {pre_linebreak_filter} or \cbk {linebreak_filter}

+callbacks, or when the original list starting at listhead was generated in

+horizontal mode):

+

+\startitemize

+\startitem

+    add an \quote {indent box} and perhaps a \nod {local_par} node at the start

+    (only if you need them)

+\stopitem

+\startitem

+    replace any found final glue by an infinite penalty (or add such a penalty,

+    if the last node is not a glue)

+\stopitem

+\startitem

+    add a glue node for the \prm {parfillskip} after that penalty node

+\stopitem

+\startitem

+    make sure all the \type {prev} pointers are OK

+\stopitem

+\stopitemize

+

+The result is a node list, it still needs to be vpacked if you want to assign it

+to a \prm {vbox}. The returned \type {info} table contains four values that are

+all numbers:

+

+\starttabulate[|l|p|]

+\DB name      \BC explanation \NC \NR

+\TB

+\NC prevdepth \NC depth of the last line in the broken paragraph \NC \NR

+\NC prevgraf  \NC number of lines in the broken paragraph \NC \NR

+\NC looseness \NC the actual looseness value in the broken paragraph \NC \NR

+\NC demerits  \NC the total demerits of the chosen solution  \NC \NR

+\LL

+\stoptabulate

+

+Note there are a few things you cannot interface using this function: You cannot

+influence font expansion other than via \type {pdfadjustspacing}, because the

+settings for that take place elsewhere. The same is true for hbadness and hfuzz

+etc. All these are in the \type {hpack} routine, and that fetches its own

+variables via globals.

+

+\subsubsection{\type {shipout}}

+

+\topicindex {shipout}

+

+\libindex{shipout}

+

+\startfunctioncall

+tex.shipout(<number> n)

+\stopfunctioncall

+

+Ships out box number \type {n} to the output file, and clears the box register.

+

+\subsubsection{\type {getpagestate}}

+

+\topicindex {pages}

+

+\libindex{getpagestate}

+

+This helper reports the current page state: \type {empty}, \type {box_there} or

+\type {inserts_only} as integer value.

+

+\subsubsection{\type {getlocallevel}}

+

+\topicindex {nesting}

+

+\libindex{getlocallevel}

+

+This integer reports the current level of the local loop. It's only useful for

+debugging and the (relative state) numbers can change with the implementation.

+

+\stopsubsection

+

+\startsubsection[title={Randomizers}]

+

+\libindex{lua_math_random}

+\libindex{lua_math_randomseed}

+\libindex{init_rand}

+\libindex{normal_rand}

+\libindex{uniform_rand}

+\libindex{uniformdeviate}

+

+For practical reasons \LUATEX\ has its own random number generator. The original

+\LUA\ random function is available as \typ {tex.lua_math_random}. You can

+initialize with a new seed with \type {init_rand} (\typ {lua_math_randomseed} is

+equivalent to this one.

+

+There are three generators: \type {normal_rand} (no argument is used), \type

+{uniform_rand} (takes a number that will get rounded before being used) and \type

+{uniformdeviate} which behaves like the primitive and expects a scaled integer, so

+

+\startfunctioncall

+tex.print(tex.uniformdeviate(65536)/65536)

+\stopfunctioncall

+

+will give a random number between zero and one.

+

+\stopsubsection

+

+\startsubsection[reference=synctex,title={Functions related to synctex}]

+

+\topicindex {synctex}

+

+\libindex{set_synctex_mode}      \libindex{get_synctex_mode}

+\libindex{set_synctex_no_files}

+\libindex{set_synctex_tag}       \libindex{get_synctex_tag}   \libindex{force_synctex_tag}

+\libindex{set_synctex_line}      \libindex{get_synctex_line}  \libindex{force_synctex_line}

+

+The next helpers only make sense when you implement your own synctex logic. Keep in

+mind that the library used in editors assumes a certain logic and is geared for

+plain and \LATEX, so after a decade users expect a certain behaviour.

+

+\starttabulate[|l|p|]

+\DB name                        \BC explanation \NC \NR

+\TB

+\NC \type{set_synctex_mode}     \NC \type {0} is the default and used normal synctex

+                                    logic, \type {1} uses the values set by the next

+                                    helpers while \type {2} also sets these for glyph

+                                    nodes; \type{3} sets glyphs and glue and \type {4}

+                                    sets only glyphs \NC \NR

+\NC \type{set_synctex_tag}      \NC set the current tag (file) value (obeys save stack) \NC \NR

+\NC \type{set_synctex_line}     \NC set the current line value (obeys save stack) \NC \NR

+\NC \type{set_synctex_no_files} \NC disable synctex file logging \NC \NR

+\NC \type{get_synctex_mode}     \NC returns the current mode (for values see above) \NC \NR

+\NC \type{get_synctex_tag}      \NC get the currently set value of tag (file) \NC \NR

+\NC \type{get_synctex_line}     \NC get the currently set value of line \NC \NR

+\NC \type{force_synctex_tag}    \NC overload the tag (file) value (\type {0} resets) \NC \NR

+\NC \type{force_synctex_line}   \NC overload the line value  (\type {0} resets) \NC \NR

+\LL

+\stoptabulate

+

+The last one is somewhat special. Due to the way files are registered in \SYNCTEX\ we need

+to explicitly disable that feature if we provide our own alternative if we want to avoid

+that overhead. Passing a value of 1 disables registering.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={The \type {texconfig} table},reference=texconfig][library=texconfig]

+

+\topicindex{libraries+\type{texconfig}}

+

+\topicindex {configuration}

+

+This is a table that is created empty. A startup \LUA\ script could

+fill this table with a number of settings that are read out by

+the executable after loading and executing the startup file.

+

+\starttabulate[|l|l|l|p|]

+\DB key                   \BC type     \BC default \BC explanation \NC \NR

+\TB

+\NC \type{kpse_init}      \NC boolean  \NC true

+\NC

+    \type {false} totally disables \KPATHSEA\ initialisation, and enables

+    interpretation of the following numeric key--value pairs. (only ever unset

+    this if you implement {\it all\/} file find callbacks!)

+\NC \NR

+\NC

+    \type{shell_escape} \NC string \NC \type {'f'} \NC Use \type {'y'} or \type

+    {'t'} or \type {'1'} to enable \type {\write18} unconditionally, \type {'p'}

+    to enable the commands that are listed in \type {shell_escape_commands}

+\NC \NR

+\NC

+    shell_escape_commands \NC string \NC \NC Comma-separated list of command

+    names that may be executed by \type {\write18} even if \type {shell_escape}

+    is set to \type {'p'}. Do {\it not\/} use spaces around commas, separate any

+    required command arguments by using a space, and use the \ASCII\ double quote

+    (\type {"}) for any needed argument or path quoting

+\NC \NR

+\NC \type{string_vacancies} \NC number   \NC  75000  \NC cf.\ web2c docs \NC \NR

+\NC \type{pool_free}        \NC number   \NC   5000  \NC cf.\ web2c docs \NC \NR

+\NC \type{max_strings}      \NC number   \NC  15000  \NC cf.\ web2c docs \NC \NR

+\NC \type{strings_free}     \NC number   \NC    100  \NC cf.\ web2c docs \NC \NR

+\NC \type{nest_size}        \NC number   \NC     50  \NC cf.\ web2c docs \NC \NR

+\NC \type{max_in_open}      \NC number   \NC     15  \NC cf.\ web2c docs \NC \NR

+\NC \type{param_size}       \NC number   \NC     60  \NC cf.\ web2c docs \NC \NR

+\NC \type{save_size}        \NC number   \NC   4000  \NC cf.\ web2c docs \NC \NR

+\NC \type{stack_size}       \NC number   \NC    300  \NC cf.\ web2c docs \NC \NR

+\NC \type{dvi_buf_size}     \NC number   \NC  16384  \NC cf.\ web2c docs \NC \NR

+\NC \type{error_line}       \NC number   \NC     79  \NC cf.\ web2c docs \NC \NR

+\NC \type{half_error_line}  \NC number   \NC     50  \NC cf.\ web2c docs \NC \NR

+\NC \type{max_print_line}   \NC number   \NC     79  \NC cf.\ web2c docs \NC \NR

+\NC \type{hash_extra}       \NC number   \NC      0  \NC cf.\ web2c docs \NC \NR

+\NC \type{pk_dpi}           \NC number   \NC     72  \NC cf.\ web2c docs \NC \NR

+\NC \type{trace_file_names} \NC boolean  \NC true

+\NC

+    \type {false} disables \TEX's normal file open|-|close feedback (the

+    assumption is that callbacks will take care of that)

+\NC \NR

+\NC \type{file_line_error}  \NC boolean  \NC false

+\NC

+    do \type {file:line} style error messages

+\NC \NR

+\NC \type{halt_on_error}    \NC boolean  \NC false

+\NC

+    abort run on the first encountered error

+\NC \NR

+\NC \type{formatname}       \NC string   \NC

+\NC

+    if no format name was given on the command line, this key will be tested first

+    instead of simply quitting

+\NC \NR

+\NC \type{jobname}        \NC string   \NC

+\NC

+    if no input file name was given on the command line, this key will be tested

+    first instead of simply giving up

+\NC \NR

+\LL

+\stoptabulate

+

+Note: the numeric values that match web2c parameters are only used if \type

+{kpse_init} is explicitly set to \type {false}. In all other cases, the normal

+values from \type {texmf.cnf} are used.

+

+\stopsection

+

+\startsection[title={The \type {texio} library}][library=texio]

+

+\topicindex{libraries+\type{texio}}

+\topicindex{\IO}

+

+This library takes care of the low|-|level I/O interface: writing to the log file

+and|/|or console.

+

+\startsubsection[title={\type {write}}]

+

+\libindex{write}

+

+\startfunctioncall

+texio.write(<string> target, <string> s, ...)

+texio.write(<string> s, ...)

+\stopfunctioncall

+

+Without the \type {target} argument, writes all given strings to the same

+location(s) \TEX\ writes messages to at this moment. If \prm {batchmode} is in

+effect, it writes only to the log, otherwise it writes to the log and the

+terminal. The optional \type {target} can be one of three possibilities: \type

+{term}, \type {log} or \type {term and log}.

+

+Note: If several strings are given, and if the first of these strings is or might

+be one of the targets above, the \type {target} must be specified explicitly to

+prevent \LUA\ from interpreting the first string as the target.

+

+\stopsubsection

+

+\startsubsection[title={\type {write_nl}}]

+

+\libindex{write_nl}

+

+\startfunctioncall

+texio.write_nl(<string> target, <string> s, ...)

+texio.write_nl(<string> s, ...)

+\stopfunctioncall

+

+This function behaves like \type {texio.write}, but make sure that the given

+strings will appear at the beginning of a new line. You can pass a single empty

+string if you only want to move to the next line.

+

+\stopsubsection

+

+\startsubsection[title={\type {setescape}}]

+

+\libindex{setescape}

+

+You can disable \type {^^} escaping of control characters by passing a value of

+zero.

+

+\stopsubsection

+

+\startsubsection[title={\type {closeinput}}]

+

+\libindex{closeinput}

+

+This function that should be used with care. It acts as \prm {endinput} but at

+the \LUA\ end. You can use it to (sort of) force a jump back to \TEX. Normally a

+\LUA\ will just collect prints and at the end bump an input level and flush these

+prints. This function can help you stay at the current level but you need to know

+what you're doing (or more precise: what \TEX\ is doing with input).

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={The \type {token} library}][library=token]

+

+\startsubsection[title={The scanner}]

+

+\topicindex{libraries+\type{token}}

+\topicindex{tokens}

+

+\libindex{scan_keyword}

+\libindex{scan_keywordcs}

+\libindex{scan_int}

+\libindex{scan_real}

+\libindex{scan_float}

+\libindex{scan_dimen}

+\libindex{scan_glue}

+\libindex{scan_toks}

+\libindex{scan_code}

+\libindex{scan_string}

+\libindex{scan_argument}

+\libindex{scan_word}

+\libindex{scan_csname}

+\libindex{scan_list}

+

+The token library provides means to intercept the input and deal with it at the

+\LUA\ level. The library provides a basic scanner infrastructure that can be used

+to write macros that accept a wide range of arguments. This interface is on

+purpose kept general and as performance is quite ok. One can build additional

+parsers without too much overhead. It's up to macro package writers to see how

+they can benefit from this as the main principle behind \LUATEX\ is to provide a

+minimal set of tools and no solutions. The scanner functions are probably the

+most intriguing.

+

+\starttabulate[|l|l|p|]

+\DB function              \BC argument           \BC result \NC \NR

+\TB

+\NC \type{scan_keyword}   \NC string             \NC returns true if the given keyword is gobbled; as with

+                                                     the regular \TEX\ keyword scanner this is case insensitive

+                                                     (and \ASCII\ based) \NC \NR

+\NC \type{scan_keywordcs} \NC string             \NC returns true if the given keyword is gobbled; this variant

+                                                     is case sensitive and also suitable for \UTF8 \NC \NR

+\NC \type{scan_int}       \NC                    \NC returns an integer \NC \NR

+\NC \type{scan_real}      \NC                    \NC returns a number from e.g.\ \type {1},  \type {1.1}, \type {.1} with optional collapsed signs \NC \NR

+\NC \type{scan_float}     \NC                    \NC returns a number from e.g.\ \type {1},  \type {1.1}, \type {.1}, \type {1.1E10}, , \type {.1e-10} with optional collapsed signs \NC \NR

+\NC \type{scan_dimen}     \NC infinity, mu-units \NC returns a number representing a dimension and or two numbers being the filler and order \NC \NR

+\NC \type{scan_glue}      \NC mu-units           \NC returns a glue spec node \NC \NR

+\NC \type{scan_toks}      \NC definer, expand    \NC returns a table of tokens tokens \NC \NR

+\NC \type{scan_code}      \NC bitset             \NC returns a character if its category is in the given bitset (representing catcodes) \NC \NR

+\NC \type{scan_string}    \NC                    \NC returns a string given between \type {{}}, as \type {\macro} or as sequence of characters with catcode 11 or 12 \NC \NR

+\NC \type{scan_argument}  \NC                    \NC this one is simular to \type {scanstring} but also accepts a \type {\cs}

+                                                     (which then get expanded) \NC \NR

+\NC \type{scan_word}      \NC                    \NC returns a sequence of characters with catcode 11 or 12 as string \NC \NR

+\NC \type{scan_csname}    \NC                    \NC returns \type {foo} after scanning \type {\foo} \NC \NR

+\NC \type{scan_list}      \NC                    \NC picks up a box specification and returns a \type {[h|v]list} node \NC \NR

+\LL

+\stoptabulate

+

+The scanners can be considered stable apart from the one scanning for a token.

+The \type {scan_code} function takes an optional number, the \type {keyword}

+function a normal \LUA\ string. The \type {infinity} boolean signals that we also

+permit \type {fill} as dimension and the \type {mu-units} flags the scanner that

+we expect math units. When scanning tokens we can indicate that we are defining a

+macro, in which case the result will also provide information about what

+arguments are expected and in the result this is separated from the meaning by a

+separator token. The \type {expand} flag determines if the list will be expanded.

+

+The string scanner scans for something between curly braces and expands on the

+way, or when it sees a control sequence it will return its meaning. Otherwise it

+will scan characters with catcode \type {letter} or \type {other}. So, given the

+following definition:

+

+\startbuffer

+\def\bar{bar}

+\def\foo{foo-\bar}

+\stopbuffer

+

+\typebuffer \getbuffer

+

+we get:

+

+\starttabulate[|l|Tl|l|]

+\DB name \BC result \NC \NR

+\TB

+\NC \type {\directlua{token.scan_string()}{foo}} \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} {foo} \NC full expansion \NC \NR

+\NC \type {\directlua{token.scan_string()}foo}   \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} foo   \NC letters and others \NC \NR

+\NC \type {\directlua{token.scan_string()}\foo}  \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")}\foo   \NC meaning \NC \NR

+\LL

+\stoptabulate

+

+The \type {\foo} case only gives the meaning, but one can pass an already

+expanded definition (\prm {edef}'d). In the case of the braced variant one can of

+course use the \prm {detokenize} and \prm {unexpanded} primitives since there we

+do expand.

+

+The \type {scan_word} scanner can be used to implement for instance a number scanner:

+

+\starttyping

+function token.scan_number(base)

+    return tonumber(token.scan_word(),base)

+end

+\stoptyping

+

+This scanner accepts any valid \LUA\ number so it is a way to pick up floats

+in the input.

+

+You can use the \LUA\ interface as follows:

+

+\starttyping

+\directlua {

+    function mymacro(n)

+        ...

+    end

+}

+

+\def\mymacro#1{%

+    \directlua {

+        mymacro(\number\dimexpr#1)

+    }%

+}

+

+\mymacro{12pt}

+\mymacro{\dimen0}

+\stoptyping

+

+You can also do this:

+

+\starttyping

+\directlua {

+    function mymacro()

+        local d = token.scan_dimen()

+        ...

+    end

+}

+

+\def\mymacro{%

+    \directlua {

+        mymacro()

+    }%

+}

+

+\mymacro 12pt

+\mymacro \dimen0

+\stoptyping

+

+It is quite clear from looking at the code what the first method needs as

+argument(s). For the second method you need to look at the \LUA\ code to see what

+gets picked up. Instead of passing from \TEX\ to \LUA\ we let \LUA\ fetch from

+the input stream.

+

+In the first case the input is tokenized and then turned into a string, then it

+is passed to \LUA\ where it gets interpreted. In the second case only a function

+call gets interpreted but then the input is picked up by explicitly calling the

+scanner functions. These return proper \LUA\ variables so no further conversion

+has to be done. This is more efficient but in practice (given what \TEX\ has to

+do) this effect should not be overestimated. For numbers and dimensions it saves

+a bit but for passing strings conversion to and from tokens has to be done anyway

+(although we can probably speed up the process in later versions if needed).

+

+\stopsubsection

+

+\startsubsection[title= {Picking up one token}]

+

+\libindex {get_next}

+\libindex {scan_token}

+\libindex {expand}

+

+The scanners look for a sequence. When you want to pick up one token from the

+input you use \type {get_next}. This creates a token with the (low level)

+properties as discussed next. This token is just the next one. If you want to

+enforce expansion first you can use \type {scan_token}. Internally tokens are

+characterized by a number that packs a lot of information. In order to access

+the bits of information a token is wrapped in a userdata object.

+

+The \type {expand} function will trigger expansion of the next token in the

+input. This can be quite unpredictable but when you call it you probably know

+enough about \TEX\ not to be too worried about that. It basically is a call to

+the internal expand related function.

+

+\stopsubsection

+

+\startsubsection[title={Creating tokens}]

+

+\libindex{create}

+\libindex{new}

+

+\libindex{is_defined}

+\libindex{is_token}

+\libindex{biggest_char}

+

+\libindex{commands}

+\libindex{command_id}

+

+\libindex{get_command}

+\libindex{get_cmdname}

+\libindex{get_csname}

+\libindex{get_id}

+\libindex{get_active}

+\libindex{get_expandable}

+\libindex{get_protected}

+\libindex{get_mode}

+\libindex{get_index}

+\libindex{get_tok}

+

+\libindex{get_next}

+

+The creator function can be used as follows:

+

+\starttyping

+local t = token.create("relax")

+\stoptyping

+

+This gives back a token object that has the properties of the \prm {relax}

+primitive. The possible properties of tokens are:

+

+\starttabulate[|l|p|]

+\DB name \BC explanation \NC \NR

+\TB

+\NC \type {command}    \NC a number representing the internal command number \NC \NR

+\NC \type {cmdname}    \NC the type of the command (for instance the catcode in case of a

+                           character or the classifier that determines the internal

+                           treatment \NC \NR

+\NC \type {csname}     \NC the associated control sequence (if applicable) \NC \NR

+\NC \type {id}         \NC the unique id of the token \NC \NR

+\NC \type {tok}        \NC the full token number as stored in \TEX \NC \NR

+\NC \type {active}     \NC a boolean indicating the active state of the token \NC \NR

+\NC \type {expandable} \NC a boolean indicating if the token (macro) is expandable \NC \NR

+\NC \type {protected}  \NC a boolean indicating if the token (macro) is protected \NC \NR

+\NC \type {mode}       \NC a number either representing a character or another entity \NC \NR

+\NC \type {index}      \NC a number running from 0x0000 upto 0xFFFF indicating a \TEX\ register index \NC \NR

+\LL

+\stoptabulate

+

+Alternatively you can use a getter \type {get_<fieldname>} to access a property

+of a token.

+

+The numbers that represent a catcode are the same as in \TEX\ itself, so using

+this information assumes that you know a bit about \TEX's internals. The other

+numbers and names are used consistently but are not frozen. So, when you use them

+for comparing you can best query a known primitive or character first to see the

+values.

+

+You can ask for a list of commands:

+

+\starttyping

+local t = token.commands()

+\stoptyping

+

+The id of a token class can be queried as follows:

+

+\starttyping

+local id = token.command_id("math_shift")

+\stoptyping

+

+If you really know what you're doing you can create character tokens by not

+passing a string but a number:

+

+\starttyping

+local letter_x = token.create(string.byte("x"))

+local other_x  = token.create(string.byte("x"),12)

+\stoptyping

+

+Passing weird numbers can give side effects so don't expect too much help with

+that. As said, you need to know what you're doing. The best way to explore the

+way these internals work is to just look at how primitives or macros or \prm

+{chardef}'d commands are tokenized. Just create a known one and inspect its

+fields. A variant that ignores the current catcode table is:

+

+\starttyping

+local whatever = token.new(123,12)

+\stoptyping

+

+You can test if a control sequence is defined with \type {is_defined}, which

+accepts a string and returns a boolean:

+

+\starttyping

+local okay = token.is_defined("foo")

+\stoptyping

+

+The largest character possible is returned by \type {biggest_char}, just in case you

+need to know that boundary condition.

+

+\stopsubsection

+

+\startsubsection[title={Macros}]

+

+\topicindex {macros}

+

+\libindex{set_macro}

+\libindex{get_macro}

+\libindex{get_meaning}

+\libindex{set_char}

+\libindex{set_lua}

+\libindex{get_functions_table}

+

+The \type {set_macro} function can get upto 4 arguments:

+

+\starttyping

+set_macro("csname","content")

+set_macro("csname","content","global")

+set_macro("csname")

+\stoptyping

+

+You can pass a catcodetable identifier as first argument:

+

+\starttyping

+set_macro(catcodetable,"csname","content")

+set_macro(catcodetable,"csname","content","global")

+set_macro(catcodetable,"csname")

+\stoptyping

+

+The results are like:

+

+\starttyping

+ \def\csname{content}

+\gdef\csname{content}

+ \def\csname{}

+\stoptyping

+

+The \type {get_macro} function can be used to get the content of a macro while

+the \type {get_meaning} function gives the meaning including the argument

+specification (as usual in \TEX\ separated by \type {->}).

+

+The \type {set_char} function can be used to do a \prm {chardef} at the

+\LUA\ end, where invalid assignments are silently ignored:

+

+\starttyping

+set_char("csname",number)

+set_char("csname",number,"global")

+\stoptyping

+

+A special one is the following:

+

+\starttyping

+set_lua("mycode",id)

+set_lua("mycode",id,"global","protected")

+\stoptyping

+

+This creates a token that refers to a \LUA\ function with an entry in the table

+that you can access with \type {lua.get_functions_table}. It is the companion

+to \lpr {luadef}.

+

+\stopsubsection

+

+\startsubsection[title={Pushing back}]

+

+\libindex{get_next}

+\libindex{put_next}

+

+There is a (for now) experimental putter:

+

+\starttyping

+local t1 = token.get_next()

+local t2 = token.get_next()

+local t3 = token.get_next()

+local t4 = token.get_next()

+-- watch out, we flush in sequence

+token.put_next { t1, t2 }

+-- but this one gets pushed in front

+token.put_next ( t3, t4 )

+\stoptyping

+

+When we scan \type {wxyz!} we get \type {yzwx!} back. The argument is either a table

+with tokens or a list of tokens. The \type {token.expand} function will trigger

+expansion but what happens really depends on what you're doing where.

+

+\stopsubsection

+

+\startsubsection[title={Nota bene}]

+

+When scanning for the next token you need to keep in mind that we're not scanning

+like \TEX\ does: expanding, changing modes and doing things as it goes. When we

+scan with \LUA\ we just pick up tokens. Say that we have:

+

+\starttyping

+\bar

+\stoptyping

+

+but \type {\bar} is undefined. Normally \TEX\ will then issue an error message.

+However, when we have:

+

+\starttyping

+\def\foo{\bar}

+\stoptyping

+

+We get no error, unless we expand \type {\foo} while \type {\bar} is still

+undefined. What happens is that as soon as \TEX\ sees an undefined macro it will

+create a hash entry and when later it gets defined that entry will be reused. So,

+\type {\bar} really exists but can be in an undefined state.

+

+\startbuffer[demo]

+bar        : \directlua{tex.print(token.scan_csname())}\bar

+foo        : \directlua{tex.print(token.scan_csname())}\foo

+myfirstbar : \directlua{tex.print(token.scan_csname())}\myfirstbar

+\stopbuffer

+

+\startlines

+\getbuffer[demo]

+\stoplines

+

+This was entered as:

+

+\typebuffer[demo]

+

+The reason that you see \type {bar} reported and not \type {myfirstbar} is that

+\type {\bar} was already used in a previous paragraph.

+

+If we now say:

+

+\startbuffer

+\def\foo{}

+\stopbuffer

+

+\typebuffer \getbuffer

+

+we get:

+

+\startlines

+\getbuffer[demo]

+\stoplines

+

+And if we say

+

+\startbuffer

+\def\foo{\bar}

+\stopbuffer

+

+\typebuffer \getbuffer

+

+we get:

+

+\startlines

+\getbuffer[demo]

+\stoplines

+

+When scanning from \LUA\ we are not in a mode that defines (undefined) macros at

+all. There we just get the real primitive undefined macro token.

+

+\startbuffer

+\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\myfirstbar

+\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\mysecondbar

+\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\mythirdbar

+\stopbuffer

+

+\startlines

+\getbuffer

+\stoplines

+

+This was generated with:

+

+\typebuffer

+

+So, we do get a unique token because after all we need some kind of \LUA\ object

+that can be used and garbage collected, but it is basically the same one,

+representing an undefined control sequence.

+

+\stopsubsection

+

+\stopsection

+

+\startsection[title={The \type {kpse} library}][library=kpse]

+

+\topicindex{libraries+\type{kpse}}

+

+This library provides two separate, but nearly identical interfaces to the

+\KPATHSEA\ file search functionality: there is a \quote {normal} procedural

+interface that shares its kpathsea instance with \LUATEX\ itself, and an object

+oriented interface that is completely on its own.

+

+\startsubsection[title={\type {set_program_name} and \type {new}}]

+

+\libindex{set_program_name}

+\libindex{default_texmfcnf}

+\libindex{new}

+

+The way the library looks up variables is driven by the \type {texmf.cmf} file

+where the currently set program name acts as filter. You can check what file is

+used by with \type {default_texmfcnf}.

+

+Before the search library can be used at all, its database has to be initialized.

+There are three possibilities, two of which belong to the procedural interface.

+

+First, when \LUATEX\ is used to typeset documents, this initialization happens

+automatically and the \KPATHSEA\ executable and program names are set to \type

+{luatex} (that is, unless explicitly prohibited by the user's startup script.

+See~\in {section} [init] for more details).

+

+Second, in \TEXLUA\ mode, the initialization has to be done explicitly via the

+\type {kpse.set_program_name} function, which sets the \KPATHSEA\ executable

+(and optionally program) name.

+

+\startfunctioncall

+kpse.set_program_name(<string> name)

+kpse.set_program_name(<string> name, <string> progname)

+\stopfunctioncall

+

+The second argument controls the use of the \quote {dotted} values in the \type

+{texmf.cnf} configuration file, and defaults to the first argument.

+

+Third, if you prefer the object oriented interface, you have to call a different

+function. It has the same arguments, but it returns a userdata variable.

+

+\startfunctioncall

+local kpathsea = kpse.new(<string> name)

+local kpathsea = kpse.new(<string> name, <string> progname)

+\stopfunctioncall

+

+Apart from these two functions, the calling conventions of the interfaces are

+identical. Depending on the chosen interface, you either call \type

+{kpse.find_file} or \type {kpathsea:find_file}, with identical arguments and

+return values.

+

+\stopsubsection

+

+\startsubsection[title={\type {find_file}}]

+

+\topicindex {files+finding}

+

+\libindex {find_file}

+

+The most often used function in the library is \type {find_file}:

+

+\startfunctioncall

+<string> f = kpse.find_file(<string> filename)

+<string> f = kpse.find_file(<string> filename, <string> ftype)

+<string> f = kpse.find_file(<string> filename, <boolean> mustexist)

+<string> f = kpse.find_file(<string> filename, <string> ftype, <boolean> mustexist)

+<string> f = kpse.find_file(<string> filename, <string> ftype, <number> dpi)

+\stopfunctioncall

+

+Arguments:

+

+\startitemize[intro]

+

+\sym{filename}

+

+the name of the file you want to find, with or without extension.

+

+\sym{ftype}

+

+maps to the \type {-format} argument of \KPSEWHICH. The supported \type {ftype}

+values are the same as the ones supported by the standalone \type {kpsewhich}

+program: \startluacode

+    local list = {

+        "afm",

+        "base",

+        "bib",

+        "bitmap font",

+        "bst",

+        "cid maps",

+        "clua",

+        "cmap files",

+        "cnf",

+        "cweb",

+        "dvips config",

+        "enc files",

+        "fmt",

+        "font feature files",

+        "gf",

+        "graphic|/|figure",

+        "ist",

+        "lig files",

+        "ls-R",

+        "lua",

+        "map",

+        "mem",

+        "MetaPost support",

+        "mf",

+        "mfpool",

+        "mft",

+        "misc fonts",

+        "mlbib",

+        "mlbst",

+        "mp",

+        "mppool",

+        "ocp",

+        "ofm",

+        "opentype fonts",

+        "opl",

+        "other binary files",

+        "other text files",

+        "otp",

+        "ovf",

+        "ovp",

+        "pdftex config",

+        "pk",

+        "PostScript header",

+        "subfont definition files",

+        "tex",

+        "TeX system documentation",

+        "TeX system sources",

+        "texmfscripts",

+        "texpool",

+        "tfm",

+        "Troff fonts",

+        "truetype fonts",

+        "type1 fonts",

+        "type42 fonts",

+        "vf",

+        "web",

+        "web2c files",

+    }

+    table.sort(list)

+    context("{\\tttf \letterpercent, t}",list)

+\stopluacode

+

+The default type is \type {tex}. Note: this is different from \KPSEWHICH, which

+tries to deduce the file type itself from looking at the supplied extension.

+

+\sym{mustexist}

+

+is similar to \KPSEWHICH's \type {-must-exist}, and the default is \type {false}.

+If you specify \type {true} (or a non|-|zero integer), then the \KPSE\ library

+will search the disk as well as the \type {ls-R} databases.

+

+\sym{dpi}

+

+This is used for the size argument of the formats \type {pk}, \type {gf}, and

+\type {bitmap font}. \stopitemize

+

+\stopsubsection

+

+\startsubsection[title={\type {lookup}}]

+

+\libindex{lookup}

+

+A more powerful (but slower) generic method for finding files is also available.

+It returns a string for each found file.

+

+\startfunctioncall

+<string> f, ... = kpse.lookup(<string> filename, <table> options)

+\stopfunctioncall

+

+The options match commandline arguments from \type {kpsewhich}:

+

+\starttabulate[|l|l|p|]

+\DB key              \BC type     \BC explanation \NC \NR

+\TB

+\NC \type{debug}     \NC number   \NC set debugging flags for this lookup\NC     \NR

+\NC \type{format}    \NC string   \NC use specific file type (see list above)\NC \NR

+\NC \type{dpi}       \NC number   \NC use this resolution for this lookup; default 600\NC \NR

+\NC \type{path}      \NC string   \NC search in the given path\NC \NR

+\NC \type{all}       \NC boolean  \NC output all matches, not just the first\NC \NR

+\NC \type{mustexist} \NC boolean  \NC search the disk as well as ls-R if necessary\NC \NR

+\NC \type{mktexpk}   \NC boolean  \NC disable/enable mktexpk generation for this lookup\NC \NR

+\NC \type{mktextex}  \NC boolean  \NC disable/enable mktextex generation for this lookup\NC \NR

+\NC \type{mktexmf}   \NC boolean  \NC disable/enable mktexmf generation for this lookup\NC \NR

+\NC \type{mktextfm}  \NC boolean  \NC disable/enable mktextfm generation for this lookup\NC \NR

+\NC \type{subdir}    \NC string

+                         or table \NC only output matches whose directory part

+                                      ends with the given string(s) \NC \NR

+\LL

+\stoptabulate

+

+\stopsubsection

+

+\startsubsection[title={\type {init_prog}}]

+

+\topicindex {initialization+bitmaps}

+

+\libindex{init_prog}

+

+Extra initialization for programs that need to generate bitmap fonts.

+

+\startfunctioncall

+kpse.init_prog(<string> prefix, <number> base_dpi, <string> mfmode)

+kpse.init_prog(<string> prefix, <number> base_dpi, <string> mfmode, <string> fallback)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {readable_file}}]

+

+\libindex{readable_file}

+

+Test if an (absolute) file name is a readable file.

+

+\startfunctioncall

+<string> f = kpse.readable_file(<string> name)

+\stopfunctioncall

+

+The return value is the actual absolute filename you should use, because the disk

+name is not always the same as the requested name, due to aliases and

+system|-|specific handling under e.g.\ \MSDOS. Returns \type {nil} if the file

+does not exist or is not readable.

+

+\stopsubsection

+

+\startsubsection[title={\type {expand_path}}]

+

+\libindex{expand_path}

+

+Like kpsewhich's \type {-expand-path}:

+

+\startfunctioncall

+<string> r = kpse.expand_path(<string> s)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {expand_var}}]

+

+\libindex{expand_var}

+

+Like kpsewhich's  \type {-expand-var}:

+

+\startfunctioncall

+<string> r = kpse.expand_var(<string> s)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {expand_braces}}]

+

+\libindex{expand_braces}

+

+Like kpsewhich's \type {-expand-braces}:

+

+\startfunctioncall

+<string> r = kpse.expand_braces(<string> s)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {show_path}}]

+

+\libindex{show_path}

+

+Like kpsewhich's \type {-show-path}:

+

+\startfunctioncall

+<string> r = kpse.show_path(<string> ftype)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {var_value}}]

+

+\libindex{var_value}

+

+Like kpsewhich's \type {-var-value}:

+

+\startfunctioncall

+<string> r = kpse.var_value(<string> s)

+\stopfunctioncall

+

+\stopsubsection

+

+\startsubsection[title={\type {version}}]

+

+\libindex{version}

+

+Returns the kpathsea version string.

+

+\startfunctioncall

+<string> r = kpse.version()

+\stopfunctioncall

+

+\stopsubsection

+

+\stopsection

+

+\stopchapter

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex-titlepage.tex b/Master/texmf-dist/doc/luatex/base/luatex-titlepage.tex
index 307741ee115..5a91f73e8b8 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex-titlepage.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex-titlepage.tex
@@ -1,70 +1,53 @@
-\environment luatex-style
-\environment luatex-logos
-
-\startcomponent luatex-titlepage
-
-\startstandardmakeup
-
-    \switchtobodyfont
-      [mainfacemedium]
-
-    \definedfont[Bold*default at \the\dimexpr.08\paperheight\relax] \setupinterlinespace
-
-    \setlayer
-      [page]
-      {\useMPgraphic{luapage}}
-
-    \setlayerframed
-      [page]
-      [preset=middletop,
-       voffset=.05\paperheight]
-      [align=middle,
-       foregroundcolor=blue,
-       frame=off]
-      {Lua\TeX\\Reference Manual}
-
-    \definedfont[Bold*default at 18pt] \setupinterlinespace
-
-    \setlayerframed
-      [page]
-      [preset=rightbottom,
-       offset=.01\paperheight]
-      [align=flushright,
-       foregroundcolor=blue,
-       frame=off]
-      {\doifsomething{\documentvariable{status}}{\documentvariable{status}\par}
-       \currentdate[month,space,year]\par
-       Version \documentvariable{version}}
-
-\stopstandardmakeup
-
-\startstandardmakeup
-
-    \start
-        \raggedleft
-        \definedfont[Bold*default at 48pt]
-        \setupinterlinespace
-        \blue Lua\TeX \endgraf Reference \endgraf Manual \endgraf
-    \stop
-
-    \vfill
-
-    \definedfont[Bold*default at 12pt]
-
-    \starttabulate[|l|l|]
-        \NC copyright \EQ Lua\TeX\ development team \NC \NR
-        \NC more info \EQ www.luatex.org \NC \NR
-        \NC version   \EQ \currentdate \doifsomething{\documentvariable{snapshot}}{(snapshot \documentvariable{snapshot})} \NC \NR
-    \stoptabulate
-
-\stopstandardmakeup
-
-\setupbackgrounds
-  [leftpage]
-  [setups=pagenumber:left]
-
-\setupbackgrounds
-  [rightpage]
-  [setups=pagenumber:right]
-
-\stopcomponent
+\environment luatex-style

+

+\startcomponent luatex-titlepage

+

+\startstandardmakeup

+

+    \switchtobodyfont

+      [mainfacemedium]

+

+    \definedfont[Bold*default at \the\dimexpr.06\paperheight\relax] \setupinterlinespace

+

+    \setlayer

+      [page]

+      {\useMPgraphic{luapage}}

+

+    \setlayerframed

+      [page]

+      [preset=righttop,

+       location=middletop,

+       hoffset=.500\measured{paperwidth},

+       voffset=.175\measured{paperheight}]

+      [align=middle,

+       foregroundcolor=white,

+       frame=off]

+      {\documentvariable{manual}\crlf Reference\crlf Manual}

+

+    \definedfont[Bold*default at 14pt] \setupinterlinespace

+

+    \setlayerframed

+      [page]

+      [preset=rightbottom,

+       offset=.025\measured{paperheight}]

+      [align=flushright,

+       foregroundcolor=white,

+       frame=off]

+      {\doifsomething{\documentvariable{status}}{\documentvariable{status}\par}

+       \currentdate[month,space,year]\par

+       Version \documentvariable{version}}

+

+    \setlayerframed

+      [page]

+      [preset=middle,

+       hoffset=-.5\dimexpr\measured{paperwidth}-\measured{spinewidth}\relax]

+      [width=.7\measured{paperwidth},

+       align=normal,

+       foregroundstyle=\bf,

+       foregroundcolor=white,

+       frame=off]

+      {\getbuffer[backpage]}

+

+\stopstandardmakeup

+

+\stopcomponent

diff --git a/Master/texmf-dist/doc/luatex/base/luatex.pdf b/Master/texmf-dist/doc/luatex/base/luatex.pdf
index cff5159b74b..dd73cae76fc 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex.pdf
+++ b/Master/texmf-dist/doc/luatex/base/luatex.pdf
diff --git a/Master/texmf-dist/doc/luatex/base/luatex.tex b/Master/texmf-dist/doc/luatex/base/luatex.tex
index aee50ec6d7b..61c075243d8 100644
--- a/Master/texmf-dist/doc/luatex/base/luatex.tex
+++ b/Master/texmf-dist/doc/luatex/base/luatex.tex
@@ -1,78 +1,103 @@
-% macros=mkvi
-
-% author    : Hans Hagen with Taco Hoekwater, Luigi Scarso & Hartmut Henkel
-% copyright : PRAGMA ADE & ConTeXt Development Team
-% license   : Creative Commons Attribution ShareAlike 4.0 International
-% reference : pragma-ade.nl | contextgarden.net | texlive (related) distributions
-% origin    : the ConTeXt distribution
-%
-% comment   : Because this manual is distributed with TeX distributions it comes with a rather
-%             liberal license. We try to adapt these documents to upgrades in the (sub)systems
-%             that they describe. Using parts of the content otherwise can therefore conflict
-%             with existing functionality and we cannot be held responsible for that. Many of
-%             the manuals contain characteristic graphics and personal notes or examples that
-%             make no sense when used out-of-context.
-%
-% comment   : Some (parts of) chapters might have been published in TugBoat, the NTG Maps, the
-%             ConTeXt Group journal or otherwise. Thanks to the editors for corrections. Also
-%             thanks to users for testing, feedback and corrections.
-%
-% 238 pages : 2017-07-06
-%             luatex 9.5 sec / luajittex 7.0 sec
-%             Dell 7600 / i7 3840QM / passmark 1.922 / Windows 10 64 bit
-
-% \tex vs \type vs \syntax vs. \luatex
-% \em \it \/
-
-% "context --nodates --nocompression luatex" can be used for comparison
-% runs, not that we do it
-
-% \enabledirectives[hyphenator.flatten]
-
-% \setupsynctex[state=start,method=max] % adds 5 pct overhead
-
-\environment luatex-style
-\environment luatex-logos
-
-\startmode[export]
-
-    \setupbackend
-      [export=luatex]
-
-\stopmode
-
-\dontcomplain
-
-\startdocument
-  [status=release,
-   version=1.07.0]
-
-\startnotmode[*export]
-    \component luatex-titlepage
-\stopnotmode
-
-\startmode[*export]
-    \component luatex-export-titlepage
-\stopmode
-
-\startfrontmatter
-    \component luatex-contents
-    \component luatex-introduction
-\stopfrontmatter
-
-\startbodymatter
-    \component luatex-enhancements
-    \component luatex-modifications
-    \component luatex-lua
-    \component luatex-languages
-    \component luatex-fonts
-    \component luatex-math
-    \component luatex-nodes
-    \component luatex-callbacks
-    \component luatex-tex
-    \component luatex-graphics
-    \component luatex-fontloader
-    \component luatex-backend
-\stopbodymatter
-
-\stopdocument
+% macros=mkvi

+

+% \nopdfcompression

+

+% \disabledirectives[vspacing.synchronizepage]

+

+% wsl /data/context/runcontext.sh --global --path=/mnt/c/data/develop/context/manuals/mkiv/external/luatex luatex.tex

+%

+% same runtime as regular context or linux

+% author    : Hans Hagen with Taco Hoekwater, Luigi Scarso & Hartmut Henkel

+% copyright : PRAGMA ADE & ConTeXt Development Team

+% license   : Creative Commons Attribution ShareAlike 4.0 International

+% reference : pragma-ade.nl | contextgarden.net | texlive (related) distributions

+% origin    : the ConTeXt distribution

+%

+% comment   : Because this manual is distributed with TeX distributions it comes with a rather

+%             liberal license. We try to adapt these documents to upgrades in the (sub)systems

+%             that they describe. Using parts of the content otherwise can therefore conflict

+%             with existing functionality and we cannot be held responsible for that. Many of

+%             the manuals contain characteristic graphics and personal notes or examples that

+%             make no sense when used out-of-context.

+%

+% comment   : Some (parts of) chapters might have been published in TugBoat, the NTG Maps, the

+%             ConTeXt Group journal or otherwise. Thanks to the editors for corrections. Also

+%             thanks to users for testing, feedback and corrections.

+

+% \tex vs \type vs \syntax vs. \luatex

+% \em \it \/

+

+% "context --nodates --nocompression luatex" can be used for comparison

+% runs, not that we do it

+

+% todo: all (sub)section to start/stop

+

+% \enabledirectives[hyphenator.flatten]

+

+% \setupsynctex[state=start,method=max] % adds 5 pct overhead

+

+% compoundhyphenmode      : looks okay

+% endlocalcontrol         : very experimental

+% fixupboxesmode          : very experimental

+% mathflattenmode         : looks okay

+% mathrulethicknessmode   : looks okay

+

+% after adding primitives: context mult-prm.mkiv

+

+\environment luatex-style

+

+\startmode[book]

+    \environment luatex-style-book

+\stopmode

+

+\environment luatex-logos

+\environment luatex-private

+

+\startmode[export]

+

+    \setupbackend

+      [export=luatex]

+

+\stopmode

+

+\startdocument

+  [manual=Lua\TeX,

+   status=experimental,

+   version=1.10]

+

+\startnotmode[*export]

+    \component luatex-titlepage

+    \component luatex-firstpage

+\stopnotmode

+

+\startmode[*export]

+    \component luatex-export-titlepage

+\stopmode

+

+\startfrontmatter

+    \component luatex-contents

+    \component luatex-introduction

+\stopfrontmatter

+

+\startbodymatter

+    \component luatex-preamble

+    \component luatex-enhancements

+    \component luatex-modifications

+    \component luatex-lua

+    \component luatex-languages

+    \component luatex-fonts

+    \component luatex-math

+    \component luatex-nodes

+    \component luatex-callbacks

+    \component luatex-tex

+    \component luatex-graphics

+    \component luatex-fontloader

+    \component luatex-backend

+\stopbodymatter

+

+\startbackmatter

+    \component luatex-registers

+    \component luatex-statistics

+\stopbackmatter

+

+\stopdocument