summaryrefslogtreecommitdiff
path: root/Master/texmf-dist/doc/context/sources/general/manuals/luatex
diff options
context:
space:
mode:
Diffstat (limited to 'Master/texmf-dist/doc/context/sources/general/manuals/luatex')
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex15
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-fonts.tex6
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-introduction.tex14
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-languages.tex202
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-math.tex270
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-modifications.tex363
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-nodes.tex673
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-style.tex6
-rw-r--r--Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex.tex24
9 files changed, 1308 insertions, 265 deletions
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex
index 35c27cfb601..19f88234a53 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex
@@ -736,7 +736,7 @@ This primitive is still experimental but can be used to assign a meaning to an a
character, as in:
\starttyping
-\def\foo{bar} \letcharcode123\foo
+\def\foo{bar} \letcharcode123=\foo
\stoptyping
This can be a bit nicer that using the uppercase tricks (using the property of
@@ -759,6 +759,13 @@ the acceptable range is from 0 to 65535.
These three primitives are like \type {\vbox}, \type {\hbox} and \type {\vtop}
but don't apply the related callbacks.
+\subsection{\type {\vsplit}}
+
+The \type {\vsplit} primitive has to be followed by a specification of the
+required height. As alternative for the \type {to} keyword you can use \type
+{upto} to get a split of the given size but result has the natural dimensions
+then.
+
\subsection{Images and Forms}
These two concepts are now core concepts and no longer whatsits. They are in fact
@@ -794,6 +801,12 @@ optional parameters are not implemented for \type {\saveboxresource}.
\useboxresource width 20mm height 10mm depth 5mm \lastsavedboxresourceindex
\stoptyping
+The box resources are of course implemented in the backend and therefore we do
+support the \type {attr} and \type {resources} keys that accept a token list. New
+is the \type {type} key. When set to non|-|zero the \type {/Type} entry is
+omitted. A value of 1 or 3 still writes a \type {/BBox}, while 2 or 3 will write
+a \type {/Matrix}.
+
\subsection{\type {\nohrule} and \type {\novrule}}
Because introducing a new keyword can cause incompatibilities, two new primitives
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-fonts.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-fonts.tex
index 7384f3b3ecc..221c0ce852c 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-fonts.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-fonts.tex
@@ -36,7 +36,9 @@ The top|-|level keys in the table are as follows:
\NC fonts \NC yes \NC no \NC yes \NC table \NC locally used fonts \NC \NR
\NC psname \NC no \NC no \NC yes \NC string \NC This is the \POSTSCRIPT\ fontname in the incoming font
source, and it's used as fontname identifier in the \PDF\
- output. \NC \NR
+ output. This has to be a valid string, e.g.\ no spaces
+ and such, as the backend will not do a cleanup. This gives
+ complete control to the loader. \NC \NR
\NC fullname \NC no \NC no \NC yes \NC string \NC output font name, used as a fallback in the \PDF\ output
if the \type {psname} is not set \NC \NR
\NC header \NC yes \NC no \NC no \NC string \NC header comments, if any \NC \NR
@@ -75,6 +77,8 @@ The top|-|level keys in the table are as follows:
\NC nomath \NC no \NC no \NC yes \NC boolean\NC This key allows a minor speedup for text fonts. If it
is present and true, then \LUATEX\ will not check the
character entries for math|-|specific keys. \NC \NR
+\NC oldmath \NC no \NC no \NC yes \NC boolean\NC This key flags a font as representing an old school \TEX\
+ math font and disables the \OPENTYPE\ code path. \NC \NR
\NC slant \NC no \NC no \NC yes \NC number \NC This has the same semantics as the \type {SlantFont}
operator in font map files. \NC \NR
\NC extent \NC no \NC no \NC yes \NC number \NC This has the same semantics as the \type {ExtendFont}
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-introduction.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-introduction.tex
index 8ab8b4463df..196a9af4ffa 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-introduction.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-introduction.tex
@@ -98,6 +98,14 @@ code in \TEX\ engines (especially code that is not needed any longer).
\stopitem
\stopitemize
+We try to keep upcoming versions compatible but intermediate releases can contain
+experimental features. A general rule is that versions that end up on \TEX live
+and|/|or are released around \CONTEXT\ meetings are stable. Future versions will
+probably become a bit leaner and meaner. Some libraries might become external as
+we don't want to bloat the binary and also don't want to add more hard coded
+solutions. After all, with \LUA\ you can extend the core functionality. The less
+dependencies, the better.
+
The \TEXLIVE\ version is to be considered the current stable version. Any version
between the yearly \TEXLIVE\ releases are to be considered beta. The beta
releases are normally available via the \CONTEXT\ distribution channels (the
@@ -112,8 +120,10 @@ Taco Hoekwater \& Luigi Scarso
\starttabulate
\NC Version \EQ \currentdate \NC \NR
-\NC \LUATEX \EQ Snapshot \number\luatexversion.\luatexrevision \NC \NR
-\NC \CONTEXT \EQ \contextversion \NC \NR
+\NC \LUATEX \EQ version \cldcontext{status.luatex_version/100},
+ revision \cldcontext{status.luatex_revision},
+ number \cldcontext{environment.luatexversion} \NC \NR
+\NC \CONTEXT \EQ MkIV \contextversion \NC \NR
\stoptabulate
\stopchapter
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-languages.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-languages.tex
index 19e3f7b1458..54a7b390d5f 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-languages.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-languages.tex
@@ -147,6 +147,38 @@ hyphenation happens is (normally) when the paragraph or a horizontal box is
constructed. When \type {\savinghyphcodes} was zero when the language got
initialized you start out with nothing, otherwise you already have a set.
+When a \type {\hjcode} is larger than $0$ but smaller than $32$ is indicates the
+to be used length. In the following example we map a character (\type {x}) onto
+another one in the patterns and tell the engine that \type {œ} counts as one
+character. Because traditionally zero itself is reserved for inhibiting
+hyphenation, a value of $32$ counts as zero.
+
+\starttyping
+% assuming french patterns:
+foobar % foo-bar
+
+\hjcode`x=`o
+
+fxxbar % fxx-bar
+
+\lefthyphenmin3
+
+œdipus % œdi-pus
+
+\lefthyphenmin4
+
+œdipus % œdipus
+
+\hjcode`œ=2
+
+œdipus % œdi-pus
+
+\hjcode`i=32
+\hjcode`d=32
+
+œdipus % œdipus
+\stoptyping
+
Carrying all this information with each glyph would give too much overhead and
also make the process of setting up thee codes more complex. A solution with
\type {hjcode} sets was considered but rejected because in practice the current
@@ -180,6 +212,134 @@ as trigger. Here are a few examples of usage:
\stopbuffer
\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop
+We only accept an explicit hyphen when there is a preceding glyph and we skip a
+sequence of explicit hyphens as that normally indicates a \type {--} or \type
+{---} ligature in which case we can in a worse case usage get bad node lists
+later on due to messed up ligature building as these dashes are ligatures in base
+fonts. This is a side effect of the separating the hyphenation, ligaturing and
+kerning steps.
+
+The start and end of a characters is signalled by a glue, penalty, kern or boundary
+node. But by default also a hlist, vlist, rule, dir, whatsit, ins, and adjust node
+indicate a start or end. You can omit the last set from the test by setting
+\type {\hyphenationbounds} to a non|-|zero value:
+
+\starttabulate[|Tl|l|]
+\NC 0 \NC not strict \NC \NR
+\NC 1 \NC strict start \NC \NR
+\NC 2 \NC strict end \NC \NR
+\NC 3 \NC strict start and strict end \NC \NR
+\stoptabulate
+
+The word start is determined as follows:
+
+\starttabulate[|Bl|l|]
+\NC boundary \NC yes when wordboundary \NC \NR
+\NC hlist \NC when hyphenationbounds 1 or 3 \NC \NR
+\NC vlist \NC when hyphenationbounds 1 or 3 \NC \NR
+\NC rule \NC when hyphenationbounds 1 or 3 \NC \NR
+\NC dir \NC when hyphenationbounds 1 or 3 \NC \NR
+\NC whatsit \NC when hyphenationbounds 1 or 3 \NC \NR
+\NC glue \NC yes \NC \NR
+\NC math \NC skipped \NC \NR
+\NC glyph \NC exhyphenchar (one only) : yes (so no -- ---) \NC \NR
+\NC otherwise \NC yes \NC \NR
+\stoptabulate
+
+The word end is determined as follows:
+
+\starttabulate[|Bl|l|]
+\NC boundary \NC yes \NC \NR
+\NC glyph \NC yes when different language \NC \NR
+\NC glue \NC yes \NC \NR
+\NC penalty \NC yes \NC \NR
+\NC kern \NC yes when not italic (for some historic reason) \NC \NR
+\NC hlist \NC when hyphenationbounds 2 or 3 \NC \NR
+\NC vlist \NC when hyphenationbounds 2 or 3 \NC \NR
+\NC rule \NC when hyphenationbounds 2 or 3 \NC \NR
+\NC dir \NC when hyphenationbounds 2 or 3 \NC \NR
+\NC whatsit \NC when hyphenationbounds 2 or 3 \NC \NR
+\NC ins \NC when hyphenationbounds 2 or 3 \NC \NR
+\NC adjust \NC when hyphenationbounds 2 or 3 \NC \NR
+\stoptabulate
+
+% (Future versions of \LUATEX\ might provide more granularity.)
+
+In traditional \TEX\ ligature building and hyphenation are interwoven with the
+line break mechanism. In \LUATEX\ these phases are isolated. As a consequence we
+deal differently with (a sequence of) explicit hyphens. We already have added
+some control over aspects of the hyphenation and yet another one concerns
+automatic hyphens (e.g.\ \type {-} characters in the input).
+
+When \type {\automatichyphenmode} has a value of 0, a hyphen will be turned into
+an automatic discretionary. The snippets before and after it will not be
+hyphenated. A side effect is that a leading hyphen can lead to a split but one
+will seldom run into that situation. Setting a pre and post character makes this
+more prominent. A value of 1 will prevent this side effect and a value of 2 will
+not turn the hyphen into a discretionary. Experiments with other options, like
+permitting hyphenation, of the words on both sides were discarded.
+
+\startbuffer[a]
+before-after \par
+before--after \par
+before---after \par
+\stopbuffer
+
+\startbuffer[b]
+-before \par
+after- \par
+--before \par
+after-- \par
+---before \par
+after--- \par
+\stopbuffer
+
+\startbuffer[c]
+before-after \par
+before--after \par
+before---after \par
+\stopbuffer
+
+We show three samples:
+
+Input A: \typebuffer[a]
+Input B: \typebuffer[b]
+Input C: \typebuffer[c]
+
+\startbuffer[demo]
+\startcombination[nx=4,ny=3,location=top]
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[a]}} {A~0~6em}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[a]}} {A~0~2pt}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone \hsize2pt \getbuffer[a]}} {A~1~2pt}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo \hsize2pt \getbuffer[a]}} {A~2~2pt}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[b]}} {B~0~6em}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[b]}} {B~0~2pt}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone \hsize2pt \getbuffer[b]}} {B~1~2pt}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo \hsize2pt \getbuffer[b]}} {B~2~2pt}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[c]}} {C~0~6em}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize2pt \getbuffer[c]}} {C~0~2pt}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plusone \hsize2pt \getbuffer[c]}} {C~1~2pt}
+ {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\plustwo \hsize2pt \getbuffer[c]}} {C~2~2pt}
+\stopcombination
+\stopbuffer
+
+\startplacefigure[reference=automatic:1,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with a \type {\hsize}
+of 6em and 2pt (which triggers a linebreak).}]
+ \dontcomplain \tt \getbuffer[demo]
+\stopplacefigure
+
+\startplacefigure[reference=automatic:2,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with \type
+{\preexhyphenchar} and \type {\postexhyphenchar} set to characters \type {A} and \type {B}.}]
+ \postexhyphenchar`A\relax
+ \preexhyphenchar `B\relax
+ \dontcomplain \tt \getbuffer[demo]
+\stopplacefigure
+
+As with primitive companions of other single character commands, the \type {\-}
+command has a more verbose primitive version in \type {\explicitdiscretionary}
+and the normally intercepted in the hyphenator character \type {-} (or whatever
+is configured) is available as \type {\automaticdiscretionary}.
+
\section{The main control loop}
In \LUATEX's main loop, almost all input characters that are to be typeset are
@@ -260,6 +420,34 @@ character|-|handling code have been moved back inline. When \type
{\tracingcommands} is on, this is visible because the full word is reported,
instead of just the initial character.
+Because we tend to make hard codes behaviour configurable a few new primitives
+have been added:
+
+\starttyping
+\hyphenpenaltymode
+\automatichyphenpenalty
+\explicithyphenpenalty
+\stoptyping
+
+The first parameter has the following consequences for automatic discs (the ones
+resulting from an \type {\exhyphenchar}:
+
+\starttabulate[|Tc|l|l|]
+\BC mode \BC automatic disc \type{-} \BC explicit disc \type{\-} \NC \NR
+\HL
+\NC 0 \NC \type {\exhyphenpenalty} \NC \type {\exhyphenpenalty} \NC \NR
+\NC 1 \NC \type {\hyphenpenalty} \NC \type {\hyphenpenalty} \NC \NR
+\NC 2 \NC \type {\exhyphenpenalty} \NC \type {\hyphenpenalty} \NC \NR
+\NC 3 \NC \type {\hyphenpenalty} \NC \type {\exhyphenpenalty} \NC \NR
+\NC 4 \NC \type {\automatichyphenpenalty} \NC \type {\explicithyphenpenalty} \NC \NR
+\NC 5 \NC \type {\exhyphenpenalty} \NC \type {\explicithyphenpenalty} \NC \NR
+\NC 6 \NC \type {\hyphenpenalty} \NC \type {\explicithyphenpenalty} \NC \NR
+\NC 7 \NC \type {\automatichyphenpenalty} \NC \type {\exhyphenpenalty} \NC \NR
+\NC 8 \NC \type {\automatichyphenpenalty} \NC \type {\hyphenpenalty} \NC \NR
+\stoptabulate
+
+other values do what we always did in \LUATEX: insert \type {\exhyphenpenalty}.
+
\section[patternsexceptions]{Loading patterns and exceptions}
The hyphenation algorithm in \LUATEX\ is quite different from the one in \TEX82,
@@ -703,3 +891,17 @@ initialized due to \type {\savinghyphcodes} being larger than zero.
\stopchapter
\stopcomponent
+
+% \parindent0pt \hsize=1.1cm
+% 12-34-56 \par
+% 12-34-\hbox{56} \par
+% 12-34-\vrule width 1em height 1.5ex \par
+% 12-\hbox{34}-56 \par
+% 12-\vrule width 1em height 1.5ex-56 \par
+% \hjcode`\1=`\1 \hjcode`\2=`\2 \hjcode`\3=`\3 \hjcode`\4=`\4 \vskip.5cm
+% 12-34-56 \par
+% 12-34-\hbox{56} \par
+% 12-34-\vrule width 1em height 1.5ex \par
+% 12-\hbox{34}-56 \par
+% 12-\vrule width 1em height 1.5ex-56 \par
+
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-math.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-math.tex
index cb8d198b1fc..3c29d960975 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-math.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-math.tex
@@ -38,6 +38,99 @@ be used as numeric values, so you can write code like this:
\fi \fi
\stoptyping
+Sometimes you won't get what you expect so a bit of explanation might help to
+understand what happens. When math is parsed and expanded it gets turned into a
+linked list. In a second pass the formula will be build. This has to do with the
+fact that in order to determine the automatically chosen sizes (in for instance
+fractions) following content can influence preceding sizes. A side effect of this
+is for instance that one cannot change the definition of a font family (and
+thereby reusing numbers) because the number that got used is stored and used in
+the second pass (so changing \type {\fam 12} mid|-|formula spoils over to
+preceding use of that family).
+
+The style switching primitives like \type {\textstyle} are turned into nodes so
+the styles set there are frozen. The \type {\mathchoice} primitive results in
+four lists being constructed of which one is used in the second pass. The fact
+that some automatic styles are not yet known also means that the \type
+{\mathstyle} primitive expands to the current style which can of course be
+different from the one really used. It's a snapshot of the first pass state. As a
+consequence in the following example you get a style number (first pass) typeset
+that can actually differ from the used style (second pass). In the case of a math
+choice used ungrouped, the chosen style is used after the choice too, unless you
+group.
+
+\startbuffer[1]
+ [a:\mathstyle]\quad
+ \bgroup
+ \mathchoice
+ {\bf \scriptstyle (x:d :\mathstyle)}
+ {\bf \scriptscriptstyle (x:t :\mathstyle)}
+ {\bf \scriptscriptstyle (x:s :\mathstyle)}
+ {\bf \scriptscriptstyle (x:ss:\mathstyle)}
+ \egroup
+ \quad[b:\mathstyle]\quad
+ \mathchoice
+ {\bf \scriptstyle (y:d :\mathstyle)}
+ {\bf \scriptscriptstyle (y:t :\mathstyle)}
+ {\bf \scriptscriptstyle (y:s :\mathstyle)}
+ {\bf \scriptscriptstyle (y:ss:\mathstyle)}
+ \quad[c:\mathstyle]\quad
+ \bgroup
+ \mathchoice
+ {\bf \scriptstyle (z:d :\mathstyle)}
+ {\bf \scriptscriptstyle (z:t :\mathstyle)}
+ {\bf \scriptscriptstyle (z:s :\mathstyle)}
+ {\bf \scriptscriptstyle (z:ss:\mathstyle)}
+ \egroup
+ \quad[d:\mathstyle]
+\stopbuffer
+
+\startbuffer[2]
+ [a:\mathstyle]\quad
+ \begingroup
+ \mathchoice
+ {\bf \scriptstyle (x:d :\mathstyle)}
+ {\bf \scriptscriptstyle (x:t :\mathstyle)}
+ {\bf \scriptscriptstyle (x:s :\mathstyle)}
+ {\bf \scriptscriptstyle (x:ss:\mathstyle)}
+ \endgroup
+ \quad[b:\mathstyle]\quad
+ \mathchoice
+ {\bf \scriptstyle (y:d :\mathstyle)}
+ {\bf \scriptscriptstyle (y:t :\mathstyle)}
+ {\bf \scriptscriptstyle (y:s :\mathstyle)}
+ {\bf \scriptscriptstyle (y:ss:\mathstyle)}
+ \quad[c:\mathstyle]\quad
+ \begingroup
+ \mathchoice
+ {\bf \scriptstyle (z:d :\mathstyle)}
+ {\bf \scriptscriptstyle (z:t :\mathstyle)}
+ {\bf \scriptscriptstyle (z:s :\mathstyle)}
+ {\bf \scriptscriptstyle (z:ss:\mathstyle)}
+ \endgroup
+ \quad[d:\mathstyle]
+\stopbuffer
+
+\typebuffer[1]
+
+% \typebuffer[2]
+
+This gives:
+
+\blank $\displaystyle \getbuffer[1]$ \blank
+\blank $\textstyle \getbuffer[1]$ \blank
+
+Using \type {\begingroup} \unknown\ \type {\endgroup} instead gives:
+
+\blank $\displaystyle \getbuffer[2]$ \blank
+\blank $\textstyle \getbuffer[2]$ \blank
+
+This might look wrong but it's just a side effect of \type {\mathstyle} expanding
+to the current (first pass) style and the number being injected in the list that
+gets converted in the second pass. It all makes sense and it illustrates the
+importance of grouping. In fact, the math choice style being effective afterwards
+has advantages. It would be hard to get it otherwise.
+
\subsection{\type {\Ustack}}
There are a few math commands in \TEX\ where the style that will be used is not
@@ -174,7 +267,7 @@ are described as follows:
\startitemize
\startitem
In any style superscripts and subscripts are taken from the next smaller style.
- Exception: in display style they are taken in script style.
+ Exception: in display style they are in script style.
\stopitem
\startitem
Subscripts are always in the cramped variant of the style; superscripts are only
@@ -314,7 +407,7 @@ needed.
The injection of \type {\abovedisplayskip} and \type {\belowdisplayskip} is not
symmetrical. An above one is always inserted, also when zero, but the below is
-only inserted when larger than zero. Especially the later mkes it sometimes hard
+only inserted when larger than zero. Especially the later makes it sometimes hard
to fully control spacing. Therefore \LUATEX\ comes with a new directive: \type
{\mathdisplayskipmode}. The following values apply:
@@ -464,6 +557,86 @@ Note 9: \type {FractionDelimiterDisplayStyleSize} and \type
{FractionDelimiterSize} do not actually exist in the \quote {standard} \OPENTYPE\
math font Cambria, but were useful enough to be added.
+\section{Nolimit correction}
+
+There are two extra math parameters \type {\Umathnolimitsupfactor} and \type
+{\Umathnolimitsubfactor} that were added to provide some control over how limits
+are spaced (for example the position of super and subscripts after integral
+operators). They relate to an extra parameter \type {\mathnolimitsmode}. The half
+corrections are what happens when scripts are placed on above and below. The
+problem with italic corrections is that officially that correction italic is used
+for above|/|below placement while advanced kerns are used for placement at the
+right end. The question is: how often is this implemented, and if so, does the
+kerns assume correction too. Anyway, with this parameter one can control it.
+
+\starttabulate[|l|ck1|ck1|ck1|ck1|ck1|ck1|]
+ \NC
+ \NC \mathnolimitsmode0 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode1 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode2 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode3 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode4 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode8000 $\displaystyle\int\nolimits^0_1$
+ \NC \NR
+ \TB
+ \NC \bf mode
+ \NC \tttf 0
+ \NC \tttf 1
+ \NC \tttf 2
+ \NC \tttf 3
+ \NC \tttf 4
+ \NC \tttf 8000
+ \NC \NR
+ \NC \bf superscript
+ \NC 0
+ \NC font
+ \NC 0
+ \NC 0
+ \NC +ic/2
+ \NC 0
+ \NC \NR
+ \NC \bf subscript
+ \NC -ic
+ \NC font
+ \NC 0
+ \NC -ic/2
+ \NC -ic/2
+ \NC 8000ic/1000
+ \NC \NR
+\stoptabulate
+
+When the mode is set to one, the math parameters are used. This way a macro
+package writer can decide what looks best. Given the current state of fonts in
+\CONTEXT\ we currently use mode 1 with factor 0 for the superscript and 750 for
+the subscripts. Positive values are used for both parameters but the subscript
+shifts to the left. A \type {\mathnolimitsmode} larger that 15 is considered to
+be a factor for the subscript correction. This feature can be handy when
+experimenting.
+
+\section{Math italic mess}
+
+The \type {\mathitalicsmode} parameter can be set to~1 to force italic correction
+before noads that represent some more complex structure (read: everything
+that is not an ord, bin, rel, open, close, punct or inner).
+
+\starttexdefinition Whatever #1
+ \NC \type{\mathitalicsmode=#1}
+ \NC \mathitalicsmode#1\ruledhbox{$\left|T^1\right|$}
+ \NC \mathitalicsmode#1\ruledhbox{$\left|T\right|$}
+ \NC \mathitalicsmode#1\ruledhbox{$T+1$}
+ \NC \mathitalicsmode#1\ruledhbox{$T{1\over2}$}
+ \NC \mathitalicsmode#1\ruledhbox{$T\sqrt{1}$}
+ \NR \NR
+\stoptexdefinition
+
+\starttabulate[|c|c|c|c|c|c|]
+ \Whatever0
+ \Whatever1
+\stoptabulate
+
+This kind of parameters relate to the fact that italic correction in \OPENTYPE\
+math is bound to fuzzy rules. So, control is the solution.
+
\section{Math spacing setting}
Besides the parameters mentioned in the previous sections, there are also 64 new
@@ -603,7 +776,7 @@ correction takes place.
Possible locations are \type {top}, \type {bottom}, \type {both} and \type
{center}. When no location is given \type {top} is assumed. An additional
parameter \type {fraction} can be specified followed by a number; a value of for
-instance 1200 means that the criterium is 1.2 times the width of the nuclues. The
+instance 1200 means that the criterium is 1.2 times the width of the nucleus. The
fraction only applies to the stepwise selected shapes and is mostly meant for the
\type {overlay} location. It also works for the other locations but then it
concerns the width.
@@ -831,12 +1004,34 @@ For Latin Modern The result looks as follows:
\stoptabulate
\stop
+\section {Last lines}
+
+There is a new primitive to control the overshoot in the calculation of the
+previous line in mid|-|paragraph display math. The default value is 2 times
+the em width of the current font:
+
+\starttyping
+\predisplaygapfactor=2000
+\stoptyping
+
+If you want to have the length of the last line independent of math i.e.\ you don't
+want to revert to a hack where you insert a fake display math formula in order to
+get the length of the last line, the following will often work too:
+
+\starttyping
+\def\lastlinelength{\dimexpr
+ \directlua {tex.sprint (
+ (nodes.dimensions(node.tail(tex.lists.page_head).list))
+ )}sp
+\relax}
+\stoptyping
+
\section {Other Math changes}
\subsection {Verbose versions of single-character math commands}
\LUATEX\ defines six new primitives that have the same function as
-\type {^}, \type {_}, \type {$}, and \type {$$}: %$
+\type {^}, \type {_}, \type {$}, and \type {$$}:
\starttabulate[|l|l|l|l|]
\NC \bf primitive \NC \bf explanation \NC \NR
@@ -869,7 +1064,57 @@ Inline math is surrounded by (optional) \type {\mathsurround} spacing but that i
dimension. There is now an additional parameter \type {\mathsurroundskip}. When set to a
non|-|zero value (or zero with some stretch or shrink) this parameter will replace
\type {\mathsurround}. By using an additional parameter instead of changing the nature
-of \type {\mathsurround}, we can remain compatible.
+of \type {\mathsurround}, we can remain compatible. In the meantime a bit more
+control has been added via \type {\mathsurroundmode}. This directive can take 6 values
+with zero being the default behaviour.
+
+\start
+
+\def\OneLiner#1#2%
+ {\NC #1
+% \NC \ruledhbox{\mathsurroundmode#1\relax\hsize 100pt x$x$x}
+% \NC \ruledhbox{\mathsurroundmode#1\relax\hsize 100pt x $x$ x}
+ \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x$x$x}
+ \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x $x$ x}
+ \NC #2
+ \NC \NR}
+
+\startbuffer
+\mathsurround 10pt
+\mathsurroundskip20pt
+\stopbuffer
+
+\typebuffer \getbuffer
+
+\starttabulate[|Tc|c|c|pl|]
+\HL
+\BC mode \BC \type {x$x$x} \BC \type {x $x$ x} \BC effect \NC \NR
+\HL
+\OneLiner{0}{obey \type {\mathsurround} when \type {\mathsurroundskip} is 0pt}
+\OneLiner{1}{only add skip to the left}
+\OneLiner{2}{only add skip to the right}
+\OneLiner{3}{add skip to the left and right}
+\OneLiner{4}{ignore the skip setting, obey \type {\mathsurround}}
+\OneLiner{5}{disable all spacing around math}
+\OneLiner{6}{only apply \type {\mathsurroundskip} when also spacing}
+\OneLiner{7}{only apply \type {\mathsurroundskip} when no spacing}
+\HL
+\stoptabulate
+
+\stop
+
+Method six omits the surround glue when there is (x)spacing glue present while
+method seven does the opposite, the glue is only applied when there is (x)space
+glue present too. Anything more fance, like checking the begining or end of a
+paragraph (or edges of a box) would not be robust anyway. If you want that you
+can write a callback that runs over a list and analyzes a paragraph. Actually, in
+that case you could also inject glue (or set the properties of a math node)
+explicitly. So, these modes are in practice mostly useful for special purposes
+and experiments (they originate in a tracker item). Keep in mind that this glue
+is part of the math node and not always treated as normal glue: it travels with
+the begin and end math nodes. Also, method 6 and 7 will zero the skip related
+fields in a node when applicable in the first occasion that checks them
+(linebreaking or packaging).
% \section{Math todo}
%
@@ -980,6 +1225,21 @@ primitive \type {\mathoption} has been introduced (so that we don't end up with
many new commands). The approach of options also permits us to see what effect a
specific solution has.
+\subsubsection {\type {\mathoption old}}
+
+This option was introduced for testing purposes when the math engine got split
+code paths and it forces the engine to treat new fonts as old ones with respect
+to italic correction etc. There are no guarantees given with respect to the final
+result and unexpected side effects are not seens as bugs as they relate to font
+properties.
+
+\startbuffer
+\mathoption old 1
+\stopbuffer
+
+The \type {oldmath} boolean flag in the \LUA\ font table is the official way to
+force old treatment as it's bound to fonts.
+
\subsubsection {\type {\mathoption noitaliccompensation}}
This option compensates placement for characters with a built|-|in italic
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-modifications.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-modifications.tex
index 0507a56e0fa..2857603ea0f 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-modifications.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-modifications.tex
@@ -67,7 +67,19 @@ most still comes from the original. But we divert a bit.
\stopitem
\startitem
- Magnification (\type {\mag}) is only supported in \DVI\ output mode.
+ Magnification (\type {\mag}) is only supported in \DVI\ output mode. You can
+ set this parameter and it even works with \type {true} units till you switch
+ to \PDF\ output mode. When you use \PDF\ output you can best not touch the
+ \type {\mag} variable. This fuzzy behaviour is not much different from using
+ \PDF\ backend related functionality while eventually \DVI\ output is
+ required.
+
+ After the output mode has been frozen (normally that happens when the first
+ page is shipped out) or when \PDF\ output is enabled, the \type {true}
+ specification is ignored. When you preload a plain format adapted to
+ \LUATEX\ it can be that the \type {\mag} parameter already has been set.
+\stopitem
+
\stopitemize
\stopsubsection
@@ -514,7 +526,9 @@ different:
\stopsubsection
-\startsubsection[reference=backendprimitives,title=The backend primitives \type {\pdf*}]
+\stopsection
+
+\startsection[reference=backendprimitives,title=The backend primitives \type {\pdf*}]
In a previous section we mentioned that some \PDFTEX\ primitives were removed and
others promoted to core \LUATEX\ primitives. That is only part of the story. In
@@ -603,6 +617,7 @@ The configuration related registers have become:
\edef\pdfinclusionerrorlevel {\pdfvariable inclusionerrorlevel}
\edef\pdfignoreunknownimages {\pdfvariable ignoreunknownimages}
\edef\pdfgentounicode {\pdfvariable gentounicode}
+\edef\pdfomitcidset {\pdfvariable omitcidset}
\edef\pdfpagebox {\pdfvariable pagebox}
\edef\pdfminorversion {\pdfvariable minorversion}
\edef\pdfuniqueresname {\pdfvariable uniqueresname}
@@ -662,6 +677,7 @@ the defaults; these are:
\pdfinclusionerrorlevel 0
\pdfignoreunknownimages 0
\pdfgentounicode 0
+\pdfomitcidset 0
\pdfpagebox 0
\pdfminorversion 4
\pdfuniqueresname 0
@@ -765,7 +781,348 @@ normally generated by the engine:
So, you even need to include the brackets!
-\stopsubsection
+Although we started from a merge of \PDFTEX\ and \ALEPH, by now the code base as
+well as functionality has diverted from those parents. Here we show the options
+that can be passed to the extensions.
+
+\starttexsyntax
+\pdfextension literal
+ [ direct | page | raw ] { tokens }
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension dest
+ num integer | name { tokens }!crlf
+ [ fitbh | fitbv | fitb | fith| fitv | fit |
+ fitr <rule spec> | xyz [ zoom <integer> ]
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension annot
+ reserveobjnum | useobjnum <integer>
+ { tokens }
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension save
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension restore
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension setmatrix
+ { tokens }
+\stoptexsyntax
+
+\starttexsyntax
+[ \immediate ] \pdfextension obj
+ reserveobjnum
+\stoptexsyntax
+
+\starttexsyntax
+[ \immediate ] \pdfextension obj
+ [ useobjnum <integer> ]
+ [ uncompressed ]
+ [ stream [ attr { tokens } ] ]
+ [ file ]
+ { tokens }
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension refobj
+ <integer>
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension colorstack
+ <integer>
+ set { tokens } | push { tokens } | pop | current
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension startlink
+ [ attr { tokens } ]
+ user { tokens } | goto | thread
+ [ file { tokens } ]
+ [ page <integer> { tokens } | name { tokens } | num integer ]
+ [ newwindow | nonewwindow ]
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension endlink
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension startthread
+ num <integer> | name { tokens }
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension endthread
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension thread
+ num <integer> | name { tokens }
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension outline
+ [ attr { tokens } ]
+ [ useobjnum <integer> ]
+ [ count <integer> ]
+ { tokens }
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension glyphtounicode
+ { tokens }
+ { tokens }
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension catalog
+ { tokens }
+ [ openaction
+ user { tokens } | goto | thread
+ [ file { tokens } ]
+ [ page <integer> { tokens } | name { tokens } | num <integer> ]
+ [ newwindow | nonewwindow ] ]
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension fontattr
+ <integer>
+ {tokens}
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension mapfile
+ {tokens}
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension mapline
+ {tokens}
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension includechars
+ {tokens}
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension info
+ {tokens}
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension names
+ {tokens}
+\stoptexsyntax
+
+\starttexsyntax
+\pdfextension trailer
+ {tokens}
+\stoptexsyntax
+
+\stopsection
+
+\startsection[title=Directions]
+
+The directional model in \LUATEX\ is inherited from \OMEGA|/|\ALEPH\ but we tried
+to improve it a bit. At some point we played with recovery of modes but that was
+disabled later on when we found that it interfered with nested directions. That
+itself had as side effect that the node list was no longer balanced with respect
+to directional nodes which in turn can give side effects when a series of dir
+changes happens without grouping.
+
+The current (0.97 onward) approach is that we again make the list balanced but
+try to avoid some side effects. What happens is quite intuitive if we forget
+about spaces (turned into glue) but even there what happens makes sense if you
+look at it in detail. However that logic makes in|-|group switching kind of
+useless when no proper nested grouping is used: switching from right to left
+several times nested, results in spacing ending up after each other due to nested
+mirroring. Of course a sane macro package will manage this for the user but here
+we are discussing the low level dir injection.
+
+This is what happens:
+
+\starttyping
+\textdir TRT nur {\textdir TLT run \textdir TRT NUR} nur
+\stoptyping
+
+This becomes stepwise:
+
+\startnarrower
+\starttyping
+injected: [+TRT]nur {[+TLT]run [+TRT]NUR} nur
+balanced: [+TRT]nur {[+TLT]run [-TLT][+TRT]NUR[-TRT]} nur[-TRT]
+result : run {RUNrun } run
+\stoptyping
+\stopnarrower
+
+And this:
+
+\starttyping
+\textdir TRT nur {nur \textdir TLT run \textdir TRT NUR} nur
+\stoptyping
+
+becomes:
+
+\startnarrower
+\starttyping
+injected: [+TRT]nur {nur [+TLT]run [+TRT]NUR} nur
+balanced: [+TRT]nur {nur [+TLT]run [-TLT][+TRT]NUR[-TRT]} nur[-TRT]
+result : run {run RUNrun } run
+\stoptyping
+\stopnarrower
+
+Now, in the following examples watch where we put the braces:
+
+\startbuffer
+\textdir TRT nur {{\textdir TLT run} {\textdir TRT NUR}} nur
+\stopbuffer
+
+\typebuffer
+
+This becomes:
+
+\startnarrower
+\getbuffer
+\stopnarrower
+
+Compare this to:
+
+\startbuffer
+\textdir TRT nur {{\textdir TLT run }{\textdir TRT NUR}} nur
+\stopbuffer
+
+\typebuffer
+
+Which renders as:
+
+\startnarrower
+\getbuffer
+\stopnarrower
+
+So how do we deal with the next?
+
+\startbuffer
+\def\ltr{\textdir TLT\relax}
+\def\rtl{\textdir TRT\relax}
+
+run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}
+run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}
+\stopbuffer
+
+\typebuffer
+
+It gets typeset as:
+
+\startnarrower
+\startlines
+\getbuffer
+\stoplines
+\stopnarrower
+
+We could define the two helpers to look back, pick up a skip, remove it and
+inject it after the dir node. But that way we loose the subtype information that
+for some applications can be handy to be kept as|-|is. This is why we now have a
+variant of \type {\textdir} which injects the balanced node before the skip.
+Instead of the previous definition we can use:
+
+\startbuffer[def]
+\def\ltr{\linedir TLT\relax}
+\def\rtl{\linedir TRT\relax}
+\stopbuffer
+
+\typebuffer[def]
+
+and this time:
+
+\startbuffer[txt]
+run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}
+run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}
+\stopbuffer
+
+\typebuffer[txt]
+
+comes out as a properly spaced:
+
+\startnarrower
+\startlines
+\getbuffer[def,txt]
+\stoplines
+\stopnarrower
+
+Anything more complex that this, like combination of skips and penalties, or
+kerns, should be handled in the input or macro package because there is no way we
+can predict the expected behaviour. In fact, the \type {\linedir} is just a
+convenience extra which could also have been implemented using node list parsing.
+
+Another adaptation to the \ALEPH\ directional model is control over shapes driven
+by \type {\hangindent} and \type {\parshape}. This is controlled by a new parameter
+\type {\shapemode}:
+
+\starttabulate[|cTB|c|]
+\NC \NC \bf \type {\hangindent} \NC \bf \type {\parshape} \NC \NR
+\NC 0 \NC normal \NC normal \NC \NR
+\NC 1 \NC mirrored \NC normal \NC \NR
+\NC 2 \NC normal \NC mirrored \NC \NR
+\NC 3 \NC mirrored \NC mirrored \NC \NR
+\stoptabulate
+
+The value is reset to zero (like \type {\hangindent} and \type {\parshape})
+after the paragraph is done with. You can use negative values to preven t
+this.
+
+In \in {figure} [fig:shapemode] a few examples are given.
+
+\startplacefigure[reference=fig:shapemode,title={The effect of \type {shapemode}.}]
+ \startcombination[2*3]
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \pardir TLT \textdir TLT
+ \hangindent 40pt \hangafter -3
+ \leftskip10pt \input tufte \par
+ \egroup} {TLT: hangindent}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \pardir TLT \textdir TLT
+ \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
+ \input tufte \par
+ \egroup} {TLT: parshape}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \pardir TRT \textdir TRT
+ \hangindent 40pt \hangafter -3
+ \leftskip10pt \input tufte \par
+ \egroup} {TRT: hangindent mode 0}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \pardir TRT \textdir TRT
+ \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
+ \input tufte \par
+ \egroup} {TRT: parshape mode 0}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \shapemode=3
+ \pardir TRT \textdir TRT
+ \hangindent 40pt \hangafter -3
+ \leftskip10pt \input tufte \par
+ \egroup} {TRT: hangindent mode 1 & 3}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \shapemode=3
+ \pardir TRT \textdir TRT
+ \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
+ \input tufte \par
+ \egroup} {TRT: parshape mode 2 & 3}
+ \stopcombination
+\stopplacefigure
\stopsection
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-nodes.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-nodes.tex
index e90813c3397..d3c2c06dae1 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-nodes.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-nodes.tex
@@ -201,6 +201,26 @@ its internal link structure is correct, otherwise an error may be result.
The subtype numbers~4 and~5 belong to the \quote {of-f-ice} explanation given
elsewhere.
+These disc nodes are kind of special as at some point they also keep information
+about breakpoints and nested ligatures. The \type {pre}, \type {post} and \type
+{replace} fields at the \LUA\ end are in fact indirectly accessed and have a
+\type {prev} pointer that is not \type {nil}. This means that when you mess
+around with the head of these (three) lists, you also need to reassign them
+because that will restore the proper \type {prev} pointer, so:
+
+\starttyping
+pre = d.pre
+-- change the list starting with pre
+d.pre = pre
+\stoptyping
+
+Otherwise you can end up with an invalid internal perception of reality and
+\LUATEX\ might even decide to crash on you. It also means that running forward
+over for instance \type {pre} is ok but backward you need to stop at \type {pre}.
+And you definitely must not mess with the node that \type {prev} points to, if
+only because it is not really an node but part of the disc data structure (so
+freeing it again might crash \LUATEX).
+
\subsubsection{math nodes}
\starttabulate[|lT|l|p|]
@@ -281,11 +301,15 @@ a \type {userskip} with subtype zero).
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
-\NC subtype \NC number \NC not used \NC \NR
+\NC subtype \NC number \NC \showsubtypes{penalty} \NC \NR
\NC attr \NC node \NC list of attributes \NC \NR
\NC penalty \NC number \NC the penalty value \NC \NR
\stoptabulate
+The subtypes are just informative and \TEX\ itself doesn't use them. When you
+run into an \type {linebreakpenalty} you need to keep in mind that it's a
+accumulation of \type {club}, \type{widow} and other relevant penalties.
+
\subsubsection[glyphnodes]{glyph nodes}
\starttabulate[|lT|l|p|]
@@ -512,6 +536,7 @@ into a single node type with separate subtypes for differentiation.
\NC sup \NC kernel node \NC superscript \NC \NR
\NC accent \NC kernel node \NC top accent \NC \NR
\NC bot_accent \NC kernel node \NC bottom accent \NC \NR
+\NC fraction \NC number \NC larger step criterium (divided by 1000) \NC \NR
\stoptabulate
\subsubsubsection{style nodes}
@@ -536,9 +561,9 @@ a trailing \type {'} to signify \quote {cramped} styles.
\NC scriptscript \NC node \NC list of scriptscriptsize alternatives \NC \NR
\stoptabulate
-A warning: never assign a node list to the display, text, script, or
-scriptscript field unless you are sure its internal link structure is
-correct, otherwise an error may be result.
+Warning: never assign a node list to the \type {display}, \type {text}, \type
+{script}, or \type {scriptscript} field unless you are sure its internal link
+structure is correct, otherwise an error may be result.
\subsubsubsection{radical nodes}
@@ -551,11 +576,13 @@ correct, otherwise an error may be result.
\NC sup \NC kernel node \NC superscript \NC \NR
\NC left \NC delimiter node \NC \NC \NR
\NC degree \NC kernel node \NC only set by \type {\Uroot} \NC \NR
+\NC width \NC number \NC required width \NC \NR
+\NC options \NC number \NC bitset of rendering options \NC \NR
\stoptabulate
-A warning: never assign a node list to the nucleus, sub, sup, left, or degree
-field unless you are sure its internal link structure is correct, otherwise an
-error may be result.
+Warning: never assign a node list to the \type {nucleus}, \type {sub}, \type
+{sup}, \type {left}, or \type {degree} field unless you are sure its internal
+link structure is correct, otherwise an error may be result.
\subsubsubsection{fraction nodes}
@@ -566,11 +593,14 @@ error may be result.
\NC num \NC kernel node \NC numerator \NC \NR
\NC denom \NC kernel node \NC denominator \NC \NR
\NC left \NC delimiter node \NC left side symbol \NC \NR
-\NC right \NC delimiter node \NC right side symbol\NC \NR
+\NC right \NC delimiter node \NC right side symbol \NC \NR
+\NC middle \NC delimiter node \NC middle symbol \NC \NR
+\NC options \NC number \NC bitset of rendering options \NC \NR
\stoptabulate
-A warning: never assign a node list to the num, or denom field unless you are
-sure its internal link structure is correct, otherwise an error may be result.
+Warning: never assign a node list to the \type {num}, or \type {denom} field
+unless you are sure its internal link structure is correct, otherwise an error
+may be result.
\subsubsubsection{fence nodes}
@@ -579,8 +609,16 @@ sure its internal link structure is correct, otherwise an error may be result.
\NC subtype \NC number \NC \showsubtypes{fence} \NC \NR
\NC attr \NC node \NC list of attributes \NC \NR
\NC delim \NC delimiter node \NC delimiter specification \NC \NR
+\NC italic \NC number \NC italic correction \NC \NR
+\NC height \NC number \NC required height \NC \NR
+\NC depth \NC number \NC required depth \NC \NR
+\NC options \NC number \NC bitset of rendering options \NC \NR
+\NC class \NC number \NC spacing related class \NC \NR
\stoptabulate
+Warning: some of these fields are used by the renderer and might get adapted in
+the process.
+
\subsection{whatsit nodes}
Whatsit nodes come in many subtypes that you can ask for by running
@@ -595,7 +633,7 @@ Whatsit nodes come in many subtypes that you can ask for by running
\stopluacode
. % period
-\subsubsection{front|-|end whatits}
+\subsubsection{front|-|end whatsits}
\subsubsubsection{open whatsits}
@@ -625,7 +663,7 @@ Whatsit nodes come in many subtypes that you can ask for by running
\NC stream \NC number \NC \TEX's stream id number \NC \NR
\stoptabulate
-\subsubsubsection{user_defined whatits}
+\subsubsubsection{user_defined whatsits}
User|-|defined whatsit nodes can only be created and handled from \LUA\ code. In
effect, they are an extension to the extension mechanism. The \LUATEX\ engine
@@ -646,14 +684,14 @@ The \type {type} can have one of six distinct values. The number is the \ASCII\
value if the first character if the type name (so you can use string.byte("l")
instead of \type {108}).
-\starttabulate[|lT|p|]
-\NC \rmbf value \NC \bf meaning \NC \bf explanation \NC \NR
-\NC 97 \NC a \NC list of attributes (a node list) \NC \NR
-\NC 100 \NC d \NC a \LUA\ number \NC \NR
-\NC 108 \NC l \NC a \LUA\ value (table, number, boolean, etc) \NC \NR
-\NC 110 \NC n \NC a node list \NC \NR
-\NC 115 \NC s \NC a \LUA\ string \NC \NR
-\NC 116 \NC t \NC a \LUA\ token list in \LUA\ table form (a list of triplets) \NC \NR
+\starttabulate[|lT|lT|p|]
+\NC \rmbf value \NC \bf meaning \NC \bf explanation \NC \NR
+\NC 97 \NC a \NC list of attributes (a node list) \NC \NR
+\NC 100 \NC d \NC a \LUA\ number \NC \NR
+\NC 108 \NC l \NC a \LUA\ value (table, number, boolean, etc) \NC \NR
+\NC 110 \NC n \NC a node list \NC \NR
+\NC 115 \NC s \NC a \LUA\ string \NC \NR
+\NC 116 \NC t \NC a \LUA\ token list in \LUA\ table form (a list of triplets) \NC \NR
\stoptabulate
\subsubsubsection{save_pos whatsits}
@@ -677,9 +715,9 @@ The difference between \type {data} and \type {string} is that on assignment, th
\type {data} field is converted to a token list, cf. use as \type {\latelua}. The
\type {string} version is treated as a literal string.
-\subsubsection{\DVI\ backend whatits}
+\subsubsection{\DVI\ backend whatsits}
-\subsubsection{special whatits}
+\subsubsection{special whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -687,9 +725,9 @@ The difference between \type {data} and \type {string} is that on assignment, th
\NC data \NC string \NC the \type {\special} information \NC \NR
\stoptabulate
-\subsubsection{\PDF\ backend whatits}
+\subsubsection{\PDF\ backend whatsits}
-\subsubsubsection{pdf_literal whatits}
+\subsubsubsection{pdf_literal whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -705,9 +743,14 @@ Possible mode values are:
\NC 0 \NC setorigin \NC \NR
\NC 1 \NC page \NC \NR
\NC 2 \NC direct \NC \NR
+\NC 3 \NC raw \NC \NR
\stoptabulate
-\subsubsubsection{pdf_refobj whatits}
+The higher the number, the less checking and the more you can run into troubles.
+Especially the \type {raw} variant can produce bad \PDF\ so you can best check
+what you generate.
+
+\subsubsubsection{pdf_refobj whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -715,7 +758,7 @@ Possible mode values are:
\NC objnum \NC number \NC the referenced \PDF\ object number \NC \NR
\stoptabulate
-\subsubsubsection{pdf_annot whatits}
+\subsubsubsection{pdf_annot whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -727,7 +770,7 @@ Possible mode values are:
\NC data \NC string \NC the annotation data \NC \NR
\stoptabulate
-\subsubsubsection{pdf_start_link whatits}
+\subsubsubsection{pdf_start_link whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -740,14 +783,14 @@ Possible mode values are:
\NC action \NC node \NC the action to perform \NC \NR
\stoptabulate
-\subsubsubsection{pdf_end_link whatits}
+\subsubsubsection{pdf_end_link whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
\NC attr \NC node \NC \NC \NR
\stoptabulate
-\subsubsubsection{pdf_dest whatits}
+\subsubsubsection{pdf_dest whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -763,7 +806,7 @@ Possible mode values are:
\NC objnum \NC number \NC the \PDF\ object number \NC \NR
\stoptabulate
-\subsubsubsection{pdf_action whatits}
+\subsubsubsection{pdf_action whatsits}
These are a special kind of item that only appears inside \PDF\ start link
objects.
@@ -795,7 +838,7 @@ Valid window types are:
\NC 2 \NC nonew \NC \NR
\stoptabulate
-\subsubsubsection{pdf_thread whatits}
+\subsubsubsection{pdf_thread whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -809,7 +852,7 @@ Valid window types are:
\NC thread_attr \NC number \NC extra thread information \NC \NR
\stoptabulate
-\subsubsubsection{pdf_start_thread whatits}
+\subsubsubsection{pdf_start_thread whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -823,14 +866,14 @@ Valid window types are:
\NC thread_attr \NC number \NC extra thread information \NC \NR
\stoptabulate
-\subsubsubsection{pdf_end_thread whatits}
+\subsubsubsection{pdf_end_thread whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
\NC attr \NC node \NC \NC \NR
\stoptabulate
-\subsubsubsection{pdf_colorstack whatits}
+\subsubsubsection{pdf_colorstack whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -840,7 +883,7 @@ Valid window types are:
\NC data \NC string \NC data \NC \NR
\stoptabulate
-\subsubsubsection{pdf_setmatrix whatits}
+\subsubsubsection{pdf_setmatrix whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
@@ -848,226 +891,20 @@ Valid window types are:
\NC data \NC string \NC data \NC \NR
\stoptabulate
-\subsubsubsection{pdf_save whatits}
+\subsubsubsection{pdf_save whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
\NC attr \NC node \NC list of attributes \NC \NR
\stoptabulate
-\subsubsubsection{pdf_restore whatits}
+\subsubsubsection{pdf_restore whatsits}
\starttabulate[|lT|l|p|]
\NC \rmbf field \NC \bf type \NC \bf explanation \NC \NR
\NC attr \NC node \NC list of attributes \NC \NR
\stoptabulate
-\section{Two access models}
-
-Deep down in \TEX\ a node has a number which is an numeric entry in a memory
-table. In fact, this model, where \TEX\ manages memory is real fast and one of
-the reasons why plugging in callbacks that operate on nodes is quite fast too.
-Each node gets a number that is in fact an index in the memory table and that
-number often gets reported when you print node related information.
-
-There are two access models, a robust one using a so called user data object that
-provides a virtual interface to the internal nodes, and a more direct access which
-uses the node numbers directly. The first model provide key based access while
-the second always accesses fields via functions:
-
-\starttyping
-nodeobject.char
-getfield(nodenumber,"char")
-\stoptyping
-
-If you use the direct model, even if you know that you deal with numbers, you
-should not depend on that property but treat it an abstraction just like
-traditional nodes. In fact, the fact that we use a simple basic datatype has the
-penalty that less checking can be done, but less checking is also the reason why
-it's somewhat faster. An important aspect is that one cannot mix both methods,
-but you can cast both models. So, multiplying a node number makes no sense.
-
-So our advice is: use the indexed (table) approach when possible and investigate
-the direct one when speed might be an real issue. For that reason we also provide
-the \type {get*} and \type {set*} functions in the top level node namespace.
-There is a limited set of getters. When implementing this direct approach the
-regular index by key variant was also optimized, so direct access only makes
-sense when we're accessing nodes millions of times (which happens in some font
-processing for instance).
-
-We're talking mostly of getters because setters are less important. Documents
-have not that many content related nodes and setting many thousands of properties
-is hardly a burden contrary to millions of consultations.
-
-Normally you will access nodes like this:
-
-\starttyping
-local next = current.next
-if next then
- -- do something
-end
-\stoptyping
-
-Here \type {next} is not a real field, but a virtual one. Accessing it results in
-a metatable method being called. In practice it boils down to looking up the node
-type and based on the node type checking for the field name. In a worst case you
-have a node type that sits at the end of the lookup list and a field that is last
-in the lookup chain. However, in successive versions of \LUATEX\ these lookups
-have been optimized and the most frequently accessed nodes and fields have a
-higher priority.
-
-Because in practice the \type {next} accessor results in a function call, there
-is some overhead involved. The next code does the same and performs a tiny bit
-faster (but not that much because it is still a function call but one that knows
-what to look up).
-
-\starttyping
-local next = node.next(current)
-if next then
- -- do something
-end
-\stoptyping
-
-If performance matters you can use an function instead:
-
-\starttabulate[|T|p|]
-\NC getnext \NC parsing nodelist always involves this one \NC \NR
-\NC getprev \NC used less but is logical companion to \type {getnext} \NC \NR
-\NC getboth \NC returns the next and prev pointer of a node \NC \NR
-\NC getid \NC consulted a lot \NC \NR
-\NC getsubtype \NC consulted less but also a topper \NC \NR
-\NC getfont \NC used a lot in \OPENTYPE\ handling (glyph nodes are consulted a lot) \NC \NR
-\NC getchar \NC idem and also in other places \NC \NR
-\NC getdisc \NC returns the \type {pre}, \type {post} and \type {replace} fields and
- optionally when true is passed also the tail fields. \NC \NR
-\NC getlist \NC we often parse nested lists so this is a convenient one too
- (only works for hlist and vlist!) \NC \NR
-\NC getleader \NC comparable to list, seldom used in \TEX\ (but needs frequent consulting
- like lists; leaders could have been made a dedicated node type) \NC \NR
-\NC getfield \NC generic getter, sufficient for the rest (other field names are
- often shared so a specific getter makes no sense then) \NC \NR
-\stoptabulate
-
-The direct variants also have setters, where the discretionary setter takes three
-(optional) arguments plus an optional fourth indicating the subtype.
-
-It doesn't make sense to add getters for all fields, also because some are not
-unique to one node type. Profiling demonstrated that these fields can get
-accesses way more times than other fields. Even in complex documents, many node
-and fields types never get seen, or seen only a few times. Most functions in the
-\type {node} namespace have a companion in \type {node.direct}, but of course not
-the ones that don't deal with nodes themselves. The following table summarized
-this:
-
-% \startcolumns[balance=yes]
-
-\def\yes{$+$} \def\nop{$-$}
-
-\starttabulate[|T|c|c|]
-\HL
-\NC \bf function \NC \bf node \NC \bf direct \NC \NR
-\HL
-\NC \type {copy_list} \NC \yes \NC \yes \NC \NR
-\NC \type {copy} \NC \yes \NC \yes \NC \NR
-\NC \type {count} \NC \yes \NC \yes \NC \NR
-\NC \type {current_attr} \NC \yes \NC \yes \NC \NR
-\NC \type {dimensions} \NC \yes \NC \yes \NC \NR
-\NC \type {do_ligature_n} \NC \yes \NC \yes \NC \NR
-\NC \type {effective_glue} \NC \yes \NC \yes \NC \NR
-\NC \type {end_of_math} \NC \yes \NC \yes \NC \NR
-\NC \type {family_font} \NC \yes \NC \nop \NC \NR
-\NC \type {fields} \NC \yes \NC \nop \NC \NR
-\NC \type {first_character} \NC \yes \NC \nop \NC \NR
-\NC \type {first_glyph} \NC \yes \NC \yes \NC \NR
-\NC \type {flush_list} \NC \yes \NC \yes \NC \NR
-\NC \type {flush_node} \NC \yes \NC \yes \NC \NR
-\NC \type {free} \NC \yes \NC \yes \NC \NR
-\NC \type {getboth} \NC \yes \NC \yes \NC \NR
-\NC \type {getbox} \NC \nop \NC \yes \NC \NR
-\NC \type {getchar} \NC \yes \NC \yes \NC \NR
-\NC \type {getdisc} \NC \yes \NC \yes \NC \NR
-\NC \type {getfield} \NC \yes \NC \yes \NC \NR
-\NC \type {getfont} \NC \yes \NC \yes \NC \NR
-\NC \type {getid} \NC \yes \NC \yes \NC \NR
-\NC \type {getleader} \NC \yes \NC \yes \NC \NR
-\NC \type {getlist} \NC \yes \NC \yes \NC \NR
-\NC \type {getnext} \NC \yes \NC \yes \NC \NR
-\NC \type {getprev} \NC \yes \NC \yes \NC \NR
-\NC \type {getsubtype} \NC \yes \NC \yes \NC \NR
-\NC \type {has_attribute} \NC \yes \NC \yes \NC \NR
-\NC \type {get_attribute} \NC \yes \NC \yes \NC \NR
-\NC \type {find_attribute} \NC \yes \NC \yes \NC \NR
-\NC \type {has_field} \NC \yes \NC \yes \NC \NR
-\NC \type {has_glyph} \NC \yes \NC \yes \NC \NR
-\NC \type {hpack} \NC \yes \NC \yes \NC \NR
-\NC \type {id} \NC \yes \NC \nop \NC \NR
-\NC \type {insert_after} \NC \yes \NC \yes \NC \NR
-\NC \type {insert_before} \NC \yes \NC \yes \NC \NR
-\NC \type {is_char} \NC \yes \NC \yes \NC \NR
-\NC \type {is_glyph} \NC \yes \NC \yes \NC \NR
-\NC \type {is_direct} \NC \nop \NC \yes \NC \NR
-\NC \type {is_node} \NC \yes \NC \yes \NC \NR
-\NC \type {kerning} \NC \yes \NC \yes \NC \NR
-\NC \type {last_node} \NC \yes \NC \yes \NC \NR
-\NC \type {length} \NC \yes \NC \yes \NC \NR
-\NC \type {ligaturing} \NC \yes \NC \yes \NC \NR
-\NC \type {mlist_to_hlist} \NC \yes \NC \nop \NC \NR
-\NC \type {new} \NC \yes \NC \yes \NC \NR
-\NC \type {next} \NC \yes \NC \nop \NC \NR
-\NC \type {prev} \NC \yes \NC \nop \NC \NR
-\NC \type {protect_glyph} \NC \yes \NC \yes \NC \NR
-\NC \type {protect_glyphs} \NC \yes \NC \yes \NC \NR
-\NC \type {protrusion_skippable} \NC \yes \NC \yes \NC \NR
-\NC \type {remove} \NC \yes \NC \yes \NC \NR
-\NC \type {set_attribute} \NC \yes \NC \yes \NC \NR
-\NC \type {setboth} \NC \yes \NC \yes \NC \NR
-\NC \type {setbox} \NC \yes \NC \yes \NC \NR
-\NC \type {setchar} \NC \yes \NC \yes \NC \NR
-\NC \type {setdisc} \NC \yes \NC \yes \NC \NR
-\NC \type {setfield} \NC \yes \NC \yes \NC \NR
-\NC \type {setlink} \NC \yes \NC \yes \NC \NR
-\NC \type {setnext} \NC \yes \NC \yes \NC \NR
-\NC \type {setprev} \NC \yes \NC \yes \NC \NR
-\NC \type {slide} \NC \yes \NC \yes \NC \NR
-\NC \type {subtype} \NC \yes \NC \nop \NC \NR
-\NC \type {subtypes} \NC \yes \NC \nop \NC \NR
-\NC \type {tail} \NC \yes \NC \yes \NC \NR
-\NC \type {todirect} \NC \yes \NC \yes \NC \NR
-\NC \type {tonode} \NC \yes \NC \yes \NC \NR
-\NC \type {tostring} \NC \yes \NC \yes \NC \NR
-\NC \type {traverse_id} \NC \yes \NC \yes \NC \NR
-\NC \type {traverse_char} \NC \yes \NC \yes \NC \NR
-\NC \type {traverse} \NC \yes \NC \yes \NC \NR
-\NC \type {types} \NC \yes \NC \nop \NC \NR
-\NC \type {type} \NC \yes \NC \nop \NC \NR
-\NC \type {unprotect_glyphs} \NC \yes \NC \yes \NC \NR
-\NC \type {unset_attribute} \NC \yes \NC \yes \NC \NR
-\NC \type {usedlist} \NC \yes \NC \yes \NC \NR
-\NC \type {vpack} \NC \yes \NC \yes \NC \NR
-\NC \type {whatsits} \NC \yes \NC \nop \NC \NR
-\NC \type {whatsitsubtypes} \NC \yes \NC \nop \NC \NR
-\NC \type {write} \NC \yes \NC \yes \NC \NR
-\NC \type {setglue} \NC \yes \NC \yes \NC \NR
-\NC \type {getglue} \NC \yes \NC \yes \NC \NR
-\NC \type {glue_is_zero} \NC \yes \NC \yes \NC \NR
-\stoptabulate
-
-% \stopcolumns
-
-The \type {node.next} and \type {node.prev} functions will stay but for
-consistency there are variants called \type {getnext} and \type {getprev}. We had
-to use \type {get} because \type {node.id} and \type {node.subtype} are already
-taken for providing meta information about nodes. Note: The getters do only basic
-checking for valid keys. You should just stick to the keys mentioned in the
-sections that describe node properties.
-
-Some nodes have indirect references. For instance a math character refers to a
-family instead of a font. In that case we provide a virtual font field as
-accessor. So, \type {getfont} and \type {.font} can be used on them. The same is
-true for the \type {width}, \type {height} and \type {depth} of glue nodes. These
-actually access the spec node properties, and here we can set as well as get the
-values.
-
\section{The \type {node} library}
The \type {node} library contains functions that facilitate dealing with (lists
@@ -1234,16 +1071,21 @@ the \TEX\ level.
This function accepts string \type {id} and \type {subtype} values as well.
-\subsubsection{\type {node.free}}
+\subsubsection{\type {node.free} and \type {node.flush_node}}
\startfunctioncall
-node.free(<node> n)
+<node> next =
+ node.free(<node> n)
+flush_node(<node> n)
\stopfunctioncall
Removes the node \type {n} from \TEX's memory. Be careful: no checks are done on
whether this node is still pointed to from a register or some \type {next} field:
it is up to you to make sure that the internal data structures remain correct.
+The \type {free} function returns the next field of the freed node, while the
+\type {flush_node} alternative returns nothing.
+
\subsubsection{\type {node.flush_list}}
\startfunctioncall
@@ -1385,7 +1227,7 @@ The second return value is the badness of the generated box.
See the description of \type {node.hpack()} for a few memory allocation caveats.
-\subsubsection{\type {node.dimensions}}
+\subsubsection{\type {node.dimensions}, \type {node.rangedimensions}}
\startfunctioncall
<number> w, <number> h, <number> d =
@@ -1438,7 +1280,17 @@ example in code like this, which prints the width of the space in between the
You need to keep in mind that this is one of the few places in \TEX\ where floats
are used, which means that you can get small differences in rounding when you
-compare the width repported by \type {hpack} with \type {dimensions}.
+compare the width reported by \type {hpack} with \type {dimensions}.
+
+The second alternative saves a few lookups and can be more convenient in some
+cases:
+
+\startfunctioncall
+<number> w, <number> h, <number> d =
+ node.rangedimensions(<node> parent, <node> first)
+<number> w, <number> h, <number> d =
+ node.rangedimensions(<node> parent, <node> first, <node> last)
+\stopfunctioncall
\subsubsection{\type {node.mlist_to_hlist}}
@@ -1574,6 +1426,25 @@ See the previous section for details. The change is in the local function \type
end
\stoptyping
+\subsubsection{\type {node.traverse_char}}
+
+This iterators loops over the glyph nodes in a list. Only nodes with a subtype
+less than 256 are seen.
+
+\startfunctioncall
+<node> n =
+ node.traverse_char(<node> n)
+\stopfunctioncall
+
+\subsubsection{\type {node.has_glyph}}
+
+This function returns the first glyph or disc node in the given list:
+
+\startfunctioncall
+<node> n =
+ node.has_glyph(<node> n)
+\stopfunctioncall
+
\subsubsection{\type {node.end_of_math}}
\startfunctioncall
@@ -1680,7 +1551,7 @@ Subtracts 256 from all glyph node subtypes. This and the next function are
helpers to convert from \type {characters} to \type {glyphs} during node
processing.
-\subsubsection{\type {node.protect_glyphs}}
+\subsubsection{\type {node.protect_glyphs} and \type {node.protect_glyph}}
\startfunctioncall
node.protect_glyphs(<node> n)
@@ -1688,7 +1559,8 @@ node.protect_glyphs(<node> n)
Adds 256 to all glyph node subtypes in the node list starting at \type {n},
except that if the value is 1, it adds only 255. The special handling of 1 means
-that \type {characters} will become \type {glyphs} after subtraction of 256.
+that \type {characters} will become \type {glyphs} after subtraction of 256. A
+single character can be marked by the singular call.
\subsubsection{\type {node.last_node}}
@@ -1750,6 +1622,9 @@ The next call will return 5 values (or northing when no glue is passed).
<integer> shrink_order = node.getglue(<node> n)
\stopfunctioncall
+When the second argument is false, only the width is returned (this is consistent
+with \type {tex.get}).
+
\subsubsection{\type {node.is_zero_glue}}
This function returns \type {true} when the width, stretch and shrink properties
@@ -1824,6 +1699,306 @@ attributes or attribute|-|value pairs are ignored.
If the attribute was actually deleted, returns its old value. Otherwise, returns
\type {nil}.
+\subsubsection{\type {node.slide}}
+
+This helper makes sure that the node lists is double linked and returns the found
+tail node.
+
+\startfunctioncall
+<node> tail =
+ node.slide(<node> n)
+\stopfunctioncall
+
+After some callbacks automatic sliding takes place. This feature can be turned
+off with \type {node.fix_node_lists(false)} but you better make sure then that
+you don't mess up lists. In most cases \TEX\ itself only uses \type {next}
+pointers but your other callbacks might expect proper \type {prev} pointers too.
+Future versions of \LUATEX\ can add more checking but this will not influence
+usage.
+
+\subsubsection{\type {node.check_discretionary} and \type {node.check_discretionaries}}
+
+When you fool around with disc nodes you need to be aware of the fact that they
+have a special internal data structure. As long as you reassign the fields when
+you have extended the lists it's ok because then the tail pointers get updated,
+but when you add to list without reassigning you might end up in troubles when
+the linebreak routien kicks in. You can call this function to check the list for
+issues with disc nodes.
+
+\startfunctioncall
+node.check_discretionary(<node> n)
+node.check_discretionaries(<node> head)
+\stopfunctioncall
+
+The plural variant runs over all disc nodes in a list, the singular variant
+checks one node only (it also checks if the node is a disc node).
+
+\subsubsection{\type {node.family_font}}
+
+When you pass it a proper family identifier the next helper will return the font
+currently associated with it. You can normally also access the font with the normal
+font field or getter because it will resolve the family automatically for noads.
+
+\startfunctioncall
+<integer> id =
+ node.family_font(<integer> fam)
+\stopfunctioncall
+
+\section{Two access models}
+
+Deep down in \TEX\ a node has a number which is an numeric entry in a memory
+table. In fact, this model, where \TEX\ manages memory is real fast and one of
+the reasons why plugging in callbacks that operate on nodes is quite fast too.
+Each node gets a number that is in fact an index in the memory table and that
+number often gets reported when you print node related information.
+
+There are two access models, a robust one using a so called user data object that
+provides a virtual interface to the internal nodes, and a more direct access which
+uses the node numbers directly. The first model provide key based access while
+the second always accesses fields via functions:
+
+\starttyping
+nodeobject.char
+getfield(nodenumber,"char")
+\stoptyping
+
+If you use the direct model, even if you know that you deal with numbers, you
+should not depend on that property but treat it an abstraction just like
+traditional nodes. In fact, the fact that we use a simple basic datatype has the
+penalty that less checking can be done, but less checking is also the reason why
+it's somewhat faster. An important aspect is that one cannot mix both methods,
+but you can cast both models. So, multiplying a node number makes no sense.
+
+So our advice is: use the indexed (table) approach when possible and investigate
+the direct one when speed might be an real issue. For that reason we also provide
+the \type {get*} and \type {set*} functions in the top level node namespace.
+There is a limited set of getters. When implementing this direct approach the
+regular index by key variant was also optimized, so direct access only makes
+sense when we're accessing nodes millions of times (which happens in some font
+processing for instance).
+
+We're talking mostly of getters because setters are less important. Documents
+have not that many content related nodes and setting many thousands of properties
+is hardly a burden contrary to millions of consultations.
+
+Normally you will access nodes like this:
+
+\starttyping
+local next = current.next
+if next then
+ -- do something
+end
+\stoptyping
+
+Here \type {next} is not a real field, but a virtual one. Accessing it results in
+a metatable method being called. In practice it boils down to looking up the node
+type and based on the node type checking for the field name. In a worst case you
+have a node type that sits at the end of the lookup list and a field that is last
+in the lookup chain. However, in successive versions of \LUATEX\ these lookups
+have been optimized and the most frequently accessed nodes and fields have a
+higher priority.
+
+Because in practice the \type {next} accessor results in a function call, there
+is some overhead involved. The next code does the same and performs a tiny bit
+faster (but not that much because it is still a function call but one that knows
+what to look up).
+
+\starttyping
+local next = node.next(current)
+if next then
+ -- do something
+end
+\stoptyping
+
+Some accessors are used frequently and for these we provide more efficient helpers:
+
+\starttabulate[|T|p|]
+\NC getnext \NC parsing nodelist always involves this one \NC \NR
+\NC getprev \NC used less but is logical companion to \type {getnext} \NC \NR
+\NC getboth \NC returns the next and prev pointer of a node \NC \NR
+\NC getid \NC consulted a lot \NC \NR
+\NC getsubtype \NC consulted less but also a topper \NC \NR
+\NC getfont \NC used a lot in \OPENTYPE\ handling (glyph nodes are consulted a lot) \NC \NR
+\NC getchar \NC idem and also in other places \NC \NR
+\NC getwhd \NC returns the \type {width}, \type {height} and \type {depth} of a list, rule or
+ (unexpanded) glyph as well as glue (its spec is looked at) and unset nodes\NC \NR
+\NC getdisc \NC returns the \type {pre}, \type {post} and \type {replace} fields and
+ optionally when true is passed also the tail fields. \NC \NR
+\NC getlist \NC we often parse nested lists so this is a convenient one too \NC \NR
+\NC getleader \NC comparable to list, seldom used in \TEX\ (but needs frequent consulting
+ like lists; leaders could have been made a dedicated node type) \NC \NR
+\NC getfield \NC generic getter, sufficient for the rest (other field names are
+ often shared so a specific getter makes no sense then) \NC \NR
+\NC getbox \NC gets the given box (a list node) \NC \NR
+\stoptabulate
+
+In the direct namespace there are more such helpers and most of them are
+accompanied by setters. The getters and setters are clever enough to see what
+node is meant. We don't deal with whatsit nodes: their fields are always accessed
+by name. It doesn't make sense to add getters for all fields, we just identifier
+the most likely candidates. In complex documents, many node and fields types
+never get seen, or seen only a few times, but for instance glyphs are candidates
+for such optimization. The \type {node.direct} interface has some more helpers.
+\footnote {We can define the helpers in the node namespace with \type {getfield}
+which is about as efficient, so at some point we might provide that as module.}
+
+The \type {setdisc} helper takes three (optional) arguments plus an optional
+fourth indicating the subtype. Its \type {getdisc} takes an optional boolean;
+when its value is \type {true} the tail nodes will also be returned. The \type
+{setfont} helper takes an optional second argument, it being the character. The
+directmode setter \type {setlink} takes a list of nodes and will link them,
+thereby ignoring \type {nil} entries. The first valid node is returned (beware:
+for good reason it assumes single nodes). For rarely used fields no helpers are
+provided and there are a few that probably are used seldom too but were added for
+consistency. You can of course always define additional accessor using \type
+{getfield} and \type {setfield} with little overhead.
+
+% \startcolumns[balance=yes]
+
+\def\yes{$+$} \def\nop{$-$}
+
+\starttabulate[|T|c|c|]
+\HL
+\NC \bf function \NC \bf node \NC \bf direct \NC \NR
+\HL
+%NC \type {do_ligature_n} \NC \yes \NC \yes \NC \NR % was never documented and experimental
+\NC \type {check_discretionaries}\NC \yes \NC \yes \NC \NR
+\NC \type {copy_list} \NC \yes \NC \yes \NC \NR
+\NC \type {copy} \NC \yes \NC \yes \NC \NR
+\NC \type {count} \NC \yes \NC \yes \NC \NR
+\NC \type {current_attr} \NC \yes \NC \yes \NC \NR
+\NC \type {dimensions} \NC \yes \NC \yes \NC \NR
+\NC \type {effective_glue} \NC \yes \NC \yes \NC \NR
+\NC \type {end_of_math} \NC \yes \NC \yes \NC \NR
+\NC \type {family_font} \NC \yes \NC \nop \NC \NR
+\NC \type {fields} \NC \yes \NC \nop \NC \NR
+\NC \type {find_attribute} \NC \yes \NC \yes \NC \NR
+\NC \type {first_glyph} \NC \yes \NC \yes \NC \NR
+\NC \type {flush_list} \NC \yes \NC \yes \NC \NR
+\NC \type {flush_node} \NC \yes \NC \yes \NC \NR
+\NC \type {free} \NC \yes \NC \yes \NC \NR
+\NC \type {get_attribute} \NC \yes \NC \yes \NC \NR
+\NC \type {getattributelist} \NC \nop \NC \yes \NC \NR
+\NC \type {getboth} \NC \yes \NC \yes \NC \NR
+\NC \type {getbox} \NC \nop \NC \yes \NC \NR
+\NC \type {getchar} \NC \yes \NC \yes \NC \NR
+\NC \type {getcomponents} \NC \nop \NC \yes \NC \NR
+\NC \type {getdepth} \NC \nop \NC \yes \NC \NR
+\NC \type {getdir} \NC \nop \NC \yes \NC \NR
+\NC \type {getdisc} \NC \yes \NC \yes \NC \NR
+\NC \type {getfield} \NC \yes \NC \yes \NC \NR
+\NC \type {getfont} \NC \yes \NC \yes \NC \NR
+\NC \type {getglue} \NC \yes \NC \yes \NC \NR
+\NC \type {getheight} \NC \nop \NC \yes \NC \NR
+\NC \type {getid} \NC \yes \NC \yes \NC \NR
+\NC \type {getkern} \NC \nop \NC \yes \NC \NR
+\NC \type {getlang} \NC \nop \NC \yes \NC \NR
+\NC \type {getleader} \NC \yes \NC \yes \NC \NR
+\NC \type {getlist} \NC \yes \NC \yes \NC \NR
+\NC \type {getnext} \NC \yes \NC \yes \NC \NR
+\NC \type {getnucleus} \NC \nop \NC \yes \NC \NR
+\NC \type {getoffsets} \NC \nop \NC \yes \NC \NR
+\NC \type {getpenalty} \NC \nop \NC \yes \NC \NR
+\NC \type {getprev} \NC \yes \NC \yes \NC \NR
+\NC \type {getproperty} \NC \yes \NC \yes \NC \NR
+\NC \type {getshift} \NC \nop \NC \yes \NC \NR
+\NC \type {getwidth} \NC \nop \NC \yes \NC \NR
+\NC \type {getwhd} \NC \nop \NC \yes \NC \NR
+\NC \type {getsub} \NC \nop \NC \yes \NC \NR
+\NC \type {getsubtype} \NC \yes \NC \yes \NC \NR
+\NC \type {getsup} \NC \nop \NC \yes \NC \NR
+\NC \type {has_attribute} \NC \yes \NC \yes \NC \NR
+\NC \type {has_field} \NC \yes \NC \yes \NC \NR
+\NC \type {has_glyph} \NC \yes \NC \yes \NC \NR
+\NC \type {hpack} \NC \yes \NC \yes \NC \NR
+\NC \type {id} \NC \yes \NC \nop \NC \NR
+\NC \type {insert_after} \NC \yes \NC \yes \NC \NR
+\NC \type {insert_before} \NC \yes \NC \yes \NC \NR
+\NC \type {is_char} \NC \yes \NC \yes \NC \NR
+\NC \type {is_direct} \NC \nop \NC \yes \NC \NR
+\NC \type {is_glue_zero} \NC \yes \NC \yes \NC \NR
+\NC \type {is_glyph} \NC \yes \NC \yes \NC \NR
+\NC \type {is_node} \NC \yes \NC \yes \NC \NR
+\NC \type {kerning} \NC \yes \NC \yes \NC \NR
+\NC \type {last_node} \NC \yes \NC \yes \NC \NR
+\NC \type {length} \NC \yes \NC \yes \NC \NR
+\NC \type {ligaturing} \NC \yes \NC \yes \NC \NR
+\NC \type {mlist_to_hlist} \NC \yes \NC \nop \NC \NR
+\NC \type {new} \NC \yes \NC \yes \NC \NR
+\NC \type {next} \NC \yes \NC \nop \NC \NR
+\NC \type {prev} \NC \yes \NC \nop \NC \NR
+\NC \type {protect_glyphs} \NC \yes \NC \yes \NC \NR
+\NC \type {protect_glyph} \NC \yes \NC \yes \NC \NR
+\NC \type {protrusion_skippable} \NC \yes \NC \yes \NC \NR
+\NC \type {rangedimensions} \NC \yes \NC \yes \NC \NR
+\NC \type {remove} \NC \yes \NC \yes \NC \NR
+\NC \type {set_attribute} \NC \nop \NC \yes \NC \NR
+\NC \type {setattributelist} \NC \nop \NC \yes \NC \NR
+\NC \type {setboth} \NC \nop \NC \yes \NC \NR
+\NC \type {setbox} \NC \nop \NC \yes \NC \NR
+\NC \type {setchar} \NC \nop \NC \yes \NC \NR
+\NC \type {setcomponents} \NC \nop \NC \yes \NC \NR
+\NC \type {setdepth} \NC \nop \NC \yes \NC \NR
+\NC \type {setdir} \NC \nop \NC \yes \NC \NR
+\NC \type {setdisc} \NC \nop \NC \yes \NC \NR
+\NC \type {setfield} \NC \yes \NC \yes \NC \NR
+\NC \type {setfont} \NC \nop \NC \yes \NC \NR
+\NC \type {setglue} \NC \yes \NC \yes \NC \NR
+\NC \type {setheight} \NC \nop \NC \yes \NC \NR
+\NC \type {setid} \NC \nop \NC \yes \NC \NR
+\NC \type {setkern} \NC \nop \NC \yes \NC \NR
+\NC \type {setlang} \NC \nop \NC \yes \NC \NR
+\NC \type {setleader} \NC \nop \NC \yes \NC \NR
+\NC \type {setlist} \NC \nop \NC \yes \NC \NR
+\NC \type {setnext} \NC \nop \NC \yes \NC \NR
+\NC \type {setnucleus} \NC \nop \NC \yes \NC \NR
+\NC \type {setoffsets} \NC \nop \NC \yes \NC \NR
+\NC \type {setpenalty} \NC \nop \NC \yes \NC \NR
+\NC \type {setprev} \NC \nop \NC \yes \NC \NR
+\NC \type {setproperty} \NC \nop \NC \yes \NC \NR
+\NC \type {setshift} \NC \nop \NC \yes \NC \NR
+\NC \type {setwidth} \NC \nop \NC \yes \NC \NR
+\NC \type {setwhd} \NC \nop \NC \yes \NC \NR
+\NC \type {setsub} \NC \nop \NC \yes \NC \NR
+\NC \type {setsubtype} \NC \nop \NC \yes \NC \NR
+\NC \type {setsup} \NC \nop \NC \yes \NC \NR
+\NC \type {slide} \NC \yes \NC \yes \NC \NR
+\NC \type {subtypes} \NC \yes \NC \nop \NC \NR
+\NC \type {subtype} \NC \yes \NC \nop \NC \NR
+\NC \type {tail} \NC \yes \NC \yes \NC \NR
+\NC \type {todirect} \NC \yes \NC \yes \NC \NR
+\NC \type {tonode} \NC \yes \NC \yes \NC \NR
+\NC \type {tostring} \NC \yes \NC \yes \NC \NR
+\NC \type {traverse_char} \NC \yes \NC \yes \NC \NR
+\NC \type {traverse_id} \NC \yes \NC \yes \NC \NR
+\NC \type {traverse} \NC \yes \NC \yes \NC \NR
+\NC \type {types} \NC \yes \NC \nop \NC \NR
+\NC \type {type} \NC \yes \NC \nop \NC \NR
+\NC \type {unprotect_glyphs} \NC \yes \NC \yes \NC \NR
+\NC \type {unset_attribute} \NC \yes \NC \yes \NC \NR
+\NC \type {usedlist} \NC \yes \NC \yes \NC \NR
+\NC \type {vpack} \NC \yes \NC \yes \NC \NR
+\NC \type {whatsitsubtypes} \NC \yes \NC \nop \NC \NR
+\NC \type {whatsits} \NC \yes \NC \nop \NC \NR
+\NC \type {write} \NC \yes \NC \yes \NC \NR
+\stoptabulate
+
+% \stopcolumns
+
+The \type {node.next} and \type {node.prev} functions will stay but for
+consistency there are variants called \type {getnext} and \type {getprev}. We had
+to use \type {get} because \type {node.id} and \type {node.subtype} are already
+taken for providing meta information about nodes. Note: The getters do only basic
+checking for valid keys. You should just stick to the keys mentioned in the
+sections that describe node properties.
+
+Some nodes have indirect references. For instance a math character refers to a
+family instead of a font. In that case we provide a virtual font field as
+accessor. So, \type {getfont} and \type {.font} can be used on them. The same is
+true for the \type {width}, \type {height} and \type {depth} of glue nodes. These
+actually access the spec node properties, and here we can set as well as get the
+values.
+
\stopchapter
\stopcomponent
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-style.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-style.tex
index 26688abaf26..c07dc3b765c 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-style.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex-style.tex
@@ -286,7 +286,7 @@
\unexpanded\def\nonterminal#1>{\mathematics{\langle\hbox{\rm #1}\rangle}}
-% taco's brainwave -)
+% taco's brainwave -) .. todo: create a typing variant so that we can avoid the !crlf
\newcatcodetable\syntaxcodetable
@@ -323,6 +323,10 @@
!egroup
+\definetyping
+ [texsyntax]
+ [color=maincolor]
+
% end of wave
\setupinteraction
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex.tex
index 59a969396ad..1613b920b20 100644
--- a/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex.tex
+++ b/Master/texmf-dist/doc/context/sources/general/manuals/luatex/luatex.tex
@@ -1,9 +1,27 @@
% macros=mkvi
+% author : Hans Hagen
+% copyright : PRAGMA ADE & ConTeXt Development Team
+% license : Creative Commons Attribution ShareAlike 4.0 International
+% reference : pragma-ade.nl | contextgarden.net | texlive (related) distributions
+% origin : the ConTeXt distribution
+%
+% comment : Because this manual is distributed with TeX distributions it comes with a rather
+% liberal license. We try to adapt these documents to upgrades in the (sub)systems
+% that they describe. Using parts of the content otherwise can therefore conflict
+% with existing functionality and we cannot be held responsible for that. Many of
+% the manuals contain characteristic graphics and personal notes or examples that
+% make no sense when used out-of-context.
+%
+% comment : Some chapters might have been published in TugBoat, the NTG Maps, the ConTeXt
+% Group journal or otherwise. Thanks to the editors for corrections. Also thanks
+% to users for testing, feedback and corrections.
+
% \tex vs \type vs \syntax vs. \luatex
% \em \it \/
-% "context --nodates --nocompression luatex" can be used for comparison runs
+% "context --nodates --nocompression luatex" can be used for comparison
+% runs, not that we do it
\environment luatex-style
\environment luatex-logos
@@ -11,8 +29,8 @@
\dontcomplain
\startdocument
- [version=0.95.0,
- status=Pre-release]
+ [status=release,
+ version=1.0.5]
\component luatex-titlepage