diff options
Diffstat (limited to 'Master/texmf-dist/doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex')
-rw-r--r-- | Master/texmf-dist/doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex | 2527 |
1 files changed, 2527 insertions, 0 deletions
diff --git a/Master/texmf-dist/doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex b/Master/texmf-dist/doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex new file mode 100644 index 00000000000..16c6e1202ab --- /dev/null +++ b/Master/texmf-dist/doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex @@ -0,0 +1,2527 @@ +% language=uk + +\environment luametatex-style + +\startcomponent luametatex-nodes + +\startchapter[reference=nodes,title={Nodes}] + +\startsection[title={\LUA\ node representation}][library=node] + +\topicindex {nodes} + +\libindex {fields} +\libindex {subtypes} +\libindex {values} + +\TEX's nodes are represented in \LUA\ as userdata objects with a variable set of +fields or by a numeric identifier when requested. When you print a node userdata +object you will see these numbers. In the following syntax tables the type of +such a userdata object is represented as \syntax {<node>}. + +\blank +\dontleavehmode {\bf The return values of \type {node.types} are:} \showtypes +\blank + +In \ETEX\ the \prm {lastnodetype} primitive has been introduced. With this +primitive the valid range of numbers is still $[-1,15]$ and glyph nodes (formerly +known as char nodes) have number~0. That way macro packages can use the same +symbolic names as in traditional \ETEX. But you need to keep in mind that these +\ETEX\ node numbers are different from the real internal ones. When you set \prm +{internalcodesmode} to a non|-|zero value, the internal codes will be used in +the \ETEX\ introspection commands \prm {lastnodetype} and \prm {currentiftype}. + +You can ask for a list of fields with \type {node.fields} and for valid subtypes +with \type {node.subtypes}. The \type {node.values} function reports some used +values. Valid arguments are \type {glue}, \type {style} and \type {math}. Keep in +mind that the setters normally expect a number, but this helper gives you a list +of what numbers matter. For practical reason the \type {pagestate} values are +also reported with this helper, but they are backend specific. + +\def\ShowValues#1{ + \blank + \dontleavehmode + {\bf The return values of \type {node.values("#1")} are:} + \showvalues{#1} + \blank +} + +\ShowValues{glue} \ShowValues{style} \ShowValues{math} \ShowValues{pagestate} + +\stopsection + +\startsection[title={Main text nodes}] + +\topicindex {nodes+text} + +These are the nodes that comprise actual typesetting commands. A few fields are +present in all nodes regardless of their type, these are: + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{next} \NC node \NC the next node in a list, or nil \NC \NR +\NC \type{id} \NC number \NC the node's type (\type {id}) number \NC \NR +\NC \type{subtype} \NC number \NC the node \type {subtype} identifier \NC \NR +\LL +\stoptabulate + +The \type {subtype} is sometimes just a dummy entry because not all nodes +actually use the \type {subtype}, but this way you can be sure that all nodes +accept it as a valid field name, and that is often handy in node list traversal. +In the following tables \type {next} and \type {id} are not explicitly mentioned. + +Besides these three fields, almost all nodes also have an \type {attr} field, and +there is a also a field called \type {prev}. That last field is always present, +but only initialized on explicit request: when the function \type {node.slide} +is called, it will set up the \type {prev} fields to be a backwards pointer in +the argument node list. By now most of \TEX's node processing makes sure that the +\type {prev} nodes are valid but there can be exceptions, especially when the +internal magic uses a leading \nod {temp} nodes to temporarily store a state. + +The \LUAMETATEX\ engine provides a lot of freedom and it is up to the user to +make sure that the node lists remain sane. There are some safeguards but there +can be cases where the engine just quits out of frustration. And, of course you +can make the engine crash. + +\startsubsection[title={\nod {hlist} and \nod {vlist} nodes}] + +\topicindex {nodes+lists} +\topicindex {lists} + +These lists share fields and subtypes although some subtypes can only occur in +horizontal lists while others are unique for vertical lists. The possible +fields are \showfields {hlist}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{list} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{width} \NC number \NC the width of the box \NC \NR +\NC \type{height} \NC number \NC the height of the box \NC \NR +\NC \type{depth} \NC number \NC the depth of the box \NC \NR +\NC \type{direction} \NC number \NC the direction of this box, see~\in [dirnodes] \NC \NR +\NC \type{shift} \NC number \NC a displacement perpendicular to the character + (hlist) or line (vlist) progression direction \NC \NR +\NC \type{glue_order} \NC number \NC a number in the range $[0,4]$, indicating the + glue order \NC \NR +\NC \type{glue_set} \NC number \NC the calculated glue ratio \NC \NR +\NC \type{glue_sign} \NC number \NC 0 = \type {normal}, 1 = \type {stretching}, 2 = + \type {shrinking} \NC \NR +\NC \type{list} \NC node \NC the first node of the body of this list \NC \NR +\LL +\stoptabulate + +The \type {orientation}, \type {woffset}, \type {hoffset}, \type {doffset}, +\type {xoffset} and \type {yoffset} fields are special. They can be used to make +the backend rotate and shift boxes which can be handy in for instance vertical +typesetting. Because they relate to (and depend on the) the backend they are not +discussed here (yet). + +A warning: never assign a node list to the \type {list} field unless you are sure +its internal link structure is correct, otherwise an error may result. + +Note: the field name \type {head} and \type {list} are both valid. Sometimes it +makes more sense to refer to a list by \type {head}, sometimes \type {list} makes +more sense. + +\stopsubsection + +\startsubsection[title={\nod {rule} nodes}] + +\topicindex {nodes+rules} +\topicindex {rules} + +Contrary to traditional \TEX, \LUATEX\ has more \prm {rule} subtypes because we +also use rules to store reuseable objects and images. User nodes are invisible +and can be intercepted by a callback. The supported fields are \showfields {rule}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes {rule} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{width} \NC number \NC the width of the rule where the special value + $-1073741824$ is used for \quote {running} glue dimensions \NC \NR +\NC \type{height} \NC number \NC the height of the rule (can be negative) \NC \NR +\NC \type{depth} \NC number \NC the depth of the rule (can be negative) \NC \NR +\NC \type{left} \NC number \NC shift at the left end (also subtracted from width) \NC \NR +\NC \type{right} \NC number \NC (subtracted from width) \NC \NR +\NC \type{dir} \NC string \NC the direction of this rule, see~\in[dirnodes] \NC \NR +\NC \type{index} \NC number \NC an optional index that can be referred to \NC \NR +\NC \type{transform} \NC number \NC an private variable (also used to specify outline width) \NC \NR +\LL +\stoptabulate + +The \type {left} and type {right} keys are somewhat special (and experimental). +When rules are auto adapting to the surrounding box width you can enforce a shift +to the right by setting \type {left}. The value is also subtracted from the width +which can be a value set by the engine itself and is not entirely under user +control. The \type {right} is also subtracted from the width. It all happens in +the backend so these are not affecting the calculations in the frontend (actually +the auto settings also happen in the backend). For a vertical rule \type {left} +affects the height and \type {right} affects the depth. There is no matching +interface at the \TEX\ end (although we can have more keywords for rules it would +complicate matters and introduce a speed penalty.) However, you can just +construct a rule node with \LUA\ and write it to the \TEX\ input. The \type +{outline} subtype is just a convenient variant and the \type {transform} field +specifies the width of the outline. + +The \type {xoffset} and \type {yoffset} fields are special. They can be used to +shift rules. Because they relate to (and depend on the) the backend they are not +discussed here (yet). + +\stopsubsection + +\startsubsection[title={\nod {ins} nodes}] + +\topicindex {nodes+insertions} +\topicindex {insertions} + +This node relates to the \prm {insert} primitive and support the fields: \showfields{ins}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC the insertion class \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{cost} \NC number \NC the penalty associated with this insert \NC \NR +\NC \type{height} \NC number \NC height of the insert \NC \NR +\NC \type{depth} \NC number \NC depth of the insert \NC \NR +\NC \type{list} \NC node \NC the first node of the body of this insert \NC \NR +\LL +\stoptabulate + +There is a set of extra fields that concern the associated glue: \type {width}, +\type {stretch}, \type {stretch_order}, \type {shrink} and \type {shrink_order}. +These are all numbers. + +A warning: never assign a node list to the \type {head} field unless you are sure +its internal link structure is correct, otherwise an error may result. You can use +\type {list} instead (often in functions you want to use local variable with similar +names and both names are equally sensible). + +\stopsubsection + +\startsubsection[title={\nod {mark} nodes}] + +\topicindex {nodes+marks} +\topicindex {marks} + +This one relates to the \prm {mark} primitive and only has a few fields: +\showfields {mark}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC unused \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{class} \NC number \NC the mark class \NC \NR +\NC \type{mark} \NC table \NC a table representing a token list \NC \NR +\LL +\stoptabulate + +\stopsubsection + +\startsubsection[title={\nod {adjust} nodes}] + +\topicindex {nodes+adjust} +\topicindex {adjust} + +This node comes from \prm {vadjust} primitive and has fields: \showfields {adjust}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{adjust} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{list} \NC node \NC adjusted material \NC \NR +\LL +\stoptabulate + +A warning: never assign a node list to the \type {head} field unless you are sure +its internal link structure is correct, otherwise an error may be the result. + +\stopsubsection + +\startsubsection[title={\nod {disc} nodes}] + +\topicindex {nodes+discretionaries} +\topicindex {discretionaries} + +The \prm {discretionary} and \prm {-}, the \type {-} character but also the +hyphenation mechanism produces these nodes. The available fields are: \showfields +{disc}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{disc} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{pre} \NC node \NC pointer to the pre|-|break text \NC \NR +\NC \type{post} \NC node \NC pointer to the post|-|break text \NC \NR +\NC \type{replace} \NC node \NC pointer to the no|-|break text \NC \NR +\NC \type{penalty} \NC number \NC the penalty associated with the break, normally + \prm {hyphenpenalty} or \prm {exhyphenpenalty} \NC \NR +\LL +\stoptabulate + +The subtype numbers~4 and~5 belong to the \quote {of-f-ice} explanation given +elsewhere. These disc nodes are kind of special as at some point they also keep +information about breakpoints and nested ligatures. + +The \type {pre}, \type {post} and \type {replace} fields at the \LUA\ end are in +fact indirectly accessed and have a \type {prev} pointer that is not \type {nil}. +This means that when you mess around with the head of these (three) lists, you +also need to reassign them because that will restore the proper \type {prev} +pointer, so: + +\starttyping +pre = d.pre +-- change the list starting with pre +d.pre = pre +\stoptyping + +Otherwise you can end up with an invalid internal perception of reality and +\LUAMETATEX\ might even decide to crash on you. It also means that running forward +over for instance \type {pre} is ok but backward you need to stop at \type {pre}. +And you definitely must not mess with the node that \type {prev} points to, if +only because it is not really a node but part of the disc data structure (so +freeing it again might crash \LUAMETATEX). + +\stopsubsection + +\startsubsection[title={\nod {math} nodes}] + +\topicindex {nodes+math} +\topicindex {math+nodes} + +Math nodes represent the boundaries of a math formula, normally wrapped into +\type {$} signs. The following fields are available: \showfields {math}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{math} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{surround} \NC number \NC width of the \prm {mathsurround} kern \NC \NR +\NC \type{width} \NC number \NC the horizontal or vertical displacement \NC \NR +\NC \type{stretch} \NC number \NC extra (positive) displacement or stretch amount \NC \NR +\NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR +\NC \type{shrink} \NC number \NC extra (negative) displacement or shrink amount\NC \NR +\NC \type{shrink_order} \NC number \NC factor applied to shrink amount \NC \NR +\LL +\stoptabulate + +The glue fields only kick in when the \type {surround} fields is zero. + +\stopsubsection + +\startsubsection[title={\nod {glue} nodes}] + +\topicindex {nodes+glue} +\topicindex {glue} + +Skips are about the only type of data objects in traditional \TEX\ that are not a +simple value. They are inserted when \TEX\ sees a space in the text flow but also +by \prm {hskip} and \prm {vskip}. The structure that represents the glue +components of a skip internally is called a \nod {glue_spec}. In \LUAMETATEX\ we +don't use the spec itself but just its values. A glue node has the fields: +\showfields {glue}. + +\starttabulate[|l|l|pA{flushleft,tolerant}|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{glue} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{leader} \NC node \NC pointer to a box or rule for leaders \NC \NR +\NC \type{width} \NC number \NC the horizontal or vertical displacement \NC \NR +\NC \type{stretch} \NC number \NC extra (positive) displacement or stretch amount \NC \NR +\NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR +\NC \type{shrink} \NC number \NC extra (negative) displacement or shrink amount\NC \NR +\NC \type{shrink_order} \NC number \NC factor applied to shrink amount \NC \NR +\LL +\stoptabulate + +Note that we use the key \type {width} in both horizontal and vertical glue. This +suits the \TEX\ internals well so we decided to stick to that naming. + +The effective width of some glue subtypes depends on the stretch or shrink needed +to make the encapsulating box fit its dimensions. For instance, in a paragraph +lines normally have glue representing spaces and these stretch or shrink to make +the content fit in the available space. The \type {effective_glue} function that +takes a glue node and a parent (hlist or vlist) returns the effective width of +that glue item. When you pass \type {true} as third argument the value will be +rounded. + +\stopsubsection + +\startsubsection[title={\nod {glue_spec} nodes}] + +\topicindex {nodes+glue} +\topicindex {gluespec} + +Internally \LUAMETATEX\ (like its ancestors) also uses nodes to store data that +is not seen in node lists. For instance the state of expression scanning (\type +{\dimexpr} etc.) and conditionals (\type {\ifcase} etc.) is also kept in lists of +nodes. A glue, which has five components, is stored in a node as well, so, where +most registers store just a number, a skip register (of internal quantity) uses a +pointer to a glue spec node. It has similar fields as glue nodes: \showfields +{glue_spec}, which is not surprising because in the past (and other engines than +\LUATEX) a glue node also has its values stored in a glue spec. This has some +advantages because often the values are the same, so for instance spacing related +skips were not resolved immediately but pointed to the current value of a space +related internal register (like \type {\spaceskip}). But, in \LUATEX\ we do +resolve these quantities immediately and we put the current values in the glue +nodes. + +\starttabulate[|l|l|pA{flushleft,tolerant}|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{width} \NC number \NC the horizontal or vertical displacement \NC \NR +\NC \type{stretch} \NC number \NC extra (positive) displacement or stretch amount \NC \NR +\NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR +\NC \type{shrink} \NC number \NC extra (negative) displacement or shrink amount\NC \NR +\NC \type{shrink_order} \NC number \NC factor applied to shrink amount \NC \NR +\LL +\stoptabulate + +You will only find these nodes in a few places, for instance when you query an +internal quantity. In principle we could do without them as we have interfaces +that use the five numbers instead. For compatibility reasons we keep glue spec +nodes exposed but this might change in the future. + +\stopsubsection + +\startsubsection[title={\nod {kern} nodes}] + +\topicindex {nodes+kerns} +\topicindex {kerns} + +The \prm {kern} command creates such nodes but for instance the font and math +machinery can also add them. There are not that many fields: \showfields {kern}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{kern} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{kern} \NC number \NC fixed horizontal or vertical advance \NC \NR +\NC \type{expansion_factor} \NC number \NC multiplier related to hz for font kerns \NC \NR +\LL +\stoptabulate + +\stopsubsection + +\startsubsection[title={\nod {penalty} nodes}] + +\topicindex {nodes+penalty} +\topicindex {penalty} + +The \prm {penalty} command is one that generates these nodes. It is one of the +type of nodes often found in vertical lists. It has the fields: \showfields +{penalty}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{penalty} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{penalty} \NC number \NC the penalty value \NC \NR +\LL +\stoptabulate + +The subtypes are just informative and \TEX\ itself doesn't use them. When you +run into an \type {linebreakpenalty} you need to keep in mind that it's a +accumulation of \type {club}, \type{widow} and other relevant penalties. + +\stopsubsection + +\startsubsection[title={\nod {glyph} nodes},reference=glyphnodes] + +\topicindex {nodes+glyph} +\topicindex {glyphs} + +These are probably the mostly used nodes and although you can push them in the +current list with for instance \prm {char} \TEX\ will normally do it for you when +it considers some input to be text. Glyph nodes are relatively large and have many +fields: \showfields {glyph}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC bit field \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{char} \NC number \NC the character index in the font \NC \NR +\NC \type{font} \NC number \NC the font identifier \NC \NR +\NC \type{lang} \NC number \NC the language identifier \NC \NR +\NC \type{left} \NC number \NC the frozen \type {\lefthyphenmnin} value \NC \NR +\NC \type{right} \NC number \NC the frozen \type {\righthyphenmnin} value \NC \NR +\NC \type{uchyph} \NC boolean \NC the frozen \prm {uchyph} value \NC \NR +\NC \type{components} \NC node \NC pointer to ligature components \NC \NR +\NC \type{xoffset} \NC number \NC a virtual displacement in horizontal direction \NC \NR +\NC \type{yoffset} \NC number \NC a virtual displacement in vertical direction \NC \NR +\NC \type{width} \NC number \NC the (original) width of the character \NC \NR +\NC \type{height} \NC number \NC the (original) height of the character\NC \NR +\NC \type{depth} \NC number \NC the (original) depth of the character\NC \NR +\NC \type{expansion_factor} \NC number \NC the to be applied expansion_factor \NC \NR +\NC \type{data} \NC number \NC a general purpose field for users (we had room for it) \NC \NR +\LL +\stoptabulate + +The \type {width}, \type {height} and \type {depth} values are read|-|only. The +\type {expansion_factor} is assigned in the par builder and used in the backend. + +A warning: never assign a node list to the components field unless you are sure +its internal link structure is correct, otherwise an error may be result. Valid +bits for the \type {subtype} field are: + +\starttabulate[|c|l|] +\DB bit \BC meaning \NC \NR +\TB +\NC 0 \NC character \NC \NR +\NC 1 \NC ligature \NC \NR +\NC 2 \NC ghost \NC \NR +\NC 3 \NC left \NC \NR +\NC 4 \NC right \NC \NR +\LL +\stoptabulate + +The \type {expansion_factor} has been introduced as part of the separation +between front- and backend. It is the result of extensive experiments with a more +efficient implementation of expansion. Early versions of \LUATEX\ already +replaced multiple instances of fonts in the backend by scaling but contrary to +\PDFTEX\ in \LUATEX\ we now also got rid of font copies in the frontend and +replaced them by expansion factors that travel with glyph nodes. Apart from a +cleaner approach this is also a step towards a better separation between front- +and backend. + +The \type {is_char} function checks if a node is a glyph node with a subtype still +less than 256. This function can be used to determine if applying font logic to a +glyph node makes sense. The value \type {nil} gets returned when the node is not +a glyph, a character number is returned if the node is still tagged as character +and \type {false} gets returned otherwise. When nil is returned, the id is also +returned. The \type {is_glyph} variant doesn't check for a subtype being less +than 256, so it returns either the character value or nil plus the id. These +helpers are not always faster than separate calls but they sometimes permit +making more readable tests. The \type {uses_font} helpers takes a node +and font id and returns true when a glyph or disc node references that font. + +\stopsubsection + +\startsubsection[title={\nod {boundary} nodes}] + +\topicindex {nodes+boundary} +\topicindex {boundary} + +This node relates to the \prm {noboundary}, \prm {boundary}, \prm +{protrusionboundary} and \prm {wordboundary} primitives. These are small +nodes: \showfields {boundary} are the only fields. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{boundary} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{value} \NC number \NC values 0--255 are reserved \NC \NR +\LL +\stoptabulate + +\stopsubsection + +\startsubsection[title={\nod {local_par} nodes}] + +\topicindex {nodes+paragraphs} +\topicindex {paragraphs} + +This node is inserted at the start of a paragraph. You should not mess +too much with this one. Valid fields are: \showfields {local_par}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{pen_inter} \NC number \NC local interline penalty (from \lpr {localinterlinepenalty}) \NC \NR +\NC \type{pen_broken} \NC number \NC local broken penalty (from \lpr {localbrokenpenalty}) \NC \NR +\NC \type{dir} \NC string \NC the direction of this par. see~\in [dirnodes] \NC \NR +\NC \type{box_left} \NC node \NC the \lpr {localleftbox} \NC \NR +\NC \type{box_left_width} \NC number \NC width of the \lpr {localleftbox} \NC \NR +\NC \type{box_right} \NC node \NC the \lpr {localrightbox} \NC \NR +\NC \type{box_right_width} \NC number \NC width of the \lpr {localrightbox} \NC \NR +\LL +\stoptabulate + +A warning: never assign a node list to the \type {box_left} or \type {box_right} +field unless you are sure its internal link structure is correct, otherwise an +error may result. + +\stopsubsection + +\startsubsection[title={\nod {dir} nodes},reference=dirnodes] + +\topicindex {nodes+direction} +\topicindex {directions} + +Direction nodes mark parts of the running text that need a change of direction +and the \prm {textdir} command generates them. Again this is a small node, we +just have \showfields {dir}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{dir} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{dir} \NC string \NC the direction (\type {0} = l2r, \type {1} = r2l) \NC \NR +\NC \type{level} \NC number \NC nesting level of this direction \NC \NR +\LL +\stoptabulate + +There are only two directions: left|-|to|-|right (\type {0}) and +right|-|to|-|left (\type {1}). This is different from \LUATEX\ that has four +directions. + +\stopsubsection + +\startsubsection[title={\nod {marginkern} nodes}] + +\topicindex {nodes+paragraphs} +\topicindex {paragraphs} +\topicindex {protrusion} + +Margin kerns result from protrusion and have: \showfields {margin_kern}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{marginkern} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{width} \NC number \NC the advance of the kern \NC \NR +\NC \type{glyph} \NC node \NC the glyph to be used \NC \NR +\LL +\stoptabulate + +\stopsubsection + +\startsubsection[title={Whatsits}] + +A whatsit node is a real simple one and it only has a subtype. It is even less +than a user node (which it actually could be) and uses hardly any memory. What +you do with it it entirely up to you: it's is real minimalistic. You can assign a +subtype and it has attributes. It is all up to the user how they are handled. + +\stopsubsection + +\startsubsection[title={Math noads}] + +\topicindex {nodes+math} +\topicindex {math+nodes} + +\startsubsubsection[title=The concept] + +These are the so||called \quote {noad}s and the nodes that are specifically +associated with math processing. When you enter a formula, \TEX\ creates a node +list with regular nodes and noads. Then it hands over the list the math +processing engine. The result of that is a nodelist without noads. Most of the +noads contain subnodes so that the list of possible fields is actually quite +small. Math formulas are both a linked list and a tree. For instance in $e = +mc^2$ there is a linked list \type {e = m c} but the \type {c} has a superscript +branch that itself can be a list with branches. + +First, there are the objects (the \TEX book calls them \quote {atoms}) that are +associated with the simple math objects: ord, op, bin, rel, open, close, punct, +inner, over, under, vcenter. These all have the same fields, and they are combined +into a single node type with separate subtypes for differentiation: \showfields +{noad}. + +Many object fields in math mode are either simple characters in a specific family +or math lists or node lists: \type {math_char}, \type {math_text_char}, {sub_box} +and \type {sub_mlist} and \type {delim}. These are endpoints and therefore the +\type {next} and \type {prev} fields of these these subnodes are unused. + +Some of the more elaborate noads have an option field. The values in this bitset +are common: + +\starttabulate[|l|r|] +\DB meaning \BC bits \NC \NR +\TB +\NC set \NC \type{0x08} \NC \NR +\NC internal \NC \type{0x00} + \type{0x08} \NC \NR +\NC internal \NC \type{0x01} + \type{0x08} \NC \NR +\NC axis \NC \type{0x02} + \type{0x08} \NC \NR +\NC no axis \NC \type{0x04} + \type{0x08} \NC \NR +\NC exact \NC \type{0x10} + \type{0x08} \NC \NR +\NC left \NC \type{0x11} + \type{0x08} \NC \NR +\NC middle \NC \type{0x12} + \type{0x08} \NC \NR +\NC right \NC \type{0x14} + \type{0x08} \NC \NR +\NC no subscript \NC \type{0x21} + \type{0x08} \NC \NR +\NC no superscript \NC \type{0x22} + \type{0x08} \NC \NR +\NC no script \NC \type{0x23} + \type{0x08} \NC \NR +\LL +\stoptabulate + +\stopsubsubsection + +\startsubsubsection[title={\nod {math_char} and \nod {math_text_char} subnodes}] + +These are the most common ones, as they represent characters, and they both have +the same fields: \showfields {math_char}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{char} \NC number \NC the character index \NC \NR +\NC \type{fam} \NC number \NC the family number \NC \NR +\LL +\stoptabulate + +The \nod {math_char} is the simplest subnode field, it contains the character and +family for a single glyph object. The family eventually resolves on a reference +to a font. The \nod {math_text_char} is a special case that you will not normally +encounter, it arises temporarily during math list conversion (its sole function +is to suppress a following italic correction). + +\stopsubsubsection + +\startsubsubsection[title={\nod {sub_box} and \nod {sub_mlist} subnodes}] + +These two subnode types are used for subsidiary list items. For \nod {sub_box}, +the \type {list} points to a \quote {normal} vbox or hbox. For \nod {sub_mlist}, +the \type {list} points to a math list that is yet to be converted. Their fields +are: \showfields {sub_box}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{list} \NC node \NC list of nodes \NC \NR +\LL +\stoptabulate + +A warning: never assign a node list to the \type {list} field unless you are sure +its internal link structure is correct, otherwise an error is triggered. + +\stopsubsubsection + +\startsubsubsection[title={\nod {delim} subnodes}] + +There is a fifth subnode type that is used exclusively for delimiter fields. As +before, the \type {next} and \type {prev} fields are unused, but we do have: +\showfields {delim}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{small_char} \NC number \NC character index of base character \NC \NR +\NC \type{small_fam} \NC number \NC family number of base character \NC \NR +\NC \type{large_char} \NC number \NC character index of next larger character \NC \NR +\NC \type{large_fam} \NC number \NC family number of next larger character \NC \NR +\LL +\stoptabulate + +The fields \type {large_char} and \type {large_fam} can be zero, in that case the +font that is set for the \type {small_fam} is expected to provide the large +version as an extension to the \type {small_char}. + +\stopsubsubsection + +\startsubsubsection[title={simple \nod {noad} nodes}] + +In these noads, the \type {nucleus}, \type {sub} and \type {sup} fields can +branch of. Its fields are: \showfields {noad}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{noad} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{nucleus} \NC kernel node \NC base \NC \NR +\NC \type{sub} \NC kernel node \NC subscript \NC \NR +\NC \type{sup} \NC kernel node \NC superscript \NC \NR +\NC \type{options} \NC number \NC bitset of rendering options \NC \NR +\LL +\stoptabulate + +\stopsubsubsection + +\startsubsubsection[title={\nod {accent} nodes}] + +Accent nodes deal with stuff on top or below a math constructs. They support: +\showfields {accent}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{accent} \NC \NR +\NC \type{nucleus} \NC kernel node \NC base \NC \NR +\NC \type{sub} \NC kernel node \NC subscript \NC \NR +\NC \type{sup} \NC kernel node \NC superscript \NC \NR +\NC \type{accent} \NC kernel node \NC top accent \NC \NR +\NC \type{bot_accent} \NC kernel node \NC bottom accent \NC \NR +\NC \type{fraction} \NC number \NC larger step criterium (divided by 1000) \NC \NR +\LL +\stoptabulate + +\stopsubsubsection + +\startsubsubsection[title={\nod {style} nodes}] + +These nodes are signals to switch to another math style. They are quite simple: +\showfields {style}. Currently the subtype is actually used to store the style +but don't rely on that for the future. Fields are: \showfields {style}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{style} \NC string \NC contains the style \NC \NR +\LL +\stoptabulate + +Valid styles are: \showvalues{style}. + +\stopsubsubsection + +\startsubsubsection[title={\nod {parameter} nodes}] + +These nodes are used to (locally) set math parameters: \showfields {parameter}. +Fields are: \showfields {parameter}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{style} \NC string \NC contains the style \NC \NR +\NC \type{name} \NC string \NC defines the parameter \NC \NR +\NC \type{value} \NC number \NC holds the value, in case of a muglue multiple \NC \NR +\LL +\stoptabulate + +\stopsubsubsection + +\startsubsubsection[title={\nod {choice} nodes}] + +Of its fields \showfields {choice} most are lists. Warning: never assign a node +list unless you are sure its internal link structure is correct, otherwise an +error can occur. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{display} \NC node \NC list of display size alternatives \NC \NR +\NC \type{text} \NC node \NC list of text size alternatives \NC \NR +\NC \type{script} \NC node \NC list of scriptsize alternatives \NC \NR +\NC \type{scriptscript} \NC node \NC list of scriptscriptsize alternatives \NC \NR +\LL +\stoptabulate + +\stopsubsubsection + +\startsubsubsection[title={\nod {radical} nodes}] + +Radical nodes are the most complex as they deal with scripts as well as +constructed large symbols. Many fields: \showfields {radical}. Warning: never +assign a node list to the \type {nucleus}, \type {sub}, \type {sup}, \type +{left}, or \type {degree} field unless you are sure its internal link structure +is correct, otherwise an error can be triggered. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{radical} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{nucleus} \NC kernel node \NC base \NC \NR +\NC \type{sub} \NC kernel node \NC subscript \NC \NR +\NC \type{sup} \NC kernel node \NC superscript \NC \NR +\NC \type{left} \NC delimiter node \NC \NC \NR +\NC \type{degree} \NC kernel node \NC only set by \lpr {Uroot} \NC \NR +\NC \type{width} \NC number \NC required width \NC \NR +\NC \type{options} \NC number \NC bitset of rendering options \NC \NR +\LL +\stoptabulate + +\stopsubsubsection + +\startsubsubsection[title={\nod {fraction} nodes}] + +Fraction nodes are also used for delimited cases, hence the \type {left} and +\type {right} fields among: \showfields {fraction}. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{width} \NC number \NC (optional) width of the fraction \NC \NR +\NC \type{num} \NC kernel node \NC numerator \NC \NR +\NC \type{denom} \NC kernel node \NC denominator \NC \NR +\NC \type{left} \NC delimiter node \NC left side symbol \NC \NR +\NC \type{right} \NC delimiter node \NC right side symbol \NC \NR +\NC \type{middle} \NC delimiter node \NC middle symbol \NC \NR +\NC \type{options} \NC number \NC bitset of rendering options \NC \NR +\LL +\stoptabulate + +Warning: never assign a node list to the \type {num}, or \type {denom} field +unless you are sure its internal link structure is correct, otherwise an error +can result. + +\stopsubsubsection + +\startsubsubsection[title={\nod {fence} nodes}] + +Fence nodes come in pairs but either one can be a dummy (this period driven empty +fence). Fields are: \showfields {fence}. Some of these fields are used by the +renderer and might get adapted in the process. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{fence} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{delim} \NC delimiter node \NC delimiter specification \NC \NR +\NC \type{italic} \NC number \NC italic correction \NC \NR +\NC \type{height} \NC number \NC required height \NC \NR +\NC \type{depth} \NC number \NC required depth \NC \NR +\NC \type{options} \NC number \NC bitset of rendering options \NC \NR +\NC \type{class} \NC number \NC spacing related class \NC \NR +\LL +\stoptabulate + +\stopsubsubsection + +\stopsubsection + +\stopsection + +\startsection[title={The \type {node} library}][library=node] + +\startsubsection[title={Introduction}] + +The \type {node} library provides methods that facilitate dealing with (lists of) +nodes and their values. They allow you to create, alter, copy, delete, and insert +node, the core objects within the typesetter. Nodes are represented in \LUA\ as +userdata. The various parts within a node can be accessed using named fields. + +Each node has at least the three fields \type {next}, \type {id}, and \type +{subtype}. The other available fields depend on the \type {id}. + +\startitemize[intro] + +\startitem + The \type {next} field returns the userdata object for the next node in a + linked list of nodes, or \type {nil}, if there is no next node. +\stopitem + +\startitem + The \type {id} indicates \TEX's \quote{node type}. The field \type {id} has a + numeric value for efficiency reasons, but some of the library functions also + accept a string value instead of \type {id}. +\stopitem + +\startitem + The \type {subtype} is another number. It often gives further information + about a node of a particular \type {id}. +\stopitem + +\stopitemize + +% Support for \nod {unset} (alignment) nodes is partial: they can be queried and +% modified from \LUA\ code, but not created. + +Nodes can be compared to each other, but: you are actually comparing indices into +the node memory. This means that equality tests can only be trusted under very +limited conditions. It will not work correctly in any situation where one of the +two nodes has been freed and|/|or reallocated: in that case, there will be false +positives. The general approach to a node related callback is as follows: + +\startitemize + +\startitem + Assume that the node list that you get is okay and properly double linked. + If for some reason the links are not right, you can apply \type {node.slide} + to the list. +\stopitem + +\startitem + When you insert a node, make sure you use a previously removed one, a new one + or a copy. Don't simply inject the same node twice. +\stopitem + +\startitem + When you remove a node, make sure that when this is permanent, you also free + the node or list. When you free a node its components are checked and when + they are nodes themselves they are also freed. +\stopitem + +\startitem + Although you can fool the system, normally you will trigger an error when you + try to copy a nonexisting node, or free an already freed node. There is some + overhead involved in this checking but the current compromise is acceptable. +\stopitem + +\startitem + When you're done, pass back (if needed) the result. It's your responsibility + to make sure that the list is properly linked (you can play safe and again + apply \type {node.slide}. In principle you can put nodes in a list that are + not acceptable in the following up actions. Some nodes get ignored, others + will trigger an error, and sometimes the engine will just crash. +\stopitem + +\stopitemize + +So, from the above it will be clear then memory management of nodes has to be +done explicitly by the user. Nodes are not \quote {seen} by the \LUA\ garbage +collector, so you have to call the node freeing functions yourself when you are +no longer in need of a node (list). Nodes form linked lists without reference +counting, so you have to be careful that when control returns back to \LUATEX\ +itself, you have not deleted nodes that are still referenced from a \type {next} +pointer elsewhere, and that you did not create nodes that are referenced more +than once. Normally the setters and getters handle this for you. + +A good example are discretionary nodes that themselves have three sublists. +Internally they use special pointers, but the user never sees them because when +you query them or set fields, this property is hidden and taken care of. You just +see a list. But, when you mess with these sub lists it is your responsibility +that it only contains nodes that are permitted in a discretionary. + +There are statistics available with regards to the allocated node memory, which +can be handy for tracing. Normally the amount of used nodes is not that large. +Typesetting a page can involve thousands of them but most are freed when the page +has been shipped out. Compared to other programs, node memory usage is not that +excessive. So, if for some reason your application leaks nodes, if at the end of +your run you lost as few hundred it's not a real problem. In fact, if you created +boxes and made copies but not flushed them for good reason, your run will for +sure end with used nodes and the statistics will mention that. The same is true +for attributes and skips (glue spec nodes): keeping the current state involves +using nodes. + +\stopsubsection + +\startsubsection[title={Housekeeping}] + +\startsubsubsection[title={\type {types}}] + +\libindex {types} + +This function returns an array that maps node id numbers to node type strings, +providing an overview of the possible top|-|level \type {id} types. + +\startfunctioncall +<table> t = node.types() +\stopfunctioncall + +When we issue this command, we get a table. The currently visible types are +\inlineluavalue { node.types() } where the numbers are the internal identifiers. +Only those nodes are reported that make sense to users so there can be gaps in +the range of numbers. + +\stopsubsubsection + +\startsubsubsection[title={\type {id} and \type {type}}] + +\libindex{id} +\libindex{type} + +This converts a single type name to its internal numeric representation. + +\startfunctioncall +<number> id = node.id(<string> type) +\stopfunctioncall + +The \type {node.id("glyph")} command returns the number \inlineluavalue { node.id +("glyph") } and \type {node.id("hlist")} returns \inlineluavalue { node.id +("hlist") } where the numbers don't relate to importance or some ordering; they +just appear in the order that is handy for the engine. Commands like this are +rather optimized so performance should be ok but you can of course always store +the id in a \LUA\ number. + +The reverse operation is: \type {node.type} If the argument is a number, then the +next function converts an internal numeric representation to an external string +representation. Otherwise, it will return the string \type {node} if the object +represents a node, and \type {nil} otherwise. + +\startfunctioncall +<string> type = node.type(<any> n) +\stopfunctioncall + +The \type {node.type(4)} command returns the string \inlineluavalue { node.type +(4) } and \type {node.id(99)} returns \inlineluavalue { node.id (99) } because +there is no node with that id. + +\stopsubsubsection + +\startsubsubsection[title={\type {fields} and \type {has_field}}] + +\libindex {fields} +\libindex {has_field} + +This function returns an indexed table with valid field names for a particular +type of node. + +\startfunctioncall +<table> t = node.fields(<number|string> id) +\stopfunctioncall + +The function accepts a string or number, so \typ {node.fields ("glyph")} returns +\inlineluavalue { node.fields ("glyph") } and \typ {node.fields (12)} gives +\inlineluavalue { node.fields (12) }. + +The \type {has_field} function returns a boolean that is only true if \type {n} +is actually a node, and it has the field. + +\startfunctioncall +<boolean> t = node.has_field(<node> n, <string> field) +\stopfunctioncall + +This function probably is not that useful but some nodes don't have a \type +{subtype}, \type {attr} or \type {prev} field and this is a way to test for that. + +\stopsubsubsection + +\startsubsubsection[title={\type {is_node}}] + +\topicindex {nodes+functions} + +\libindex {is_node} + +\startfunctioncall +<boolean|integer> t = node.is_node(<any> item) +\stopfunctioncall + +This function returns a number (the internal index of the node) if the argument +is a userdata object of type \type {<node>} and false when no node is passed. + +\stopsubsubsection + +\startsubsubsection[title={\type {new}}] + +\libindex{new} + +The \type {new} function creates a new node. All its fields are initialized to +either zero or \type {nil} except for \type {id} and \type {subtype}. Instead of +numbers you can also use strings (names). If you pass a second argument the +subtype will be set too. + +\startfunctioncall +<node> n = node.new(<number|string> id) +<node> n = node.new(<number|string> id, <number|string> subtype) +\stopfunctioncall + +As already has been mentioned, you are responsible for making sure that nodes +created this way are used only once, and are freed when you don't pass them +back somehow. + +\stopsubsubsection + +\startsubsubsection[title={\type {free}, \type {flush_node} and \type {flush_list}}] + +\libindex{free} +\libindex{flush_node} +\libindex{flush_list} + +The next one frees node \type {n} from \TEX's memory. Be careful: no checks are +done on whether this node is still pointed to from a register or some \type +{next} field: it is up to you to make sure that the internal data structures +remain correct. Fields that point to nodes or lists are flushed too. So, when +you used their content for something else you need to set them to nil first. + +\startfunctioncall +<node> next = node.free(<node> n) +flush_node(<node> n) +\stopfunctioncall + +The \type {free} function returns the next field of the freed node, while the +\type {flush_node} alternative returns nothing. + +A list starting with node \type {n} can be flushed from \TEX's memory too. Be +careful: no checks are done on whether any of these nodes is still pointed to +from a register or some \type {next} field: it is up to you to make sure that the +internal data structures remain correct. + +\startfunctioncall +node.flush_list(<node> n) +\stopfunctioncall + +When you free for instance a discretionary node, \type {flush_list} is applied to +the \type {pre}, \type {post}, \type {replace} so you don't need to do that +yourself. Assigning them \type {nil} won't free those lists! + +\stopsubsubsection + +\startsubsubsection[title={\type {copy} and \type {copy_list}}] + +\libindex{copy} +\libindex{copy_list} + +This creates a deep copy of node \type {n}, including all nested lists as in the case +of a hlist or vlist node. Only the \type {next} field is not copied. + +\startfunctioncall +<node> m = node.copy(<node> n) +\stopfunctioncall + +A deep copy of the node list that starts at \type {n} can be created too. If +\type {m} is also given, the copy stops just before node \type {m}. + +\startfunctioncall +<node> m = node.copy_list(<node> n) +<node> m = node.copy_list(<node> n, <node> m) +\stopfunctioncall + +Note that you cannot copy attribute lists this way. However, there is normally no +need to copy attribute lists as when you do assignments to the \type {attr} field +or make changes to specific attributes, the needed copying and freeing takes +place automatically. When you change a value of an attribute {\em in} a list, it will +affect all the nodes that share that list. + +\stopsubsubsection + +\startsubsubsection[title={\type {write}}] + +\libindex {write} + +\startfunctioncall +node.write(<node> n) +\stopfunctioncall + +This function will append a node list to \TEX's \quote {current list}. The node +list is not deep|-|copied! There is no error checking either! You might need to +enforce horizontal mode in order for this to work as expected. + +\stopsubsubsection + +\stopsubsection + +\startsubsection[title={Manipulating lists}] + +\startsubsubsection[title={\type {slide}}] + +\libindex {slide} + +This helper makes sure that the node list is double linked and returns the found +tail node. + +\startfunctioncall +<node> tail = node.slide(<node> n) +\stopfunctioncall + +After some callbacks automatic sliding takes place. This feature can be turned +off with \type {node.fix_node_lists(false)} but you better make sure then that +you don't mess up lists. In most cases \TEX\ itself only uses \type {next} +pointers but your other callbacks might expect proper \type {prev} pointers too. +Future versions of \LUATEX\ can add more checking but this will not influence +usage. + +\stopsubsubsection + +\startsubsubsection[title={\type {tail}}] + +\libindex {tail} + +\startfunctioncall +<node> m = node.tail(<node> n) +\stopfunctioncall + +Returns the last node of the node list that starts at \type {n}. + +\stopsubsubsection + +\startsubsubsection[title={\type {length} and \type {count}}] + +\libindex {length} +\libindex {count} + +\startfunctioncall +<number> i = node.length(<node> n) +<number> i = node.length(<node> n, <node> m) +\stopfunctioncall + +Returns the number of nodes contained in the node list that starts at \type {n}. +If \type {m} is also supplied it stops at \type {m} instead of at the end of the +list. The node \type {m} is not counted. + +\startfunctioncall +<number> i = node.count(<number> id, <node> n) +<number> i = node.count(<number> id, <node> n, <node> m) +\stopfunctioncall + +Returns the number of nodes contained in the node list that starts at \type {n} +that have a matching \type {id} field. If \type {m} is also supplied, counting +stops at \type {m} instead of at the end of the list. The node \type {m} is not +counted. This function also accept string \type {id}'s. + +\stopsubsubsection + +\startsubsubsection[title={\type {remove}}] + +\libindex {remove} + +\startfunctioncall +<node> head, current, removed = + node.remove(<node> head, <node> current) +<node> head, current = + node.remove(<node> head, <node> current, <boolean> true) +\stopfunctioncall + +This function removes the node \type {current} from the list following \type +{head}. It is your responsibility to make sure it is really part of that list. +The return values are the new \type {head} and \type {current} nodes. The +returned \type {current} is the node following the \type {current} in the calling +argument, and is only passed back as a convenience (or \type {nil}, if there is +no such node). The returned \type {head} is more important, because if the +function is called with \type {current} equal to \type {head}, it will be +changed. When the third argument is passed, the node is freed. + +\stopsubsubsection + +\startsubsubsection[title={\type {insert_before}}] + +\libindex {insert_before} + +\startfunctioncall +<node> head, new = node.insert_before(<node> head, <node> current, <node> new) +\stopfunctioncall + +This function inserts the node \type {new} before \type {current} into the list +following \type {head}. It is your responsibility to make sure that \type +{current} is really part of that list. The return values are the (potentially +mutated) \type {head} and the node \type {new}, set up to be part of the list +(with correct \type {next} field). If \type {head} is initially \type {nil}, it +will become \type {new}. + +\stopsubsubsection + +\startsubsubsection[title={\type {insert_after}}] + +\libindex {insert_after} + +\startfunctioncall +<node> head, new = node.insert_after(<node> head, <node> current, <node> new) +\stopfunctioncall + +This function inserts the node \type {new} after \type {current} into the list +following \type {head}. It is your responsibility to make sure that \type +{current} is really part of that list. The return values are the \type {head} and +the node \type {new}, set up to be part of the list (with correct \type {next} +field). If \type {head} is initially \type {nil}, it will become \type {new}. + +\stopsubsubsection + +\startsubsubsection[title={\type {last_node}}] + +\libindex {last_node} + +\startfunctioncall +<node> n = node.last_node() +\stopfunctioncall + +This function pops the last node from \TEX's \quote{current list}. It returns +that node, or \type {nil} if the current list is empty. + +\stopsubsubsection + +\startsubsubsection[title={\type {traverse}}] + +\libindex {traverse} + +\startfunctioncall +<node> t, id, subtype = node.traverse(<node> n) +\stopfunctioncall + +This is a \LUA\ iterator that loops over the node list that starts at \type {n}. +Typically code looks like this: + +\starttyping +for n in node.traverse(head) do + ... +end +\stoptyping + +is functionally equivalent to: + +\starttyping +do + local n + local function f (head,var) + local t + if var == nil then + t = head + else + t = var.next + end + return t + end + while true do + n = f (head, n) + if n == nil then break end + ... + end +end +\stoptyping + +It should be clear from the definition of the function \type {f} that even though +it is possible to add or remove nodes from the node list while traversing, you +have to take great care to make sure all the \type {next} (and \type {prev}) +pointers remain valid. + +If the above is unclear to you, see the section \quote {For Statement} in the +\LUA\ Reference Manual. + +\stopsubsubsection + +\startsubsubsection[title={\type {traverse_id}}] + +\libindex {traverse_id} + +\startfunctioncall +<node> t, subtype = node.traverse_id(<number> id, <node> n) +\stopfunctioncall + +This is an iterator that loops over all the nodes in the list that starts at +\type {n} that have a matching \type {id} field. + +See the previous section for details. The change is in the local function \type +{f}, which now does an extra while loop checking against the upvalue \type {id}: + +\starttyping + local function f(head,var) + local t + if var == nil then + t = head + else + t = var.next + end + while not t.id == id do + t = t.next + end + return t + end +\stoptyping + +\stopsubsubsection + +\startsubsubsection[title={\type {traverse_char} and \type {traverse_glyph}}] + +\libindex {traverse_char} +\libindex {traverse_glyph} + +The \type{traverse_char} iterator loops over the \nod {glyph} nodes in a list. +Only nodes with a subtype less than 256 are seen. + +\startfunctioncall +<node> n, font, char = node.traverse_char(<node> n) +\stopfunctioncall + +The \type{traverse_glyph} iterator loops over a list and returns the list and +filters all glyphs: + +\startfunctioncall +<node> n, font, char = node.traverse_glyph(<node> n) +\stopfunctioncall + +\stopsubsubsection + +\startsubsubsection[title={\type {traverse_list}}] + +\libindex {traverse_list} + +This iterator loops over the \nod {hlist} and \nod {vlist} nodes in a list. + +\startfunctioncall +<node> n, id, subtype, list = node.traverse_list(<node> n) +\stopfunctioncall + +The four return values can save some time compared to fetching these fields but +in practice you seldom need them all. So consider it a (side effect of +experimental) convenience. + +\stopsubsubsection + +\startsubsubsection[title={\type {find_node}}] + +\libindex {find_node} + +This helper returns the location of the first match at or after node \type {n}: + +\startfunctioncall +<node> n = node.find_node(<node> n, <integer> subtype) +<node> n, subtype = node.find_node(<node> n) +\stopfunctioncall + +\stopsubsubsection + + +\stopsubsection + +\startsubsection[title={Glue handling}][library=node] + +\startsubsubsection[title={\type {setglue}}] + +\libindex {setglue} + +You can set the properties of a glue in one go. If you pass no values, the glue +will become a zero glue. + +\startfunctioncall +node.setglue(<node> n) +node.setglue(<node> n,width,stretch,shrink,stretch_order,shrink_order) +\stopfunctioncall + +When you pass values, only arguments that are numbers are assigned so + +\starttyping +node.setglue(n,655360,false,65536) +\stoptyping + +will only adapt the width and shrink. + +When a list node is passed, you set the glue, order and sign instead. + +\stopsubsubsection + +\startsubsubsection[title={\type {getglue}}] + +\libindex {getglue} + +The next call will return 5 values or nothing when no glue is passed. + +\startfunctioncall +<integer> width, <integer> stretch, <integer> shrink, <integer> stretch_order, + <integer> shrink_order = node.getglue(<node> n) +\stopfunctioncall + +When the second argument is false, only the width is returned (this is consistent +with \type {tex.get}). + +When a list node is passed, you get back the glue that is set, the order of that +glue and the sign. + +\stopsubsubsection + +\startsubsubsection[title={\type {is_zero_glue}}] + +\libindex {is_zero_glue} + +This function returns \type {true} when the width, stretch and shrink properties +are zero. + +\startfunctioncall +<boolean> isglue = node.is_zero_glue(<node> n) +\stopfunctioncall + +\stopsubsubsection + +\stopsubsection + +\startsubsection[title={Attribute handling}][library=node] + +\startsubsubsection[title={Attributes}] + +\topicindex {attributes} + +Assignments to attributes registers result in assigning lists with set attributes +to nodes and the implementation is non|-|trivial because the value that is +attached to a node is essentially a (sorted) sparse array of key|-|value pairs. +It is generally easiest to deal with attribute lists and attributes by using the +dedicated functions in the \type {node} library. + +\stopsubsubsection + +\startsubsubsection[title={\nod {attribute_list} nodes}] + +\topicindex {nodes+attributes} + +An \type {attribute_list} item is used as a head pointer for a list of attribute +items. It has only one user|-|visible field: + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{next} \NC node \NC pointer to the first attribute \NC \NR +\LL +\stoptabulate + +\stopsubsubsection + +\startsubsubsection[title={\nod {attr} nodes}] + +A normal node's attribute field will point to an item of type \nod +{attribute_list}, and the \type {next} field in that item will point to the first +defined \quote {attribute} item, whose \type {next} will point to the second +\quote {attribute} item, etc. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{next} \NC node \NC pointer to the next attribute \NC \NR +\NC \type{number} \NC number \NC the attribute type id \NC \NR +\NC \type{value} \NC number \NC the attribute value \NC \NR +\LL +\stoptabulate + +As mentioned it's better to use the official helpers rather than edit these +fields directly. For instance the \type {prev} field is used for other purposes +and there is no double linked list. + +\stopsubsubsection + +\startsubsubsection[title={\type {current_attr}}] + +\libindex{current_attr} + +This returns the currently active list of attributes, if there is one. + +\startfunctioncall +<node> m = node.current_attr() +\stopfunctioncall + +The intended usage of \type {current_attr} is as follows: + +\starttyping +local x1 = node.new("glyph") +x1.attr = node.current_attr() +local x2 = node.new("glyph") +x2.attr = node.current_attr() +\stoptyping + +or: + +\starttyping +local x1 = node.new("glyph") +local x2 = node.new("glyph") +local ca = node.current_attr() +x1.attr = ca +x2.attr = ca +\stoptyping + +The attribute lists are ref counted and the assignment takes care of incrementing +the refcount. You cannot expect the value \type {ca} to be valid any more when +you assign attributes (using \type {tex.setattribute}) or when control has been +passed back to \TEX. + +\stopsubsubsection + +\startsubsubsection[title={\type {has_attribute}}] + +\libindex {has_attribute} + +\startfunctioncall +<number> v = node.has_attribute(<node> n, <number> id) +<number> v = node.has_attribute(<node> n, <number> id, <number> val) +\stopfunctioncall + +Tests if a node has the attribute with number \type {id} set. If \type {val} is +also supplied, also tests if the value matches \type {val}. It returns the value, +or, if no match is found, \type {nil}. + +\stopsubsubsection + +\startsubsubsection[title={\type {get_attribute}}] + +\libindex {get_attribute} + +\startfunctioncall +<number> v = node.get_attribute(<node> n, <number> id) +\stopfunctioncall + +Tests if a node has an attribute with number \type {id} set. It returns the +value, or, if no match is found, \type {nil}. If no \type {id} is given then the +zero attributes is assumed. + +\stopsubsubsection + +\startsubsubsection[title={\type {find_attribute}}] + +\libindex {find_attribute} + +\startfunctioncall +<number> v, <node> n = node.find_attribute(<node> n, <number> id) +\stopfunctioncall + +Finds the first node that has attribute with number \type {id} set. It returns +the value and the node if there is a match and otherwise nothing. + +\stopsubsubsection + +\startsubsubsection[title={\type {set_attribute}}] + +\libindex {set_attribute} + +\startfunctioncall +node.set_attribute(<node> n, <number> id, <number> val) +\stopfunctioncall + +Sets the attribute with number \type {id} to the value \type {val}. Duplicate +assignments are ignored. + +\stopsubsubsection + +\startsubsubsection[title={\type {unset_attribute}}] + +\libindex {unset_attribute} + +\startfunctioncall +<number> v = + node.unset_attribute(<node> n, <number> id) +<number> v = + node.unset_attribute(<node> n, <number> id, <number> val) +\stopfunctioncall + +Unsets the attribute with number \type {id}. If \type {val} is also supplied, it +will only perform this operation if the value matches \type {val}. Missing +attributes or attribute|-|value pairs are ignored. + +If the attribute was actually deleted, returns its old value. Otherwise, returns +\type {nil}. + +\stopsubsubsection + +\stopsubsection + +\startsubsection[title={Glyph handling}][library=node] + +\startsubsubsection[title={\type {first_glyph}}] + +\libindex {first_glyph} + +\startfunctioncall +<node> n = node.first_glyph(<node> n) +<node> n = node.first_glyph(<node> n, <node> m) +\stopfunctioncall + +Returns the first node in the list starting at \type {n} that is a glyph node +with a subtype indicating it is a glyph, or \type {nil}. If \type {m} is given, +processing stops at (but including) that node, otherwise processing stops at the +end of the list. + +\stopsubsubsection + +\startsubsubsection[title={\type {is_char} and \type {is_glyph}}] + +\libindex {is_char} +\libindex {is_glyph} + +The subtype of a glyph node signals if the glyph is already turned into a character reference +or not. + +\startfunctioncall +<boolean> b = node.is_char(<node> n) +<boolean> b = node.is_glyph(<node> n) +\stopfunctioncall + +\stopsubsubsection + +\startsubsubsection[title={\type {has_glyph}}] + +\libindex {has_glyph} + +This function returns the first glyph or disc node in the given list: + +\startfunctioncall +<node> n = node.has_glyph(<node> n) +\stopfunctioncall + +\stopsubsubsection + +\startsubsubsection[title={\type {ligaturing}}] + +\libindex {ligaturing} + +\startfunctioncall +<node> h, <node> t, <boolean> success = node.ligaturing(<node> n) +<node> h, <node> t, <boolean> success = node.ligaturing(<node> n, <node> m) +\stopfunctioncall + +Apply \TEX-style ligaturing to the specified nodelist. The tail node \type {m} is +optional. The two returned nodes \type {h} and \type {t} are the new head and +tail (both \type {n} and \type {m} can change into a new ligature). + +\stopsubsubsection + +\startsubsubsection[title={\type {kerning}}] + +\libindex {kerning} + +\startfunctioncall +<node> h, <node> t, <boolean> success = node.kerning(<node> n) +<node> h, <node> t, <boolean> success = node.kerning(<node> n, <node> m) +\stopfunctioncall + +Apply \TEX|-|style kerning to the specified node list. The tail node \type {m} is +optional. The two returned nodes \type {h} and \type {t} are the head and tail +(either one of these can be an inserted kern node, because special kernings with +word boundaries are possible). + +\stopsubsubsection + +\startsubsubsection[title={\type {unprotect_glyph[s]}}] + +\libindex {unprotect_glyphs} +\libindex {unprotect_glyph} + +\startfunctioncall +node.unprotect_glyph(<node> n) +node.unprotect_glyphs(<node> n,[<node> n]) +\stopfunctioncall + +Subtracts 256 from all glyph node subtypes. This and the next function are +helpers to convert from \type {characters} to \type {glyphs} during node +processing. The second argument is optional and indicates the end of a range. + +\stopsubsubsection + +\startsubsubsection[title={\type {protect_glyph[s]}}] + +\libindex {protect_glyphs} +\libindex {protect_glyph} + +\startfunctioncall +node.protect_glyph(<node> n) +node.protect_glyphs(<node> n,[<node> n]) +\stopfunctioncall + +Adds 256 to all glyph node subtypes in the node list starting at \type {n}, +except that if the value is 1, it adds only 255. The special handling of 1 means +that \type {characters} will become \type {glyphs} after subtraction of 256. A +single character can be marked by the singular call. The second argument is +optional and indicates the end of a range. + +\stopsubsubsection + +\startsubsubsection[title={\type {protrusion_skippable}}] + +\libindex {protrusion_skippable} + +\startfunctioncall +<boolean> skippable = node.protrusion_skippable(<node> n) +\stopfunctioncall + +Returns \type {true} if, for the purpose of line boundary discovery when +character protrusion is active, this node can be skipped. + +\stopsubsubsection + +\startsubsubsection[title={\type {check_discretionary}, \type {check_discretionaries}}] + +\libindex{check_discretionary} +\libindex{check_discretionaries} + +When you fool around with disc nodes you need to be aware of the fact that they +have a special internal data structure. As long as you reassign the fields when +you have extended the lists it's ok because then the tail pointers get updated, +but when you add to list without reassigning you might end up in trouble when +the linebreak routine kicks in. You can call this function to check the list for +issues with disc nodes. + +\startfunctioncall +node.check_discretionary(<node> n) +node.check_discretionaries(<node> head) +\stopfunctioncall + +The plural variant runs over all disc nodes in a list, the singular variant +checks one node only (it also checks if the node is a disc node). + +\stopsubsubsection + +\startsubsubsection[title={\type {flatten_discretionaries}}] + +\libindex {flatten_discretionaries} + +This function will remove the discretionaries in the list and inject the replace +field when set. + +\startfunctioncall +<node> head, count = node.flatten_discretionaries(<node> n) +\stopfunctioncall + +\stopsubsubsection + +\stopsubsection + +\startsubsection[title={Packaging}][library=node] + +\startsubsubsection[title={\type {hpack}}] + +\libindex {hpack} + +This function creates a new hlist by packaging the list that begins at node \type +{n} into a horizontal box. With only a single argument, this box is created using +the natural width of its components. In the three argument form, \type {info} +must be either \type {additional} or \type {exactly}, and \type {w} is the +additional (\type {\hbox spread}) or exact (\type {\hbox to}) width to be used. +The second return value is the badness of the generated box. + +\startfunctioncall +<node> h, <number> b = + node.hpack(<node> n) +<node> h, <number> b = + node.hpack(<node> n, <number> w, <string> info) +<node> h, <number> b = + node.hpack(<node> n, <number> w, <string> info, <string> dir) +\stopfunctioncall + +Caveat: there can be unexpected side|-|effects to this function, like updating +some of the \prm {marks} and \type {\inserts}. Also note that the content of +\type {h} is the original node list \type {n}: if you call \type {node.free(h)} +you will also free the node list itself, unless you explicitly set the \type +{list} field to \type {nil} beforehand. And in a similar way, calling \type +{node.free(n)} will invalidate \type {h} as well! + +\stopsubsubsection + +\startsubsubsection[title={\type {vpack}}] + +\libindex {vpack} + +This function creates a new vlist by packaging the list that begins at node \type +{n} into a vertical box. With only a single argument, this box is created using +the natural height of its components. In the three argument form, \type {info} +must be either \type {additional} or \type {exactly}, and \type {w} is the +additional (\type {\vbox spread}) or exact (\type {\vbox to}) height to be used. + +\startfunctioncall +<node> h, <number> b = + node.vpack(<node> n) +<node> h, <number> b = + node.vpack(<node> n, <number> w, <string> info) +<node> h, <number> b = + node.vpack(<node> n, <number> w, <string> info, <string> dir) +\stopfunctioncall + +The second return value is the badness of the generated box. See the description +of \type {hpack} for a few memory allocation caveats. + +\stopsubsubsection + +\startsubsubsection[title={\type {prepend_prevdepth}}] + +\libindex {prepend_prevdepth} + +This function is somewhat special in the sense that it is an experimental helper +that adds the interlinespace to a line keeping the baselineskip and lineskip into +account. + +\startfunctioncall +<node> n, <number> delta = + node.prepend_prevdepth(<node> n,<number> prevdepth) +\stopfunctioncall + +\stopsubsubsection + +\startsubsubsection[title={\type {dimensions}, \type {rangedimensions}, \type {naturalwidth}}] + +\libindex{dimensions} +\libindex{rangedimensions} + +\startfunctioncall +<number> w, <number> h, <number> d = + node.dimensions(<node> n) +<number> w, <number> h, <number> d = + node.dimensions(<node> n, <node> t) +\stopfunctioncall + +This function calculates the natural in|-|line dimensions of the node list starting +at node \type {n} and terminating just before node \type {t} (or the end of the +list, if there is no second argument). The return values are scaled points. An +alternative format that starts with glue parameters as the first three arguments +is also possible: + +\startfunctioncall +<number> w, <number> h, <number> d = + node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order, + <node> n) +<number> w, <number> h, <number> d = + node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order, + <node> n, <node> t) +\stopfunctioncall + +This calling method takes glue settings into account and is especially useful for +finding the actual width of a sublist of nodes that are already boxed, for +example in code like this, which prints the width of the space in between the +\type {a} and \type {b} as it would be if \type {\box0} was used as-is: + +\starttyping +\setbox0 = \hbox to 20pt {a b} + +\directlua{print (node.dimensions( + tex.box[0].glue_set, + tex.box[0].glue_sign, + tex.box[0].glue_order, + tex.box[0].head.next, + node.tail(tex.box[0].head) +)) } +\stoptyping + +You need to keep in mind that this is one of the few places in \TEX\ where floats +are used, which means that you can get small differences in rounding when you +compare the width reported by \type {hpack} with \type {dimensions}. + +The second alternative saves a few lookups and can be more convenient in some +cases: + +\startfunctioncall +<number> w, <number> h, <number> d = + node.rangedimensions(<node> parent, <node> first) +<number> w, <number> h, <number> d = + node.rangedimensions(<node> parent, <node> first, <node> last) +\stopfunctioncall + +A simple and somewhat more efficient variant is this: + +\startfunctioncall +<number> w = + node.naturalwidth(<node> start, <node> stop) +\stopfunctioncall + +\stopsubsubsection + +\stopsubsection + +\startsubsection[title={Math}][library=node] + +\startsubsubsection[title={\type {mlist_to_hlist}}] + +\libindex {mlist_to_hlist} + +\startfunctioncall +<node> h = + node.mlist_to_hlist(<node> n, <string> display_type, <boolean> penalties) +\stopfunctioncall + +This runs the internal mlist to hlist conversion, converting the math list in +\type {n} into the horizontal list \type {h}. The interface is exactly the same +as for the callback \cbk {mlist_to_hlist}. + +\stopsubsubsection + +\startsubsubsection[title={\type {end_of_math}}] + +\libindex {end_of_math} + +\startfunctioncall +<node> t = node.end_of_math(<node> start) +\stopfunctioncall + +Looks for and returns the next \type {math_node} following the \type {start}. If +the given node is a math end node this helper returns that node, else it follows +the list and returns the next math endnote. If no such node is found nil is +returned. + +\stopsubsubsection + +\stopsubsection + +\stopsection + +\startsection[title={Two access models}][library=node] + +\topicindex{nodes+direct} +\topicindex{direct nodes} + +\libindex {todirect} +\libindex {tonode} +\libindex {tostring} + +Deep down in \TEX\ a node has a number which is a numeric entry in a memory +table. In fact, this model, where \TEX\ manages memory is real fast and one of +the reasons why plugging in callbacks that operate on nodes is quite fast too. +Each node gets a number that is in fact an index in the memory table and that +number often is reported when you print node related information. You go from +userdata nodes and there numeric references and back with: + +\startfunctioncall +<integer> d = node.todirect(<node> n)) +<node> n = node.tonode(<integer> d)) +\stopfunctioncall + +The userdata model is rather robust as it is a virtual interface with some +additional checking while the more direct access which uses the node numbers +directly. However, even with userdata you can get into troubles when you free +nodes that are no longer allocated or mess up lists. if you apply \type +{tostring} to a node you see its internal (direct) number and id. + +The first model provides key based access while the second always accesses fields +via functions: + +\starttyping +nodeobject.char +getfield(nodenumber,"char") +\stoptyping + +If you use the direct model, even if you know that you deal with numbers, you +should not depend on that property but treat it as an abstraction just like +traditional nodes. In fact, the fact that we use a simple basic datatype has the +penalty that less checking can be done, but less checking is also the reason why +it's somewhat faster. An important aspect is that one cannot mix both methods, +but you can cast both models. So, multiplying a node number makes no sense. + +So our advice is: use the indexed (table) approach when possible and investigate +the direct one when speed might be a real issue. For that reason \LUATEX\ also +provide the \type {get*} and \type {set*} functions in the top level node +namespace. There is a limited set of getters. When implementing this direct +approach the regular index by key variant was also optimized, so direct access +only makes sense when nodes are accessed millions of times (which happens in some +font processing for instance). + +We're talking mostly of getters because setters are less important. Documents +have not that many content related nodes and setting many thousands of properties +is hardly a burden contrary to millions of consultations. + +Normally you will access nodes like this: + +\starttyping +local next = current.next +if next then + -- do something +end +\stoptyping + +Here \type {next} is not a real field, but a virtual one. Accessing it results in +a metatable method being called. In practice it boils down to looking up the node +type and based on the node type checking for the field name. In a worst case you +have a node type that sits at the end of the lookup list and a field that is last +in the lookup chain. However, in successive versions of \LUATEX\ these lookups +have been optimized and the most frequently accessed nodes and fields have a +higher priority. + +Because in practice the \type {next} accessor results in a function call, there +is some overhead involved. The next code does the same and performs a tiny bit +faster (but not that much because it is still a function call but one that knows +what to look up). + +\starttyping +local next = node.next(current) +if next then + -- do something +end +\stoptyping + +In the direct namespace there are more helpers and most of them are accompanied +by setters. The getters and setters are clever enough to see what node is meant. +We don't deal with whatsit nodes: their fields are always accessed by name. It +doesn't make sense to add getters for all fields, we just identifier the most +likely candidates. In complex documents, many node and fields types never get +seen, or seen only a few times, but for instance glyphs are candidates for such +optimization. The \type {node.direct} interface has some more helpers. \footnote +{We can define the helpers in the node namespace with \type {getfield} which is +about as efficient, so at some point we might provide that as module.} + +The \type {setdisc} helper takes three (optional) arguments plus an optional +fourth indicating the subtype. Its \type {getdisc} takes an optional boolean; +when its value is \type {true} the tail nodes will also be returned. The \type +{setfont} helper takes an optional second argument, it being the character. The +directmode setter \type {setlink} takes a list of nodes and will link them, +thereby ignoring \type {nil} entries. The first valid node is returned (beware: +for good reason it assumes single nodes). For rarely used fields no helpers are +provided and there are a few that probably are used seldom too but were added for +consistency. You can of course always define additional accessors using \type +{getfield} and \type {setfield} with little overhead. When the second argument of +\type {setattributelist} is \type {true} the current attribute list is assumed. + +In \CONTEXT\ some of the not performance|-|critical userdata variants are +emulated in \LUA\ and not in the engine, so we retain downward compatibility. + +\def\yes{$+$} \def\nop{$-$} + +\def\supported#1#2#3#4% + {\NC \type{#1} + \NC \ifx#2\yes\lix{node} {#1}\fi #2 + \NC \ifx#3\yes\lix{node.direct}{#1}\fi #3 \NC + \NC #4 \NC + \NR} + +\starttabulate[|l|c|c|] +\DB function \BC node \BC direct \NC emulated \NC \NR +\TB +\supported {check_discretionaries} \nop \yes \yes +\supported {check_discretionary} \nop \yes \yes +\supported {copy} \yes \yes \relax +\supported {copy_list} \yes \yes \relax +%supported {copy_only} \nop \yes \relax +\supported {count} \nop \yes \yes +\supported {current_attr} \yes \yes \relax +\supported {dimensions} \nop \yes \yes +\supported {effective_glue} \nop \yes \yes +\supported {end_of_math} \nop \yes \yes +\supported {find_attribute} \nop \yes \yes +%supported {find_attribute_range} \nop \yes \relax +%supported {find_node} \nop \yes \relax +\supported {first_glyph} \nop \yes \yes +\supported {flatten_discretionaries} \nop \yes \yes +%supported {flush_components} \nop \yes \relax +\supported {flush_list} \yes \yes \relax +\supported {flush_node} \yes \yes \relax +\supported {free} \yes \yes \relax +\supported {get_attribute} \yes \yes \relax +\supported {get_properties_table} \yes \yes \relax +\supported {get_synctex_fields} \nop \yes \relax +\supported {getattributelist} \nop \yes \relax +\supported {getboth} \nop \yes \relax +\supported {getbox} \nop \yes \relax +\supported {getchar} \nop \yes \relax +\supported {getcomponents} \nop \yes \relax +\supported {getdata} \nop \yes \relax +\supported {getdepth} \nop \yes \relax +\supported {getdirection} \nop \yes \relax +\supported {getdisc} \nop \yes \relax +\supported {getexpansion} \nop \yes \relax +\supported {getfam} \nop \yes \relax +\supported {getfield} \yes \yes \relax +\supported {getfont} \nop \yes \relax +\supported {getglue} \nop \yes \yes +%supported {getglyphdata} \nop \yes \relax % experiment +\supported {getheight} \nop \yes \relax +\supported {getid} \nop \yes \relax +\supported {getkern} \nop \yes \relax +\supported {getlang} \nop \yes \relax +\supported {getleader} \nop \yes \relax +\supported {getlist} \nop \yes \relax +\supported {getnext} \nop \yes \relax +\supported {getnormalizedline} \nop \yes \relax +\supported {getnucleus} \nop \yes \relax +\supported {getoffsets} \nop \yes \relax +\supported {getorientation} \nop \yes \relax +\supported {getpenalty} \nop \yes \relax +\supported {getpost} \nop \yes \relax +\supported {getpre} \nop \yes \relax +\supported {getprev} \nop \yes \relax +\supported {getproperty} \yes \yes \relax +\supported {getreplace} \nop \yes \relax +\supported {getshift} \nop \yes \relax +\supported {getsub} \nop \yes \relax +\supported {getsubtype} \nop \yes \relax +\supported {getsup} \nop \yes \relax +\supported {getwhd} \nop \yes \relax +\supported {getwidth} \nop \yes \relax +\supported {has_attribute} \yes \yes \relax +\supported {has_dimensions} \nop \yes \relax +\supported {has_field} \yes \yes \relax +\supported {has_glyph} \nop \yes \yes +\supported {hpack} \nop \yes \yes +%supported {ignore_math_skip} \nop \yes \relax +\supported {insert_after} \yes \yes \relax +\supported {insert_before} \yes \yes \relax +\supported {is_char} \nop \yes \relax +\supported {is_direct} \nop \yes \relax +\supported {is_glyph} \nop \yes \relax +\supported {is_node} \yes \yes \relax +\supported {is_valid} \nop \yes \relax +\supported {is_zero_glue} \nop \yes \yes +\supported {kerning} \nop \yes \yes +\supported {last_node} \nop \yes \yes +\supported {length} \nop \yes \yes +\supported {ligaturing} \nop \yes \yes +\supported {make_extensible} \nop \yes \yes +\supported {mlist_to_hlist} \nop \yes \yes +\supported {naturalwidth} \nop \yes \yes +\supported {new} \yes \yes \relax +\supported {prepend_prevdepth} \nop \yes \yes +\supported {protect_glyphs} \nop \yes \yes +\supported {protect_glyph} \nop \yes \yes +\supported {protrusion_skippable} \nop \yes \yes +\supported {rangedimensions} \nop \yes \yes +\supported {remove} \yes \yes \relax +\supported {set_attribute} \yes \yes \relax +\supported {set_synctex_fields} \nop \yes \relax +\supported {setattributelist} \nop \yes \relax +\supported {setboth} \nop \yes \relax +\supported {setbox} \nop \yes \relax +\supported {setchar} \nop \yes \relax +\supported {setcomponents} \nop \yes \relax +\supported {setdata} \nop \yes \relax +\supported {setdepth} \nop \yes \relax +\supported {setdirection} \nop \yes \relax +\supported {setdisc} \nop \yes \relax +\supported {setexpansion} \nop \yes \relax +\supported {setfam} \nop \yes \relax +\supported {setfield} \yes \yes \relax +\supported {setfont} \nop \yes \relax +\supported {setglue} \yes \yes \relax +%supported {setglyphdata} \nop \yes \relax % experiment +\supported {setheight} \nop \yes \relax +\supported {setkern} \nop \yes \relax +\supported {setlang} \nop \yes \relax +\supported {setleader} \nop \yes \relax +\supported {setlink} \nop \yes \relax +\supported {setlist} \nop \yes \relax +\supported {setnext} \nop \yes \relax +\supported {setnucleus} \nop \yes \relax +\supported {setoffsets} \nop \yes \relax +\supported {setorientation} \nop \yes \relax +\supported {setpenalty} \nop \yes \relax +\supported {setprev} \nop \yes \relax +\supported {setproperty} \yes \yes \relax +\supported {setshift} \nop \yes \relax +\supported {setsplit} \nop \yes \relax +\supported {setsub} \nop \yes \relax +\supported {setsubtype} \nop \yes \relax +\supported {setsup} \nop \yes \relax +\supported {setwhd} \nop \yes \relax +\supported {setwidth} \nop \yes \relax +\supported {slide} \nop \yes \yes +\supported {start_of_par} \nop \yes \relax +\supported {subtype} \nop \nop \relax +\supported {tail} \yes \yes \relax +\supported {todirect} \nop \yes \relax +\supported {tonode} \nop \yes \relax +\supported {tostring} \yes \nop \relax +\supported {traverse} \yes \yes \relax +\supported {traverse_char} \yes \yes \relax +\supported {traverse_glyph} \yes \yes \relax +\supported {traverse_id} \yes \yes \relax +\supported {traverse_list} \yes \yes \relax +\supported {type} \yes \nop \relax +\supported {unprotect_glyphs} \nop \yes \yes +\supported {unprotect_glyph} \nop \yes \yes +\supported {unset_attribute} \yes \yes \relax +\supported {usedlist} \nop \yes \yes +\supported {uses_font} \nop \yes \yes +\supported {vpack} \nop \yes \yes +\supported {write} \yes \yes \relax +\LL +\stoptabulate + +The \type {node.next} and \type {node.prev} functions will stay but for +consistency there are variants called \type {getnext} and \type {getprev}. We had +to use \type {get} because \type {node.id} and \type {node.subtype} are already +taken for providing meta information about nodes. Note: The getters do only basic +checking for valid keys. You should just stick to the keys mentioned in the +sections that describe node properties. + +Some of the getters and setters handle multiple node types, given that the field +is relevant. In that case, some field names are considered similar (like \type +{kern} and \type {width}, or \type {data} and \type {value}. In retrospect we +could have normalized field names better but we decided to stick to the original +(internal) names as much as possible. After all, at the \LUA\ end one can easily +create synonyms. + +Some nodes have indirect references. For instance a math character refers to a +family instead of a font. In that case we provide a virtual font field as +accessor. So, \type {getfont} and \type {.font} can be used on them. The same is +true for the \type {width}, \type {height} and \type {depth} of glue nodes. These +actually access the spec node properties, and here we can set as well as get the +values. + +In some places \LUATEX\ can do a bit of extra checking for valid node lists and +you can enable that with: + +\startfunctioncall +node.fix_node_lists(<boolean> b) +\stopfunctioncall + +You can set and query the \SYNCTEX\ fields, a file number aka tag and a line +number, for a glue, kern, hlist, vlist, rule and math nodes as well as glyph +nodes (although this last one is not used in native \SYNCTEX). + +\startfunctioncall +node.set_synctex_fields(<integer> f, <integer> l) +<integer> f, <integer> l = + node.get_synctex_fields(<node> n) +\stopfunctioncall + +Of course you need to know what you're doing as no checking on sane values takes +place. Also, the synctex interpreter used in editors is rather peculiar and has +some assumptions (heuristics). + +\stopsection + +\startsection[title={Normalization}][library=node] + +As an experiment the lines resulting from paragraph construction can be normalized. +There are several modes, that can be set and queried with: + +\startfunctioncall +node.direct.setnormalize(<integer> n) +<integer> n = node.direct.getnormalize() +\stopfunctioncall + +The state of a line (a hlist) can be queried with: + +\startfunctioncall +<integer> leftskip, <integer> rightskip, + <integer> lefthangskip, <integer> righthangskip, + <node> head, <node> tail, + <integer> parindent, <integer> parfillskip = node.direct.getnormalized() +\stopfunctioncall + +The modes accumulate, so mode \type {4} includes \type {1} upto \type {3}: + +\starttabulate[|l|p|] +\DB value \BC explanation \NC \NR +\TB +\NC \type{1} \NC left and right skips and directions \NC \NR +\NC \type{2} \NC indentation and parfill skip \NC \NR +\NC \type{3} \NC hanging indentation and par shapes \NC \NR +\NC \type{4} \NC idem but before left and right skips \NC \NR +\NC \type{5} \NC inject compensation for overflow \NC \NR +\LL +\stoptabulate + +This is experimental code and might take a while to become frozen. + +\stopsection + +\startsection[title={Properties}][library=node] + +\topicindex {nodes+properties} +\topicindex {properties} + +\libindex{flush_properties_table} +\libindex{get_properties_table} +\libindex{set_properties_mode} + +Attributes are a convenient way to relate extra information to a node. You can +assign them at the \TEX\ end as well as at the \LUA\ end and and consult them at +the \LUA\ end. One big advantage is that they obey grouping. They are linked +lists and normally checking for them is pretty efficient, even if you use a lot +of them. A macro package has to provide some way to manage these attributes at the +\TEX\ end because otherwise clashes in their usage can occur. + +Each node also can have a properties table and you can assign values to this +table using the \type {setproperty} function and get properties using the \type +{getproperty} function. Managing properties is way more demanding than managing +attributes. + +Take the following example: + +\starttyping +\directlua { + local n = node.new("glyph") + + node.setproperty(n,"foo") + print(node.getproperty(n)) + + node.setproperty(n,"bar") + print(node.getproperty(n)) + + node.free(n) +} +\stoptyping + +This will print \type {foo} and \type {bar} which in itself is not that useful +when multiple mechanisms want to use this feature. A variant is: + +\starttyping +\directlua { + local n = node.new("glyph") + + node.setproperty(n,{ one = "foo", two = "bar" }) + print(node.getproperty(n).one) + print(node.getproperty(n).two) + + node.free(n) +} +\stoptyping + +This time we store two properties with the node. It really makes sense to have a +table as property because that way we can store more. But in order for that to +work well you need to do it this way: + +\starttyping +\directlua { + local n = node.new("glyph") + + local t = node.getproperty(n) + + if not t then + t = { } + node.setproperty(n,t) + end + t.one = "foo" + t.two = "bar" + + print(node.getproperty(n).one) + print(node.getproperty(n).two) + + node.free(n) +} +\stoptyping + +Here our own properties will not overwrite other users properties unless of +course they use the same keys. So, eventually you will end up with something: + +\starttyping +\directlua { + local n = node.new("glyph") + + local t = node.getproperty(n) + + if not t then + t = { } + node.setproperty(n,t) + end + t.myself = { one = "foo", two = "bar" } + + print(node.getproperty(n).myself.one) + print(node.getproperty(n).myself.two) + + node.free(n) +} +\stoptyping + +This assumes that only you use \type {myself} as subtable. The possibilities are +endless but care is needed. For instance, the generic font handler that ships +with \CONTEXT\ uses the \type {injections} subtable and you should not mess with +that one! + +There are a few helper functions that you normally should not touch as user: \typ +{flush_properties_table} will wipe the table (normally a bad idea), \typ +{get_properties_table} and will give the table that stores properties (using +direct entries) and you can best not mess too much with that one either because +\LUATEX\ itself will make sure that entries related to nodes will get wiped when +nodes get freed, so that the \LUA\ garbage collector can do its job. In fact, the +main reason why we have this mechanism is that it saves the user (or macro +package) some work. One can easily write a property mechanism in \LUA\ where +after a shipout properties gets cleaned up but it's not entirely trivial to make +sure that with each freed node also its properties get freed, due to the fact +that there can be nodes left over for a next page. And having a callback bound to +the node deallocator would add way to much overhead. + +When we copy a node list that has a table as property, there are several +possibilities: we do the same as a new node, we copy the entry to the table in +properties (a reference), we do a deep copy of a table in the properties, we +create a new table and give it the original one as a metatable. After some +experiments (that also included timing) with these scenarios we decided that a +deep copy made no sense, nor did nilling. In the end both the shallow copy and +the metatable variant were both ok, although the second one is slower. The most +important aspect to keep in mind is that references to other nodes in properties +no longer can be valid for that copy. We could use two tables (one unique and one +shared) or metatables but that only complicates matters. + +When defining a new node, we could already allocate a table but it is rather easy +to do that at the lua end e.g.\ using a metatable \type {__index} method. That +way it is under macro package control. When deleting a node, we could keep the +slot (e.g. setting it to false) but it could make memory consumption raise +unneeded when we have temporary large node lists and after that only small lists. +Both are not done. + +So in the end this is what happens now: when a node is copied, and it has a table +as property, the new node will share that table. If the second argument of \typ +{set_properties_mode} is \type {true} then a metatable approach is chosen: the +copy gets its own table with the original table as metatable. If you use the +generic font loader the mode is enabled that way. + +A few more xperiments were done. For instance: copy attributes to the properties +so that we have fast access at the \LUA\ end. In the end the overhead is not +compensated by speed and convenience, in fact, attributes are not that slow when +it comes to accessing them. So this was rejected. + +Another experiment concerned a bitset in the node but again the gain compared to +attributes was neglectable and given the small amount of available bits it also +demands a pretty strong agreement over what bit represents what, and this is +unlikely to succeed in the \TEX\ community. It doesn't pay off. + +Just in case one wonders why properties make sense: it is not so much speed that +we gain, but more convenience: storing all kinds of (temporary) data in attributes +is no fun and this mechanism makes sure that properties are cleaned up when a +node is freed. Also, the advantage of a more or less global properties table is +that we stay at the \LUA\ end. An alternative is to store a reference in the node +itself but that is complicated by the fact that the register has some limitations +(no numeric keys) and we also don't want to mess with it too much. + +\stopsection + +\stopchapter + +\stopcomponent |