summaryrefslogtreecommitdiff
path: root/Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex
diff options
context:
space:
mode:
Diffstat (limited to 'Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex')
-rw-r--r--Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex853
1 files changed, 853 insertions, 0 deletions
diff --git a/Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex b/Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex
new file mode 100644
index 00000000000..4127f93300b
--- /dev/null
+++ b/Master/texmf-dist/doc/luatex/base/luatex-enhancements.tex
@@ -0,0 +1,853 @@
+% language=uk
+
+\environment luatex-style
+\environment luatex-logos
+
+\startcomponent luatex-enhancements
+
+\startchapter[reference=enhancements,title={Basic \TEX\ enhancements}]
+
+\section{Introduction}
+
+From day one, \LUATEX\ has offered extra features compared to the superset of
+\PDFTEX\ and \ALEPH. This has not been limited to the possibility to execute
+\LUA\ code via \type {\directlua}, but \LUATEX\ also adds functionality via new
+\TEX|-|side primitives or extensions to existing ones.
+
+When \LUATEX\ starts up in \quote {iniluatex} mode (\type {luatex -ini}), it
+defines only the primitive commands known by \TEX82 and the one extra command
+\type {\directlua}. As is fitting, a \LUA\ function has to be called to add the
+extra primitives to the user environment. The simplest method to get access to
+all of the new primitive commands is by adding this line to the format generation
+file:
+
+\starttyping
+\directlua { tex.enableprimitives('',tex.extraprimitives()) }
+\stoptyping
+
+But be aware that the curly braces may not have the proper \type {\catcode}
+assigned to them at this early time (giving a \quote {Missing number} error), so
+it may be needed to put these assignments before the above line:
+
+\starttyping
+\catcode `\{=1
+\catcode `\}=2
+\stoptyping
+
+More fine|-|grained primitives control is possible and you can look up the
+details in \in {section} [luaprimitives]. For simplicity's sake, this manual
+assumes that you have executed the \type {\directlua} command as given above.
+
+The startup behaviour documented above is considered stable in the sense that
+there will not be backward|-|incompatible changes any more. We have promoted some
+rather generic \PDFTEX\ primitives to core \LUATEX\ ones, and the ones inherited
+frome \ALEPH\ (\OMEGA) are also promoted. Effectively this means that we now only
+have the \type {tex}, \type {etex} and \type {luatex} sets left.
+
+In \in {Chapter} [modifications] we discuss several primitives that are derived
+from \PDFTEX\ and \ALEPH\ (\OMEGA). Here we stick to real new ones. In the
+chapters on fonts and math we discuss a few more new ones.
+
+\section{Version information}
+
+There are three new primitives to test the version of \LUATEX:
+
+\starttabulate[|l|pl|pl|]
+\NC \bf primitive \NC \bf explanation \NC \bf value \NC \NR
+\NC \type {\luatexbanner} \NC the banner reported on the command line \NC \luatexbanner \NC \NR
+\NC \type {\luatexversion} \NC a combination of major and minor number \NC \the\luatexversion \NC \NR
+\NC \type {\luatexrevision} \NC the revision number, the current value is \NC \luatexrevision \NC \NR
+\stoptabulate
+
+The official \LUATEX\ version is defined as follows:
+
+\startitemize
+\startitem
+ The major version is the integer result of \type {\luatexversion} divided by
+ 100. The primitive is an \quote {internal variable}, so you may need to prefix
+ its use with \type {\the} depending on the context.
+\stopitem
+\startitem
+ The minor version is the two-digit result of \type {\luatexversion} modulo 100.
+\stopitem
+\startitem
+ The revision is the given by \type {\luatexrevision}. This primitive expands to
+ a positive integer.
+\stopitem
+\startitem
+ The full version number consists of the major version, minor version and
+ revision, separated by dots.
+\stopitem
+\stopitemize
+
+\section{\UNICODE\ text support}
+
+Text input and output is now considered to be \UNICODE\ text, so input characters
+can use the full range of \UNICODE\ ($2^{20}+2^{16}-1 = \hbox{0x10FFFF}$). Later
+chapters will talk of characters and glyphs. Although these are not
+interchangeable, they are closely related. During typesetting, a character is
+always converted to a suitable graphic representation of that character in a
+specific font. However, while processing a list of to|-|be|-|typeset nodes, its
+contents may still be seen as a character. Inside \LUATEX\ there is no clear
+separation between the two concepts. Because the subtype of a glyph node can be
+changed in \LUA\ it is up to the user: subtypes larger than 255 indicate that
+font processing has happened.
+
+A few primitives are affected by this, all in a similar fashion: each of them has
+to accommodate for a larger range of acceptable numbers. For instance, \type
+{\char} now accepts values between~0 and $1{,}114{,}111$. This should not be a
+problem for well|-|behaved input files, but it could create incompatibilities for
+input that would have generated an error when processed by older \TEX|-|based
+engines. The affected commands with an altered initial (left of the equals sign)
+or secondary (right of the equals sign) value are: \type {\char}, \type
+{\lccode}, \type {\uccode}, \type {\catcode}, \type {\sfcode}, \type {\efcode},
+\type {\lpcode}, \type {\rpcode}, \type {\chardef}.
+
+As far as the core engine is concerned, all input and output to text files is
+\UTF-8 encoded. Input files can be pre|-|processed using the \type {reader}
+callback. This will be explained in a later chapter.
+
+Output in byte|-|sized chunks can be achieved by using characters just outside of
+the valid \UNICODE\ range, starting at the value $1{,}114{,}112$ (0x110000). When
+the time comes to print a character $c>=1{,}114{,}112$, \LUATEX\ will actually
+print the single byte corresponding to $c$ minus 1{,}114{,}112.
+
+Output to the terminal uses \type {^^} notation for the lower control range
+($c<32$), with the exception of \type {^^I}, \type {^^J} and \type {^^M}. These
+are considered \quote {safe} and therefore printed as|-|is. You can disable
+escaping with \type {texio.setescape(false)} in which case you get the normal
+characters on the console.
+
+Normalization of the \UNICODE\ input can be handled by a macro package during
+callback processing (this will be explained in \in {section} [iocallback]).
+
+\section{Extended tables}
+
+All traditional \TEX\ and \ETEX\ registers can be 16-bit numbers. The affected
+commands are:
+
+\startfourcolumns
+\starttyping
+\count
+\dimen
+\skip
+\muskip
+\marks
+\toks
+\countdef
+\dimendef
+\skipdef
+\muskipdef
+\toksdef
+\insert
+\box
+\unhbox
+\unvbox
+\copy
+\unhcopy
+\unvcopy
+\wd
+\ht
+\dp
+\setbox
+\vsplit
+\stoptyping
+\stopfourcolumns
+
+Because font memory management has been rewritten, character properties in fonts
+are no longer shared among fonts instances that originate from the same metric
+file.
+
+\section{Attributes}
+
+\subsection{Attribute registers}
+
+Attributes are a completely new concept in \LUATEX. Syntactically, they behave a
+lot like counters: attributes obey \TEX's nesting stack and can be used after
+\type {\the} etc.\ just like the normal \type {\count} registers.
+
+\startsyntax
+\attribute <16-bit number> <optional equals> <32-bit number>!crlf
+\attributedef <csname> <optional equals> <16-bit number>
+\stopsyntax
+
+Conceptually, an attribute is either \quote {set} or \quote {unset}. Unset
+attributes have a special negative value to indicate that they are unset, that
+value is the lowest legal value: \type {-"7FFFFFFF} in hexadecimal, a.k.a.
+$-2147483647$ in decimal. It follows that the value \type {-"7FFFFFFF} cannot be
+used as a legal attribute value, but you {\it can\/} assign \type {-"7FFFFFFF} to
+\quote {unset} an attribute. All attributes start out in this \quote {unset}
+state in \INITEX.
+
+Attributes can be used as extra counter values, but their usefulness comes mostly
+from the fact that the numbers and values of all \quote {set} attributes are
+attached to all nodes created in their scope. These can then be queried from any
+\LUA\ code that deals with node processing. Further information about how to use
+attributes for node list processing from \LUA\ is given in~\in {chapter}[nodes].
+
+Attributes are stored in a sorted (sparse) linked list that are shared when
+possible. This permits efficient testing and updating.
+
+\subsection{Box attributes}
+
+Nodes typically receive the list of attributes that is in effect when they are
+created. This moment can be quite asynchronous. For example: in paragraph
+building, the individual line boxes are created after the \type {\par} command has
+been processed, so they will receive the list of attributes that is in effect
+then, not the attributes that were in effect in, say, the first or third line of
+the paragraph.
+
+Similar situations happen in \LUATEX\ regularly. A few of the more obvious
+problematic cases are dealt with: the attributes for nodes that are created
+during hyphenation, kerning and ligaturing borrow their attributes from their
+surrounding glyphs, and it is possible to influence box attributes directly.
+
+When you assemble a box in a register, the attributes of the nodes contained in
+the box are unchanged when such a box is placed, unboxed, or copied. In this
+respect attributes act the same as characters that have been converted to
+references to glyphs in fonts. For instance, when you use attributes to implement
+color support, each node carries information about its eventual color. In that
+case, unless you implement mechanisms that deal with it, applying a color to
+already boxed material will have no effect. Keep in mind that this
+incompatibility is mostly due to the fact that separate specials and literals are
+a more unnatural approach to colors than attributes.
+
+It is possible to fine-tune the list of attributes that are applied to a \type
+{hbox}, \type {vbox} or \type {vtop} by the use of the keyword \type {attr}. An
+example:
+
+\starttyping
+\attribute2=5
+\setbox0=\hbox {Hello}
+\setbox2=\hbox attr1=12 attr2=-"7FFFFFFF{Hello}
+\stoptyping
+
+This will set the attribute list of box~2 to $1=12$, and the attributes of box~0
+will be $2=5$. As you can see, assigning the maximum negative value causes an
+attribute to be ignored.
+
+The \type {attr} keyword(s) should come before a \type {to} or \type {spread}, if
+that is also specified.
+
+\section{\LUA\ related primitives}
+
+\subsection{\type {\directlua}}
+
+In order to merge \LUA\ code with \TEX\ input, a few new primitives are needed.
+The primitive \type {\directlua} is used to execute \LUA\ code immediately. The
+syntax is
+
+\startsyntax
+\directlua <general text>!crlf
+\directlua <16-bit number> <general text>
+\stopsyntax
+
+The \syntax {<general text>} is expanded fully, and then fed into the \LUA\
+interpreter. After reading and expansion has been applied to the \syntax
+{<general text>}, the resulting token list is converted to a string as if it was
+displayed using \type {\the\toks}. On the \LUA\ side, each \type {\directlua}
+block is treated as a separate chunk. In such a chunk you can use the \type
+{local} directive to keep your variables from interfering with those used by the
+macro package.
+
+The conversion to and from a token list means that you normally can not use \LUA\
+line comments (starting with \type {--}) within the argument. As there typically
+will be only one \quote {line} the first line comment will run on until the end
+of the input. You will either need to use \TEX|-|style line comments (starting
+with \%), or change the \TEX\ category codes locally. Another possibility is to
+say:
+
+\starttyping
+\begingroup
+\endlinechar=10
+\directlua ...
+\endgroup
+\stoptyping
+
+Then \LUA\ line comments can be used, since \TEX\ does not replace line endings
+with spaces.
+
+Likewise, the \syntax {<16-bit number>} designates a name of a \LUA\ chunk and is
+taken from the \type {lua.name} array (see the documentation of the \type {lua}
+table further in this manual). When a chunk name starts with a \type {@} it will
+be displayed as a file name. This is a side effect of the way \LUA\ implements
+error handling.
+
+The \type {\directlua} command is expandable. Since it passes \LUA\ code to the
+\LUA\ interpreter its expansion from the \TEX\ viewpoint is usually empty.
+However, there are some \LUA\ functions that produce material to be read by \TEX,
+the so called print functions. The most simple use of these is \type
+{tex.print(<string> s)}. The characters of the string \type {s} will be placed on
+the \TEX\ input buffer, that is, \quote {before \TEX's eyes} to be read by \TEX\
+immediately. For example:
+
+\startbuffer
+\count10=20
+a\directlua{tex.print(tex.count[10]+5)}b
+\stopbuffer
+
+\typebuffer
+
+expands to
+
+\getbuffer
+
+Here is another example:
+
+\startbuffer
+$\pi = \directlua{tex.print(math.pi)}$
+\stopbuffer
+
+\typebuffer
+
+will result in
+
+\getbuffer
+
+Note that the expansion of \type {\directlua} is a sequence of characters, not of
+tokens, contrary to all \TEX\ commands. So formally speaking its expansion is
+null, but it places material on a pseudo-file to be immediately read by \TEX, as
+\ETEX's \type {\scantokens}. For a description of print functions look at \in
+{section} [sec:luaprint].
+
+Because the \syntax {<general text>} is a chunk, the normal \LUA\ error handling
+is triggered if there is a problem in the included code. The \LUA\ error messages
+should be clear enough, but the contextual information is still pretty bad.
+Often, you will only see the line number of the right brace at the end of the
+code.
+
+While on the subject of errors: some of the things you can do inside \LUA\ code
+can break up \LUATEX\ pretty bad. If you are not careful while working with the
+node list interface, you may even end up with assertion errors from within the
+\TEX\ portion of the executable.
+
+The behaviour documented in the above subsection is considered stable in the sense
+that there will not be backward-incompatible changes any more.
+
+\subsection{\type {\latelua}}
+
+Contrary to \type {\directlua}, \type {\latelua} stores \LUA\ code in a whatsit
+that will be processed at the time of shipping out. Its intended use is a cross
+between \PDF\ literals (often available as \type {\pdfliteral}) and the
+traditional \TEX\ extension \type {\write}. Within the \LUA\ code you can print
+\PDF\ statements directly to the \PDF\ file via \type {pdf.print}, or you can
+write to other output streams via \type {texio.write} or simply using \LUA\ \IO\
+routines.
+
+\startsyntax
+\latelua <general text>!crlf
+\latelua <16-bit number> <general text>
+\stopsyntax
+
+Expansion of macros in the final \type {<general text>} is delayed until just
+before the whatsit is executed (like in \type {\write}). With regard to \PDF\
+output stream \type {\latelua} behaves as \PDF\ page literals. The \syntax
+{name <general text>} and \syntax {<16-bit number>} behave in the same way as
+they do for \type {\directlua}
+
+\subsection{\type {\luaescapestring}}
+
+This primitive converts a \TEX\ token sequence so that it can be safely used as
+the contents of a \LUA\ string: embedded backslashes, double and single quotes,
+and newlines and carriage returns are escaped. This is done by prepending an
+extra token consisting of a backslash with category code~12, and for the line
+endings, converting them to \type {n} and \type {r} respectively. The token
+sequence is fully expanded.
+
+\startsyntax
+\luaescapestring <general text>
+\stopsyntax
+
+Most often, this command is not actually the best way to deal with the
+differences between the \TEX\ and \LUA. In very short bits of \LUA\
+code it is often not needed, and for longer stretches of \LUA\ code it
+is easier to keep the code in a separate file and load it using \LUA's
+\type {dofile}:
+
+\starttyping
+\directlua { dofile('mysetups.lua') }
+\stoptyping
+
+\subsection{\type {\luafunction}}
+
+The \type {\directlua} commands involves tokenization of its argument (after
+picking up an optional name or number specification). The tokenlist is then
+converted into a string and given to \LUA\ to turn into a function that is
+called. The overhead is rather small but when you use this primitive hundreds of
+thousands of times, it can become noticeable. For this reason there is a variant
+call available: \type {\luafunction}. This command is used as follows:
+
+\starttyping
+\directlua {
+ local t = lua.get_functions_table()
+ t[1] = function() tex.print("!") end
+ t[2] = function() tex.print("?") end
+}
+
+\luafunction1
+\luafunction2
+\stoptyping
+
+Of course the functions can also be defined in a separate file. There is no limit
+on the number of functions apart from normal \LUA\ limitations. Of course there
+is the limitation of no arguments but that would involve parsing and thereby give
+no gain. The function, when called in fact gets one argument, being the index, so
+in the following example the number \type {8} gets typeset.
+
+\starttyping
+\directlua {
+ local t = lua.get_functions_table()
+ t[8] = function(slot) tex.print(slot) end
+}
+\stoptyping
+
+\section{\type {\clearmarks}}
+
+This primitive complements the \ETEX\ mark primitives and clears a mark class
+completely, resetting all three connected mark texts to empty. It is an
+immediate command.
+
+\startsyntax
+\clearmarks <16-bit number>
+\stopsyntax
+
+\section{\type {\noligs} and \type {\nokerns}}
+
+These primitives prohibit ligature and kerning insertion at the time when the
+initial node list is built by \LUATEX's main control loop. You can enable these
+primitives when you want to do node list processing of \quote {characters}, where
+\TEX's normal processing would get in the way.
+
+\startsyntax
+\noligs <integer>!crlf
+\nokerns <integer>
+\stopsyntax
+
+These primitives can also be implemented by overloading the ligature building and
+kerning functions, i.e.\ by assigning dummy functions to their associated
+callbacks. Keep in mind that when you define a font (using \LUA) you can also
+omit the kern and ligature tables, which has the same effect as the above.
+
+\section{\type {\formatname}}
+
+The \type {\formatname} syntax is identical to \type {\jobname}. In \INITEX, the
+expansion is empty. Otherwise, the expansion is the value that \type {\jobname} had
+during the \INITEX\ run that dumped the currently loaded format.
+
+\section{\type {\scantextokens}}
+
+The syntax of \type {\scantextokens} is identical to \type {\scantokens}. This
+primitive is a slightly adapted version of \ETEX's \type {\scantokens}. The
+differences are:
+
+\startitemize
+\startitem
+ The last (and usually only) line does not have a \type {\endlinechar}
+ appended.
+\stopitem
+\startitem
+ \type {\scantextokens} never raises an EOF error, and it does not execute
+ \type {\everyeof} tokens.
+\stopitem
+\startitem
+ There are no \quote {\unknown\ while end of file \unknown} error tests
+ executed. This allows the expansion to end on a different grouping level or
+ while a conditional is still incomplete.
+\stopitem
+\stopitemize
+
+\section {Alignments}
+
+\subsection{\tex {alignmark}}
+
+This primitive duplicates the functionality of \type {#} inside alignment
+preambles.
+
+\subsection{\tex {aligntab}}
+
+This primitive duplicates the functionality of \type {&} inside alignments and
+preambles.
+
+\section{Catcode tables}
+
+Catcode tables are a new feature that allows you to switch to a predefined
+catcode regime in a single statement. You can have a practically unlimited number
+of different tables. This subsystem is backward compatible: if you never use the
+following commands, your document will not notice any difference in behaviour
+compared to traditional \TEX. The contents of each catcode table is independent
+from any other catcode tables, and their contents is stored and retrieved from
+the format file.
+
+\subsection{\type {\catcodetable}}
+
+\startsyntax
+\catcodetable <15-bit number>
+\stopsyntax
+
+The primitive \type {\catcodetable} switches to a different catcode table. Such a
+table has to be previously created using one of the two primitives below, or it
+has to be zero. Table zero is initialized by \INITEX.
+
+\subsection{\type {\initcatcodetable}}
+
+\startsyntax
+\initcatcodetable <15-bit number>
+\stopsyntax
+
+The primitive \type {\initcatcodetable} creates a new table with catcodes identical
+to those defined by \INITEX:
+
+\starttabulate[|r|l|l|l|]
+\NC 0 \NC \tttf \letterbackslash \NC \NC \type {escape} \NC\NR
+\NC 5 \NC \tttf \letterhat\letterhat M \NC return \NC \type {car_ret} \NC\NR
+\NC 9 \NC \tttf \letterhat\letterhat @ \NC null \NC \type {ignore} \NC\NR
+\NC 10 \NC \tttf <space> \NC space \NC \type {spacer} \NC\NR
+\NC 11 \NC {\tttf a} \endash\ {\tttf z} \NC \NC \type {letter} \NC\NR
+\NC 11 \NC {\tttf A} \endash\ {\tttf Z} \NC \NC \type {letter} \NC\NR
+\NC 12 \NC everything else \NC \NC \type {other} \NC\NR
+\NC 14 \NC \tttf \letterpercent \NC \NC \type {comment} \NC\NR
+\NC 15 \NC \tttf \letterhat\letterhat ? \NC delete \NC \type {invalid_char} \NC\NR
+\stoptabulate
+
+The new catcode table is allocated globally: it will not go away after the
+current group has ended. If the supplied number is identical to the currently
+active table, an error is raised.
+
+\subsection{\type {\savecatcodetable}}
+
+\startsyntax
+\savecatcodetable <15-bit number>
+\stopsyntax
+
+\type {\savecatcodetable} copies the current set of catcodes to a new table with
+the requested number. The definitions in this new table are all treated as if
+they were made in the outermost level.
+
+The new table is allocated globally: it will not go away after the current group
+has ended. If the supplied number is the currently active table, an error is
+raised.
+
+\section{Suppressing errors}
+
+\subsection{\type {\suppressfontnotfounderror}}
+
+\startsyntax
+\suppressfontnotfounderror = 1
+\stopsyntax
+
+If this integer parameter is non|-|zero, then \LUATEX\ will not complain about
+font metrics that are not found. Instead it will silently skip the font
+assignment, making the requested csname for the font \type {\ifx} equal to \type
+{\nullfont}, so that it can be tested against that without bothering the user.
+
+\subsection{\type {\suppresslongerror}}
+
+\startsyntax
+\suppresslongerror = 1
+\stopsyntax
+
+If this integer parameter is non|-|zero, then \LUATEX\ will not complain about
+\type {\par} commands encountered in contexts where that is normally prohibited
+(most prominently in the arguments of non-long macros).
+
+\subsection{\type {\suppressifcsnameerror}}
+
+\startsyntax
+\suppressifcsnameerror = 1
+\stopsyntax
+
+If this integer parameter is non|-|zero, then \LUATEX\ will not complain about
+non-expandable commands appearing in the middle of a \type {\ifcsname} expansion.
+Instead, it will keep getting expanded tokens from the input until it encounters
+an \type {\endcsname} command. If the input expansion is unbalanced with respect
+to \type {\csname} \ldots \type {\endcsname} pairs, the \LUATEX\ process may hang
+indefinitely.
+
+\subsection{\type {\suppressoutererror}}
+
+\startsyntax
+\suppressoutererror = 1
+\stopsyntax
+
+If this new integer parameter is non|-|zero, then \LUATEX\ will not complain
+about \type {\outer} commands encountered in contexts where that is normally
+prohibited.
+
+\subsection{\type {\suppressmathparerror}}
+
+The following setting will permit \type {\par} tokens in a math formula:
+
+\startsyntax
+\suppressmathparerror = 1
+\stopsyntax
+
+So, the next code is valid then:
+
+\starttyping
+$ x + 1 =
+
+a $
+\stoptyping
+
+\section{\type {\matheqnogapstep}}
+
+By default \TEX\ will add one quad between the equation and the number. This is
+hard coded. A new primitive can control this:
+
+\startsyntax
+\matheqnogapstep = 1000
+\stopsyntax
+
+Because a math quad from the math text font is used instead of a dimension, we
+use a step to control the size. A value of zero will suppress the gap. The step
+is divided by 1000 which is the usual way to mimmick floating point factors in
+\TEX.
+
+\section{\type {\outputbox}}
+
+\startsyntax
+\outputbox = 65535
+\stopsyntax
+
+This new integer parameter allows you to alter the number of the box that will be
+used to store the page sent to the output routine. Its default value is 255, and
+the acceptable range is from 0 to 65535.
+
+\section{\type {\fontid} and \type {\setfontid}}
+
+\startsyntax
+\fontid\font
+\stopsyntax
+
+This primitive expands into a number. It is not a register so there is no need to
+prefix with \type {\number} (and using \type {\the} gives an error). The currently
+used font id is \fontid\font. Here are some more:
+
+\starttabulate[|l|c|]
+\NC \type {\bf} \NC \bf \fontid\font \NC \NR
+\NC \type {\it} \NC \it \fontid\font \NC \NR
+\NC \type {\bi} \NC \bi \fontid\font \NC \NR
+\stoptabulate
+
+These numbers depend on the macro package used because each one has its own way
+of dealing with fonts. They can also differ per run, as they can depend on the
+order of loading fonts. For instance, when in \CONTEXT\ virtual math \UNICODE\
+fonts are used, we can easily get over a hundred ids in use. Not all ids have to
+be bound to a real font, after all it's just a number.
+
+The primitive \type {\setfontid} can be used to enable a font with the given id
+(which of course needs to be a valid one).
+
+\section{\type {\gleaders}}
+
+This type of leaders is anchored to the origin of the box to be shipped out. So
+they are like normal \type {\leaders} in that they align nicely, except that the
+alignment is based on the {\it largest\/} enclosing box instead of the {\it
+smallest\/}. The \type {g} stresses this global nature.
+
+\section{\type {\nohrule} and \type {\novrule}}
+
+Because internally box resources and image resources are now stored as a special
+kind of rule, we also introduced an empty rule variant. Because introducing a new
+keyword can cause incompatibilities, two new primitives were introduced: \type
+{\nohrule} and \type {\novrule}. These can be used to reserve space. This is
+often more efficient than creating an empty box with fake dimensions).
+
+\section{\type {\Uchar}}
+
+The expandable command \type {\Uchar} reads a number between~0 and $1{,}114{,}111$
+and expands to the associated \UNICODE\ character.
+
+\section{\type {\hyphenationmin}}
+
+This primitive can be used to set the minimal word length, so setting it to a value
+of~$5$ means that only words of 6 characters and more will be hyphenated, of course
+within the constraints of the \type {\lefthyphenmin} and \type {\righthyphenmin}
+values (as stored in the glyph node). This primitive accepts a number and stores
+the value with the language.
+
+\section{\type {\boundary}, \type {\noboundary}, \type {\protrusionboundary} and \type
+{\wordboundary}}
+
+The \type {\noboundary} commands used to inject a whatsit node but now injects a normal
+node with type \type {boundary} and subtype~0. In addition you can say:
+
+\starttyping
+x\boundary 123\relax y
+\stoptyping
+
+This has the same effect but the subtype is now~1 and the value~123 is stored.
+The traditional ligature builder still sees this as a cancel boundary directive
+but at the \LUA\ end you can implement different behaviour. The added benefit of
+passing this value is a side effect of the generalization. The subtypes~2 and~3
+are used to control protrusion and word boundaries in hyphenation.
+
+\section{\type {\vpack}, \type {\hpack} and \type {\tpack}}
+
+These three primitives are like \type {\vbox}, \type {\hbox} and \type {\vtop}
+but don't apply the related callbacks.
+
+\section{\type {\csstring}, \type {\begincsname} and \type {\lastnamedcs}}
+
+These are somewhat special. The \type {\csstring} primitive is like
+\type {\string} but it omits the leading escape character. This can be
+somewhat more efficient that stripping it of afterwards.
+
+The \type {\begincsname} primitive is like \type {\csname} but doesn't create
+a relaxed equivalent when there is no such name. It is equivalent to
+
+\starttyping
+\ifcsname foo\endcsname
+ \csname foo\endcsname
+\fi
+\stoptyping
+
+The advantage is that it saves a lookup (don't expect much speedup) but more
+important is that it avoids using the \type {\if}.
+
+The \type {\lastnamedcs} is one that should be used with care. The above
+example could be written as:
+
+\starttyping
+\ifcsname foo\endcsname
+ \lastnamedcs
+\fi
+\stoptyping
+
+This is slightly more efficient than constructing the string twice (deep down in
+\LUATEX\ this also involves some \UTF8 juggling), but probably more relevant is
+that it saves a few tokens and can make code a bit more more readable.
+
+\section{\type {\toksapp}, \type {\tokspre}, \type {\etoksapp} and \type {\etokspre}}
+
+Instead of:
+
+\starttyping
+\toks0\expandafter{\the\toks0 foo}
+\stoptyping
+
+you can use:
+
+\starttyping
+\etoksapp0{foo}
+\stoptyping
+
+The \type {pre} variants prepend instead of append, and the \type {e} variants
+expand the passed general text.
+
+\section{Debugging}
+
+If \type {\tracingonline} is larger than~2, the node list display will also print
+the node number of the nodes.
+
+\section{Images and Forms}
+
+These two concepts are now core concepts and no longer whatsits. They are in fact
+now implemented as rules with special properties. Normal rules have subtype~0,
+saved boxes have subtype~1 and images have subtype~2. This has the positive side
+effect that whenever we need to take content with dimensions into account, when we
+look at rule nodes, we automatically also deal with these two types.
+
+The syntax of the \type {\save...resource} is the same as in \PDFTEX\ but you
+should consider them to be backend specific. This means that a macro package
+should treat them as such and check for the current output mode if applicable.
+Here are the equivalents:
+
+\starttabulate[|l|l|]
+\NC \type {\saveboxresource} \EQ \type {\pdfxform} \NC \NR
+\NC \type {\saveimageresource} \EQ \type {\pdfximage} \NC \NR
+\NC \type {\useboxresource} \EQ \type {\pdfrefxform} \NC \NR
+\NC \type {\useimageresource} \EQ \type {\pdfrefximage} \NC \NR
+\NC \type {\lastsavedboxresourceindex} \EQ \type {\pdflastxform} \NC \NR
+\NC \type {\lastsavedimageresourceindex} \EQ \type {\pdflastximage} \NC \NR
+\NC \type {\lastsavedimageresourcepages} \EQ \type {\pdflastximagepages} \NC \NR
+\stoptabulate
+
+\LUATEX\ accepts optional dimension parameters for \type {\use...resource} in the
+same format as for rules. With images, these dimensions are then used instead of
+the ones given to \type {\useimageresource} but the original dimensions are not
+overwritten, so that a \type {\useimageresource} without dimensions still
+provides the image with dimensions defined by \type {\saveimageresource}. These
+optional parameters are not implemented for \type {\saveboxresource}.
+
+\starttyping
+\pdfrefximage width 20mm height 10mm depth 5mm \pdflastximage
+\pdfrefxform width 20mm height 10mm depth 5mm \pdflastxform
+\stoptyping
+
+\section{\type {\outputmode} and \type {\draftmode}}
+
+The \type {\outputmode} variable tells \LUATEX\ what it has to produce:
+
+\starttabulate[|l|l|]
+\NC \type {0} \NC \DVI\ code \NC \NR
+\NC \type {1} \NC \PDF\ code \NC \NR
+\stoptabulate
+
+The value of the \type {\draftmode} counter signals the backend if it should
+output less. The \PDF\ backend accepts a value of~$1$, while the \DVI\ backend
+ignores the value.
+
+\section{File syntax}
+
+\LUATEX\ will accept a braced argument as a file name:
+
+\starttyping
+\input {plain}
+\openin 0 {plain}
+\stoptyping
+
+This allows for embedded spaces, without the need for double quotes. Macro
+expansion takes place inside the argument.
+
+\section{Font syntax}
+
+\LUATEX\ will accept a braced argument as a font name:
+
+\starttyping
+\font\myfont = {cmr10}
+\stoptyping
+
+This allows for embedded spaces, without the need for double quotes. Macro
+expansion takes place inside the argument.
+
+\section{Writing to file}
+
+You can now open upto 127 files with \type {\openout}. When no file is open
+writes will go to the console and log. As a consequence a system command is
+no longer possible but one can use \type {os.execute} to do the same.
+
+\section{\type{\nospaces}}
+
+This new primitive can be used to overrule the usual \type {\spaceskip}
+related heuristics when a space character is seen in a text flow. The
+value~\type{1} triggers no injection while \type{2} results in injection of
+a zero skip. Below we see the results for four characters separated by a
+space.
+
+\startlinecorrection
+\startcombination[3*2]
+ {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 10mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 10mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 10mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 1mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 1mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 1mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 1mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 1mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 1mm}}
+\stopcombination
+\stoplinecorrection
+
+\section{\type{\letcharcode}}
+
+This primitive is still experimental but can be used to assign a meaning to an active
+character, as in:
+
+\starttyping
+\def\foo{bar} \letcharcode123\foo
+\stoptyping
+
+This can be a bit nicer that using the uppercase tricks (using the property of
+\type {\uppercase} that it treats active characters special).
+
+\stopchapter
+
+\stopcomponent