summaryrefslogtreecommitdiff
path: root/Master/texmf-dist/doc/latex/lualatex-doc/lualatex-doc.tex
blob: e28ec7619a090efe410225e9ffe67de6c8ed5399 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
% lualatex-doc: a guide to LuaLaTeX
%
% Written by Manuel Pégourié-Gonnard <mpg@elzevir.fr>, 2010.
%
% Distributed under the terms of the GNU free documentation licence:
%   http://www.gnu.org/licenses/fdl.html
% without any invariant section or cover text.

\documentclass{lltxdoc}

\title{A guide to \lualatex}
\author{Manuel Pégourié-Gonnard \email{mpg@elzevir.fr}}
\date{\today}

\begin{document}

\maketitle

\begin{abstract}
  This document is a map, or touristic guide, for the new world of
  \lualatex.\footnote{Though focusing on \lualatex, it includes relevant
    information about \luatex with the Plain format, too.} The intended
  audience ranges from complete newcomers (with a working knowledge of
  conventional \latex) to package developers. This guide is intended to be
  comprehensive in the following sense: it contains pointers to all relevant
  sources, gathers information that is otherwise scattered, and adds
  introductory material.

  Feedback and suggestions for improvement are most welcome.
\end{abstract}

\vspace{\stretch{1}}
\setcounter{tocdepth}{2}
\listoftoc*{toc}
\vspace*{\stretch{2}}
\clearpage


\section{Introduction}\label{intro}

\subsection{Just what is \lualatex?}\label{what}

To answer this question, we need to mention a few details about the \tex world
that you may usually ignore: the difference between an \emph{engine} and a
\emph{format}. An engine is an actual computer program, while a format is a
set of macros executed by an engine, usually preloaded when the engine is
invoked with a particular name.

Actually, a format is more or less like a document class or a package, except
it is associated with a particular command name. Imagine there is a command
\cmd{latex-article} that would do the same as \cmd{latex}, except you wouldn't
need to say ©\documentclass{article}© at the beginning of your file.
Similarly, in current distributions, the command \cmd{pdflatex} is the same as
the command \cmd{pdftex} except that you don't need to put the instructions to
load \latex at the beginning of your source file. This is convenient, and
slightly more efficient too.

Formats are great because they implement powerful commands using the basic
tools an engine provides.  However the power of the format is sometimes
limited by the engine's tools set, so people started developing more
powerful engines so that other people can implement even more powerful formats
(or packages). The most famous engines now (except the original \tex) are
\pdftex, \xetex and \luatex.

To further complicate the picture, the original \tex engine produced only DVI
files, while its successors may (also) produce PDF file.  Each command in your
system corresponds to a particular engine with a particular format and a
particular output mode. The following table summarizes this: rows are indexed
by format, columns by engine, and in each cell the first line is the command
for this engine with this format in DVI mode, and the second for the same PDF
mode.

\begin{center}
  \newcommand*\cell [2] {%
    \parbox{6em}{\centering\leavevmode\color{code}\ttfamily
      \strut\maybe{#1} \\ \strut\maybe{#2}}}
  \makeatletter
  \newcommand*\maybe [1] {%
    \@ifmtarg{#1} {\textcolor{gray}{\normalfont (none)}} {#1}}
  \begin{tabular}{l|cccc}
                & \tex & \pdftex & \xetex & \luatex
    \\ \hline
    Plain
                &  \cell{tex}{}
                &  \cell{etex}{pdftex}
                &  \cell{}{xetex}
                &  \cell{dviluatex}{luatex}
    \\ \hline
    \latex
                &  \cell{}{}
                &  \cell{latex}{pdflatex}
                &  \cell{}{xelatex}
                &  \cell{dvilualatex}{lualatex}
    \\
  \end{tabular}
\end{center}

We can now answer the title question: \lualatex is the \luatex engine with the
\latex format. Well, this answer isn't very satisfying if you don't know what
\luatex and \latex are.

\medskip

As you probably know, \latex is the general framework
in which documents begin with ©\documentclass©, packages are loaded with
©\usepackage©, fonts are selected in a clever way (so that you can switch to
boldface while preserving italics), pages are build with complicated algorithm
including support for headers, footers, footnotes, margin notes, floating
material, etc. This mostly doesn't change with \lualatex, but new and more
powerful packages are available to make parts of the system work in a better
way.

Let's look at \luatex now. It is the designated successor of \pdftex and
includes of all its core features: direct generation of PDF files with support
for advanced PDF features and micro-typographic enhancements to \tex
typographic algorithms. The main new features of \luatex are:
\begin{enumerate}
  \item Native support of Unicode, the modern standard for character
    classification and encoding, supporting all characters in the world, from
    English to traditional Chinese through Arabic, including a lot a
    mathematical (or otherwise specialised) symbols.
  \item Inclusion of Lua as an embedded scripting language (see
    section~\ref{luaintex} for details).
  \item A lot of wonderful Lua libraries, including:
    \begin{itemize}
      \item ©fontloader©, supporting modern font formats such as
        TrueType and OpenType;
      \item ©font©, allowing advanced manipulation of the fonts
        from within the document;
      \item ©mplib©, an embedded version of the graphic program MetaPost;
      \item ©callback©, providing hooks into parts of the \tex engine that
        were previously inaccessible to the programmer;
      \item utility libraries for manipulating images, pdf files, etc.
    \end{itemize}
\end{enumerate}
Some of these features, such as Unicode support, directly impact all
documents, while others merely provide tools that package authors will use to
provide you with more powerful commands and other enhancements.

\subsection{Switching from \latex to \lualatex}\label{switch}

As the previous section explains, \lualatex is mostly like \latex, with a few
differences, and more powerful packages and tools available. Here we present
absolute minimum you should know to produce a document with \lualatex, while
the rest of the document provides more details about the available packages.

There are only three differences:
\begin{enumerate}
  \item Don't load \pf{inputenc}, just encode your source in UTF-8.
  \item Don't load \pf{fontenc}, but load \pf{fontspec}.
  \item Don't use any package that changes the fonts, but use \pf{fontspec}'s
    commands instead.
\end{enumerate}
So, you only need to get familiar with \pf{fontspec}, which is
easy: select the main (serif) font with ©\setmainfont©, the sans serif font
with ©\setsansfont© and the mono-spaced (typewriter) font with ©\setmonofont©.
The argument to these commands is the human-friendly name of the font, for
example ©Latin modern Roman© rather than ©ec-lmr10©. You probably want to use
©\defaultfontfeatures{Ligatures=TeX}© before these commands to keep the usual
\tex substitutions (such as ©---© for an em-dash) working.

The good news is: you can directly access any font on your operating system
(in addition to those of your \tex distribution) including TrueType and
OpenType fonts and have access to their most advanced features. It means it
is now easy to install for use with \lualatex any modern font you may download
or purchase from an editor and benefit from their full potential.

Now for the bad news: it is not always easy to get a list of all available
fonts. Under Windows with \texlive, the command-line tool \cmd{fc-list} lists
them all, but is not very friendly. Under Mac OS X, the \emph{Fontbook}
application lists the fonts of your system, but not those of your
distribution. Same with \cmd{fc-list} on Linux. More bad news: it is not easy
to access your old fonts that way. Happily, more fonts are available in modern
formats every day (well, month or year, actually, if you count only good
fonts).

\emph{En passant}, let's mention that the content of this section so far also
holds for \xelatex, that is, \latex over \xetex. Indeed, \xetex shares two of
the essential features of \luatex: native Unicode and support for modern font
formats (but doesn't have the other features of \luatex; on the other hand, it
is more stable right now). Though their implementations concerning fonts are
vastly different, \pf{fontspec} manages to offer a mostly unified font
interface for both \xelatex and \lualatex.

\medskip

So, to benefit from the new features of \luatex, you must drop a few parts of
the old world, namely the fonts that are not available in a modern format (and
also the liberty to encode your source how you want, but UTF-8 is so much
superior that this one hardly counts). The package \pk{luainputenc} provides
various trade-offs that allow you to regain these parts\footnote{While the
  name suggests it is only about input encodings, the details of \latex's font
  encoding implementation imply this package is needed (and works) for old
  fonts too.} possibly at the expense of loosing real Unicode support.

That's all you need to know to start producing documents with \lualatex. I
recommend you have a look at the \pf{fontspec} manual and actually try to
compile a small document using funny fonts. You can then skim over the rest of
this document as you wish. Section~\ref{workornot} lists all the other
differences between conventional \latex and \lualatex that I'm aware of.

\subsection{A Lua-in-\tex primer}\label{luaintex}

Lua is a nice, small language, obviously less surprising and much easier to
learn than \tex as a programming language. The essential reference is the very
readable book \emph{Programming in Lua}, whose first edition is freely
\href{http://www.lua.org/pil/} {available online}. For a quick start, I
recommend you read chapters~1 to~5 and have a quick glance at part~3. Note
that all the libraries mentioned in chapter~3 are included in \luatex, but the
©os© library is restricted for security reasons.

Depending on your programming culture, you may be directly interested in the
rest of part~1 and part~2 which present more advanced features of the
language, but part~4 is useless in a \luatex context, unless of course you
want to hack \luatex itself. Finally, the \emph{Lua reference manual} is
\href{http://www.lua.org/manual/}{available online in various languages} and
comes with a handy index.

\medskip

Now, let's turn to Lua in \luatex. The main way to execute Lua code from the
\tex end is the ©\directlua© command, which takes arbitrary Lua code as an
argument. Conversely, you can pass information from Lua to \tex with
©tex.sprint©.\footnote{The name probably means ``string print'' as opposed to
  ``run very fast for a short period of time.''} For example,
\begin{Verbatim}
  the standard approximation $\pi = \direclua{tex.sprint(math.pi)}$
\end{Verbatim}
prints ``the standard  approximation $\pi = \directlua{tex.sprint(math.pi)}$''
in your document. See how easy it is to mix \tex and Lua?

Actually, there are a few gotchas. Let's look at the Lua to \tex way first,
it's the simplest (since it's more Lua than \tex). If you look at the \luatex
manual, you'll see there is another function with a simpler name, ©tex.print©,
to pass information this way. It works by virtually inserting a full line into
you \tex source, whose contents are its argument. In case you didn't know,
\tex does all kinds of nasty\footnote{Okay, these are usually nice and helpful
  actions, but it this case they are most probably unexpected so I call them
  nasty.} things with full lines of the source, like
ignoring spaces at the beginning and end of line and appending an end-of-line
character. Most of the time, you don't want this to happen, so I recommend
using ©tex.sprint© which virtually inserts its argument in the current line,
so is much more predictable.

If you're enough of a \tex{}nician to know about catcodes, you'll be happy to
know that ©tex.print© and its variants give you nearly full control over the
catcodes used for tokenizing the argument, since you can specify a catcode
table as the first argument. You'll probably want to learn about catcode
tables (currently~2.7.6 in the \luatex manual) before you're fully happy. If
you don't know about catcodes, just skip this paragraph.\footnote{Erf, too
  late, you already read it.}

\medskip

Let's look at ©\direclua© now. To get an idea about how it works, imagine that
it's a ©\write© command, but it writes only to a virtual file and immediately
arranges for this file to be fed to the Lua interpreter. On the Lua end, the
consequence is that each argument of a ©\directlua© command has its own scope:
local variables from one will not be visible to the other. (Which is rather
sane, but always good to know.)

Now, the major gotcha is that before being fed to the Lua interpreter, the
argument is first read and tokenised by \tex, then fully expanded and turned
back into a plain string. Being read by \tex has several consequences. One of
them is that end of lines are turned into spaces, so the Lua interpreter sees
only one (long) line of input. Since Lua is a free-from language, it doesn't
usually matter, but it does if you use comments:
\begin{Verbatim}
  \directlua{a_function()
    -- a comment
    another_function()}
\end{Verbatim}
won't do what you probably expect: ©another_function()© will be seen as part
of the comment (it's only one line, remember).

Another consequence of being read by \tex is that successive spaces are merged
into one space, and \tex comments are discarded. So, here is a surprisingly
correct version of the previous example.
\begin{Verbatim}
  \directlua{a_function()
    % a comment
    another_function()}
\end{Verbatim}
It is also worth noticing that, since since the argument basically is inside a
©\write©, it's in expansion-only context. If you don't know what it means, let
me say that expansion issues are mostly what makes \tex programming so
difficult to avoid expanding further on that matter.

\medskip

I'm sorry if the last three paragraphs were a bit \tex{}nical in nature but
I thought you had to know. To reward you for staying with me, here is a
debugging trick. Put the following code near the beginning of your document:
\begin{Verbatim}
  \newrite\luadebug
  \immediate\openout\luadebug luadebug.lua
  \AtEndDocument{\immediate\closeout\luadebug}
  \newcommand\directluadebug{\immediate\write\luadebug}
\end{Verbatim}
Then, when you have a hard time understanding why a particular call to
©\directlua© doesn't do what you expect, replace this instance of the command
with ©\directluadebug©, compile as usual and look in the file
\file{luadebug.lua} produced what the Lua interpreter actually read.

For everything but trivial pieces of Lua code, it is much wiser to use an
external file containing only Lua code defining functions, then load it and
use its functions. For example:
\begin{Verbatim}
  \directlua{dofile("my-lua-functions.lua")}
  \newcommand*{\greatmacro}[2]{%
    \directlua{my_great_function("\luatexluaescapestring{#1}", #2)}}
\end{Verbatim}
The example assumes that ©my_great_function© takes a string and a number as
arguments. Notice that we carefully use the ©\luatexluaescapestring© primitives
on the string argument to escape any backslash or double-quote it might contain
and which would confuse the Lua parser.\footnote{If you ever used SQL
  then the concept of escaping strings is hopefully not new to you.}

\medskip

That's all concerning Lua in \tex. If you're wondering why
©\luatexluaescapestring© has such a long and silly name, you might want to
read the next section.

\subsection{Other things you should know}\label{things}

Just in case it isn't obvious, the \luatex manual, \file{luatexref-t.pdf}, is
a great source of information about \luatex and you'll probably want to
consult it at some point (though it is a bit arid and technical).

It is important to know that the name of the new primitives of \luatex as you
read them in the manual are not the actual names you'll be able to use in
\lualatex. To prevent clashes with existing macro names, all new primitives
have been prefixed with ©\luatex© unless they already start with it, so
©\luaescapestring© becomes ©\luatexluaescapetring© while ©\luatexversion©
remains ©\luatexversion©. The rationale is detailed in section~\ref{formats}.


\section{Essential packages and practices}\label{essential}

This section presents the packages you'll probably want to always load as a
user, or that you should absolutely know about as a developer.

\subsection{User-level}

\pkdesc{fontspec}{\WSPR}{\xetex, \luatex}{\latex}{%
  macros/latex/contrib/fontspec/}[https://github.com/wspr/fontspec/]
Nice interface to font management, well-integrated in to the \latex font
selection scheme. Already presented in the previous section.

\subsection{Developer-level}

\subsubsection{Naming conventions}

On the \tex end, control sequences starting with ©\luatex© are reserved for
primitives. It is strongly recommended that you do \emph{not} define any such
control sequence, to prevent name clashes with future versions of \luatex. If
you want to a emphasize that a macro is specific to \luatex, we recommend that
you use the ©\lua© prefix (without a following ©tex©) instead. It is okay to
use the ©\luatex@© prefix for internal macros, since primitive names never
contain ©@©, but it might be confusing. Moreover, you're already using a
unique prefix for internal macros in all of your packages, aren't you?

On the Lua end, please keep the global namespace as clean as possible. That
is, use a table ©mypackage© and put all your public functions and objects in
this table. You might want to use Lua's
\href{http://www.lua.org/manual/5.1/manual.html#pdf-module} {\code{module()}}.
Other strategies for Lua module management are discussed in
\href{http://www.lua.org/pil/15.html} {chapter~15 of \emph{Programming in
    Lua}}.  Also, it is probably a good idea to use ©local© for your internal
variables and functions. Finally, to avoid clashes with future versions of
\luatex, it is recommended to avoid modifying the namespaces of \luatex's
default libraries.

\subsubsection{Engine and mode detection}\label{detect}

Various packages allow to detect the engine currently processing the document.

\pkdesc{ifluatex}{\HO}{all}{\latex, Plain}{%
  macros/latex/contrib/oberdiek/}
Provides ©\ifluatex© and makes sure ©\luatexversion© is available.

\pkdesc{iftex}{\VK}{all}{\latex, Plain}{%
  macros/latex/contrib/iftex/}[http://bitbucket.org/vafa/iftex]
Provides ©\ifPDFTeX©, ©\ifXeTeX©, ©\ifLuaTeX© and corresponding ©\Require©
commands.

\pkdesc{expl3}{The \LaTeX3 Project}{all}{\latex}{%
  macros/latex/contrib/expl3/}[http://www.latex-project.org/code.html]
Amongst \emph{many} other things, provides ©\luatex_if_engine:TF©,
©\xetex_if_engine:TF© and their variants.

\pkdesc{ifpdf}{\HO}{all}{\latex, Plain}{%
  macros/latex/contrib/oberdiek/}
Provides ©\ifpdf© switch. \luatex, like \pdftex, can produce either PDF or DVI
output; the later is not very useful with \luatex as it doesn't support any
advanced feature such as Unicode and modern font formats. The ©\ifpdf© switch
is true if and only if you are running \pdftex-or-\luatex in PDF mode (note
that this doesn't include \xetex, whose support for PDF is different).

\subsubsection{Basic resources}

\pkdesc{luatexbase}{\ER \& \MPG}{\luatex}{\latex, Plain}{%
  macros/luatex/generic/luatexbase/}[https://github.com/mpg/luatexbase]
The Plain and \latex formats provide macros to manage \tex basic resources,
such as count or box registers. \luatex introduces new resources that need to
be shared gracefully by packages. This package provides the basic tools to
manage: the extended conventional \tex resources, catcode tables, attributes,
callbacks, Lua modules loading and identification. It also provides basic
tools to handle a few compatibility issues with older version of \luatex.

\note{Warning} This package is currently in conflict with the \pk{luatex}
package, since they both do almost the same thing. The respective package
authors are well aware of this situation and plan to somehow merge the two
packages in the near future, though the timeline is not clear.

\pkdesc{luatex}{\HO}{\luatex}{\latex, Plain}{%
  macros/latex/contrib/oberdiek/}
See the description of \pk{luatexbase} above. This package provides the same
core features except for callback management and Lua module identification.

\pkdesc{lualibs}{\ER}{\luatex}{Lua}{%
  macros/luatex/generic/lualibs/}[https://github.com/mpg/lualibs]
Collection of Lua libraries and additions to the standard libraries; mostly
derived from the \context libraries. If you need a basic function that Lua
doesn't provide, check this package before rolling your own implementation.

\subsubsection{Font internals}\label{fontint}

Those packages are loaded by \pk{fontspec} to handle some low-level font and
encoding issues. A normal user should only use \pk{fontspec}, but a developer
may need to know about them.

\pkdesc{luaotfload}{\ER \& \KH}{\luatex}{\latex, Plain}{%
  macros/luatex/generic/luaotfload/}[https://github.com/khaledhosny/luaotfload]
Low-level OpenType font loading. Basically, it uses the ©fontloader© Lua
library and the appropriate callbacks to implement a syntax for the ©\font©
primitive very similar to that of \xetex and implement the corresponding font
features. It also handles a font database for transparent access to fonts from
the system and the \tex distribution either by family name or by file name, as
well as a font cache for faster loading.

\pkdesc{euenc}{\WSPR, \ER \& \KH}{\xetex, \luatex}{\latex}{%
  macros/latex/contrib/euenc/}[https://github.com/wspr/euenc]
Implements the ©EUx© Unicode font encodings for \latex's \pf{fontenc} system.
Currently, \xelatex is using ©EU1© and \luatex is using ©EU2©. Includes
definitions (\file{fd} files) for Latin Modern, the default font loaded by
\pk{fontspec}.

Also includes a stripped version of \pf{xunicode} so that the usual control
sequences for non-ASCII characters (such as ©\'e©) do the right thing.


\section{Other packages}\label{other}

\subsection{User-level}

\pkdesc{luatextra}{\ER \& \MPG}{\luatex}{\latex}{%
  macros/luatex/generic/luatextra/}[https://github.com/mpg/luatextra]
Loads various packages such as \pk{luatexbase}, \pk{luaotfload}, \pf{metalogo}
(providing the ©\LuaTeX© and ©\LuaLaTeX© commands) and adds a few goodies,
such as a ©luacode© environment.

\pkdesc{luainputenc}{\ER \& \MPG}{\luatex, \xetex, \pdftex}{\latex}{%
  macros/luatex/latex/luainputenc/}[https://github.com/mpg/luainputenc]
Helps compiling documents relying on legacy encodings (either in the source or
with the fonts). Already presented in the introduction. When running \xetex,
just loads \pf{xetex-inputenc}; under \pdftex, loads the standard
\pf{inputenc}.

\pkdesc{luamplib}{\HH, \Taco \& \ER}{\luatex}{\latex, Plain}{%
  macros/luatex/generic/luamplib/}[https://github.com/mpg/luamplib]
Provides a nice interface for the ©mplib© Lua library that embeds metapost in
\luatex.

\pkdesc{luacolor}{\HO}{\luatex}{\latex}{%
  macros/latex/contrib/oberdiek/}
Changes low-level color implementation to use \luatex attributes in place of
whatsits. This makes the implementation more robust and fixes strange bugs
such as wrong alignment when ©\color© happens at the beginning of a ©\vbox©.

\pkdesc{luadirections}{\KH}{\luatex}{\latex, Plain, \context}{}
[https://github.com/khaledhosny/luadirections]
Higher-level interface to \luatex's multi-directional support. Currently not
released on CTAN.

\subsection{Developer-level}

\pkdesc{pdftexcmds}{\HO}{\luatex, \pdftex, \xetex}{\latex, Plain}{%
  macros/latex/contrib/oberdiek/}
Though \luatex is mostly a superset of \pdftex, a few utility primitives were
removed (those that are sort of superseded by Lua) or renamed. This package
provides them with consistent names across engines, including \xetex which
recently implemented some of these primitives, such as ©\strcmp©.

\pkdesc{magicnum}{\HO}{\luatex, \pdftex, \xetex}{\latex, Plain}{%
  macros/latex/contrib/oberdiek/}
Provides hierarchical access to ``magic numbers'' such as catcodes, group
types, etc. used internally by \tex and its successors. Under \luatex, a more
efficient implementation is used and a Lua interface is provided.

\pkdesc{lua-alt-getopt}{Aleksey Cheusov}{\cmd{texlua}}{command-line}{%
  support/lua/lua-alt-getopt}[http://luaforge.net/project/lua_altgetopt]
Command-line option parser, mostly compatible with POSIX and GNU getopt, to be
used in command-line Lua scripts such as \cmd{mkluatexfontdb} from
\pk{luaotfload}.


\section{The \cmd{luatex} and \cmd{lualatex} formats}\label{formats}

This section is for developers and curious users only; normal users can safely
skip it. The following informations apply to \texlive 2010, and most likely to
\miktex 2.9 too, though I didn't actually check. Earlier versions of \texlive
had slightly different and less complete arrangements.

\para{Primitive names}
As mentioned in section~\ref{things}, the name of the \luatex-specific
primitives are not the same in the \cmd{lualatex} format as in the \luatex
manual. In the \cmd{luatex} format (that is, \luatex with the Plain format),
primitives are available with their natural name, but also with the prefixed
name, in order to ease development of generic packages.

The rationale, copy-pasted from the file \file{lualatexiniconfig.tex} that
implements this for the \cmd{lualatex} format, is:

\begin{myquote}
  \begin{enumerate}
    \item All current macro packages run smoothly on top of pdf(e)TeX, so
      those primitives are left untouched.
    \item Other non-TeX82 primitives in \luatex may cause name clashes with
      existing macros in macro packages, especially when they use very
      ``natural'' names such as ©\outputbox©, ©\mathstyle© etc. Such a
      probability for name clashes is undesirable, since the most existing
      LaTeX documents that run without change under \luatex, the better.
    \item The \luatex team doesn't want to apply a systematic prefixing policy,
      but kindly provided a tool allowing prefixes to be applied. So we chose
      to use it.  Previously, we even disabled the extra primitives, but now
      we feel it's better to enable them with systematic prefixing, in order
      to avoid that each and every macro package (or user) enables them with
      various and inconsistent prefixes (including the empty prefix).
    \item The ©luatex© prefix was chosen since it is already used as a prefix
      for some primitives, such as ©\luatexversion©: this way, those
      primitives don't end up with a double prefix (for details, see
      ©tex.enableprimitives© in the \luatex manual).
    \item The ©\directlua© primitive is provided both with its natural name
      (allowing easy detection of \luatex) and a prefixed version
      ©\luatexdirectlua© (for consistency with ©\luatexlatelua©).
    \item Various remarks:
      \begin{itemize}
        \item The obvious drawback of such a prefixing policy is that the
          names used by \latex or generic macro writer won't match the names
          used in the manual.  We hope this is compensated by the gain in
          backwards compatibility.
        \item All primitives dealing with Unicode math already begin with ©\U©,
          and maybe will match the names of \xetex primitives some day, so
          maybe prefixing was not necessary/desirable for them. However, we
          tried to make the prefixing rule as simple as possible, so that
          the previous point doesn't get even worse.
        \item The final name of some primitive may sound strange, namely those
          already containing the name of an engine, such as
          ©\luatexOmegaVersion©.  However, since \luatex is not a drop-in
          replacement for Omega/Aleph, we felt it wrong to provide
          ©\OmegaVersion©.
        \item Maybe some day we'll feel it's better to provide all primitives
          without prefixing. If this happens, it will be easy to add the
          unprefixed primitives in the format while keeping the prefixed names
          for compatibility. It wouldn't work the other way round; i.e.,
          belatedly realizing that we should not provide the unprefixed
          primitives would then break any \luatex-specific macro packages
          that had been written.
      \end{itemize}
  \end{enumerate}
\end{myquote}

\para{\cs{jobname}}[jobname]
The \latex kernel (and a lot of packages) use constructs like
©\input\jobname.aux© for various purposes. When ©\jobname© contains spaces,
this doesn't do the right thing, since the argument of ©\input© ends at the
first space. To work around this, \pdftex automagically quotes ©\jobname© when
needed, but \luatex doesn't for some reason. A nearly complete workaround is
included in \latex-based (as opposed to Plain-based) \luatex formats.

It doesn't work, however, if \luatex is invoked as ©lualatex '\input name'©,
as opposed to the more usual ©lualatex name©. To work around this
limitation of the workaround included in the format, specifying a jobname
explicitly, as in ©lualatex jobname=name '\input name'©. Or even better, just
don't use spaces in the names of your \tex files.

For more details, see
\href{http://www.ntg.nl/pipermail/dev-luatex/2009-April/002549.html}{this old
  thread} and
\href{http://tug.org/pipermail/luatex/2010-August/001986.html}{this newer one}
on the \luatex mailing lists, and \file{lualatexquotejobname.tex} for the
implementation of the workaround.


\section{Things that just work, partially work, or don't work (yet)}
\label{workornot}

\subsection{Just working}\label{working}

\para{Unicode}
Conventional \latex offers some level of support for UTF-8 in input files.
However, at a low level, non-ASCII characters are not atomic in this case:
they consist of several elementary pieces (known as \emph{tokens} to
\tex{}nicians). Hence, packages that scan text character by character
often have problems with UTF-8 in conventional \latex. Such packages include:
\pf{fancyvrb}, \pf{listings}, \pf{soul} and probably others.

The good new is, with \lualatex, these packages start working on arbitrary
Unicode characters without even being modified. (Disclaimer: I didn't run
thorough tests, but I'm pretty sure they do.) You don't need to use tricky
wrapper packages such as \pf{listingsutf8} or \pf{soulutf8} (they won't work
with \luatex, anyway).

\subsection{Partially working}\label{partial}

\para{microtype}
Package \pf{microtype} has limited support for \luatex: more precisely, as of
version 2.4 2010/01/10, protrusion and expansion are available and activated
by default in PDF mode, but kerning, spacing and tracking are not supported
(sse table~1 in section~3.1 of \file{microtype.pdf}).

On the other hand, \pk{luaotfload}, loaded by \pk{fontspec}, supports a lot of
microtypographic features. So the only problem is the lack of a unified
interface.

\para{xunicode}
Package \pf{xunicode}'s main feature is to ensure that the usual control
sequences for non-ASCII characters (such as ©\'e©) do the right thing in a
Unicode context. It could \emph{probably} work with \luatex, but explicitly
checks for \xetex only. Since \pk{euenc}, loaded by \pk{fontspec}, includes a
modified version of this package, you usually don't need \pf{xunicode} so this
is not a problem.

\subsection{Not working (yet)}\label{notworking}

\para{old encodings}[oldenco] Packages playing with input (source files) or
output (fonts) encodings are very likely to break with \luatex. The good news
is, Unicode is a more powerful way to handle encoding problems that old
packages were trying to solve, so you most likely don't need these packages
anyway. However, not everything is already ported to the shiny new world of
Unicode, so you may have a more limited (or just different) set of choices
available for some time (this is especially true for fonts).

\para{spaces} Spaces in file names are not really well supported in the \tex
world in general. This doesn't really get better with \luatex. Also, due to
tricky reasons, things my be worse if you have spaces in the name of your main
\tex file \emph{and} don't invoke \luatex in the usual way. If you do you
invoke it in the usual way, everything should work, and I won't tell you what
the unusual invocation looks like. Otherwise, read the point about
\pararef{jobname} in section~\ref{formats} for a workaround and technical
details. Or even better, don't use spaces in the names of your \tex files.

\end{document}

% vim: spell spelllang=en