\iffalse meta-comment File: source3body.tex Copyright (C) 1990-2012,2014-2020 The LaTeX3 Project It may be distributed and/or modified under the conditions of the LaTeX Project Public License (LPPL), either version 1.3c of this license or (at your option) any later version. The latest version of this license is in the file https://www.latex-project.org/lppl.txt This file is part of the "l3kernel bundle" (The Work in LPPL) and all files in that bundle must be distributed together. The released version of this bundle is available from CTAN. ----------------------------------------------------------------------- The development version of the bundle can be found at https://github.com/latex3/latex3 for those people who are interested. \fi %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % This file is used by % % source3.tex % documentation including implementation % % interface3.tex % only interface documentation % % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{abstract} \setlength\parindent{0pt} \setlength\parskip{\baselineskip} \noindent This is the reference documentation for the \pkg{expl3} programming environment. The \pkg{expl3} modules set up an experimental naming scheme for \LaTeX{} commands, which allow the \LaTeX{} programmer to systematically name functions and variables, and specify the argument types of functions. The \TeX{} and \eTeX{} primitives are all given a new name according to these conventions. However, in the main direct use of the primitives is not required or encouraged: the \pkg{expl3} modules define an independent low-level \LaTeX3 programming language. At present, the \pkg{expl3} modules are designed to be loaded on top of \LaTeXe{}. In time, a \LaTeX3 format will be produced based on this code. This allows the code to be used in \LaTeXe{} packages \emph{now} while a stand-alone \LaTeX3 is developed. \begin{bfseries} While \pkg{expl3} is still experimental, the bundle is now regarded as broadly stable. The syntax conventions and functions provided are now ready for wider use. There may still be changes to some functions, but these will be minor when compared to the scope of \pkg{expl3}. New modules will be added to the distributed version of \pkg{expl3} as they reach maturity. \end{bfseries} \end{abstract} \clearpage {% \def\\{:}% fix "newlines" in the ToC \tableofcontents } \clearpage \pagenumbering{arabic} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % Each of the following \DocInput lines includes a file with extension % .dtx. Each of these files may be typeset separately. For instance % pdflatex l3box.dtx % will typeset the source of the LaTeX3 box commands. If you use the % Makefile, the index will be generated automatically; e.g., % make doc F=l3box % % If this file is processed, each of these separate dtx files will be % contained as a part of a single document. % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \makeatletter \def\partname{Part} \def\maketitle{\clearpage\part{\@title}} \let\thanks\@gobble \let\DelayPrintIndex\PrintIndex \let\PrintIndex\@empty \providecommand*{\hexnum}[1]{\text{\texttt{\char`\"}#1}} \makeatother \part{Introduction to \pkg{expl3} and this document} This document is intended to act as a comprehensive reference manual for the \pkg{expl3} language. A general guide to the \LaTeX3 programming language is found in \href{expl3.pdf}{expl3.pdf}. \section{Naming functions and variables} \LaTeX3 does not use \texttt{@} as a \enquote{letter} for defining internal macros. Instead, the symbols |_| and \texttt{:} are used in internal macro names to provide structure. The name of each \emph{function} is divided into logical units using \texttt{_}, while \texttt{:} separates the \emph{name} of the function from the \emph{argument specifier} (\enquote{arg-spec}). This describes the arguments expected by the function. In most cases, each argument is represented by a single letter. The complete list of arg-spec letters for a function is referred to as the \emph{signature} of the function. Each function name starts with the \emph{module} to which it belongs. Thus apart from a small number of very basic functions, all \pkg{expl3} function names contain at least one underscore to divide the module name from the descriptive name of the function. For example, all functions concerned with comma lists are in module \texttt{clist} and begin |\clist_|. Every function must include an argument specifier. For functions which take no arguments, this will be blank and the function name will end \texttt{:}. Most functions take one or more arguments, and use the following argument specifiers: \begin{description} \item[\texttt{N} and \texttt{n}] These mean \emph{no manipulation}, of a single token for \texttt{N} and of a set of tokens given in braces for \texttt{n}. Both pass the argument through exactly as given. Usually, if you use a single token for an \texttt{n} argument, all will be well. \item[\texttt{c}] This means \emph{csname}, and indicates that the argument will be turned into a csname before being used. So |\foo:c| |{ArgumentOne}| will act in the same way as |\foo:N| |\ArgumentOne|. \item[\texttt{V} and \texttt{v}] These mean \emph{value of variable}. The \texttt{V} and \texttt{v} specifiers are used to get the content of a variable without needing to worry about the underlying \TeX{} structure containing the data. A \texttt{V} argument will be a single token (similar to \texttt{N}), for example |\foo:V| |\MyVariable|; on the other hand, using \texttt{v} a csname is constructed first, and then the value is recovered, for example |\foo:v| |{MyVariable}|. \item[\texttt{o}] This means \emph{expansion once}. In general, the \texttt{V} and \texttt{v} specifiers are favoured over \texttt{o} for recovering stored information. However, \texttt{o} is useful for correctly processing information with delimited arguments. \item[\texttt{x}] The \texttt{x} specifier stands for \emph{exhaustive expansion}: every token in the argument is fully expanded until only unexpandable ones remain. The \TeX{} \tn{edef} primitive carries out this type of expansion. Functions which feature an \texttt{x}-type argument are \emph{not} expandable. \item[\texttt{e}] The \texttt{e} specifier is in many respects identical to \texttt{x}, but with a very different implementation. Functions which feature an \texttt{e}-type argument may be expandable. The drawback is that \texttt{e} is extremely slow (often more than $200$ times slower) in older engines, more precisely in non-\LuaTeX{} engines older than 2019. \item[\texttt{f}] The \texttt{f} specifier stands for \emph{full expansion}, and in contrast to \texttt{x} stops at the first non-expandable token (reading the argument from left to right) without trying to expand it. If this token is a \meta{space token}, it is gobbled, and thus won't be part of the resulting argument. For example, when setting a token list variable (a macro used for storage), the sequence \begin{verbatim} \tl_set:Nn \l_mya_tl { A } \tl_set:Nn \l_myb_tl { B } \tl_set:Nf \l_mya_tl { \l_mya_tl \l_myb_tl } \end{verbatim} will leave |\l_mya_tl| with the content |A\l_myb_tl|, as |A| cannot be expanded and so terminates expansion before |\l_myb_tl| is considered. \item[\texttt{T} and \texttt{F}] For logic tests, there are the branch specifiers \texttt{T} (\emph{true}) and \texttt{F} (\emph{false}). Both specifiers treat the input in the same way as \texttt{n} (no change), but make the logic much easier to see. \item[\texttt{p}] The letter \texttt{p} indicates \TeX{} \emph{parameters}. Normally this will be used for delimited functions as \pkg{expl3} provides better methods for creating simple sequential arguments. \item[\texttt{w}] Finally, there is the \texttt{w} specifier for \emph{weird} arguments. This covers everything else, but mainly applies to delimited values (where the argument must be terminated by some specified string). \item[\texttt{D}] The \texttt{D} specifier means \emph{do not use}. All of the \TeX{} primitives are initially \cs{let} to a \texttt{D} name, and some are then given a second name. Only the kernel team should use anything with a \texttt{D} specifier! \end{description} Notice that the argument specifier describes how the argument is processed prior to being passed to the underlying function. For example, |\foo:c| will take its argument, convert it to a control sequence and pass it to |\foo:N|. Variables are named in a similar manner to functions, but begin with a single letter to define the type of variable: \begin{description} \item[\texttt{c}] Constant: global parameters whose value should not be changed. \item[\texttt{g}] Parameters whose value should only be set globally. \item[\texttt{l}] Parameters whose value should only be set locally. \end{description} Each variable name is then build up in a similar way to that of a function, typically starting with the module\footnote{The module names are not used in case of generic scratch registers defined in the data type modules, e.g., the \texttt{int} module contains some scratch variables called \cs{l_tmpa_int}, \cs{l_tmpb_int}, and so on. In such a case adding the module name up front to denote the module and in the back to indicate the type, as in \texttt{\string\l_int_tmpa_int} would be very unreadable.} name and then a descriptive part. Variables end with a short identifier to show the variable type: \begin{description} \item[\texttt{clist}] Comma separated list. \item[\texttt{dim}] \enquote{Rigid} lengths. \item[\texttt{fp}] Floating-point values; \item[\texttt{int}] Integer-valued count register. \item[\texttt{muskip}] \enquote{Rubber} lengths for use in mathematics. \item[\texttt{seq}] \enquote{Sequence}: a data-type used to implement lists (with access at both ends) and stacks. \item[\texttt{skip}] \enquote{Rubber} lengths. \item[\texttt{str}] String variables: contain character data. \item[\texttt{tl}] Token list variables: placeholder for a token list. \end{description} Applying \texttt{V}-type or \texttt{v}-type expansion to variables of one of the above types is supported, while it is not supported for the following variable types: \begin{description} \item[\texttt{bool}] Either true or false. \item[\texttt{box}] Box register. \item[\texttt{coffin}] A \enquote{box with handles} --- a higher-level data type for carrying out \texttt{box} alignment operations. \item[\texttt{flag}] Integer that can be incremented expandably. \item[\texttt{fparray}] Fixed-size array of floating point values. \item[\texttt{intarray}] Fixed-size array of integers. \item[\texttt{ior}/\texttt{iow}] An input or output stream, for reading from or writing to, respectively. \item[\texttt{prop}] Property list: analogue of dictionary or associative arrays in other languages. \item[\texttt{regex}] Regular expression. \end{description} \subsection{Terminological inexactitude} A word of warning. In this document, and others referring to the \pkg{expl3} programming modules, we often refer to \enquote{variables} and \enquote{functions} as if they were actual constructs from a real programming language. In truth, \TeX{} is a macro processor, and functions are simply macros that may or may not take arguments and expand to their replacement text. Many of the common variables are \emph{also} macros, and if placed into the input stream will simply expand to their definition as well~--- a \enquote{function} with no arguments and a \enquote{token list variable} are almost the same.\footnote{\TeX{}nically, functions with no arguments are \tn{long} while token list variables are not.} On the other hand, some \enquote{variables} are actually registers that must be initialised and their values set and retrieved with specific functions. The conventions of the \pkg{expl3} code are designed to clearly separate the ideas of \enquote{macros that contain data} and \enquote{macros that contain code}, and a consistent wrapper is applied to all forms of \enquote{data} whether they be macros or actually registers. This means that sometimes we will use phrases like \enquote{the function returns a value}, when actually we just mean \enquote{the macro expands to something}. Similarly, the term \enquote{execute} might be used in place of \enquote{expand} or it might refer to the more specific case of \enquote{processing in \TeX's stomach} (if you are familiar with the \TeX{}book parlance). If in doubt, please ask; chances are we've been hasty in writing certain definitions and need to be told to tighten up our terminology. \section{Documentation conventions} This document is typeset with the experimental \pkg{l3doc} class; several conventions are used to help describe the features of the code. A number of conventions are used here to make the documentation clearer. Each group of related functions is given in a box. For a function with a \enquote{user} name, this might read: \begin{function}[label = ]{\ExplSyntaxOn, \ExplSyntaxOff} \begin{syntax} |\ExplSyntaxOn| \dots{} |\ExplSyntaxOff| \end{syntax} The textual description of how the function works would appear here. The syntax of the function is shown in mono-spaced text to the right of the box. In this example, the function takes no arguments and so the name of the function is simply reprinted. \end{function} For programming functions, which use \texttt{_} and \texttt{:} in their name there are a few additional conventions: If two related functions are given with identical names but different argument specifiers, these are termed \emph{variants} of each other, and the latter functions are printed in grey to show this more clearly. They will carry out the same function but will take different types of argument: \begin{function}[label = ]{\seq_new:N, \seq_new:c} \begin{syntax} |\seq_new:N| \meta{sequence} \end{syntax} When a number of variants are described, the arguments are usually illustrated only for the base function. Here, \meta{sequence} indicates that |\seq_new:N| expects the name of a sequence. From the argument specifier, |\seq_new:c| also expects a sequence name, but as a name rather than as a control sequence. Each argument given in the illustration should be described in the following text. \end{function} \paragraph{Fully expandable functions} \hypertarget{expstar}{Some functions are fully expandable}, which allows them to be used within an \texttt{x}-type or \texttt{e}-type argument (in plain \TeX{} terms, inside an \tn{edef} or \tn{expanded}), as well as within an \texttt{f}-type argument. These fully expandable functions are indicated in the documentation by a star: \begin{function}[EXP, label = ]{\cs_to_str:N} \begin{syntax} |\cs_to_str:N| \meta{cs} \end{syntax} As with other functions, some text should follow which explains how the function works. Usually, only the star will indicate that the function is expandable. In this case, the function expects a \meta{cs}, shorthand for a \meta{control sequence}. \end{function} \paragraph{Restricted expandable functions} \hypertarget{rexpstar}{A few functions are fully expandable} but cannot be fully expanded within an \texttt{f}-type argument. In this case a hollow star is used to indicate this: \begin{function}[rEXP, label = ]{\seq_map_function:NN} \begin{syntax} |\seq_map_function:NN| \meta{seq} \meta{function} \end{syntax} \end{function} \paragraph{Conditional functions} \hypertarget{explTF}{Conditional (\texttt{if}) functions} are normally defined in three variants, with \texttt{T}, \texttt{F} and \texttt{TF} argument specifiers. This allows them to be used for different \enquote{true}/\enquote{false} branches, depending on which outcome the conditional is being used to test. To indicate this without repetition, this information is given in a shortened form: \begin{function}[EXP,TF, label = ]{\sys_if_engine_xetex:} \begin{syntax} |\sys_if_engine_xetex:TF| \Arg{true code} \Arg{false code} \end{syntax} The underlining and italic of \texttt{TF} indicates that three functions are available: \begin{itemize} \item |\sys_if_engine_xetex:T| \item |\sys_if_engine_xetex:F| \item |\sys_if_engine_xetex:TF| \end{itemize} Usually, the illustration will use the \texttt{TF} variant, and so both \meta{true code} and \meta{false code} will be shown. The two variant forms \texttt{T} and \texttt{F} take only \meta{true code} and \meta{false code}, respectively. Here, the star also shows that this function is expandable. With some minor exceptions, \emph{all} conditional functions in the \pkg{expl3} modules should be defined in this way. \end{function} Variables, constants and so on are described in a similar manner: \begin{variable}[label = ]{\l_tmpa_tl} A short piece of text will describe the variable: there is no syntax illustration in this case. \end{variable} In some cases, the function is similar to one in \LaTeXe{} or plain \TeX{}. In these cases, the text will include an extra \enquote{\textbf{\TeX{}hackers note}} section: \begin{function}[EXP, label = ]{\token_to_str:N} \begin{syntax} |\token_to_str:N| \meta{token} \end{syntax} The normal description text. \begin{texnote} Detail for the experienced \TeX{} or \LaTeXe\ programmer. In this case, it would point out that this function is the \TeX{} primitive |\string|. \end{texnote} \end{function} \paragraph{Changes to behaviour} When new functions are added to \pkg{expl3}, the date of first inclusion is given in the documentation. Where the documented behaviour of a function changes after it is first introduced, the date of the update will also be given. This means that the programmer can be sure that any release of \pkg{expl3} after the date given will contain the function of interest with expected behaviour as described. Note that changes to code internals, including bug fixes, are not recorded in this way \emph{unless} they impact on the expected behaviour. \section{Formal language conventions which apply generally} As this is a formal reference guide for \LaTeX3 programming, the descriptions of functions are intended to be reasonably \enquote{complete}. However, there is also a need to avoid repetition. Formal ideas which apply to general classes of function are therefore summarised here. For tests which have a \texttt{TF} argument specification, the test if evaluated to give a logically \texttt{TRUE} or \texttt{FALSE} result. Depending on this result, either the \meta{true code} or the \meta{false code} will be left in the input stream. In the case where the test is expandable, and a predicate (|_p|) variant is available, the logical value determined by the test is left in the input stream: this will typically be part of a larger logical construct. \section{\TeX{} concepts not supported by \LaTeX3{}} The \TeX{} concept of an \enquote{\cs{outer}} macro is \emph{not supported} at all by \LaTeX3{}. As such, the functions provided here may break when used on top of \LaTeXe{} if \cs{outer} tokens are used in the arguments. \DisableImplementation \DocInput{l3bootstrap.dtx} \DocInput{l3names.dtx} \ExplSyntaxOn \clist_gput_right:Nn \g_docinput_clist { l3kernel-functions.dtx } \ExplSyntaxOff \DocInput{l3basics.dtx} \DocInput{l3expan.dtx} \DocInput{l3quark.dtx} \DocInput{l3tl.dtx} \DocInput{l3str.dtx} \DocInput{l3str-convert.dtx} \DocInput{l3seq.dtx} \DocInput{l3int.dtx} \DocInput{l3flag.dtx} \DocInput{l3prg.dtx} \DocInput{l3sys.dtx} \DocInput{l3clist.dtx} \DocInput{l3token.dtx} \DocInput{l3prop.dtx} \DocInput{l3msg.dtx} \DocInput{l3file.dtx} \DocInput{l3skip.dtx} \DocInput{l3keys.dtx} \DocInput{l3intarray.dtx} \DocInput{l3fp.dtx} % To get the various submodules of l3fp to appear in the implementation % part only, they have to be added to the documentation list after typesetting % the 'user' part just for the main module. \ExplSyntaxOn \clist_gput_right:Nn \g_docinput_clist { l3fp-aux.dtx , l3fp-traps.dtx , l3fp-round.dtx , l3fp-parse.dtx , l3fp-assign.dtx , l3fp-logic.dtx , l3fp-basics.dtx , l3fp-extended.dtx , l3fp-expo.dtx , l3fp-trig.dtx , l3fp-convert.dtx , l3fp-random.dtx , } \ExplSyntaxOff \DocInput{l3fparray.dtx} \DocInput{l3sort.dtx} \DocInput{l3tl-analysis.dtx} \DocInput{l3regex.dtx} \DocInput{l3box.dtx} \DocInput{l3coffins.dtx} \DocInput{l3color-base.dtx} \DocInput{l3luatex.dtx} \DocInput{l3unicode.dtx} \DocInput{l3text.dtx} \ExplSyntaxOn \clist_gput_right:Nn \g_docinput_clist { l3text-case.dtx , l3text-purify.dtx } \ExplSyntaxOff \DocInput{l3legacy.dtx} \DocInput{l3candidates.dtx} \ExplSyntaxOn \clist_gput_right:Nn \g_docinput_clist { l3deprecation.dtx } \ExplSyntaxOff \endinput