% \iffalse meta-comment % % Copyright (C) 2020-2021 % The LaTeX Project and any individual authors listed elsewhere % in this file. % % This file is part of the LaTeX base system. % ------------------------------------------- % % It may be distributed and/or modified under the % conditions of the LaTeX Project Public License, either version 1.3c % of this license or (at your option) any later version. % The latest version of this license is in % http://www.latex-project.org/lppl.txt % and version 1.3c or later is part of all distributions of LaTeX % version 2008 or later. % % This file has the LPPL maintenance status "maintained". % % The list of all files belonging to the LaTeX base distribution is % given in the file `manifest.txt'. See also `legal.txt' for additional % information. % % The list of derived (unpacked) files belonging to the distribution % and covered by LPPL is defined by the unpacking scripts (with % extension .ins) which are part of the distribution. % % \fi % Filename: usrguide3.tex \documentclass{ltxguide} \usepackage[T1]{fontenc} % needed for \textbackslash in tt \title{New \LaTeX\ methods for authors (starting 2020)} \author{\copyright~Copyright 2020-2021, \LaTeX\ Project Team.\\ All rights reserved.} \date{2021-04-12} \NewDocumentCommand\cs{m}{\texttt{\textbackslash\detokenize{#1}}} \NewDocumentCommand\marg{m}{\arg{#1}} \NewDocumentCommand\meta{m}{\ensuremath{\langle}\textit{#1}\ensuremath{\rangle}} \NewDocumentCommand\pkg{m}{\textsf{#1}} \NewDocumentCommand\text{m}{\ifmmode\mbox{#1}\else#1\fi} % Fix a 'feature' \makeatletter \renewcommand \verbatim@font {\normalfont \ttfamily} \makeatother \begin{document} \maketitle \tableofcontents \section{Introduction} \LaTeXe{} was released in 1994 and added a number of then-new concepts to \LaTeX{}. These are described in \texttt{usrguide}, which has largely remained unchanged. Since then, the \LaTeX{} team have worked on a number of ideas, firstly a programming language for \LaTeX{} (\pkg{expl3}) and then a range of tools for document authors which build on that language. Here, we describe \emph{stable} and \emph{widely-usable} concepts that have resulted from that work. These `new' ideas have been transferred from development packages into the \LaTeXe{} kernel. As such, they are now available to \emph{all} \LaTeX{} users and have the \emph{same stability} as any other part of the kernel. The fact that `behind the scenes' they are built on \pkg{expl3} is useful for the development team, but is not directly important to users. \section{Creating document commands and environments} \subsection{Overview} Creating document commands and environments using the \LaTeX3 toolset is based around the idea that a common set of descriptions can be used to cover almost all argument types used in real documents. Thus parsing is reduced to a simple description of which arguments a command takes: this description provides the `glue' between the document syntax and the implementation of the command. First, we will describe the argument types, then move on to explain how these can be used to create both document commands and environments. Various more specialized features are then described, which allow an even richer application of a simple interface set up. The details here are intended to help users create document commands in general. More technical detail, suitable for \TeX{} programmers, is included in \texttt{interface3}. \subsection{Describing argument types} In order to allow each argument to be defined independently, the parser does not simply need to know the number of arguments for a function, but also the nature of each one. This is done by constructing an \emph{argument specification}, which defines the number of arguments, the type of each argument and any additional information needed for the parser to read the user input and properly pass it through to internal functions. The basic form of the argument specifier is a list of letters, where each letter defines a type of argument. As will be described below, some of the types need additional information, such as default values. The argument types can be divided into two, those which define arguments that are mandatory (potentially raising an error if not found) and those which define optional arguments. The mandatory types \begin{itemize} \item[\texttt{m}] A standard mandatory argument, which can either be a single token alone or multiple tokens surrounded by curly braces |{}|. Regardless of the input, the argument will be passed to the internal code without the outer braces. This is the type specifier for a normal \TeX{} argument. \item[\texttt{r}] Given as \texttt{r}\meta{token1}\meta{token2}, this denotes a `required' delimited argument, where the delimiters are \meta{token1} and \meta{token2}. If the opening delimiter \meta{token1} is missing, the default marker |-NoValue-| will be inserted after a suitable error. \item[\texttt{R}] Given as \texttt{R}\meta{token1}\meta{token2}\marg{default}, this is a `required' delimited argument as for~\texttt{r}, but it has a user-definable recovery \meta{default} instead of |-NoValue-|. \item[\texttt{v}] Reads an argument `verbatim', between the following character and its next occurrence, in a way similar to the argument of the \LaTeXe{} command \cs{verb}. Thus a \texttt{v}-type argument is read between two identical characters, which cannot be any of |%|, |\|, |#|, |{|, |}| or \verb*| |. The verbatim argument can also be enclosed between braces, |{| and |}|. A command with a verbatim argument will produce an error when it appears within an argument of another function. \item[\texttt{b}] Only suitable in the argument specification of an environment, it denotes the body of the environment, between |\begin|\marg{environment} and |\end|\marg{environment}. See Section~\ref{sec:cmd:body} for details. \end{itemize} The types which define optional arguments are: \begin{itemize} \item[\texttt{o}] A standard \LaTeX{} optional argument, surrounded with square brackets, which will supply the special |-NoValue-| marker if not given (as described later). \item[\texttt{d}] Given as \texttt{d}\meta{token1}\meta{token2}, an optional argument which is delimited by \meta{token1} and \meta{token2}. As with \texttt{o}, if no value is given the special marker |-NoValue-| is returned. \item[\texttt{O}] Given as \texttt{O}\marg{default}, is like \texttt{o}, but returns \meta{default} if no value is given. \item[\texttt{D}] Given as \texttt{D}\meta{token1}\meta{token2}\marg{default}, it is as for \texttt{d}, but returns \meta{default} if no value is given. Internally, the \texttt{o}, \texttt{d} and \texttt{O} types are short-cuts to an appropriated-constructed \texttt{D} type argument. \item[\texttt{s}] An optional star, which will result in a value \cs{BooleanTrue} if a star is present and \cs{BooleanFalse} otherwise (as described later). \item[\texttt{t}] An optional \meta{token}, which will result in a value \cs{BooleanTrue} if \meta{token} is present and \cs{BooleanFalse} otherwise. Given as \texttt{t}\meta{token}. \item[\texttt{e}] Given as \texttt{e}\marg{tokens}, a set of optional \emph{embellishments}, each of which requires a \emph{value}. If an embellishment is not present, |-NoValue-| is returned. Each embellishment gives one argument, ordered as for the list of \meta{tokens} in the argument specification. All \meta{tokens} must be distinct. \item[\texttt{E}] As for \texttt{e} but returns one or more \meta{defaults} if values are not given: \texttt{E}\marg{tokens}\marg{defaults}. See Section~\ref{sec:cmd:embellishment} for more details. \end{itemize} \subsection{Modifying argument descriptions} In addition to the argument \emph{types} discussed above, the argument description also gives special meaning to three other characters. First, \texttt{+} is used to make an argument long (to accept paragraph tokens). In contrast to \cs{newcommand}, this applies on an argument-by-argument basis. So modifying the example to `|s o o +m O{default}|' means that the mandatory argument is now \cs{long}, whereas the optional arguments are not. Secondly, \texttt{!} is used to control whether spaces are allowed before optional arguments. There are some subtleties to this, as \TeX{} itself has some restrictions on where spaces can be `detected': more detail is given in Section~\ref{sec:cmd:opt-space}. Finally, the character \texttt{>} is used to declare so-called `argument processors', which can be used to modify the contents of an argument before it is passed to the macro definition. The use of argument processors is a somewhat advanced topic, (or at least a less commonly used feature) and is covered in Section~\ref{sec:cmd:processors}. \subsection{Creating document commands and environments} \begin{decl} |\NewDocumentCommand| \arg{cmd} \arg{arg spec} \arg{code} \\ |\RenewDocumentCommand| \arg{cmd} \arg{arg spec} \arg{code} \\ |\ProvideDocumentCommand| \arg{cmd} \arg{arg spec} \arg{code} \\ |\DeclareDocumentCommand| \arg{cmd} \arg{arg spec} \arg{code} \end{decl} This family of commands are used to create a \meta{cmd}. The argument specification for the function is given by \meta{arg spec}, and the command uses the \meta{code} with |#1|, |#2|, etc.\ replaced by the arguments found by the parser. An example: \begin{verbatim} \NewDocumentCommand\chapter{s o m} {% \IfBooleanTF{#1}% {\typesetstarchapter{#3}}% {\typesetnormalchapter{#2}{#3}}% } \end{verbatim} would be a way to define a \cs{chapter} command which would essentially behave like the current \LaTeXe{} command (except that it would accept an optional argument even when a \texttt{*} was parsed). The \cs{typesetnormalchapter} could test its first argument for being |-NoValue-| to see if an optional argument was present. (See Section~\ref{sec:cmd:special} for details of \cs{IfBooleanTF} and testing for |-NoValue-|.) The difference between the \cs{New...} \cs{Renew...}, \cs{Provide...} and \cs{Declare...} versions is the behavior if \meta{cmd} is already defined. \begin{itemize} \item \cs{NewDocumentCommand} will issue an error if \meta{cmd} has already been defined. \item \cs{RenewDocumentCommand} will issue an error if \meta{cmd} has not previously been defined. \item \cs{ProvideDocumentCommand} creates a new definition for \meta{function} only if one has not already been given. \item \cs{DeclareDocumentCommand} will always create the new definition, irrespective of any existing \meta{cmd} with the same name. This should be used sparingly. \end{itemize} \begin{decl} |\NewDocumentEnvironment| \arg{env} \arg{arg spec} \arg{beg-code} \arg{end-code} \\ |\RenewDocumentEnvironment| \arg{env} \arg{arg spec} \arg{beg-code} \arg{end-code} \\ |\ProvideDocumentEnvironment| \arg{env} \arg{arg spec} \arg{beg-code} \arg{end-code} \\ |\DeclareDocumentEnvironment| \arg{env} \arg{arg spec} \arg{beg-code} \arg{end-code} \end{decl} These commands work in the same way as \cs{NewDocumentCommand}, etc.\@, but create environments (\cs{begin}\arg{env} \ldots{} \cs{end}\arg{environment}). Both the \meta{beg-code} and \meta{end-code} may access the arguments as defined by \meta{arg spec}. The arguments will be given following \cs{begin}\arg{environment}. \subsection{Optional arguments} \label{sec:cmd:opt} In contrast to commands created using \LaTeXe{}'s \cs{newcommand}, optional arguments created using \cs{NewDocumentCommand} may safely be nested. Thus for example, following \begin{verbatim} \NewDocumentCommand\foo{om}{I grabbed `#1' and `#2'} \NewDocumentCommand\baz{o}{#1-#1} \end{verbatim} using the command as \begin{verbatim} \foo[\baz[stuff]]{more stuff} \end{verbatim} will print \begin{quote} I grabbed `stuff-stuff' and `more stuff' \end{quote} This is particularly useful when placing a command with an optional argument \emph{inside} the optional argument of a second command. When an optional argument is followed by a mandatory argument with the same delimiter, the parser issues a warning because the optional argument could not be omitted by the user, thus becoming in effect mandatory. This can apply to \texttt{o}, \texttt{d}, \texttt{O}, \texttt{D}, \texttt{s}, \texttt{t}, \texttt{e}, and \texttt{E} type arguments followed by \texttt{r} or \texttt{R}-type required arguments. The default for \texttt{O}, \texttt{D} and \texttt{E} arguments can be the result of grabbing another argument. Thus for example \begin{verbatim} \NewDocumentCommand\foo{O{#2} m} \end{verbatim} would use the mandatory argument as the default for the leading optional one. \subsection{Spacing and optional arguments} \label{sec:cmd:opt-space} \TeX{} will find the first argument after a function name irrespective of any intervening spaces. This is true for both mandatory and optional arguments. So |\foo[arg]| and \verb*|\foo [arg]| are equivalent. Spaces are also ignored when collecting arguments up to the last mandatory argument to be collected (as it must exist). So after \begin{verbatim} \NewDocumentCommand\foo{m o m}{ ... } \end{verbatim} the user input |\foo{arg1}[arg2]{arg3}| and \verb*|\foo{arg1} [arg2] {arg3}| will both be parsed in the same way. The behavior of optional arguments \emph{after} any mandatory arguments is selectable. The standard settings will allow spaces here, and thus with \begin{verbatim} \NewDocumentCommand\foobar{m o}{ ... } \end{verbatim} both |\foobar{arg1}[arg2]| and \verb*|\foobar{arg1} [arg2]| will find an optional argument. This can be changed by giving the modified |!| in the argument specification: \begin{verbatim} \NewDocumentCommand\foobar{m !o}{ ... } \end{verbatim} where \verb*|\foobar{arg1} [arg2]| will not find an optional argument. There is one subtly here due to the difference in handling by \TeX{} of `control symbols', where the command name is made up of a single character, such as `\texttt{\textbackslash\textbackslash}'. Spaces are not ignored by \TeX{} here, and thus it is possible to require an optional argument directly follow such a command. The most common example is the use of \texttt{\textbackslash\textbackslash} in \pkg{amsmath} environments, which in the terms here would be defined as \begin{verbatim} \NewDocumentCommand\\{!s !o}{ ... } \end{verbatim} \subsection{`Embellishments'} \label{sec:cmd:embellishment} The \texttt{E}-type argument allows one default value per test token. This is achieved by giving a list of defaults for each entry in the list, for example: \begin{verbatim} E{^_}{{UP}{DOWN}} \end{verbatim} If the list of default values is \emph{shorter} than the list of test tokens, the special |-NoValue-| marker will be returned (as for the \texttt{e}-type argument). Thus for example \begin{verbatim} E{^_}{{UP}} \end{verbatim} has default \texttt{UP} for the |^| test character, but will return the |-NoValue-| marker as a default for |_|. This allows mixing of explicit defaults with testing for missing values. \subsection{Testing special values} \label{sec:cmd:special} Optional arguments make use of dedicated variables to return information about the nature of the argument received. \begin{decl} |\IfNoValueTF| \arg{arg} \arg{true code} \arg{false code} \\ |\IfNoValueT| \arg{arg} \arg{true code} \\ |\IfNoValueF| \arg{arg} \arg{false code} \end{decl} The \cs{IfNoValue(TF)} tests are used to check if \meta{argument} (|#1|, |#2|, \emph{etc.}) is the special |-NoValue-| marker. For example \begin{verbatim} \NewDocumentCommand\foo{o m} {% \IfNoValueTF {#1}% {\DoSomethingJustWithMandatoryArgument{#2}}% {\DoSomethingWithBothArguments{#1}{#2}}% } \end{verbatim} will use a different internal function if the optional argument is given than if it is not present. Note that three tests are available, depending on which outcome branches are required: \cs{IfNoValueTF}, \cs{IfNoValueT} and \cs{IfNoValueF}. As the \cs{IfNoValue(TF)} tests are expandable, it is possible to test these values later, for example at the point of typesetting or in an expansion context. It is important to note that |-NoValue-| is constructed such that it will \emph{not} match the simple text input |-NoValue-|, i.e.~that \begin{verbatim} \IfNoValueTF{-NoValue-} \end{verbatim} will be logically \texttt{false}. When two optional arguments follow each other (a syntax we typically discourage), it can make sense to allow users of the command to specify only the second argument by providing an empty first argument. Rather than testing separately for emptiness and for |-NoValue-| it is then best to use the argument type~|O| with an empty default value, and simply test for emptiness using the \pkg{expl3} conditional \cs{tl_if_blank:nTF} or its \pkg{etoolbox} analogue \cs{ifblank}. \begin{decl} |\IfValueTF| \arg{arg} \arg{true code} \arg{false code} \\ |\IfValueT| \arg{arg} \arg{true code} \\ |\IfValueF| \arg{arg} \arg{false code} \end{decl} The reverse form of the \cs{IfNoValue(TF)} tests are also available as \cs{IfValue(TF)}. The context will determine which logical form makes the most sense for a given code scenario. \begin{decl} |\BooleanFalse| \\ |\BooleanTrue| \end{decl} The \texttt{true} and \texttt{false} flags set when searching for an optional character (using \texttt{s} or \texttt{t\meta{char}}) have names which are accessible outside of code blocks. \begin{decl} |\IfBooleanTF| \arg{arg} \arg{true code} \arg{false code} \\ |\IfBooleanT| \arg{arg} \arg{true code} \\ |\IfBooleanF| \arg{arg} \arg{false code} \end{decl} Used to test if \meta{argument} (|#1|, |#2|, \emph{etc.}) is \cs{BooleanTrue} or \cs{BooleanFalse}. For example \begin{verbatim} \NewDocumentCommand\foo{sm} {% \IfBooleanTF {#1}% {\DoSomethingWithStar{#2}}% {\DoSomethingWithoutStar{#2}}% } \end{verbatim} checks for a star as the first argument, then chooses the action to take based on this information. \subsection{Argument processors} \label{sec:cmd:processors} Argument processor are applied to an argument \emph{after} it has been grabbed by the underlying system but before it is passed to \meta{code}. An argument processor can therefore be used to regularize input at an early stage, allowing the internal functions to be completely independent of input form. Processors are applied to user input and to default values for optional arguments, but \emph{not} to the special |-NoValue-| marker. Each argument processor is specified by the syntax \texttt{>}\marg{processor} in the argument specification. Processors are applied from right to left, so that \begin{verbatim} >{\ProcessorB} >{\ProcessorA} m \end{verbatim} would apply \cs{ProcessorA} followed by \cs{ProcessorB} to the tokens grabbed by the \texttt{m} argument. \begin{decl} |\SplitArgument| \arg{number} \arg{token(s)} \end{decl} This processor splits the argument given at each occurrence of the \meta{tokens} up to a maximum of \meta{number} tokens (thus dividing the input into $\text{\meta{number}} + 1$ parts). An error is given if too many \meta{tokens} are present in the input. The processed input is placed inside $\text{\meta{number}} + 1$ sets of braces for further use. If there are fewer than \arg{number} of \arg{tokens} in the argument then |-NoValue-| markers are added at the end of the processed argument. \begin{verbatim} \NewDocumentCommand \foo {>{\SplitArgument{2}{;}} m} {\InternalFunctionOfThreeArguments#1} \end{verbatim} If only a single character \meta{token} is used for the split, any category code $13$ (active) character matching the \meta{token} will be replaced before the split takes place. Spaces are trimmed at each end of each item parsed. \begin{decl} |\SplitList| \arg{token(s)} \end{decl} This processor splits the argument given at each occurrence of the \meta{token(s)} where the number of items is not fixed. Each item is then wrapped in braces within |#1|. The result is that the processed argument can be further processed using a mapping function (see below). \begin{verbatim} \NewDocumentCommand \foo {>{\SplitList{;}} m} {\MappingFunction#1} \end{verbatim} If only a single character \meta{token} is used for the split, any category code $13$ (active) character matching the \meta{token} will be replaced before the split takes place. Spaces are trimmed at each end of each item parsed. \begin{decl} |\ProcessList| \arg{list} \arg{function} \end{decl} To support \cs{SplitList}, the function \cs{ProcessList} is available to apply a \meta{function} to every entry in a \meta{list}. The \meta{function} should absorb one argument: the list entry. For example \begin{verbatim} \NewDocumentCommand \foo {>{\SplitList{;}} m} {\ProcessList{#1}{\SomeDocumentCommand}} \end{verbatim} \begin{decl} |\ReverseBoolean| \end{decl} This processor reverses the logic of \cs{BooleanTrue} and \cs{BooleanFalse}, so that the example from earlier would become \begin{verbatim} \NewDocumentCommand\foo{>{\ReverseBoolean} s m} {% \IfBooleanTF#1% {\DoSomethingWithoutStar{#2}}% {\DoSomethingWithStar{#2}}% } \end{verbatim} \begin{decl} |\TrimSpaces| \end{decl} Removes any leading and trailing spaces (tokens with character code~$32$ and category code~$10$) for the ends of the argument. Thus for example declaring a function \begin{verbatim} \NewDocumentCommand\foo {>{\TrimSpaces} m} {\showtokens{#1}} \end{verbatim} and using it in a document as \begin{flushleft} \verb= =\verb*=\foo{ hello world }= \end{flushleft} will show `\verb*=hello world=' at the terminal, with the space at each end removed. \cs{TrimSpaces} will remove multiple spaces from the ends of the input in cases where these have been included such that the standard \TeX{} conversion of multiple spaces to a single space does not apply. \subsection{Body of an environment} \label{sec:cmd:body} While environments |\begin|\marg{environment}\ \dots{}\,|\end|\marg{environment} are typically used in cases where the code implementing the \meta{environment} does not need to access the contents of the environment (its `body'), it is sometimes useful to have the body as a standard argument. This is achieved by ending the argument specification with~\texttt{b}, which is a dedicated argument type for this situation. For instance \begin{verbatim} \NewDocumentEnvironment{twice} {O{\ttfamily} +b} {#2#1#2} {} \end{verbatim} \begin{verbatim} \begin{twice}[\itshape] Hello world! \end{twice} \end{verbatim} typesets `Hello world!{\itshape Hello world!}'. The prefix |+| is used to allow multiple paragraphs in the environment's body. Argument processors can also be applied to \texttt{b}~arguments. By default, spaces are trimmed at both ends of the body: in the example there would otherwise be spaces coming from the ends the lines after |[\itshape]| and |world!|. Putting the prefix |!| before \texttt{b} suppresses space-trimming. When \texttt{b} is used in the argument specification, the last argument of the environment declaration (e.g., \cs{NewDocumentEnvironment}), which consists of an \meta{end code} to insert at |\end|\marg{environment}, is redundant since one can simply put that code at the end of the \meta{start code}. Nevertheless this (empty) \meta{end code} must be provided. Environments that use this feature can be nested. \subsection{Fully-expandable document commands} Document commands created using \cs{NewDocumentCommand}, etc.\@, are normally created so that they do not expand unexpectedly. This is done using engine features, so is more powerful than \LaTeXe{}'s \cs{protect} mechanism. There are \emph{very rare} occasion when it may be useful to create functions using a expansion-only grabber. This imposes a number of restrictions on the nature of the arguments accepted by a function, and the code it implements. This facility should only be used when \emph{absolutely necessary}. \begin{decl} |\NewExpandableDocumentCommand| \arg{cmd} \arg{arg spec} \arg{code} \\ |\RenewExpandableDocumentCommand| \arg{cmd} \arg{arg spec} \arg{code} \\ |\ProvideExpandableDocumentCommand| \arg{cmd} \arg{arg spec} \arg{code} \\ |\DeclareExpandableDocumentCommand| \arg{cmd} \arg{arg spec} \arg{code} \end{decl} This family of commands is used to create a document-level \meta{function}, which will grab its arguments in a fully-expandable manner. The argument specification for the function is given by \meta{arg spec}, and the function will execute \meta{code}. In general, \meta{code} will also be fully expandable, although it is possible that this will not be the case (for example, a function for use in a table might expand so that \cs{omit} is the first non-expandable non-space token). Parsing arguments by pure expansion imposes a number of restrictions on both the type of arguments that can be read and the error checking available: \begin{itemize} \item The last argument (if any are present) must be one of the mandatory types \texttt{m}, \texttt{r} or \texttt{R}. \item The `verbatim' argument type \texttt{v} is not available. \item Argument processors (using \texttt{>}) are not available. \item It is not possible to differentiate between, for example |\foo[| and |\foo{[}|: in both cases the \texttt{[} will be interpreted as the start of an optional argument. As a result, checking for optional arguments is less robust than in the standard version. \end{itemize} \subsection{Details about argument delimiters} In normal (non-expandable) commands, the delimited types look for the initial delimiter by peeking ahead (using \pkg{expl3}'s |\peek_...| functions) looking for the delimiter token. The token has to have the same meaning and `shape' of the token defined as delimiter. There are three possible cases of delimiters: character tokens, control sequence tokens, and active character tokens. For all practical purposes of this description, active character tokens will behave exactly as control sequence tokens. \subsubsection{Character tokens} A character token is characterized by its character code, and its meaning is the category code~(|\catcode|). When a command is defined, the meaning of the character token is fixed into the definition of the command and cannot change. A command will correctly see an argument delimiter if the open delimiter has the same character and category codes as at the time of the definition. For example in: \begin{verbatim} \NewDocumentCommand { \foobar } { D<>{default} } {(#1)} \end{verbatim} \begin{verbatim} \foobar \par \char_set_catcode_letter:N < \foobar \end{verbatim} the output would be: \begin{verbatim} (hello) (default) \end{verbatim} as the open-delimiter |<| changed in meaning between the two calls to |\foobar|, so the second one doesn't see the |<| as a valid delimiter. Commands assume that if a valid open-delimiter was found, a matching close-delimiter will also be there. If it is not (either by being omitted or by changing in meaning), a low-level \TeX{} error is raised and the command call is aborted. \subsubsection{Control sequence tokens} A control sequence (or control character) token is characterized by is its name, and its meaning is its definition. A token cannot have two different meanings at the same time. When a control sequence is defined as delimiter in a command, it will be detected as delimiter whenever the control sequence name is found in the document regardless of its current definition. For example in: \begin{verbatim} \cs_set:Npn \x { abc } \NewDocumentCommand { \foobar } { D\x\y{default} } {(#1)} \foobar \x hello\y \par \cs_set:Npn \x { def } \foobar \x hello\y \end{verbatim} the output would be: \begin{verbatim} (hello) (hello) \end{verbatim} with both calls to the command seeing the delimiter |\x|. \subsection{Creating new argument processors} \begin{decl} |\ProcessedArgument| \end{decl} Argument processors allow manipulation of a grabbed argument before it is passed to the underlying code. New processor implementations may be created as functions which take one trailing argument, and which leave their result in the \cs{ProcessedArgument} variable. For example, \cs{ReverseBoolean} is defined as \begin{verbatim} \ExplSyntaxOn \cs_new_protected:Npn \ReverseBoolean #1 { \bool_if:NTF #1 { \tl_set:Nn \ProcessedArgument { \c_false_bool } } { \tl_set:Nn \ProcessedArgument { \c_true_bool } } } \ExplSyntaxOff \end{verbatim} [As an aside: the code is written in \pkg{expl3}, so we don't have to worry about spaces creeping into the definition.] \subsection{Access to the argument specification} The argument specifications for document commands and environments are available for examination and use. \begin{decl} |\GetDocumentCommandArgSpec| \arg{function} \\ |\GetDocumentEnvironmentArgSpec| \arg{environment} \end{decl} These functions transfer the current argument specification for the requested \meta{function} or \meta{environment} into the token list variable \cs{ArgumentSpecification}. If the \meta{function} or \meta{environment} has no known argument specification then an error is issued. The assignment to \cs{ArgumentSpecification} is local to the current \TeX{} group. \begin{decl} |\ShowDocumentCommandArgSpec| \arg{function} \\ |\ShowDocumentEnvironmentArgSpec| \arg{environment} \end{decl} These functions show the current argument specification for the requested \meta{function} or \meta{environment} at the terminal. If the \meta{function} or \meta{environment} has no known argument specification then an error is issued. \end{document}