diff options
Diffstat (limited to 'Master/texmf-dist/doc/latex/expl3/l32eproc.tex')
-rw-r--r-- | Master/texmf-dist/doc/latex/expl3/l32eproc.tex | 523 |
1 files changed, 523 insertions, 0 deletions
diff --git a/Master/texmf-dist/doc/latex/expl3/l32eproc.tex b/Master/texmf-dist/doc/latex/expl3/l32eproc.tex new file mode 100644 index 00000000000..d560c0c2338 --- /dev/null +++ b/Master/texmf-dist/doc/latex/expl3/l32eproc.tex @@ -0,0 +1,523 @@ +\documentclass + {ltugproc} +% {ltxguide} + +\usepackage{shortvrb} +\MakeShortVerb{\|} +\setlength\hfuzz{10pt} + +% A couple of \provide.. so document runs with +% both ltugproc and ltxguide classes +% +\providecommand\m[1]{$\langle$\textit{#1}$\rangle$} +\providecommand\netaddress{\date} +\providecommand\acro[1]{\textsc{\MakeLowercase{#1}}} + +\begin{document} +\title{The \LaTeX3 Programming Language\\ +A syntax proposal for \TeX\ macro programming} + + +\author{The \LaTeX3 Project} +\netaddress{latex-l@urz.uni-heidelberg.de} + + + +\begin{abstract} +This paper proposes a new set of programming conventions suitable for +implementing large scale \TeX\ programming projects such as \LaTeX. +(This syntax is not suitable for either document +markup, or as a style specification language.) + +The main features include a systematic naming scheme for all commands, +and a systematic mechanism for controlling the expansion of arguments +to functions. + +The syntax is under consideration as a basis for programming within +the \LaTeX3 project. + +This paper is based on a talk given by David +Carlisle but describes the work of several people, principally: + Frank Mittelbach, + Denys Duchier, + Rainer Sch\"opf, + Chris Rowley, + Michael Downes, + Johannes Braams, + David Carlisle and + Alan Jeffrey. +\end{abstract} + +\maketitle + + +\section{Introduction} +This paper describes a \TeX\ based language which is intended to +provide a more consistent and rational programming environment for the +construction of large scale \TeX\ macro projects such as \LaTeX. + +Variants of this language have been in use within the \LaTeX3 project +since around 1990 but the syntax specification to be outlined here +should \emph{not} be considered final. This is an experimental +language, and the syntax and command names may (and probably will) +change as more experience is gained with using the language in +practice. + +\section{Programming Interface levels for \LaTeX} + +One may identify several distinct languages that one might want to see +in a \TeX\ based system. This paper will \emph{only} be concerned with +the last of these three. + +\begin{description} +\item[Document Markup] This language consists of the commands that are + to be embedded in the document instance. It is generally accepted + that such a language should be essentially \emph{declarative}. + One might consider a traditional \TeX\ based markup such as the + \LaTeX2 markup as described in \cite{A-W:LLa94}, or alternatively one + might consider an \acro{SGML} based markup. + + One problem with more traditional \TeX\ coding conventions is that + the command names and syntax of the \TeX\ primitives are designed to + have a `natural' syntax when used directly by the author as document + markup. In fact one almost never uses the primitives in this + way, rather they are just used to define higher level commands. + +\item[Designer's Interface] In order to easily translate a (human) + designer's design specification into a program that accepts the + document instance one would ideally like to have a declarative + language that allows the relationships and spacing rules of the + various document elements to be easily expressed. As this language + is not embedded within the document text, it may take a rather + different form to the markup language described above. For + \acro{SGML} based systems one may consider the \acro{DSSSL} language + as playing this role. For \LaTeX2, then this level was essentially + missing in \LaTeX2.09. \LaTeXe\ made some improvements in this area + but it is still the case that implementing a design specification in + \LaTeX\ requires more `low level' coding than would be desired. +\item[Programmer's Interface] + This language, the subject of this paper, is the implementation + language in which the basic typesetting functionality is + implemented, building upon the primitives supplied by \TeX\ (or a + successor program). It may also be used to implement the previous + two languages `within' \TeX, as in the current \LaTeX\ system. +\end{description} + + +\section{Programming language: Main Features} + +The language outlined in this paper aims to provide a suitable +base for coding large (and small) scale projects in \TeX. +Its main distinguishing features are the following. +\begin{itemize} +\item Consistent naming scheme for all functions, including \TeX\ + primitives. +\item Standard mechanisms for controlling argument expansion. +\item Provision of sufficiently rich set of core functions for + handling programming constructs such as: + sequences, sets, stacks, property lists, etc. +\item White space ignored. +\end{itemize} + +\section{Naming Scheme} + +The name space is divided between \emph{Functions} and +\emph{Parameters}. Functions (normally) take arguments and are +executed, Parameters are usually passed as arguments to functions. +They are not directly executed, but accessed though accessor functions. + +Functions and parameters associated with a similar function (for +example accessing counters, or manipulating lists, etc.) are arranged +into \emph{modules}. Before giving the details of the form of the +command names, we give a few examples. + +|\l_tmpa_box| is a local parameter (hence the |l_| prefix) +corresponding to a box register. + +|\g_tmpa_box| is a global parameter (hence the |g_| prefix) +corresponding to a box register. + +|\c_one| is the constant (|c_|) parameter with value one. + +|\cnt_add:Nn| is the function which adds the value specified by +its second argument to the count register specified by its first +argument. The different natures of the two arguments are indicated by +the |:Nn| suffix. The first argument must be a single token specifying +the name of the count parameter. Such single token arguments are +denoted |N|. The second argument is a normal \TeX\ `non-delimited +argument' which may either be a single token, or a brace delimited +token list containing an expression for the value to be added. +The |n| denotes such `normal' argument forms. + +|\cnt_add:cn| is similar to the above, but in this case the the +counter is specified in the first argument by a list of tokens that +expands to the \emph{name} of the count parameter. + +These examples should give the basic flavour of the scheme. Parameters +are classified into local, global or constant (there are further more +technical cases in addition to these three) and this access type is +shown by |\l_|, |\g_| or |\c_|. Functions are arranged by +\emph{module} (The |cnt| module in these cases) with a descriptive +name followed by an indication of the type of argument to be passed. + +In more detail the specification of the names is as follows. + +Functions have the following general syntax: + \begin{quote} + |\|\m{module}|_|\m{description}|:|\m{arg-spec} + \end{quote} + The programmer can chose an arbitrary \m{module} name (consisting + of letters only) a group of functions with related functionality are + then all given names prefixed by this module name. The \m{description} + is a description of the functionality of the function, and should + consist of letters, and possibly |_| characters. \m{arg-spec} + describes the type of arguments as will be described below. + +The syntax of parameters is as follows: + \begin{quote} + |\|\m{access}|_|\m{module}|_|\m{description}|_|\m{type} + \end{quote} + \m{access} describes how the parameter can be accessed. The + principal access types are \emph{constant}, \emph{local} or + \emph{global}. As described below, some special access types relate + to \TeX\ primitive parameters. The meanings of \m{module} and + \m{description} in the parameter syntax is the same as that for + functions. Finally \m{type} should denote the type of parameter, + such as |cnt| for count registers, etc. + + + +Typical \m{module} names in the kernel include |cnt| for integer count +related functions, |seq| for functions relating to sequences, |box|, +etc. Normally additional packages adding new functionality would add +new modules as needed. + +The \m{description} is an arbitrary name for the function or +parameter, consisting of letters, or the |_| character. + +Function names always end with an \m{arg-spec} after a final colon. +This gives an indication of the types of argument that a function +takes, and provides a convenient method of naming similar functions +that just differ by their argument forms, as will be explained below. + +The \m{arg-spec} consists of a (possibly empty) list of characters +each denoting one argument that the function takes. It is important to +note that `argument' here refers to the conceptual argument of the +function. +The top level \TeX\ macro that has this name typically has no +arguments. This is similar to the existing \LaTeX\ convention where one +says that |\section| has an optional argument and a mandatory +argument, whereas the \TeX\ macro |\section| actually takes no +parameters at the \TeX\ level, it merely calls some standard \LaTeX\ +internal functions which look ahead for star forms and optional +arguments. The list of possible argument specifiers includes: +\begin{description} +\item[n] Unexpanded token (or token-list if in braces). In other words + this is a standard \TeX\ undelimited macro argument. +\item[o] One time expanded token or token-list. In the case of a token + list then only the first token in the list is expanded. +\item[x] Fully expanded token or token-list. Typically this means that + the argument is expanded in the style of |\edef| (|\def:Npx|) before + being passed to the function. +\item[c] A character string used (after expansion) as a command name. + The argument (a token or braced token list) should expand to a + sequence of characters which is then used to construct a command + name (via |\csname|, |\cs:w|). This command token is passed as the + argument to the function. + \item[N] A single token. (Unlike |n|, this argument must not be + surrounded by braces). A typical example of a command taking an N + argument is |\def|, in which the command being defined must be + unbraced. + \item[O] Single unbraced token that is expanded once and passed (as a + braced token list) to the function. + \item[X] Single unbraced token that is fully expanded and passed (as a + braced token list) to the function. + \item[C] A character string used as for |c| arguments but the + resulting command token is then expanded (as for |O|) and the + result passed as a braced token list to the function. + \item[p] A primitive \TeX\ parameter specification. This can be + something simple like |#1#2#3| but may be arbitrary delimited + argument syntax, such as |#1,#2\q_stop#3|. + \item[T, F] These are special cases of |n| arguments, used as the + true and false cases in conditional tests. + \item[D] `Do not use'. This special case is used for \TeX\ primitives + that are only used while bootstrapping the \LaTeX\ kernel. If the + \TeX\ primitive needs to be used in other contexts it will be given + an alternative name with a more appropriate argument specification. + \item[w] `weird' syntax. Used for arguments that take non standard + forms, usually delimited arguments that are needed internally to + implement certain modules, and also the boolean tests of many of the + primitive |\if|\ldots\ tests. +\end{description} + + +For parameters, the \m{type} should be from the list of available +data types (which include the primitive \TeX\ registers, but also +data types built within the system). + +Possible values for \m{type} include: +\begin{description} +\item[cnt] Integer valued counter. +\item[toks] Token register. +\item[box] Box register. +\item[fcnt] `Fake' count register. A data type supplied by the kernel + to avoid problems with the limited number of available count + registers in (standard) \TeX. +\end{description} + +The \m{access} codes that are used in parameter names include +\begin{description} +\item[c] Constants. +\item[l] Parameters that should only be set locally. +\item[g] Parameters that should only be set globally. +\end{description} + +\section{Checking Parameter assignments} +One of the advantages of having a consistent scheme is that the system +can provide more extensive error checking and debugging facilities. +For example a function that makes a global assignment can check that +it is not passed a local parameter as argument by checking that the +name of the command to be assigned starts with |\g_|. Such checking is +probably too slow for production runs, but the kernel has hooks built +in to allow a format to be made in which all functions perform this +kind of check. A typical section of code might look like +\begin{verbatim} +%<*check> +\def_new:Npn \toks_gset:Nn #1 { + \chk_global:N #1 + \pref_global:D #1 +} +%</check> +%<*!check> +\let_new:NN + \toks_gset:Nn \pref_global:D +%</!check> +\end{verbatim} +The function |\toks_gset:Nn| takes a single token (|N|) specifying a +token register, and globally sets it to the value passed in the second +argument. So typical use would be +\begin{verbatim} +\toks_gset \g_xxx_toks {some value} +\end{verbatim} +In the normal definition, |\toks_gset| can be defined just to be |\let| +to |\global|, as the primitive token register does not require any +explicit assignment function. This is the |%<*!check>| code above. +However the alternative definition first checks that the argument +passed as |#1| is a global parameter and raises an error if it is not. +It does this by taking apart the command name passed as |#1| and +checking that it starts |\g_|. + +\section{Consistent use of accessor functions} +The primitive \TeX\ syntax for register assignments has a very minimal +syntax, and apart from box functions there are no explicit functions +for assignment or use of the registers. This makes it very difficult +to implement alternative data types with a syntax that is at all +similar to the syntax for the primitives, and also encourages a coding +style that is very error prone. + +As noted in the example given above, The \LaTeX\ data types are +provided with explicit functions for setting and using the parameters +even when these have essentially empty definitions. This allows for +better error checking as described above, and also allows the +construction of alternative data types with a similar interface. +For example the `fake counter' data type mentioned previously works at +the user level just like the data type based on primitive count +registers, internally it does not use count registers though. +Typical functions in the |fcnt| module include: + + +\verb|\fcnt_new:N \l_tempa_fcnt|\\ +Declare the local parameter |\l_tempa_fcnt| as a fake counter. + +\verb|\fcnt_add:Nn \l_tempa_fcnt \c_thirty_two|\\ +Increment the counter by 32. + +\section{Expansion Control} +Anyone who programs in \TeX\ is used to the problem of arranging that +arguments to functions are suitably expanded before the function is +called. A couple of real examples copied from \texttt{latex.ltx}: +\begin{verbatim} +\global + \expandafter\expandafter\expandafter + \let + \expandafter + \reserved@a + \csname\curr@fontshape\endcsname + +\expandafter + \in@ + \csname sym#3\expandafter\endcsname + \expandafter{\group@list}% +\end{verbatim} +The first piece of code is a global |\let|. The token to be defined +is obtained by expanding \verb|\reserved@a| one level. The command +that it is to be let too is obtained by fully expanding +\verb|\curr@fontshape| and then using the tokens produced by that +expansion to construct a command name. This results in the mess +of interwoven \verb|\expandafter| and \verb|\csname| beloved of all +\TeX\ programmers, and code that is essentially unreadable. + +A similar construction using the conventions outlined here would be +\begin{verbatim} +\glet:Oc + \reserved_a: \l_current_font_shape_tlp +\end{verbatim} +The command \verb|\glet:Oc| is a global |\let| that expands its +argument once, and generates a command name out of its second +argument, before making the definition. This produces coe that +is far more readable. + +Similarly the second piece of code above produces a token list by +expanding \verb|\group@list| once, and then creates a command name out +of `\verb|sym#3|' (this is inside the definition of another fumction). +The function |\in@| is called which tests if its first argument occurs +in the token list of its second argument. + +Again it would be much clearer, if the above function \verb|\in@| was +called (say) \verb|\test_if_in:nn| (a function taking two normal +`\texttt{n}' arguments) and then a variant function was defined with +the appropriate argument types and simply called as follows: +\begin{verbatim} +\test_if_in:co {sym#3} \group_list: +\end{verbatim} +Note that apart from the lack of \verb|\expandafter| the space after +\verb|}| will be silently ignored. + + +For many common functions the kernel will provide functions with a +range of argument forms, and similarly it is expected that extension +packages providing new functions will make then available in the more +common forms. However There will be occasions where it is necessary +to construct such a variant form. + +A consistent mechanism is provided by the kernel to produce functions +with any argument type, starting from a function that takes `normal' +\TeX\ delimited arguments. Suppose you have a function \verb|\cmd:nnn| +that takes two arguments, and you need to construct \verb|\cmd:cnx| +a variant form in which the first argument is passed as a \emph{name} +of a command, and the third argument must be fully expanded before +being passed to \verb|\cmd:nnn|. + +One simply defines \verb|\cmd:cx| as follows: +\begin{verbatim} +\def:Nn \cmd:cnx {\exp_args:Ncnx \cmd:nnn} +\end{verbatim} +The function \verb|\exp_args:Ncnx| takes as its first (\texttt{N}) +argument the `base' function, and then grabs the next three arguments +from the token stream, acts on the first with \verb|\csname|, and the +last with \verb|\edef| and then constructs a call to the base function +with suitably transformed arguments. So +\begin{verbatim} +\cmd:cnx {abc}{pq}{\rst\xyz} +\end{verbatim} +is equivalent, but emminently more readable, to +\begin{verbatim} +\edef\temp{\rst\xyz} +\expandafter\cmd:nnn + \csname abc\expandafter\endcsname + \expandafter{% + \expandafter p\expandafter q% + \expandafter}% + \expandafter{\temp} +\end{verbatim} + +A large range of argument processing functions are provided in +addition to \verb|\exp_args:Ncnx|. If you need a particular argument +combination for which a function is not provided, one may be +constructed in a simple way. For example you need to construct +\verb|\exp_args:Nxcxcxc| a function that fully expands arguments +1,~3 and~5 of a given function, and produces commands to pass as +arguments 2,~4 and~6 using \verb|\csname|. The definition is simply +\begin{verbatim} +\def:Npn \exp_args:Nxcxcxc + {\::x\::c\::x\::c\::x\::c\:::} +\end{verbatim} +Similar functions, |::o| etc exist for all the other argument types, +and they may be strung together in any order, terminated by +\verb|\:::| to create a function which processes arguments in the +desired way. + +As hopefully demonstrated, the use of variant forms greatly improves +the readability of the code, and experience shows that the longer +command names which result from the new syntax do not realy make the +process of \emph{writing} the code any harder. + +\section{The Current Experimental Distribution} +The initial implementations of a \TeX\ format using this kind of +syntax were made with an unreleased (and non functional) format +(which pre-dates \LaTeXe!) The current distribution consists of a +subset of the functionality of that format, converted to run as +packages on top of \LaTeXe. + +The intention is to allow experienced \TeX\ programmers to experiment +on the system and to comment on the interface. This means that +\textbf{\itshape the interface will change}. No part of this system, +including the names of any commands should be relied upon as being +available in a later release. Please do \emph{experiment} with these +packages, but do not use them for documents that you expect to keep +unchanged over a long period. + +In view of the proposed experimental use for this distribution, we +currently have only converted a few modules for use with \LaTeXe. +These set up the basic conventions, and then implement a few basic +programming constructs such as lists and sequences. These are really +to give a flavour of the code, and to indicate that the intention is +that the kernel provide a sufficiently rich set of programming +constructs so that packages may use them and thus more efficiently +share code, unlike the situation in the current \LaTeX\ where every +large package implements its own version of lists, stacks etc. + +The current packages are: +\begin{description} +\item[l3names] +Sets up the basic naming scheme, including naming the \TeX\ +primitives. If used with the option \texttt{[removeoldnames]} +then the old primitive names such as \verb|\box| are \emph{undefined} +and thus made available for user definitions. Use of this option might +possibly break existing \TeX\ code! + +\item [l3basics] +Some basic definitions that are used by the other packages. + +\item[l3chk] +Functions to check (and make) definitions (comparable to the existing +\verb|\newcommand| or \verb|\renewcommand|). + +\item[l3tlp] +Token List Pointers. A basic \LaTeX3 data type for storing token lists. +(These are essentially macros with no arguments.) + + +\item[l3expan] +The argument expansion module discussed in the previous section. + + +\item[l3quarq] + +A `quark' is a command that is defined to expand to itself. So it may +not be directly used (it would generate an infinite loop) but has many +uses as special markers within \LaTeX\ code. + + +\item[l3seq] +A module implementing the basic list and stack data types. + +\item[l3prop] +Property lists are the data type for handling key/value assignments. +\end{description} + +The distribution also contains the \TeX\ source for this document, a +docstrip install file and two small test files. + +\begin{thebibliography}{1} + +\bibitem{A-W:LLa94} +Leslie Lamport. +\newblock {\em {\LaTeX:} A Document Preparation System}. +\newblock Addison-Wesley, Reading, Massachusetts, second edition, 1994. + +\end{thebibliography} + +\end{document} + + |