summaryrefslogtreecommitdiff
path: root/support/splint/examples/ld/ldman.w
diff options
context:
space:
mode:
Diffstat (limited to 'support/splint/examples/ld/ldman.w')
-rw-r--r--support/splint/examples/ld/ldman.w297
1 files changed, 183 insertions, 114 deletions
diff --git a/support/splint/examples/ld/ldman.w b/support/splint/examples/ld/ldman.w
index b6eb733dbd..af55b19111 100644
--- a/support/splint/examples/ld/ldman.w
+++ b/support/splint/examples/ld/ldman.w
@@ -1,5 +1,4 @@
-% Copyright 2012-2020, Alexander Shibakov
-% Copyright 2002-2014 Free Software Foundation, Inc.
+% Copyright 2012-2024, Alexander Shibakov
% This file is part of SPLinT
%
% SPLinT is free software: you can redistribute it and/or modify
@@ -15,27 +14,22 @@
% You should have received a copy of the GNU General Public License
% along with SPLinT. If not, see <http://www.gnu.org/licenses/>.
@s TeX_ TeX
-\def\MRI{{\sc MRI}}
\def\optimization{5}
\input ldman.sty
+\input ldsetup.sty
\modenormal
\input ldfrontmatter.sty
-\input noweb.sty
- \xreflocaltrue
- \readlxrefs % read the reference file if exists before any sections appear
- \newwrite\xrefstream % references for noweb like style of sectioning.
- \immediate\openout\xrefstream=\jobname.xxr
% multi-column output
\input dcols.sty
+\topskip=9pt
\let\oldN\N
\let\N\chapterN
\let\M\textM
+\initauxstream
\showlastactiontrue
-\immediate\openout\gindex=\jobname.gdx
@** Introduction.
-\setupfootnotes
This is a manual documenting the development of a parser that can be
used to typeset \ld\ files (linker scripts) with or without the help
of \CWEB. An existing parser for \ld\ has been adopted as a base, with
@@ -50,14 +44,15 @@ limitations).
Nearly every aspect of the design is discussed, including the
supporting \TeX\ macros that make both the parser and this
documentation possible. The \TeX\ macros presented here are collected
-in \.{ldman.sty} which is later included in the \TeX\ file produced by \CWEAVE.
+in \.{ldman.sty} and \.{ldsetup.sty} which are later included in the
+\TeX\ file produced by \CWEAVE.
@<Set up the generic parser machinery@>=
@G(t)
\ifx\optimization\UNDEFINED %/* this trick is based on the premise that \.{\\UNDEFINED} */
\def\optimization{0} %/* is never defined nor created with \.{\\csname$\ldots$\\endcsname} */
\fi
-\let\nx\noexpand %/* convenient */
+\let\nx\noexpand %/* a convenient shortcut*/
\input limbo.sty %/* general setup macros */
\input yycommon.sty %/* general routines for stack and array access */
@@ -69,23 +64,32 @@ in \.{ldman.sty} which is later included in the \TeX\ file produced by \CWEAVE.
\ifnum\optimization>\tw@
\input yyfaststack.sty
-\else
- \let\stashnext\stashnextwithnothing
\fi
\input yystype.sty %/* scanner auxiliary types and functions */
\input yyunion.sty %/* parser data structures */
-\input yxunion.sty %/* extended parser data structures */
-\expandafter\def %/* adjust the \.{\\yyinput} to recognize \.{\\yyendgame} */
- \expandafter\multicharswitch\expandafter
- {\multicharswitch\yyendgame{\yyinput\yyeof\yyeof\endparseinput\removefinalvb}}
+
+\amendswitch %/* adjust the \.{\\yyinput} to recognize \.{\\yyendgame} */
+ \multicharswitch\near\yyeof\by\yyendgame\to\multicharswitch % /* add a new label */
+\replaceaction\multicharswitch\at\yyendgame % /* replace the new empty action */
+ \by{{\yyinput\yyeof\yyeof\endparseinput\removefinalvb}}\to\multicharswitch
+
+\def\MRI{{\sc MRI}} %/* so we can say that \MRI\ script typesetting is not supported$\ldots$ */
+@g
+
+@ Up to this point, the parser initialization process has been non parser specific (although
+sensitive to the namespace chosen). The next command inputs the data structures used
+by the \ld\ parser. The reader should consult the file below for the details on the \AST\ produced
+by the parser and its semantics.
+@<Define the normal mode@>=
+@G(t)
\input ldunion.sty %@>/* \ld\ parser data structures */@+
@g
@*1 Bootstrapping.
To produce a usable parser/scanner duo, several pieces of code must
be generated. The most important of these are the {\it table files\/}
-(\.{ptab.tex} and \.{ltab.tex}) for the parser and the scanner. These
+(\.{ldptab.tex} and \.{ldltab.tex}) for the parser and the scanner. These
consist of the integer tables defining the operation of the parser and
scanner automata, the values of some constants, and the `action switch'.
@@ -101,10 +105,10 @@ parsing our grammar once again, this time with a \bison\ grammar
designed for typesetting \bison\ files. A special {\it
bootstrapping\/} mode is used to extract the appropriate
information. The name `bootstrapping' notwithstanding, the parser and
-lexer used in the bootstrapping phase are not the minimized versions
+lexer used in the bootstrapping phase are not necessarily the minimized versions
used in bootstrapping the \bison\ parser.
-The first component generated during the bootstrapping pass is a list
+One component generated during the bootstrapping pass is a list
of `token equivalences' (or `aliases') to be used by the lexer. Every
token (to be precise, every {\it named token type}) used in
a \bison\ grammar is declared using one of the
@@ -117,50 +121,44 @@ the token names with their numerical values (accomplished by
\.{\\settokens}). If all tokens are named tokens, no token equivalence
list is necessary to set up the interaction between the lexer and the
parser. In this case (the present \ld\ parser is a typical
-example), the token list serves a secondary role: it provides hints
+example), the token list may also serve a secondary role: to provide hints
for the macros that typeset the grammar terms, after the \.{\\tokeneq}
-macro is redefined to serve this purpose.
+macro is redefined for this purpose.
On the other hand, after a declaration such as `\prodstyle{\%token}
\.{CHAR} \.{"char"}' the string \.{"char"} becomes an alias
for the named token \.{CHAR}. Only the string version gets recorded in
the |yytname| array. Establishing the equivalence between the two
-token forms can now only be accomplished by examining the grammar
+token forms now can be accomplished only by examining the grammar
source file and is delegated to the bootstrapping phase parser.
-\leavevmode\namedspot{bootstrapstates}The other responsibility of the bootstrapping parser is to extract
-the information about \flex\ {\it states\/} used by the lexer from the
-appropriate source file. As is the case with token names, this
-information is output in a rather chaotic fashion by the scanner
-generator and is all but useless for our purposes. The original
-bootstrapping macros were designed to handle \flex's \prodstyle{\%x}
-and \prodstyle{\%s} state declarations and produce a \Cee\ file with the
-appropriate definitions. This file can later be included by the
-`driver' routine to produce the appropriate table file for the lexer.
-To round off the bootstrapping mode we only need to establish the
-output streams for the tokens and the states, supply the appropriate file
-names for the two lists, flag the bootstrapping mode for the
-bootstrapping macros and inline typesetting (\.{\\prodstyle} macros)
-and input the appropriate machinery.
-
-This is done by the macros
-below. The bootstrap lexer setup (\.{\\bootstraplexersetup}) consists
-of inputting the token equivalence table for the \bison\ parser
-(i.e.~the parser that processes the \bison\ grammar file) and defining
-a robust token output function which simply ignores the token values
-the lexer is not aware of (it should not be necessary in our case
-since we are using full featured lexer and parser).
+\leavevmode\namedspot{bootstrapstates}Another (perhaps most important)
+goal of the bootstrapping phase is to extract the information about \flex\
+{\it states\/} used by the lexer from the appropriate source file. As
+is the case with token names, this information is output in a rather
+chaotic fashion by the scanner generator and is all but useless for
+our purposes. The bootstrapping macros are designed to
+handle \flex's \prodstyle{\%x} and \prodstyle{\%s} state declarations
+and produce a \Cee\ file with the appropriate definitions. This file
+can later be included by the `driver' routine to generate the
+appropriate table file for the lexer. To round off the bootstrapping
+mode we only need to establish the output streams for the tokens and
+the states, supply the appropriate file names for the two lists, flag
+the bootstrapping mode for the bootstrapping macros and inline
+typesetting (\.{\\prodstyle} macros) and input the appropriate
+machinery.
+
+This is done by the macros below. The bootstrap lexer setup
+(\.{\\bootstraplexersetup}) consists of inputting the token
+equivalence table for the \bison\ parser (i.e.~the parser that
+processes the \bison\ grammar file) and defining a robust token output
+function which simply ignores the token values the lexer is not aware
+of (it should not be necessary in our case since we are using a full
+featured lexer).
@<Define the bootstrapping mode@>=
@G(t)
-\newwrite\tokendefs %/* token list */
-\newwrite\stlist %/* \flex\ state list */
-\newwrite\gindex %/* index entries */
-
\def\modebootstrap{%
- \edef\bstrapparser{dyytab.tex}%
- \bootstrapmodetrue
\def\bootstraplexersetup{%
- \input bo.tok%
\let\yylexreturn\yylexreturnbootstrap /* only return tokens whose value is known */
%\let\yylexreturn\yylexreturnregular /* should also work */
}%
@@ -173,7 +171,7 @@ since we are using full featured lexer and parser).
Every parser/lexer pair (as well as some other macros) operates
within a set of dedicated {\it namespaces\/}. This simply means that the macros
that output token values, switch lexer states and access various
-tables `tack on' the string of characters representing the current
+tables `tack on' a string of characters representing the current
namespace to the `low level' control sequence name that performs the
actual output or access. Say, \.{\\yytname} becomes an alias of
\.{\\yytname[main]} while in the \.{[main]} namespace. When a parser
@@ -208,16 +206,21 @@ grammar describing prologue declarations uses the \.{[prologue]}
namespace.
\aterm{index}the \.{[index]} namespace is used for typesetting the
-index entries and is not necessarily associated with any parser or
-lexer. Somewhat confusingly, the macros that typeset \TeX\ entries, use
-\.{index} (without the brackets) as a pseudonamespace to display \TeX\
-terms in the index (due to the design of these typesetting macros, many
-of them take parameters, which can lead to chaos in the index). These
-two namespaces are not related but due to `hystorical' reasons (and the
-poorly thought out \TeX\ typesetting macro design) the \.{index} name has
-been retained. In addition, \.{index:visual} is used to adjust the sort order
-of \TeX\ terms (similar to the way \.{\\prettywordpairwvis} macro does).
+index entries for the terms that are automatically inserted by the main parser
+(such as an empty right hand side of a production or an inline action).
+\aterm{index:tex{\rm, }index:visual{\rm, and }texline}
+the macros that typeset \TeX\ entries, use
+\.{index:tex} as a pseudonamespace to display \TeX\
+terms in the index (due to the design of these typesetting macros, many
+of them take parameters, which can lead to chaos in the index).
+The \.{index:visual} pseudonamespace is used for ordering the index entries for
+\TeX\ macros (this is how the entry for \.{\\getsecond} (\texref{/getsecond})
+ends up in the {\ssf P} section of the index). Finally, the \TeX\ typesetting macros
+use the \.{texline} pseudonamespace to process the names of \TeX\ command sequences
+in action code. The reason we refer to these as {\it pseudo\/}namespaces is that
+only the token (or term) names are aliased, and not, say, the finite automata names
+in the case of the \TeX\ typesetting.
\aterm{flexre{\rm, }flexone{, \rm and} flextwo}the parsers for
\flex\ input use the \.{[flexre]}, \.{[flexone]}, and~\.{[flextwo]} namespaces for
@@ -259,14 +262,37 @@ initializing the \bison\ parser (accomplished by inputting
\newtoks\ldcmds
\def\modenormal{%
+ \commonstartup
+ \input ldtexlex.sty% /* \TeX\ typesetting specific to \ld */
+ \expandafter\def\csname index domain translation [L]\endcsname{%
+ {\noexpand\ld\ index}% /* used to typeset the table of contents */
+ {ld index}% /* outline entry */
+ {L\sc D INDEX}% /* index header */
+ }%
+ \def\otherlangindexheader{%
+ B{\sc ISON}, F{\sc LEX}, \TeX, {\sc AND} L{\sc D INDICES}%
+ }% /* modify the index header */
+ @>@[@<Initialize \ld\ parsers@>@]
+ @>@[@<Modified name parser for \ld\ grammar@>@]
+}
+@g
+
+@ @<Common startup routine@>=
+@G(t)
+\def\commonstartup{
\def\appendr##1##2{\edef\appnext{##1{\the##1##2}}\appnext}%
\def\appendl##1##2{\edef\appnext{##1{##2\the##1}}\appnext}%
- \input yyinit.sty%
+ \input yyinit.sty %
\input yytexlex.sty% /* \TeX\ typesetting macros */
- \input ldtexlex.sty% /* \TeX\ typesetting specific to \ld */
+ \input gindex.sty % /* indexing macros specific to \flex, \bison, and \ld */
+ \input noweb.sty % /* \noweb\ style references */
+ \xreflocaltrue
+ \let\sectionlistsetup\lxrefseparator
+ \let\inx\inxmod
+ \let\fin\finmod
+ \termindextrue
+ \immediate\openout\gindex=\jobname.gdx
\let\hostparsernamespace\ldnamespace /* the namespace where tokens are looked up for typesetting purposes */
- @>@[@<Initialize \ld\ parsers@>@]
- @>@[@<Modified name parser for \ld\ grammar@>@]
}
@g
@@ -288,53 +314,88 @@ wreaking havoc on the lexer state stack.
@g
@ This is the \ld\ parser invocation routine. It is coded according to
-a straightforward sequence initialize-invoke-execute-or-fall back.
+a straightforward sequence initialize-invoke-execute-or-fall back. The
+\.{\\preparseld} macro is invoked by \.{\\lsectionbegin} (see \.{limbo.sty}).
+It starts by defining the postprocessing and typesetting macro (\.{\\postparseld})
+followed by the parser setup.
@<Define the normal mode@>=
@G(t)
\def\preparseld{%
\let\postparse\postparseld
- \expandafter\hidecs\expandafter{\ldunion}% /* inhibit expansion so that fewer \.{\\noexpand}s are necessary */
+ \hidecslist\ldunion % /* inhibit expansion so that fewer \.{\\noexpand}s are necessary */
\toldparser
+ \displayrawtable % /* do this after the parser namespaces are setup */
\ldparserinit
\yyparse
}
+@g
+@ The postprocessing macro defines a procedure for typesetting the output and saving the
+parsed result to a file. After that, a generic postprocessing macro is executed.
+@<Define the normal mode@>=
+@G(t)
\def\postparseld{%
- \ifsaveparseoutput
- {\newlinechar=`^^J\immediate\write\exampletable{^^J\harmlesscomment
- parsed table: \the\ldcmds^^J^^J\harmlesscomment
- stashed stream:^^J\the\yystash^^J^^J\harmlesscomment
- format stream: ^^J\the\yyformat}%
- }%
- \fi
- \ifchecktable
- \errmessage{parsed table: \the\ldcmds^^J^^J%
- stashed stream: \the\yystash^^J^^J%
- format stream: \the\yyformat}%
- \fi
- \restorecslist{ld-parser:restash}\ldunion % /* mark variables, preprocess stash */
- \setprodtable
- \the\ldcmds
- \restorecslist{ld-display}\ldunion
- \setprodtable /* use the \bison's parser typesetting definitions */
- \restorecs{ld-display}{\anint\bint\hexint} % /* $\ldots$ except for integer typesetting */
- \the\ldcmds
- \par
- \vskip-\baselineskip
- \the\lddisplay
+ \let\saveparsedtable\saveldtable
+ \let\typesetparsedtables\typesetldtables
+ \postparsegeneric{(ld script)}%
+}
+@g
+
+@ The table output and typesetting macros are responsible for the parsed table output
+to a log file and the typographic representation of the result. The table is output
+after expanding its contents. The commands
+\begindemo
+^\hidecslist\cwebstreamchars
+^\restorecslist{ld-parser-debug}\ldunion
+\enddemo
+are meant to inhibit most expansions so that only the list (iterator) macros are expanded.
+The parser for \ld\ does not currently use the list macros to speed up the parsing process
+but the general set up is here in case it is needed in the future. Unfortunately, the same technique
+cannot be applied to the display of the \.{\\yystash} and \.{\\yyformat} streams (that use linked lists
+by default), since there are too many random sequences that can appear in such streams. Therefore,
+to facilitate debugging, one should expand the lists before displaying them (see \.{yyinput.sty} for
+details; lists with iterators are a bit more convenient due to their flexibility).
+@<Define the normal mode@>=
+@G(t)
+\def\saveldtable#1{{%
+ \hidecslist\cwebstreamchars
+ \restorecslist{ld-parser-debug}\ldunion
+ \expandafter\saveoutputcode\expandafter{\the\ldcmds}\exampletable{#1}%
+}}
+
+\def\typesetldtables{%
+ \begingroup
+ \displayparsedoutput\ldcmds
+ \restorecslist{ld-parser:restash}\ldunion % /* mark variables, preprocess stash */
+ \setprodtable
+ \the\ldcmds
+ \restorecslist{ld-display}\ldunion
+ \setprodtable /* use the \bison's parser typesetting definitions */
+ \restorecs{ld-display}{\anint\bint\hexint} % /* $\ldots$ except for integer typesetting */
+ \the\ldcmds
+ \par
+ \vskip-\baselineskip
+ \the\lddisplay
+ \endgroup
}
+@g
+@ The parsing routine defined above is the first macro in the \ld\ parser stack.
+The only remaining procedure on the stack is an error reporting macro in case
+the parsing pass failed.
+@<Define the normal mode@>=
+@G(t)
\fillpstack{l}{%
\preparseld
{\preparsefallback{++}}% /* skip this section if parsing failed, put \.{++} on the screen */
- \relax % /* this \.{\\relax} serves as a `guard' for the braces */
+ \relax % /* this \.{\\relax} `guards' the braces above during \.{\\poppstack} */
}
@g
@ Unless they are being bootstrapped, the \ld\ parser and its
term parser are initialized by the normal mode. The token typesetting
of \ld\ grammar tokens is adjusted at the same time (see the remarks
-above about the mechanism that is responsible for this). Most nonterminals
+above about the mechanism that is responsible for this). Most terminals
(such as keywords, etc.) may be displayed unchanged (provided the
names used by the lexer agree with their appearance in the script file,
see below), while the typeseting of others is modified in
@@ -358,8 +419,8 @@ examining \.{ltokenset.sty}.
@G(t)
\genericparser
name: ld,
- ptables: ptab.tex,
- ltables: ltab.tex,
+ ptables: ldptab.tex,
+ ltables: ldltab.tex,
tokens: {},
asetup: {},
dsetup: {},
@@ -372,12 +433,28 @@ examining \.{ltokenset.sty}.
host: ld;%
@g
-@ The macros are collected in a single file included at the beginning
-of this documentation.
+@ We also need some modifications to the indexing macros in order to typeset \ld\ terms
+and variables separately.
+@<Additional macros for the \ld\ lexer/parser@>=
+@G(t)
+@g
+
+@ The macros are collected in two files included at the beginning
+of this documentation\footnote{An attentive reader may have noticed that the files
+have the extension \.{.stx} instead of the traditional \.{.sty}. The reason for this
+is the postprocessing step that `cleans' the generated files of various \Cee\
+artifacts output by \CWEB\ and turns \.{ldman.stx} and \.{ldsetup.stx} into
+\.{ldman.sty} and \.{ldsetup.sty} included by \.{ldman.tex}$\ldots$}.
@(ldman.stx@>=
@<Set up the generic parser machinery@>@;
@<Begin namespace setup@>@;
@<Define the bootstrapping mode@>@;
+@<Common startup routine@>@;
+
+@ The macros collected in the file below are not specific to this manual but are needed in
+order to use the \ld\ parser generated and are thus put in a separate which can be included
+by other \TeX\ programs that wish to use them.
+@(ldsetup.stx@>=
@<Define the normal mode@>@;
@<Additional macros for the \ld\ lexer/parser@>@;
@@ -404,23 +481,22 @@ MEMORY
need for a `parser stack' as in the case of the \bison\
parser. If one must be able to display still smaller segments of \ld\
code, using `hidden context' tricks (discussed elsewhere) seems to be
-a better approach.
-%
+a better approach.
+%\saveparseoutputtrue\checktabletrue
+%\saveparseoutputfalse\checktablefalse
@<Example \ld\ script@>=
@i ldexample_l.hx
@ @<The same example of an \ld\ script@>=
-@i ldexample_b.hx
+@i ldexample_t.hx
@ @<Some random portion of \ld\ code@>=
@i ldnp.x
@** Appendix. The original code of the \ld\ parser and lexer is reproduced below. It
-is mostly left intact and is typeset by the pretty printing parser for
-\bison\ input. The lexer (\flex) input is reproduced verbatim and is
-left mostly unformatted with the exception of spacing and the embedded
-\Cee\ code.
+is left mostly intact and is typeset by the pretty printing parser for
+\bison\ input. The lexer (\flex) input is given a similar treatment.
The treatment of comments is a bit more invasive. \CWEB\ silently
assumes that the comment refers to the preceding statement or a group
@@ -446,7 +522,7 @@ creators of \ld.%\checktabletrue\saveparseoutputtrue
@q of index entries depends on the placement of this inclusion @>
@i alphas.hx
-@** Index. \checktablefalse\saveparseoutputtrue
+@** Index. \checktablefalse\saveparseoutputtrue \global\let\secrangedisplay\empty
This section lists the variable names and (in some cases)
the keywords used inside the `language sections' of the \CWEB\
source. It takes advantage of the built-in facility of \CWEB\ to supply
@@ -477,15 +553,8 @@ $$
index entries, as well, mimicking \CWEB's behavior for the
{\it inline \Cee\/} (\.{\yl}$\ldots$\.{\yl}). Such entries are labeled
with $^\circ$, to provide a reminder of their origin.
-\unsetfootnotes
-\def\next{\expandafter\eatone\string}
-\edef\unindexable{{\next\the}{\next\nx}{\next\yy}{\next\yylexnext}{\next\else}{\next\fi}{\next\yyBEGIN}{\next\next}}
-\input gindex.sty
-\closeout\gindex
-\termindexfalse
-\let\inx\inxmod
-\let\fin\finmod
+\makeunindexablex{{\csstring\the}{\csstring\nx}{\csstring\yy}{\csstring\yylexnext}%
+ {\csstring\else}{\csstring\fi}{\csstring\yyBEGIN}{\csstring\next}}
\let\oldMRL\MRL
\def\MRL#1{\smash{\oldMRL{#1}}} % a more sophisticated way to handle it woud be to add a \smash whenever we are
% in the [index] namespace but this is simpler and works as well
-\def\topofcontents{\null\vskip-3\baselineskip\centerline{C{\sc ONTENTS} (\sc\uppercase\expandafter{\title})}\medskip}