@q Copyright 2012-2020, Alexander Shibakov@> @q This file is part of SPLinT@> @q SPLinT is free software: you can redistribute it and/or modify@> @q it under the terms of the GNU General Public License as published by@> @q the Free Software Foundation, either version 3 of the License, or@> @q (at your option) any later version.@> @q SPLinT is distributed in the hope that it will be useful,@> @q but WITHOUT ANY WARRANTY; without even the implied warranty of@> @q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> @q GNU General Public License for more details.@> @q You should have received a copy of the GNU General Public License@> @q along with SPLinT. If not, see .@> \input limbo.sty \def\optimization{5} \input yy.sty \modenormal \let\oldN\N \let\N\textN \let\M\textM % uncomment the next line to leave the typesetting of tokens in its raw state. %\def\hostparsernamespace{[unreacheable]} \let\hostparsernamespace\mainnamespace \input btokenset.sty @** A simple scanner for \flex\ options. This is a `bare-bones' scanner for a subset of the `extended' \bison\ grammar that parses some of the `extensions', namely, the \flex\ state declarations. It does not use the state mechanism itself, and is supposed to be used with the bootstrapping parser, even though this is not strictly necessary. It parses state declarations as long as they are separated into their own \CWEB\ sections and extracts the {\it names\/} of the states. The \flex\ scanner output `driver' does the rest after including the produced header file. If reusing the existing scanner for \bison\ were not a priority a proper way to design a scanner like this is to make it a subset of the existing scanner code. This way portions of the program would be made more reusable and the overall design made more consistent. @s TeX_ TeX @(ssffo.ll@>= @G @> @ @= %{@> @ @=%} @> @ @= %% @> @ @= %% @g @ A couple of handy abbreviations to get started. Note that the definition of a letter is more restrictive in this case since we only need to grab the states of an existing \bison\ lexer. For a bootstrapping scanner like this it is beneficial to fail early while scanning something that is not in its attention domain: it results in faster bootstrapping and lower chance of accidentally parsing something that should not have been. Making the syntax and the grammar more restrictive helps to acheive this, as well as makes the overall design simpler. @= @G(fs1) letter [_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] id {letter}({letter}|[-0-9])* @g @ @= #include #include void define_all_states( void ){} @ A standard combination of options to match the \TeX\ code that drives the scanner. @= @G %option bison-bridge %option noyywrap nounput noinput reentrant %option noyy_top_state %option debug %option stack %option outfile="ssffo.c" @g @ There are not that many regular expressions to list, since the range of tokens recognized by this routine is not very wide. @= @@; @@; @ White space skipping. @= @G(fs2) [ \f\n\t\v] {@> @[TeX_( "/yylexnext" );@]@=} @g @ The rest of it are either identifiers or \.{\%}-options. The typesetting of the appropriate \bison\ tokens below was arranged by inputting \.{btokenset.sty} and relying on the pretty-printing macros for {\it \TeX} to do their job. @= @G(fs2) {id} {@> @ @=} "%x" {@> @[TeX_( "/yylexreturnptr{FLEX_STATE_X}" );@] @=} "%s" {@> @[TeX_( "/yylexreturnptr{FLEX_STATE_S}" );@] @=} . {@> @ @=} @g @ @= @[TeX_( "/iftracebadchars" );@]@; @[TeX_( " /yycomplain{invalid character(s): /the/yytext}" );@]@; @[TeX_( "/fi" );@]@; @[TeX_( "/yylexreturn{$undefined}" );@]@; @ The lexer returns standard \.{\\yyunion} types. @= @[TeX_( "/edef/next{/yylval{{/the/yytextpure}{/the/yytext}" );@]@; @[TeX_( " {/the/yyfmark}{/the/yysmark}}}/next" );@]@; @[TeX_( "/yylexreturn{ID}" );@]@; @** Index. \def\TeXx{\TeX\ material}