% progdoc.doc 21 Sep 90 %------------------------------------------------------------ % (c) 1988-1990 by J.Schrod. Copyright conditions see below. % % Macro package for the documentation of programs (and TeX macros) % MAKEPROG is needed % % % VERSION HISTORY (MSCF -- most significant change first) % % DATE WHO REMARK % 90-09-21 js table of contents and headline added. % 89-09-08 js repaired initialization of \char_count. % 89-06-30 js Definition of \beginverbatim/\endverbatim. % 88-12-05 js first version (for ftp/Bitnet). % bug reports to: % % Detig$\,\cdot\,$Schrod \TeX{}sys % Joachim Schrod % Kranichweg 1 % % D-6074 R\"odermark-Urberach % FR Germany % % Tel. (+6074) 1617 % Bitnet: XITIJSCH@DDATHD21 % author: % js: Joachim Schrod % contributors: % (-: place enough for your name! :-) % documented with itself... \input progdoc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % local macros % \font\mc=cmr9 % for names like ASCII \font\sc=cmcsc10 % caps and small caps 10pt \def\WEB{{\tt WEB\/}} \def\LaTeX{{\rm L\kern-.36em\raise.3ex\hbox{\sc a}\kern-.15em\TeX}} \def\GNU{{\mc GNU}} % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \title{{\tt progdoc} --- A {\tt WEB\/}-like Markup} \chap Introduction. \WEB{} systems allow the documentation of programs by supporting the separation in program fragments which can be collected and rearranged afterwards. This allows {\it top-down\/} programming as well as the {\it bottom-up\/} design of programs. Each program fragment can be documented, usually with \TeX{}. The creation of a new \WEB{} system has become easy with the introduction of Spider \WEB{} but even within Spider \WEB{} the lexical analysis is bound to languages of the Algol family. Building up \WEB{} systems for languages which are line oriented or even for ``exotic'' programming languages like \TeX{} (with a dynamic lexical, syntactical and semantical structure) remains to be difficult. This macro package was built to allow good documentation for programs in languages for which \WEB{} does not exist. It separates a program text in sections that can be documented. All sections, collected sequentially, will result in the complete program. In each section begin and end of the program part are marked with |\beginprog| and |\endprog|, this program part will be formatted as is input (``verbatim''). The separation of the sections is done in this version (which is based on Plain~\TeX{}) by the macros |\chap| and |\sect|. In the documentation part of a section text pieces, e.g.\ names of variables, can be inserted verbatim to demonstrate the connection to the program text. These parts are included in vertical bars (`{\tt\vbar}') while the vertical bar itself can be obtained by |\origvert|. Longer text parts can be printed verbatim by enclosing them in |\beginverbatim| and |\endverbatim| which behaves like |\beginprog| and |\endprog| except that the text between is not copied to the program file. This macro package does not offer the creation of an index because it is not known which syntactical tokens the language has that should be mentioned in an index. As well the creation of a table of contents and of headings would be preferable to allow better orientation in the document. Some remarks about future enhancements which should be done urgently are set in slanted type. \sect This program is free software; you can redistribute it and/or modify it under the terms of the \GNU{} General Public License as published by the Free Software Foundation; either version~1, or (at your option) any later version. This program is distributed in the hope that it will be useful, but {\bf without any warranty\/}; without even the implied warranty of {\bf merchantability\/} or {\bf fitness for a particular purpose}. See the \GNU{} General Public License for more details. You should have received a copy of the \GNU{} General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675~Mass Ave, Cambridge, MA~02139, USA. \sect We have to realize three parts: (1)~the formatting of rather small verbatim texts in a line, (2)~the formatting of larger parts of program and (3)~the document structuring elements for the separation of the sections. But before we start we declare some shorthands for category codes. By declaring the at sign~(`|@|') as well as the underscore~`(|_|)' as letters we can use them in our macros. (I agree with D.~Knuth that |\identifier_several_words_long| is more readable than |\IdentifierSeveralWordsLong| and in every case better than |\p@@@s|.) With the at sign we can use the ``private'' Plain macros and with the underscore we can make our own macros more readable. But as we have to restore these category codes at the end of this macro file we store their former values in the control sequences |\atcode| and |\uscode|. This method is better than to use a group because not all macros have to be defined global this way. \beginprog \chardef\escape=0 \chardef\subscript=8 \chardef\letter=11 \chardef\other=12 %\chardef\active=13 % is defined in Plain already \chardef\atcode=\catcode`\@ \chardef\uscode=\catcode`\_ \catcode`\@=\letter \catcode`\_=\letter \endprog \chap Local Verbatim Formatting. The main point of every verbatim formatting is the switching of the character codes of all characters that have a special meaning for \TeX{}. This can be done with the control sequence |\dospecials| that applies the control sequence |\do| to all special characters. Additionally, every line is regarded as a paragraph without indentation. Between two paragraphs, i.e.\ between two lines, no extra space is set. Finally all blanks and tab characters shall be obeyed and the inter word space after sentence terminators shall not be enlarged. The activation of the tab character expansion with |\obeytabs| is analog to |\obeyspaces| in {\tt plain.tex\/}. Verbatim texts are set in monospace, we use (like in \WEB{}) the character set with the extended {\mc ASCII}. \beginprog \font\tentex=cmtex10 % typewriter extended ASCII 10pt \let\ttex=\tentex % only for PLAIN with base size 10pt \def\setup_verbatim{% \def\do##1{\catcode`##1\other}\dospecials \parskip\z@skip \parindent\z@ \obeylines \obeyspaces \obeytabs \frenchspacing \ttex } \let\tab=\space \begingroup \catcode`\^^I=\active% % Attention: no tabs! \gdef\obeytabs{\catcode`\^^I=\active\def^^I{\tab}} \global\let^^I=\tab% % if an active tab appears in a \write \endgroup \endprog \sect After having saved the old meaning of `{\tt\vbar}' in |\origvert| and after declaring |\vbar| as a synonym for the character that has the code of a vertical bar in the current font, the vertical bar can be made active. Then we call |\setup_verbatim|. But the newline characters shall not be processed, they shall be treated as blank space. This can be done by defining |\par| as |\space|. The next vertical bar in the input closes the group which becomes an (unbreakable) |\hbox| then. The old meanings of the special characters and of the vertical bar are restored and \TeX{} is in normal (horizontal) mode again. \beginprog \let\origvert=| \chardef\vbar=`\| \catcode`\|=\active \def|{% \leavevmode \hbox\bgroup \let\par\space \setup_verbatim \let|\egroup } \endprog \chap Program Fragments in Verbatim. We need macros to format the program fragments without any linebreaking. Such a text area shall start with the macro |\beginprog| resp.\ |\beginverbatim| and end with |\endprog| resp.\ |\endverbatim|. The end macros must stand at the very beginning of a line and must be followed by white space (blank, tab or newline character). After all these macros the rest of the line is ignored. Two demands must be regarded: There should be no length restrictions for the processed text, and the tab characters should be expanded so that this macro works on PC's and on VAXes, too. \sect The implementation method is quite simple: We read the next line, test, wether the end is reached (by comparing with the end line) and otherwise set the line actually read. Every character is inspected and tabular characters are expanded. The verbatim text is started with |\begin_verbatim| which will be either called by |\beginprog| or by |\beginverbatim|. These macros will also define the contents of the end line. Whether a line is set or whether the end of the processed area is reached is indicated by the switch |\if@print|. At the beginning of the |\begin_verbatim| macro most settings are done with |\setup_verbatim| (the vertical bar must be handled separately) and the rest of the line is ignored. As everything is done within a group, the end of the verbatim text can be processed by simply closing this group. For the user it looks as if |\endprog| or |\endverbatim| terminates the processing, but it just serves for the identification of the end, the true processing is done with the internal macro |\end_verbatim|. \beginprog \newif\if@print \def\begin_verbatim{% \endgraf \bigbreak \begingroup \setup_verbatim \catcode`\|\other \@printtrue \ignore_rest_line } \let\end_verbatim=\endgroup % internal command ! \endprog \sect The first line is ignored, all the other lines are identified with |\set_next_line| and processed with |\do_set|. This separation in identification and processing allows that the line end character is active in the definition only for a short time. When a line is to be formatted, we first check with |\check_print| whether it contains |\endprog|, otherwise it is printed with |\print_char|. The printing must be done for every character individually because we want to check for tab characters; the exact algorithm is described below. Here we just have to note that |\print_char| is used with two parameters of which the second one is terminated with the token |\end_line|. The first parameter is the first character of the line, the second parameter is the rest of the line. If the line is empty, the argument of |\do_set| is empty, too; so the activation of |\print_char| must be finished with two |\end_line|. Then the first |\end_line| is the first argument for |\print_char| and the second argument is empty. But if the line did contain something, the second |\end_line| is evaluated, for this case it is defined as |\relax|. Finally we call |\set_next_line| again to format the next line. If the end is reached, i.e.\ if a line with |\endprog| was found, |\set_next_line| is redefined as |\relax|. This can be done because the original meaning is restored when the group is closed by |\end_verbatim|. \beginprog \begingroup \obeylines% % ^^M is active! ==> every line must end with % \gdef\ignore_rest_line#1^^M{\set_next_line}% \gdef\set_next_line#1^^M{\do_set{#1}}% \endgroup \def\do_set#1{% \endgraf \check_print{#1}% \if@print \indent \print_char#1\end_line\end_line \else \let\set_next_line\end_verbatim \fi \set_next_line } \let\end_line=\relax \endprog \sect Before we attack the problem of formatting a line, we declare |\check_print| that checks the end of the verbatim mode. We have to do two things: we must split everything in front of the first blank or tab character and compare it with |\endprog|. The splitting is easy because the line which is our first argument contains blanks and tabulators as active characters. First we call |\cut_at_tab| that demands a tab character as separator for its two parameters so that everything in the line in front of the first tabulator is part of the first parameter. If there is no tab character in the line, we append one so that the second parameter is empty. The same trick is used to separate the part in front of the first blank character from the resulting first part. The check is done with |\do_check|. We use a separate macro here so that we can indent it (in the following definition blanks are active!) \beginprog \begingroup \obeyspaces\obeytabs \gdef\check_print#1{\cut_at_tab#1^^I\end_line} \gdef\cut_at_tab#1^^I#2\end_line{\check_first_part#1 \end_line}% blank ! \gdef\check_first_part#1 #2\end_line{\do_check{#1}} \endgroup \endprog \sect |\do_check| compares the line with a sample line that is available in |\end_verbatim_line|. This macro will be defined later. \beginprog \def\do_check#1{% \def\@line{#1}% \ifx \@line\end_verbatim_line \@printfalse \fi } \endprog \sect Now we can set a line: we start with the first character, followed by the rest of the line. Each character is counted in |\char_count|. At the beginning of a line |\char_count| is~0, it is reset at the end of the line. \beginprog \newcount\char_count \char_count\z@ \def\print_char#1#2\end_line{% \print_first_char{#1}% \print_rest_of_line{#2}% } \endprog \sect For each character that is set |\char_count| is incremented. If a character is a tabulator, we set with |\print_tab| the required amount of blank characters, otherwise the character itself. We must compare the character that is stored in |\@char| with a macro whose ``first-level'' expansion is an active tabulator. For this case we declare |\@tab|. \beginprog {\obeytabs\gdef\@tab{^^I}} \def\print_first_char#1{% \def\@char{#1}% \advance \char_count\@ne \ifx \@char\@tab \print_tab \else \@char \fi } \endprog \sect If we want to fill the line with blank spaces up to the next column with a number that can be divided by~8, we must be able to compute the column number modulo~8, but \TeX{} has no modulo operator. So we define the macro |\mod_viii| that computes its argument modulo~8 and returns the result in the counter |\count_mod_viii|. For the computation we use the temporary counter |\count@|. \beginprog \newcount\count_mod_viii \def\mod_viii#1{% \count@ #1\relax \count_mod_viii\count@ \divide \count@ 8\relax \multiply \count@ 8\relax \advance \count_mod_viii -\count@ } \endprog \sect Now we can declare |\print_tab|. We must remember that |\char_count| was incremented already, if we set only one blank character the counter keeps untouched. \beginprog \def\print_tab{% \loop \space \mod_viii\char_count \ifnum \count_mod_viii>\z@ \advance \char_count\@ne \repeat } \endprog \sect If the rest of the line is empty, we have finished the task. |\char_count| is reset to~0 for the next line. Inside the |\else| part of |\ifx| |\print_char| should not be used directly because this costs too much storage of \TeX{}. Instead we set a control sequence |\next| that is processed afterwards, depending on the result of the comparison. If there is still something to set, we use |\print_char| again, otherwise a syntactically similar macro that expands to |\relax|. \beginprog \def\print_rest_of_line#1{% \def\@line{#1}% \ifx \@line\empty \char_count\z@ \def\next##1\end_line{\relax}% \else \let\next\print_char \fi \next#1\end_line } \endprog \sect Now we are ready to define the two ``user accessible'' macros |\beginprog| and |\beginverbatim|. They must define the prototyp end line |\end_verbatim_line| which will be compared against every line in the verbatim text. During the definition of |\end_verbatim_line| it must be cared for that the escape character~`|\|' is a printable character: A comparison with |\ifx| demands identical category codes. As a temporary escape character we use the slash. \beginprog {\catcode`\/=\escape \catcode`\\=\other % / is temporary escape char /gdef/beginprog{% /gdef/end_verbatim_line{\endprog}% /begin_verbatim } /gdef/beginverbatim{% /gdef/end_verbatim_line{\endverbatim}% /begin_verbatim } } % here \endgroup can't be used \endprog \chap Document Structuring. The layout of the document shall be like in \WEB{}; this can be done easily in Plain. A document -- which may have a title -- consists of sections which are numbered. We distinguish between main sections which start a group of sections and between normal sections within a group. Main sections have a title, a table of contents with all titles is output at the end. \sect The number of the current section is stored in the counter |\sectno|. \beginprog \newcount\sectno \sectno=\@ne \newtoks\title \title{} \endprog \sect As usual, the name |\toc_file_name| of the table of contents file |\toc_file| is built from the jobname and the extension |toc|. We open the file and output a |\relax| so we are save that something is in the file and it can be read in later without any difficulties. In this file lines are written with the main section numbers, their titles, and their page numbers, respectively. Every line is marked with |\tocline| which is assumed to have the given three parameters. Now we define |\tocline| to be |\relax| so that |\write| does not expand it. \beginprog \def\toc_file_name{\jobname.toc } % <--- Space!! \newwrite\toc_file \immediate\openout\toc_file=\toc_file_name \immediate\write\toc_file{\relax} \let\tocline=\relax \endprog \sect We want a headline with the current section number, the document title, the last main section title, and the page number. The footline is empty. To realize this headline every section must |\mark| it's section number {\it in front of\/} a potentiel page break (so the number may be accessed by |\topmark|) and every main section must define |\main_title| to be it's title. \beginprog \headline{% \tenrm \S\topmark \qquad \the\title \hfil \main_title \qquad \tenrm \folio } \footline{\hfil} \endprog \sect But this does not work the way it should: On the first page |\topmark| will be always empty! Here we borrow a trick from |webmac.tex|, we insert a first garbage page which will be discarded by the output routine. \beginprog \vbox to \vsize{} \output={% \setbox\z@ \vbox{\unvbox\@cclv}% % throw away the first page \global\output{\plainoutput}% % restore Plain output routine } \endprog \sect The main sections are started with the macro |\chap|. It has one parameter, the title of the section group. This parameter must be terminated by a dot. We start a new page, typeset the title in bold face and separate it from the section text with a |\medskip|. This text, the first paragraph of the documentation part of the section, is formatted without paragraph indentation. At the beginning of the new page we output the section number, the title, and the page number to the contents file. We must take care to expand the section number right now because it will be incremented and would not be the correct value during the output routine. The deletion of the paragraph indentation is done with |\everypar|. We do not care if |\everypar| is used for something else and delete it after usage. \beginprog \def\chap#1.{% \endgraf \mark{\number\sectno}% % gather information for headline \vfill\supereject \begingroup \def\protect##1{\string ##1 }% % <--- Space!! \edef\write_toc_sectno{\write\toc_file{\tocline{\number\sectno}}}% \write_toc_sectno \write\toc_file{{#1}{\folio}}% \endgroup \gdef\main_title{#1}% % more information for headline \noindent {\bf \number\sectno.\quad #1.}% \global\advance \sectno\@ne \endgraf \medskip \nobreak \everypar{% % remove next parindent \setbox0\lastbox \global\everypar{}% }% } \endprog \sect Normal sections are started with |\sect|. This macro has no parameter. Between two paragraphs we set 2~pica space (if possible) and prefer the page breaking. Between the section number and the text one quad of space is set. \begingroup \sl The page break manipulation should be more sophisticated. \endgroup \beginprog \def\sect{% \endgraf \mark{\number\sectno}% % gather information for headline \vskip 2pc plus 1pc minus 6dd \goodbreak \noindent {\bf \number\sectno.}\quad \ignorespaces \global\advance \sectno\@ne } \endprog \sect At the end we add a page with the table of contents. This is triggered by |\bye|. If the user does not want this table he may use |\byebye|. \beginprog \def\byebye{\par\vfil\supereject\end} % original definition of \bye \def\bye{% \endgraf \vfill \supereject \immediate\closeout\toc_file \message{Table of contents:}% \leftskip\z@skip \rightskip\z@skip \parskip\z@skip \parindent\z@ \parfillskip\z@skip \normalbaselines \headline{\hfil}% \footline{\hfil \folio \hfil}% \def\HeaderToc{\rightline{Section\hbox to\thr@@ em{\hss Page}}}% \def\tocline##1##2##3{% \line{% \ignorespaces ##2% \leaders\hbox to .5em{.\hfil}\hfil \ ##1% \hbox to \thr@@ em{\hss ##3}% }% } \pageno\m@ne % table of contents gets roman page numbers \topofcontents % may redefine \HeaderToc and \tocline! \HeaderToc \catcode`\@\letter % @ and _ are letters (for reading private macros) \catcode`\_\letter \input \toc_file_name \catcode`\@\atcode % @ and _ are no more letters \catcode`\_\uscode \botofcontents \byebye % print the contents page(s) and terminate } \font\titlefont=cmr7 scaled\magstep4 % title on the contents page \font\ttitlefont=cmtt10 scaled\magstep2 % typewriter type in title \def\topofcontents{ \null \vfill \centerline{\let\tt\ttitlefont \titlefont \the\title} \vskip 1cm } \def\botofcontents{ \vfill\vfill } \endprog \sect We are finished and just have to restore the category codes. \beginprog \catcode`\@=\atcode \catcode`\_=\uscode \endinput \endprog \bye