diff options
Diffstat (limited to 'info/digests/tex-implementors/message.19')
-rw-r--r-- | info/digests/tex-implementors/message.19 | 628 |
1 files changed, 628 insertions, 0 deletions
diff --git a/info/digests/tex-implementors/message.19 b/info/digests/tex-implementors/message.19 new file mode 100644 index 0000000000..0d430a6284 --- /dev/null +++ b/info/digests/tex-implementors/message.19 @@ -0,0 +1,628 @@ +>From BNB@math.ams.com Mon Oct 23 08:40:43 1989 +Return-Path: <root@ese> +Date: Sun 22 Oct 89 10:51:40-EST +From: bbeeton <BNB@math.ams.com> +Subject: Knuth's description of new TeX and Metafont +To: TeX-implementors@math.ams.com +Mail-System-Version: <VAX-MM(229)+TOPSLIB(132)+PONY(228)@MATH.AMS.COM> +Sender: BNB@vax01.ams.com + +Date: 22 October 89 Message No: 019 + +To: TeX implementors and distributors + +From: Barbara Beeton + +Subject: Knuth's description of new TeX and Metafont + + +Knuth has sent me the article below for TUGboat and to disseminate to +the world. You are welcome to distribute this further, but if you do, +please mark it "draft", and note that it will appear in TUGboat. + +I have successfully retrieved the main TeX README file from the directory +/pub/tex at labrea.stanford.edu and scanned the contents of the TeX +directories. Nearly everything was brought from Score with the original +timestamp more-or-less intact. The directory structure is changed somewhat +from Score, but it is logical, and should be easy enough to navigate. +However, nothing new seems to have appeared, in particular the new errata +(it will be called "errata.fiv") and the additions to the bug files (which +will be called tex82.89 and mf84.89). The errata and TeX bug files will +be in the subdirectory /pub/tex/tex and the MF bug file in /pub/tex/mf . +The README file is included for information. + +I will send a notice as soon as I see that the new .WEB files and +documentation have been installed at labrea. However, I will be away +from tomorrow through 1 November, so you won't hear anything until at +least the beginning of November. + + +######################################################################## + +[ README file from /pub/tex at labrea.stanford.edu ] + +This is the official TeX distribution directory at Stanford University. +Most of the files here were moved from Score.Stanford.EDU in September +1989 when that host was retired from service. File modification times +were preserved as much as possible. + +All of the files here are available for anonymous FTP by anyone and +may be freely redistributed. Some are copyrighted; see the files +themselves for more details. If you have any questions or comments, +please send mail to "tex@labrea.stanford.edu". + +(Note: at present, things are still somewhat unsettled, so some files +may be missing or have problems. Some of the directories are being +reorganized. By the end of October 1989 it should be more stable.) + +Here is a short description of what is contained in the subdirect- +ories. For further information, refer to the file README (or a +similar name) in each subdirectory. + +amsfonts AMS Cyrillic and math symbols fonts + +amstex The AMSTeX macros + +bibtex The BibTeX program and related files + +cm Metafont sources for the Computer Modern fonts + +fonts TFM files for standard (and some non-standard) fonts + +gf GF font files (for Imagen and Lasewriter) + +imagen Various files to support Imagen printers + +latex LaTeX macros, font sources, related files + +lib Input files read by TeX, Metafont and other programs + +ln03 Various files to support LN03 printers + +mf Metafont source code and documentation + +mfware Metafont utilities + +misc Miscellaneous files (temporary) + +tex TeX source code and documentation + +texware TeX utilities + +texhax TeXhax archives + +tugboat Files for things mentioned in TUGboat + +unix Unix TeX distribution from University of Washington + +web The WEB system + + +######################################################################## + +% Knuth's article + +\font\logo=logo10 % font used for the METAFONT logo +\font\logosl=logosl10 % font used for slanted METAFONT logo + +\def\MF{{\logo META}\-{\logo FONT}} +\def\MFbook{{\sl The {\logosl METAFONT}\kern1pt book}} +\def\TeX{T\hbox{\hskip-.1667em\lower.424ex\hbox{E}\hskip-.125em X}} +\def\ldt{\mathinner{\ldotp\ldotp}} + +\line{\bf The New Versions of \TeX\ and \MF\ \hfill by Donald E. Knuth} +\bigskip +\noindent +For more than five years I held firm to my conviction that a stable system +was far better than a system that continues to evolve. But during the TUG +meeting at Stanford in August, 1989, I~was persuaded to make one last set of +changes, in order to bring \TeX\ and \MF\ to a state of completion consistent +with their overall philosophy and goals. + +The main reason for the changes was the fact that I~had guessed wrong about +7-bit character sets versus 8-bit character sets. I~believed that standard text +input would continue indefinitely to be confined to at most 128~characters, +since I~did not think a keyboard with 256~different outputs would be +especially efficient. Needless to say, I~was proved wrong, especially by +developments in Europe and Asia. As soon as I~realized that a text formatting +program with 7-bit input would rapidly begin to seem as archaic as the 6-bit +systems we once had, I~knew that a fundamental revision was necessary. + +But the 7-bit assumption pervaded everything, so I needed to take the programs +apart and redo them thoroughly in 8-bit style. This put \TeX\ +onto the operating table and under the knife +for the first time since 1984, and I~had a final +opportunity to include a few new features that had occurred to me or been +suggested by users since then. + +The new extensions are entirely upward compatible with previous versions +of \TeX\ and \MF\ (with a few small exceptions mentioned below). +This means that error-free inputs to the old \TeX\ and \MF\ will still +be error-free inputs to the new systems, and they will still produce the +same outputs. + +However, anybody who dares to use the new extensions will be unable to get +the desired results from old versions of \TeX\ and \MF\null. I~am therefore +asking the \TeX\ community to update all copies of the old versions +as soon as possible. Let us root out and destroy the obsolete 7-bit systems, +even though we were able to do many fine things with them. + +In this note I'll discuss the changes, one by one; then I'll describe +the exceptions to upward compatibility. + +\bigskip +\noindent +{\bf 1. The character set.} +Up to 256 distinct characters are now allowed in input files. The codes that +were formerly limited to the range $0\ldt 127$ are now in the range +$0\ldt 255$. All characters are alike; you are free to use any character +for any purpose in \TeX, assigning appropriate values to its +{\tt{\char'134}catcode}, +{\tt{\char'134}mathcode}, +{\tt{\char'134}lccode}, +{\tt{\char'134}uccode}, +{\tt{\char'134}sfcode}, +and +{\tt{\char'134}delcode}. +Plain \TeX\ initializes these code values for characters above~127 just as +it initializes the codes for ordinary punctuation characters +like~`{\tt{\char'041}}'. + +There's a new convention for inputting an arbitrary 8-bit character +to \TeX\ when you can't necessarily type~it: The four consecutive +characters +{\tt{\char'136\char'136}}$\alpha\beta$, where $\alpha$ and~$\beta$ are +any of the ``lowercase hexadecimal digits'' +{\tt{0}}, +{\tt{1}}, +{\tt{2}}, +{\tt{3}}, +{\tt{4}}, +{\tt{5}}, +{\tt{6}}, +{\tt{7}}, +{\tt{8}}, +{\tt{9}}, +{\tt{a}}, +{\tt{b}}, +{\tt{c}}, +{\tt{d}}, +{\tt{e}}, +or +{\tt{f}}, +are treated by \TeX\ on input as if they were a single character with +specified code digits. For example, +{\tt{\char'136\char'136}80} +gives character code~128; the entire character set +is available from +{\tt{\char'136\char'136}00} +to +{\tt{\char'136\char'136}ff}. +The old convention discussed in Appendix~C, under which character~0 was +{\tt{\char'136\char'136\char'100}}, +character~1 (control--A) was +{\tt{\char'136\char'136}A}, +\dots, +and character~127 was +{\tt{\char'136\char'136}?}, +still works for the first 128~character codes, except that the +character following +{\tt{\char'136\char'136}} +should not be a lowercase hexadecimal digit when the immediately following +character is another such digit. + +The existence of 8-bit characters has less effect +in \MF\ than in \TeX, because \MF's character classes are built in to each +installation. The normal set of 95~printing characters described on +page~51 of +\MFbook\ +can be supplemented by extended characters as discussed on page~282, but this +is rarely done because it leads to problems of portability. \MF's +{\bf char} operator is now redefined to operate modulo~256 instead +of modulo~128. + +\bigskip\noindent +{\bf 2. Hyphenation tables.} +Up to 256 distinct sets of rules for hyphenation are now allowed in \TeX. +There's a new integer parameter called +{\tt{\char'134}language}, +whose current value specifies the hyphenation convention in force. If +{\tt{\char'134}language} +is negative or greater than~255, \TeX\ acts as if +$\hbox{\tt{\char'134}language}=0$. + +When you list hyphenation exceptions with \TeX's +{\tt{\char'134}hyphenation} +primitive, those exceptions apply to the current language only. Similarly, +the +{\tt{\char'134}patterns} +primitive tells \TeX\ to remember new hyphenation patterns for the current +language; this operation is allowed only in the special ``initialization'' +program called {\tt INITEX}\null. Hyphenation exceptions can be added at any +time, but new patterns cannot be added after a paragraph has been typeset. + +When \TeX\ reads the text of a paragraph, it automatically inserts +``whatsit nodes'' into the horizontal list for that paragraph whenever +a character comes from a different +{\tt{\char'134}language} +than its predecessor. In that way \TeX\ can tell what hyphenation +rules to use on each word of the paragraph even if you switch +frequently back and forth among many different languages. + +The special whatsit nodes are inserted automatically in unrestricted horizontal +mode (i.e., when you are creating a paragraph, but not when you are +specifying the contents of an hbox). You can insert a special whatsit +yourself in restricted horizontal mode by saying +{\tt{\char'134}language}$\langle$number$\rangle$. +This is needed only if you are doing something tricky, like unboxing some +contribution to a paragraph. + +\bigskip\noindent +{\bf 3. Hyphenated fragment control.} +\TeX\ has new parameters +{\tt{\char'134}lefthyphenmin} +and +{\tt{\char'134}righthyphenmin}, +which specify the smallest word fragments that will appear at the beginning +or end of a word that has been hyphenated. Previously the values +{\tt{\char'134}lefthyphenmin=2} +and +{\tt{\char'134}righthyphenmin=3} +were hard-wired into \TeX\ and impossible to change. Now plain \TeX\ +format supplies the old values, which are still recommended for most +American publications; but you can get more hyphens by decreasing these +parameters, and you can get fewer hyphens by increasing them. If the sum of +{\tt{\char'134}lefthyphenmin} +and +{\tt{\char'134}righthyphenmin} +is~63 or more, all hyphenation is suppressed. (You can also suppress +hyphenation by using a font with +{\tt{\char'134}hyphenchar=-1}, +or by switching to a +{\tt{\char'134}language} +that has no hyphenation patterns or exceptions.) + +\bigskip\noindent +{\bf 4. Smarter ligatures.} +Now here's the most radical change. +Previous versions of \TeX\ had only one kind of ligature, in which two +characters like~`f' and~`i' were changed into a single character like~`fi' +when they appeared consecutively. The new \TeX\ understands much more +complex constructions by which, for example, we could change +an~`i' following~`f' to a dotless~`\i' while the~`f' remains + unchanged:~`f\i'. + +As before, you get ligatures only if they have been provided in the font +you are using. So let's look at the new features of \MF\ by which +enhanced ligatures can be created. A~\MF\ programmer can specify a +``ligature/kerning program'' for any character of the font being +created. If, for example, the~`fi' combination appears in font +position~12, the replacement of~`f' and~`\i' by~`fi' is specified by +including the statement +$$\hbox{\tt{"i"~=:~12}}$$ +in the ligature/kerning program for {\tt{"f"}}; this is \MF's present +convention. + +The new ligatures allow you to retain one or both of the original characters +while inserting a new one. Instead of {\tt{=:}} you can also write +{\tt{\char'174}=:} if you wish to retain the left character, or +{\tt{=:{\char'174}}} if you wish to retain the right character, +or {\tt{\char'174}=:{\char'174}} if you want to keep them both. +For example, if the dotless~\i\ appears in font position~16, you can +get the behavior mentioned above by having +$$\hbox{% +{\tt{"i" {\char'174}=: 16}} +}$$ +in f's program. + +There also are four additional operators +$$\hbox{% +{\tt{\char'174}=:{\char'076}},\qquad +{\tt{=:{\char'174\char'076}}},\qquad +{\tt{\char'174}=:{\char'174\char'076}},\qquad +{\tt{\char'174}=:{\char'174\char'076\char'076}}, +}$$ +where each {\tt\char'076} tells \TeX\ to shift its focus one position +to the right. For example, if~f and~i had been replaced by~f +and dotless~\i\ as above, \TeX\ would begin again to execute f's +ligature/kern program, possibly inserting a kern before the dotless~\i, +or possibly changing the~f to an entirely different character, etc. +But if the instruction had been +$$\hbox{% +{\tt{"i" {\char'174}=:{\char'076} 16}} +}$$ +instead, \TeX\ would turn immediately to the ligature/kern program for +characters following character~16 (the dotless \i); +no further change would be made between~f and~\i\ even if the font +had something specified there. + +\bigskip\noindent +{\bf 5. Boundary ligatures.} +Every consecutive string of `characters' read by \TeX\ in horizontal mode +(after macro expansion) can be called a `word'. (Technically we consider +a `character' in this definition to be either a character whose +{\tt{\char'134}catcode} +is a +letter or otherchar, or a control sequence that has been +{\tt{\char'134}let} +equal to such a character, or a control sequence that has been defined by +{\tt{\char'134}chardef}, +or the construction +{\tt{\char'134}char}$\langle$number$\rangle$.) +The new \TeX\ now imagines that there is an invisible ``left boundary +character'' just before every such word, and an invisible ``right boundary +character'' just after it. These boundary characters take effect if the font +designer has specified ligatures and/or kerning between them and the +adjacent letters. Thus, the first or last character of a word can +now be made to change its shape automatically. + +A ligature/kern program for the left boundary character is specified within +\MF\ by using the special label~ +{\tt{\char'174\char'174}:} +in a {\bf ligtable} command. A~ligature or kern with the right +boundary character is specified by assigning a value to the new internal +\MF\ parameter +{\it boundarychar}, +and by specifying a ligature or kern with respect to this character. +The +{\it boundarychar\/} +may or may not exist as a real character in the font. + +For example, suppose we want to change the first letter of a word from~`F' +to~`ff' if we are doing some olde English. The \MF\ font designer could then +say +$$\hbox{ligtable {\tt{\char'174\char'174}: "F" {\char'174}:= 11}}$$ +if character 11 is the `ff'. The same ligtable instruction should +appear in the programs for characters like~( and~` and~`` and~- that can +precede strings of letters; then `{\tt Bassington-French}' will +yield `Bassington-ffrench'. + +If the `s' of our font is the pre-19th +century~s that looks like a mutilated~`f', and if we have a modern~`s' +in position~128, we can convert the final~s's as Ben Franklin did by +introducing ligature instructions such as +$$\vcenter{\halign{{\tt{#}}\hfil$\;$&{\tt{#}}\hfil\cr +boundarychar :=&255;\cr +ligtable "s":&255 =:{\char'174} 128,\cr +&"." =:{\char'174} 128,\cr +&"," =:{\char'174} 128,\cr +&")" =:{\char'174} 128,\cr +&"'" =:{\char'174} 128,\cr}}$$ +and so on. (A true oldstyle font would also have +ligatures for +ss and si and sl and ssi and ssl +and~st; it would be fun to create a Computer Modern Oldstyle.) + +The implicit left boundary character is omitted by \TeX\ if you say +{\tt{\char'134}noboundary} +just before the word; the implicit right boundary is omitted if you say +{\tt{\char'134}noboundary} +just after it. + +\bigskip\noindent +{\bf 6. More compact ligatures.} +Two or more ligtables can now share common code. To do this in \MF, you +say `{\bf skipto}~$\langle n\rangle$' at the end of one {\bf ligtable} +command, then you say `$\langle n\rangle$::' within another. Such local labels +can be reused; e.g., you can say {\bf skipto}~1 again after {\tt 1::} has +appeared, and this skips to the {\it next\/} appearance of~{\tt 1::}. There +are 256~local labels, numbered~0 to~255. Restriction: At most 128 ligature +or kern commands can intervene between a {\bf skipto} and its matching label. + +The {\tt TFM} file format has been upwardly extended to allow more than 32,500 +ligature/kern commands per font. (Previously there was an effective limit +of 256.) + +\bigskip\noindent +{\bf 7. Better looking sloppiness.} +There is now a better way to avoid overfull boxes, for people who don't want +to look at their documents to fix unfeasible line breaks manually. Previously +people tried to do this by setting +{\tt{\char'134}tolerance=10000}, +but the result was terrible because \TeX\ would tend to consolidate +all the badness in one truly horrible line. (\TeX\ considers all badness +$\ge10000$ to be infinitely bad, and all these infinities are equal.) + +The new feature is a dimension parameter called +{\tt{\char'134}emergencystretch}. +If +{\tt{\char'134}emergencystretch} +is positive and if \TeX\ has been unable to typeset a paragraph without +exceeding the given tolerances, another pass over the paragraph is made +in which \TeX\ pretends that additional stretchability equal to +{\tt{\char'134}emergencystretch} +is present in every line. The effect of this is to scale down all the +badnesses into a range where previously infinite cases become finite; +\TeX\ will find an optimum solution to the scaled-down problem, and this +will be about as good as possible in a practical sense. (The extra stretching +is not really present; therefore underfull boxes will be reported in warning +messges unless +{\tt{\char'134}hbadness} +is increased.) + +\bigskip\noindent +{\bf 8. Looking at badness.} +\TeX\ has a new internal integer parameter called +{\tt{\char'134}badness} +that records the badness of the box it has most recently constructed. +If that box was overfull, +{\tt{\char'134}badness} +will be 1000000; otherwise +{\tt{\char'134}badness} +will be between~0 and~10000. + +\bigskip\noindent +{\bf 9. Looking at the line number.} +\TeX\ also has a new internal integer parameter called +{\tt{\char'134}inputlineno}, +which contains the number of the line that \TeX\ would show on an error message +if an error occurred now. (This parameter and +{\tt{\char'134}badness} +are ``read only'' in the same way as +{\tt{\char'134}lastpenalty}: +You can use them in the context of a $\langle$number$\rangle$, e.g., by saying +`{\tt{\char'134}ifnum{\char'134}inputlineno{\char'076\char'134}badness ...\ +{\char'134}fi}' +or +`{\tt{\char'134}the{\char'134}inputlineno}', +but you cannot set them to new values.) + +\bigskip\noindent +{\bf 10. Not looking at error context.} +There's a new integer parameter called +{\tt{\char'134}errorcontextlines} +that specifies the maximum number of two-line pairs of context displayed with +\TeX's error messages (in addition to the top and bottom lines, which always +appear). Plain \TeX\ now sets +{\tt{\char'134}errorcontextlines=5}, +but higher level format packages might prefer +{\tt{\char'134}errorcontextlines=1} +or even +{\tt{\char'134}errorcontextlines=0}. +In the latter case, an error that previously involved three or more pairs of +context would now appear as follows: + +\halign{\qquad\qquad{\tt{#}}\hfil\cr +{\char'041} Error.\cr +$\langle$somewhere$\rangle$ The {\char'134}top\cr +\phantom{$\langle$somewhere$\rangle$ The {\char'134}top\ }line\cr +...\cr +1.123 {\char'134}The\cr +\phantom{1.123 {\char'134}The\ }bottom line.\cr} + +\noindent +(If +{\tt{\char'134}errorcontextlines{\char'074}0} +you wouldn't even see the `{\tt{...}}' here.) + +\bigskip\noindent +{\bf 11. Output recycling.} +One more new integer parameter completes the set. If +{\tt{\char'134}holdinginserts{\char'076}0} +when \TeX\ is putting the current page into +{\tt{\char'134}box255} +for the +{\tt{\char'134}output} +routine, \TeX\ will not move anything from insertion nodes into the +corresponding boxes; all insertion nodes will stay in place. Designers of +output routines can use this when they want to put the contents of box~255 back +into the current page to be re-broken (because they might want to change +{\tt{\char'134}vsize} +or something). + +\bigskip\noindent +{\bf 12. Exceptions to upward compatibility.} +The new features of \TeX\ and \MF\ imply that a few things work differently +than before. I~will try to list all such cases here (except when the +previous behavior was erroneous due to a bug in \TeX\ or \MF\null). +I~don't know of any cases where users will actually be affected, because +all of these exceptions are pretty esoteric. + +\medskip $\bullet$\enspace +\TeX\ used to convert the character strings +{\tt{\char'136\char'136}0}, +{\tt{\char'136\char'136}1}, +\dots, +{\tt{\char'136\char'136}9}, +{\tt{\char'136\char'136}a}, +{\tt{\char'136\char'136}b}, +{\tt{\char'136\char'136}c}, +{\tt{\char'136\char'136}d}, +{\tt{\char'136\char'136}e}, +{\tt{\char'136\char'136}f} +into the respective single characters +{\tt p}, +{\tt q}, +\dots, +{\tt y}, +{\tt{\char'041}}, +{\tt "}, +{\tt{\char'043}}, +{\tt{\char'044}}, +{\tt{\char'045}}, +{\tt{\char'046}}. +It will no longer do this if the following character is one of the characters +{\tt 0123456789abcdef}. + +\medskip $\bullet$\enspace +\TeX\ used to insert no character at the end of an input line if +{\tt{\char'134}endlinechar{\char'076}127}. +It will now insert a character unless +{\tt{\char'134}endlinechar{\char'076}255}. +(As previously, +{\tt{\char'134}endlinechar{\char'074}0} +suppresses the end-of-line character. This character is normally +$13=$ ASCII control--M $=$ carriage return.) + +\medskip $\bullet$\enspace +Some diagnostic messages from \TeX\ used to have the notation +{\tt ["80]} \dots {\tt ["FF]} +when referring to characters $128\ldots 255$ (for example when displaying the +contents of an overfull box involving fonts that include such characters). +The notation +{\tt{\char'136\char'136}80} $\ldots$ +{\tt{\char'136\char'136}ff} +is now used instead. + +\medskip $\bullet$\enspace +The expressions +{\tt{char128}} and {\tt{char0}} used to be equivalent in \MF; now +{\bf char} is defined modulo~256 instead. Hence {\tt{char-1}} $=$ +{\tt{char255}}, etc. + +\medskip $\bullet$\enspace +{\tt INITEX} used to forget all previous hyphenation patterns each time +you specified +{\tt{\char'134}patterns}. +Now all hyphenation pattern specifications are cummulative, and you are not +permitted to use +{\tt{\char'134}patterns} +after a paragraph has been hyphenated by {\tt INITEX}. + +\medskip $\bullet$\enspace +\TeX\ used to act a bit differently when you tried to typeset missing +characters of a font. A~missing character is now considered to be a word +boundary, so you will get slightly more diagnostic output when +{\tt{\char'134}tracingcommands{\char'076}0}. + +\medskip $\bullet$\enspace +\TeX\ and \MF\ will report different statistics at the end of a run because +they now have a different number of primitives. + +\medskip $\bullet$\enspace +Programs that use the string pool feature of {\tt TANGLE} will no longer run +without changes, because the new {\tt TANGLE} starts numbering multicharacter +strings at~256 instead of~128. + +\medskip $\bullet$\enspace +{\tt INITEX} programs must now set +{\tt{\char'134}lefthyphenmin=2} and +{\tt{\char'134}righthyphenmin=3} +in order to reproduce their previous behavior. + +\bye + + +######################################################################## + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% Character code reference +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% +% Upper case letters: ABCDEFGHIJKLMNOPQRSTUVWXYZ +% Lower case letters: abcdefghijklmnopqrstuvwxyz +% Digits: 0123456789 +% Square, curly, angle braces, parentheses: [] {} <> () +% Backslash, slash, vertical bar: \ / | +% Punctuation: . ? ! , : ; +% Underscore, hyphen, equals sign: _ - = +% Quotes--right left double: ' ` " +%"at", "number" "dollar", "percent", "and": @ # $ % & +% "hat", "star", "plus", "tilde": ^ * + ~ +% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +[ end of message 019 ] +------- + + |