diff options
author | Norbert Preining <norbert@preining.info> | 2019-09-02 13:46:59 +0900 |
---|---|---|
committer | Norbert Preining <norbert@preining.info> | 2019-09-02 13:46:59 +0900 |
commit | e0c6872cf40896c7be36b11dcc744620f10adf1d (patch) | |
tree | 60335e10d2f4354b0674ec22d7b53f0f8abee672 /info/digests/tex-implementors/033 |
Initial commit
Diffstat (limited to 'info/digests/tex-implementors/033')
-rw-r--r-- | info/digests/tex-implementors/033 | 499 |
1 files changed, 499 insertions, 0 deletions
diff --git a/info/digests/tex-implementors/033 b/info/digests/tex-implementors/033 new file mode 100644 index 0000000000..dffe11ce22 --- /dev/null +++ b/info/digests/tex-implementors/033 @@ -0,0 +1,499 @@ +Date: 1 November 91 Message No: 033 + +To: TeX implementors and distributors + +From: Barbara Beeton + +Subject: Messages from DEK, part 2 + + +Here is the second installment of DEK's September comments. + + +######################################################################## + +Incompatibility of positive/negative integer values + +Date: Tue, 20 Aug 91 16:58:09 MDT +From: Nelson H.F. Beebe <beebe@math.utah.edu> +Subject: Perhaps a bug (design flaw) in TeX + +I think that the `feature' described below qualifies as a design flaw +in TeX, and should be reported to Don Knuth if it has not come up +before; I came across it while testing the statement on p. 178, l. +-11, of + +@string{SV = "Spring{\-}er-Ver{\-}lag"} + +@Book{Seroul:beginners-tex, + author = "Raymond Seroul and Silvio Levy", + title = "A Beginner's Book of {\TeX}", + publisher = SV, + year = "1991", + ISBN = "0-387-97562-4, 3-540-7562-4", + note = "This is a translation and adaption by Silvio Levy of + \cite{Seroul:tex}.", +} + +about the maximum and minimum integers that TeX can handle. +[The text has an error there; I've just completed a comprehensive +errata list for it.] + +TeX does not permit the input of the most negative 32-bit integer +(-2^{31}) on two's complement machines, but you can generate it by +subtraction ((-2^{-31}+1) - 1) and output it correctly. + +This makes the statement at the top of p. 118 of the TeXbook a lie: +registers are capable of containing the number -2147483648, NOT +-2147483647, provided the host architecture has two's complement +arithmetic, which is true for almost all machines today, and certainly +the vast majority of TeX implementations. UNIVAC and CDC mainframes +had one's complement arithmetic, but also had words of more than 32 +bits, and as far as I am aware, only some calculators may use +sign-magnitude representation; both of these systems have signed +zeros, and extreme values that are equal in magnitude. + +I believe that a programming language, which TeX surely is, ought to +be able to read what it can write. + +This asymmetry could be avoided if the code in section 445 of TeX: The +Program accumulated the number as a negative value, then flipped the +sign if necessary. Authors of textbooks, computer programs, and +language run-time libraries, should not make this mistake, yet the +error continues to be repeated. + +While TeX detects overflow from multiplication, it does not detect +overflow from negation. + +Here is an example: + +This is TeX, C Version 3.0 + +% Try the value -(2^{31}) (most negative two's complement number) +*\count0=-2147483648 +! Number too big. +<*> \count0=-2147483648 + +% Input the value -(2^{31}-1) +*\count0=-2147483647 + +*\showthe\count0 +> -2147483647. + +% Now generate the most negative two's complement number +*\advance \count0 by -1 + +*\showthe\count0 +> -2147483648. + +% Now demonstrate that integer overflow is undetected on sign inversion: +\count1=-\count0 + +*\showthe\count1 +> -2147483648. + +% However, integer overflow is caught on multiplication: +\multiply\count0 by \count0 +! Arithmetic overflow. +<*> \multiply\count0 by \count0 + +======================================================================== + + [ dek: + TeX is _not_ a programming language in the general sense of + supporting arithmetic at extreme values. There are lots of + _dimen_ values that TeX can write but not read. + Probably a flaw, but a permanent one. In general, arithmetic + in TeX is not supposed to push the handling conditions; + making that all work would cause significant performance penalty. + ] + +************************************************************************ + +File name overflow of string pool + +[ Since this report, I have seen a couple of other reports on this + topic in the electronic discussion lists, mostly from Europe. + While not a bug, it can certainly be a serious inconvenience. + A couple of the reports have mentioned building nonstandard + versions of TeX with a separate pool of file names; not good for + compatibility. ] + +Date: Fri, 12 Jul 91 19:06 +0200 +From: "Johannes L. Braams" <J.L.Braams%pttrnl.nl@pucc.PRINCETON.EDU> +Subject: Bug/misfeature in TeX? + + We have run into a problem with TeX. + We have an application where we would like to \input about + 2400 files. We can't do that because TeX runs out of string pool + space. This application is rather important because it concerns + the reports the lab has to make each quarter of a year. + + When I studied TeX the program to find out what happens when a file + is being \input I found that the name of the file is stored in + string pool. AND it never gets removed from the string pool (as far + as I could find out). + What I don't understand is why filenames are written to string pool + in the first place. + Isn't it possible to use some kind of stack or array mechanism to + store filenames? It should then be possible to free the memory + used to store a filename when the file gets closed and the filename + is no longer needed. + + Do you know the answer or someone who does? Or is this a bug? I would + rather call it a design flaw actually. + + Regards, + + Johannes Braams + +PTT Research Neher Laboratorium, P.O. box 421, +2260 AK Leidschendam, The Netherlands. +Phone : +31 70 3325051 E-mail : JL_Braams@pttrnl.nl +Fax : +31 70 3326477 + ------- +Date: Mon, 15 Jul 91 01:59:22 BST +From: Chris Thompson <CET1@phoenix.cambridge.ac.uk> +Subject: Re: Bug/misfeature in TeX? + +I agree that it's a design flaw, not a bug. People do keep falling +over it from time to time, though, so maybe Don could be asked to +think about it again. I suspect, however, that there is no easy fix, +for reasons I will explain below. + +Johannes asks why the names go in the string pool in the first place: +the answer to that is "why not?"... it is the convenient place to keep +more or less arbitrarily long strings. The space occupied by things +added to the string pool can be reclaimed, provided it is done straight +away, before other parts of TeX have been exercised that may add other +strings (especially, control sequence names) to the pool. There are +two types of file name to think about (neither of which are reclaimed +at the moment, with one partial---and wrong---exception): + +1. The 1, 2 or 3 strings generated by |scan_file_name|. Usually these + are used in some implementation-dependant way to open a file, and + maybe then as arguments to |*_make_name_string|, and are then never + needed again; and all this would usually happen straight away. + Exception: deferred (non-\immediate) \openout's. + +2. The string generated by |*_make_name_string|. For things like the + log and DVI files, this has to be kept for ever (printing them is + almost the last thing TeX does). The interesting case, however, is + \input. The string is printed (immediately), and then stored in the + |name_field| of the current input stack entry. *Almost* the only + thing TeX uses it for thereafter is as a number > 17 (to distinguish + the case of an input level being an \input file (as opposed to + terminal input or a \read level). The sole exception is in section + 84 where it is used to deal with the "E" response to the error + prompt: in distribution TeX as part of a message, but in practice + as input to the implementation-dependant way of invoking an editor. + +(BEGIN ASIDE + +The ``partial and wrong exception'' is the code in section 537 +introduced by change 283. |start_input| reclaims the space occupied +by the result of |a_make_name_string|, if that is still the top string +in the pool, and replaces it by the `name' part of the results of +|scan_file_name|. I have had to undo this "fix" in my implementations: +the *only* thing that the ``file name'' is needed for is as an argument +to the editor, and it is an unwarranted assumption that + +a. The values of the `area' and `extension' parts of the name are + irrelevant to that purpose, and + +b. The output of |a_make_name_string| doesn't contain extra information, + available as a result of the opening process, that may also be + relevant. + +END ASIDE) + +In theory the contents of the strings of type 2 for \input files could +be kept on some sort of separate stack, as Johannes suggests (parallel +to the |input_file| and |line_stack| arrays), but this would be quite +convoluted and involve a lot of duplication of code. More plausible +would be an attempt to reclaim them if they are still the top string +in the pool when the file is closed (in |end_file_reading|); this isn't +so unlikely in cases like Johannes'... presumably not all 2400 files +can use never-before-encountered control sequences, or he will be +running out of other things besides the string pool! + +The strings of type 1 create a difficulty, however, unless they can +be got rid of just after the call of |a_make_name_string| (a certain +amount of permuting of the string pool would be required to do that). +If they, also, are to be got rid of when the file is closed, again +subject to the condition that they are at the top of the pool, one +will have to (at least) remember how many of them there were. + +Some of this would, in fact, be rather easier in METAFONT than TeX. +METAFONT's string pool entries have a use count, and reclaiming space +consists of purging consecutive entries at the top of the pool whose +use counts have all fallen to zero. One could easily arrange that the +strings of type 1 had use counts of zero after the opening process was +over, and that the strings of type 2 for "input" files had a use count +of 1 which was decremented to 0 at close time; then the right things +would happen more or less automatically. However, TeX *doesn't* have +such use counts, and I don't really suppose Don is going to introduce +them in order to solve this problem. + +Chris Thompson + ------- + + [ dek: + I think the strings are also needed for font file names. + For ordinary input files I put the special code into \S537 + [which CET1 disabled] so that the Math Reviews could input + lots of files. + Of course there's a workaround (using the operating system + to concatenate files!) but otherwise all I can suggest is a + local change-file routine that tries to reclaim string space + when closing files if the unneeded strings are still at the + end of the string pool. You could introduce a new array + indexed by 1..max_in_open to keep relevant status information + if it isn't already present (see \S304). + ] + +************************************************************************ + +TeX -- handling of \newlinechar within \special + +Date: Thu 9 May 91 09:42:09-EST +From: Ron Whitney <RFW@VAX01.AMS.COM> +Subject: \newlinechar within \special + +Recently I've seen an inconsistency in the way a couple of +versions of TeX for the PC handle \newlinechar within +\special commands. + +One (Fuchs, \mu-TeX) gives the same treatment in this case +as it does with \write-streams. The others use a more literal +interpretation of Knuth's statement on p.228 of The TeXBook +regarding what TeX does as it writes out \special information: + +" TeX doesn't look at the token list to see if it makes sense; +the list is simply copied to the output." + +So if one has \newlinechar=`\^^J, + + \special{ooh^^Jaah} + +puts this 9-character sequence into the .dvi file instead of +"ooh<newline>aah". (Of course, the ^^J gets contracted to single +token first, then gets blown back up to the 3.) + +I would have said that \mu-TeX's treatment is the proper one, but +perhaps it's understood that the string within the \special is not to +be tampered with other than to eat the tokens and then spit them out. +Is this an old issue? Is it open to interpretation? + ------- +Date: Thu, 9 May 91 13:09:08 EDT +From: karl@cs.umb.edu (Karl Berry) +To: RFW@vax01.ams.com +Subject: \newlinechar within \special + +ron> I would have said that \mu-TeX's treatment is the proper one, but +> perhaps it's understood that the string within the \special is not to +> be tampered with other than to eat the tokens and then spit them out. +> Is this an old issue? Is it open to interpretation? +trip.tex seems not to test this. I guess it's open to interpretation, +although Knuth should probably be asked. + +My personal opinion is that ^^J should get turned into a newline +character(s); it's easy to turn this feature off (in fact, I suppose +it's off by default in plain), after all. + +karl@cs.umb.edu + ------- +Date: Thu, 09 May 91 23:54:07 BST +From: Chris Thompson <CET1@phoenix.cambridge.ac.uk> +Cc: Ron Whitney <RFW@vax01.ams.com>, Karl Berry <karl@cs.umb.edu> +Subject: Re: \newlinechar within \special + +I am afraid that I don't really understand what the postings by Ron +Whitney and Karl Berry are saying. The suitably processed token list +in a \special ends up in the DVI file. So what does it mean to replace +characters equal to \newlinechar in this conext by "newline"? What or +whose "newline"? DVI files aren't text files. And if you are going to +say "ASCII CR, of course" or "ASCII LF, of course", be prepared to + [ dek: ^ _or_ _both_ ] +fight off the other 50% of the world :-) If you are going to say +"should depend on the implementation", then don't: the contents of +the DVI file produced are meant to be implementation-independant. + +Reference-level TeX does not treat characters equal to \newlinechar +specially in \special's; they appear unchanged in the DVI file. The +mechanical reason for this is that although |special_out| writes the +token list to the string pool (|selector:=new_string|), the special +treatment of \newlinechar in TeX sections 58--60 only applies when +|selector<pseudo|. To my mind, an implementation that deviates from +this behaviour has a bug, and I would favour enhancing the TRIP test +accordingly. + + [ dek: correct ^^^^^^^^^^^^^^^^^^^^^^^ + will do + In other words, \mu TeX is doing the wrong thing. + [and it's hard to see how they do it without a lot of trouble!] + No; I think Ron is misled. (You can't see what's in a special + using tracingall because newlinechar will obscure it) You have + to look at output of dvitype. + Probably \mu TeX is doing the right thing. + +Chris Thompson +JANET: cet1@uk.ac.cam.phx +Internet: cet1%phx.cam.ac.uk@nsfnet-relay.ac.uk + ------- +Date: 05/10/91 15:42:35 GMT+1 +From: MITTELBACH FRANK <PZF5HZ%RUIPC1E.BITNET@CUNYVM.CUNY.EDU> +Subject: RE: RE: \NEWLINECHAR WITHIN \SPECIAL and \message + +I think that Chris remark that dvi files are to be device independent +is questionable as far as specials are concerned. In fact the +special is supposed to pass some string to the dvi driver and this +means that this program is supposed to understand it. Now this means +that the driver needs to interprete the bytes inside the special +in the same way as the TeX that writes them out. But if we assume +that this is done under some ascii conversion table then why not +accept ascii <newline>. Not that I see many applications for this. + +Do I miss something? + +The whole discussion reminded me of some related business with the +newline char of TeX which I think is a bug although one can +surely plea for a questionable feature. + +Compare the output of + +\newlinechar=`\@ +\message{foo@bar} + +to + +\newlinechar=`\^^J +\message{foo^^Jbar} + +The first message is broken into two lines the second comes out as is. + + [ dek: + I guess because of certain UNIX implementations coercing all + tabs to spaces, those implementation cannot possibly "see" + a tab. + ? Wait, tab is ^^I. + What _is_ going on? + Oh, I see; Mittelbach and Sch\"opf are right, see below + $10.24 + ] + +Same discrepancy happens with \errormessage which is quite unfortunate +and certainly makes macro packages non portable if certain characters +can't be entered directly. + +Whether or not this is covered by the documentation in the TeX book +is difficult to say since there are quite a few places where Don +leaves things open to interpretation. + +Frank Mittelbach + ------- +Date: Fri, 10 May 91 17:09:12 +0200 +From: schoepf@sc.ZIB-Berlin.DE (Rainer Schoepf) +Cc: PZF5HZ@RUIPC1E.BITNET +Subject: RE: RE: \NEWLINECHAR WITHIN \SPECIAL and \message + +Frank writes: + + I think that Chris remark that dvi files are to be device independent + is questionable as far as specials are concerned. In fact the + special is supposed to pass some string to the dvi driver and this + means that this program is supposed to understand it. Now this means + that the driver needs to interprete the bytes inside the special + in the same way as the TeX that writes them out. But if we assume + that this is done under some ascii conversion table then why not + accept ascii <newline>. Not that I see many applications for this. + + Do I miss something? + +Yes, you do--at least as far as the new line character is concerned. +The point here is that normally the meaning of the \newlinechar is +"TeX's internal end-of-line marker", full stop. When writing to a text +file (irregardless of the code table) this has a definite meaning, +namely: start a new line here, full stop. When it comes to \specials, +the notion of "lines" seems at least questionable, even more since the +sequence of characters inside a \special need not be anything legible. + + [ dek: + Well, I don't intend 8-bit codes to be going there; I hope they + are input from other files by DVI drivers. People might develop + binary-coded special conventions but they are too non-portable. + The main point in Chris's message is that newline is handled in + three completely different ways (on PC, MAC, and UNIX) + ] + +\specials are device-dependent, true. But the consequence of your +argumentation is that the same device (say, a PostScript printer) +would see a different command on a Unix workstation and an IBM +mainframe. Keep in mind that the \special string is not written under +the control of the character conversion tables. + + The whole discussion reminded me of some related business with the + newline char of TeX which I think is a bug although one can + surely plea for a questionable feature. + + Compare the output of + + \newlinechar=`\@ + \message{foo@bar} + + to + + \newlinechar=`\^^J + \message{foo^^Jbar} + + The first message is broken into two lines the second comes out as is. + +New, this is something different, since it applies to text files where +(as I said above) the notion of "start a new line here" is perfectly +sensible. In my eyes this is a bug and should be fixed, even if this +behaviour is in conformance with the TeXbook. + +Rainer Sch\"opf + ------- + + [ dek: + Well I've thought about it some more and decided that \special + should send 8-bit codes to DVI file without changing the + printable ASCII form. This applies also to font file names + in case future users want 8-bit codes in those names (nonportable + but perhaps important to somebody to see the name in Cyrillic + or something). + I am changing TeX 3.14 to do this more logically, basically + by making 8-bit codes more equal to their printable cousins. + At present there are several anomalies [like the string one + mentioned w.r.t. Piff's work] [also when you have file names, + job names, etc. with nonstandard 8-bit codes], and I think I + see how to make it all come out right, ... + as a byproduct, Mittelbach's \message problem goes away too. + Internally characters will not be translated to ^^A form until + the last minute when they simply must be translated. + ] + + +######################################################################## + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% Character code reference +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% +% Upper case letters: ABCDEFGHIJKLMNOPQRSTUVWXYZ +% Lower case letters: abcdefghijklmnopqrstuvwxyz +% Digits: 0123456789 +% Square, curly, angle braces, parentheses: [] {} <> () +% Backslash, slash, vertical bar: \ / | +% Punctuation: . ? ! , : ; +% Underscore, hyphen, equals sign: _ - = +% Quotes--right left double: ' ` " +%"at", "number" "dollar", "percent", "and": @ # $ % & +% "hat", "star", "plus", "tilde": ^ * + ~ +% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +[ end of message 033 ] +------- + |