path: root/info/digests/tex-implementors/message.40

Date:     14 March 91                           Message No:     040

To:       TeX implementors and distributors

From:     Barbara Beeton

Subject:  Messages from DEK, part 5


************************************************************************

>>> WEAVE -- Peter Breitenlohner

Date: Sun, 10 Nov 91 17:11:15 GMT
From: Peter Breitenlohner <PEB%DM0MPI11.bitnet@CUNYVM.CUNY.EDU>
Organization: Max-Planck-Institut fuer Physik, Muenchen
Subject: message for dek

Barbara, can you please forward this message. Thanks  Peter
 ----------------

To:   Donald E. Knuth
From: Peter Breitenlohner

1. Thanks for your comments on patgen. I mostly wanted to make sure
that you don't object to the character translation scheme. (You had
quite a few argumantss against my first ideas on how to do that.)

2. Yet another bug/misfeature in WEAVE.

After a discussion with Bart Childs I looked at the way WEAVE marks
modules as changed and did some experiments. Consider the following web
and change file (maybe somewhat atypical, but no modules are removed or
inserted):
================  cut here ====================================
@ One.
11111
@
Two-c.
22222
@ Three.
33333
@ Four-c.
44444
@ Five-c.
55555
@ Six.
66666
@ Seven.
77777
@ Eight-c. @ Nine-c. @ Ten-c.
@ Eleven.
@ Twelve-c?.
 @ Thirteen-c.
@ Fourteen.
@*Index.
================  cut here ====================================
@x - this changes Two
@
Two-c.
22222
@y
 [ dek:  not much of a change ]
@
Two-c.
22222
@z
@x - this changes Four and Five
44444
@ Five-c.
55555
@y
@ Five-c.
55555
@z
@x - this changes Eight, Nine, and Ten
@ Eight-c. @ Nine-c. @ Ten-c.
@y
@ Eight-c. @ Nine-c. @ Ten-c.
@z
@x - this changes Thirteen (and maybe also Twelve)
 @ Thirteen-c.
@y
 @ Thirteen-c.
@z
================  cut here ====================================
One would think that modules Two, Four, Five, Eight, Nine, Ten, and
the index are changed. One might argue whether module Twelve has been
changed or not. Technically the blank preceding '@ Thirteen' is part
of module Twelve. That blank could, however, also be a form feed with
xord[form_feed]=" ", a situation not uncommon until recently in many web
files. Thus one might prefer not to count blanks, tabs, form feeds, and
blank lines as changes.

Incidentally I don't quite see how a tab_mark can enter into the input
buffer of WEAVE or TANGLE unless through modified xchr/xord arrays.

Actually modules Four, Eight, and Nine are not marked as changed by
 [ dek:          ^^^^
        this is the only one I personally would care about, of course
 ]
WEAVE 4.2
Unfortunately I see no simple fix for that, which would not also mark
modules One and Seven as changed (I guess that would just undo the
Aug 83 changes).

The code below uses a completely revised changed-module-algorithm:
 [ dek:  @ 1 ]
1. a module is marked as changed if it starts while changing is true
 [ dek:  @ 2 ]
2. when a matching change is detected, the current module is marked as
   changed unless the first non blank line after the '@x' and after the
   '@y' both start with '@ ' or '@*' (possibly preceeded by blanks/tabs).
 [ dek:                                        ^^^^^^^^^    ^^^^^^^^^^^
                                               preceded     whitespace
 ]

================  cut here ====================================
% All line numbers refer to WEAVE.WEB 4.2, as of September 5, 1990.

@x [9] m.71 l.1274 - fix bug in changed_module reckoning
@!changing: boolean; {if |true|, the current line is from |change_file|}
@y
@!changing: boolean; {if |true|, the current line is from |change_file|}
@!change_pending: boolean; {if |true|, the current change is not yet
  recorded in |changed_module[module_count]|}
@z
%---------------------------------------
@x [9] m.79 l.1364 - fix bug in changed_module reckoning
prepares to read the next line from |change_file|.
@y
prepares to read the next line from |change_file|.

In addition the current module is marked as changed unless the first
line after the \.{@@x} and after the \.{@@y} both start with either
|'@@*'| or |'@@ '| (possibly preceded by blanks and\slash or blank
lines). The \.{WEB} macro |if_start_of_module_then_change_pending|
tests for this condition.

@d if_start_of_module_then_change_pending==
 [ dek:                                 ^^ (#) would be less tricky ]
  loc:=0; buffer[limit]:=" ";
  while (loc<limit)and((buffer[loc]=" ")or(buffer[loc]=tab_mark)) do
    incr(loc);
  if buffer[loc]="@@" then
    if (buffer[loc+1]="*")or@|
      (buffer[loc+1]=" ")or@|(buffer[loc+1]=tab_mark) then
        change_pending
 [ dek:                :=#
@z
%---------------------------------------
@x [9] m.79 l.1370 check_change - fix bug in changed_module reckoning
begin if lines_dont_match then return;
@y
begin if lines_dont_match then return;
change_pending:=false;
if not changed_module[module_count] then
  begin if_start_of_module_then_change_pending:=true;
  if not change_pending then changed_module[module_count]:=true;
  end;
@z
%---------------------------------------
@x [9] m.82 l.1427 get_line - fix bug in changed_module reckoning
begin restart: if changing then changed_module[module_count]:=true
else  @<Read from |web_file| and maybe turn on |changing|@>;
if changing then
  begin @<Read from |change_file| and maybe turn off |changing|@>;
  if not changing then
    begin changed_module[module_count]:=true; goto restart;
    end;
@y this is identical to the corresponding code in TANGLE
begin restart: if changing then
  @<Read from |change_file| and maybe turn off |changing|@>;
if not changing then
  begin @<Read from |web_file| and maybe turn on |changing|@>;
  if changing then goto restart;
@z
%---------------------------------------
@x [9] m.84 l.1453 - fix bug in changed_module reckoning
if limit>1 then {check if the change has ended}
@y
if limit>1 then {check if the change has ended}
 [ dek:  ^
        Here I think it should be 0
 ]
  begin if change_pending then
    begin if_start_of_module_then_change_pending:=false;
    if change_pending then
      begin changed_module[module_count]:=true; change_pending:=false;
      end;
    end;
 [ dek:
        And I added   buffer[limit]:=" ";   here
@z
%---------------------------------------
@x [9] m.84 l.1464 - fix bug in changed_module reckoning
    end;
@y
    end;
  end;
@z
%---------------------------------------
 [ dek:
        Suppose change-pending and limit = 1, then the change won't
        have ended.  If the line comes in as `.' say and the _next_
        line begins `@*' then we won't have marked the current module
        as changed.
        In your test the line comes in as `@' and you seem to be
        lucky because you increase module_count
 ]
@x [11] m.110 l.1934 - fix bug in changed_module reckoning
changed_module[module_count]:=false; {it will become |true| if any line changes}
@y
changed_module[module_count]:=changing;
  {it will become |true| if any line changes}
@z
%---------------------------------------
================  cut here ====================================
Regards Peter

<<< end WEAVE -- Peter Breitenlohner

************************************************************************

>>> WEAVE -- Brian {Hamilton Kelly}

Date: Mon, 18 NOV 91 14:44:15 GMT
Originally-from: TEX "Brian {Hamilton Kelly} <TeX@Uk.Ac.Cranfield.RMCS>"
From: TEX@rmcs.cranfield.ac.uk
Subject: RE: Bugs or mis-features in WEAVE

barbara,

I was more than somewhat disappointed by DEK's response to my message
about WEAVE, which you (re-)relayed to him on 8 May 91 (your message-id
<673745280.0.BNB@MATH.AMS.COM>).

(I'll adopt below your usual artifice of interpolating DEK's pencilled
notes.)

In a message of 29-SEP-1989 13:07 BST, I wrote:

 
> It looks as though fixing the long line bug has introduced one!!!
> 
> I think I've engineered a fix: can you feed it through to DEK please?
> 
> @x [Section 122 |flush_buffer|]
> write_ln(tex_file); incr(out_line);
> if b<out_ptr then for k:=b+1 to out_ptr do out_buf[k-b]:=out_buf[k];
> @y
> write_ln(tex_file); incr(out_line);
> per_cent:=false;
> if b<out_ptr then     {See if the \&{user} has started a comment with \.\%}
  [dek: begin...]
>   for k:=1 to j do
>     if (out_buf[k] = "%") and ((k=1) or (out_buf[k-1] <> "\")) 
>     then per_cent:=true;
> if per_cent then      {Ensure continuation line starts in comment mode}
> begin
>   out_buf[b]:="%";
>   decr(b)
> end;
> if b<out_ptr then for k:=b+1 to out_ptr do out_buf[k-b]:=out_buf[k];
> @z
  [dek:
    Sorry, I do not have time to update WEAVE and TANGLE...
    (I prefer to shorten the long lines and get back to Vol. 4)
    but if Brian feels so strongly about that he *has* to see this in,
    he should send me *debugged* code!...
    The above looks flaky (has undeclared variable [anyway I'd probably
    use a goto!] and missing begin end at least.]


OK, after more experience of actually writing in WEB ab initio (vs. just
writing change files) I'm prepared to accept that the other `misfeature'
reported in the same message isn't such a bad idea after all [viz.
actually listing multiple references to the same WEB section when a
section is invoked more than once in-line].

However, I do think that the above problem really is a bug, and one
that's likely to cause all sorts of untold grief to someone who only
ever uses WEAVE to pretty-print someone else's changes; especially if
the author hasn't him/herself done this (cf. TeX V2.991 when DEK had
index entry macros used without accounting for \_ requirements)

I'll admit that I've never compiled the code above per se, because the
 [ dek: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        AHA, just as I guessed.
 ]
correction to the bug for the VMS implementation is subtly different,
since that implementation uses a separate auxiliary buffer for outputs
to the .tex file.  But the same phenomenon (of Weave inserting an
infelicitous line-wrap) was observed on two separate implementations of
Weave for the IBM-PC: my colleague Niel Kempson fed through the above
suggested fix to each of the authors, and it was included, compiled
quite happily, and fixed the bug in each version without any further
problem.
 [ dek:
        And this should have been stated so that I wouldn't have
        started with the wrong impression.
 ]

Therefore, I cannot see why DEK says that there's an undeclared
 [ dek:
        because I don't have the foggiest idea what WEAVE does anymore
 ]
variable: I've just gone through the above suggested correction with a
fine-toothed comb, and every variable used therein is not only declared,
but already in *use* in \S122 of weave.web.  (OK, |per_cent| is a
parameter, but it's a call-by-value one, and therefore can legitimately
be re-used, after it's done it's initial job] as a local variable for
 [ dek:                     ^^^ its ]
after all, it already has a most apposite name).  Moreover, at the risk of
some comment about grandmothers and sucking eggs, why should it require
a begin...end pair?...
 [ dek:
        Well if he hadn't said ``I think i've engineered a fix'' I would
        have had more confidence.  But this going back to maintain old
        programs always takes several days off my life...  I have come
        to really hate doing it, please excuse me but it is the truth.
        Talk about stress!
 ]

Forgetting about the comment, and rearranging...

> if b<out_ptr then
>   for k:=1 to j do
>     if (out_buf[k] = "%") and ((k=1) or (out_buf[k-1] <> "\")) then
>       per_cent:=true;

can be parsed into:

> if b<out_ptr then
>   for k:=1 to j do
>     if <expression> then <statement>;

thence to:

> if b<out_ptr then
>   for k:=1 to j do
>     <if-statement>;

and

> if b<out_ptr then
>   for <variable-identifier> := <expression> to <expression> do
>     <statement>;

then

> if b<out_ptr then
>   <for-statement>;

and finally

> if <expression> then
>   <statement>;

So what need for a compound statment?  If Weave et al made a habit of
always surrounding the single statment controlled by a conditional or
 [ dek:
        Not sure what I meant but I guess I may have been thinking
        about efficiency -- no need to test per_cent if b>=out_ptr
        in that code
 ]
iterative statement into a compound statement (as is done by many writers
in C, to avoid potential problems when multiple statements are later
inserted in place of the original single one), then I'd be prepared to
agree that this too should become a compound statment.  But there are
many places in Weave, TeX and elsewhere where an if or for controls but
a single unardorned statement.  Of course, had this been Modula-2 (a
 [ dek:     ^  unadorned ]
language that I cannot commend too highly) there'd be no possibility of
confusion about the statements controlled by any construct, although no
BEGIN would ever be required, and I don't agree with Wirth's overloading
of END.

 [ dek:
        Anyway I looked at it again and found a real bug in Brian's
        ``engineering'': Sometimes flush_buffer is called with b=out_ptr
        when you want to propagate a %.  (You see why I can't satisfy
        everybody's whims without spending a lot of time that I haven't
        got.)  The next time anybody asks for anything I just won't
        even try; I don't like to disappoint people, but a h*** of a lot
        more people will be disappointed if I don't finish Volume 4.
        [But after wasting this much time I _did_ put in a patch that
        should satisfy Brian.  Hoping that he will be so happy he will
        help me with Vol. 4 by convincing all his friends never to
        pester me again.]

        Barbara please excuse me for getting so upset.
        Some day I will no doubt be happy again!
 ]

> Again, could you please pass this to DEK?

Best regards,

                        Brian {Hamilton Kelly}

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ JANET:     tex@uk.ac.cranfield.rmcs                                     +
+ BITNET:    tex%uk.ac.cranfield.rmcs@ac.uk                               +
+ INTERNET:  tex%uk.ac.cranfield.rmcs@nsfnet-relay.ac.uk                  +
+ Smail:     School of Electrical Engineering & Science, Royal Military   +
+            College of Science, Shrivenham, SWINDON SN6 8LA, U.K.        +
+ Phone:     Swindon (0793) 785252 (UK), +44-793-785252 (International)   +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 -------
(reply, 19 Nov 91)
brian,
you would like me to send it around one more time?
-------
Date: Wed, 20 NOV 91 10:41:14 GMT
Originally-from: TEX "Brian {Hamilton Kelly} <TeX@Uk.Ac.Cranfield.RMCS>"
From: TEX@rmcs.cranfield.ac.uk
Subject: RE: Bugs or mis-features in WEAVE

Dear barbara,

In your message <690606220.0.BNB@MATH.AMS.COM> 
of Tue 19 Nov 91 22:03:40-EST, you wrote:

> you would like me to send it around one more time?

Yes please; Chris Thompson has since come back to me (I circulated my
message for you to him before sending it for you to forward to DEK) and
suggested that perhaps trailing comments (apart from the percent itself)
should be eliminated from the woven TeX before even reaching the output
routine, so that this problem wouldn't arise.  I'm not so sure that I
agree, although the problem HAS only raised its ugly head because of TeX
comments in the Web that were only there to explain why the Web was
written that way, not why the program was so written (or in limbo
material providing a revision history), so perhaps nothing's lost by
discarding them.

>>> WEAVE -- Brian {Hamilton Kelly}

 [ dek:
        Please tell Peter B and Brian H~K that the new WEAVE.WEB is
        now available for alphatesting
        on labrea   ~ftp/alpha  directory
        I hope they can check it before February
        thanks
 ]

************************************************************************

>>> Barry Smith, |history| variable in TeX.WEB

Date: Mon 6 Jan 92 21:36:47-EST
From: bbeeton <BNB@MATH.AMS.COM>
To: winkler@cs.stanford.edu
Subject: another tex report for don

this report just arrived.
                ---------------

Date: Mon, 6 Jan 92 17:15 PST
From: barry@reed.edu (Barry Smith)
To: BNB@MATH.AMS.COM
Subject: Re:  flash!!! notice that tex update is about to happen

hmmm, mail here was down over the weekend, so this may be too late, but...

I noticed (while working on Lighting Textures) that in section 1335
(final_cleanup), the |history| variable is not updated when either of
these error messages is produced.  Only a problem when someone (something?)
external to TeX is looking at history, of course.

(might even qualify as a high-value bug?)

Barry Smith, Blue Sky Research
barry@reed.edu
-------

 [ dek:
        I don't think I'll make any change here, since this sort of
        ``warning'' is not the kind for which there is info in the
        transcript file ... so it would look funny if TeX then said
        ``(see the transcript file for info)''.
        Of course a system dependent error level can be signalled
        here if somebody wants.
 ]

<<< end Barry Smith, |history| variable in TeX.WEB

************************************************************************

>>> Chris Rowley, two items: \fontdimen2 and marks

Date: Tue 7 Jan 92 11:02:12-EST
From: bbeeton <BNB@MATH.AMS.COM>
To: winkler@cs.stanford.edu
Subject: another tex comment for don

little comments keep dribbling in ...
                ---------------

Date: Mon, 6 JAN 92 23:43:37 GMT
From: CA_ROWLEY@vax.acs.open.ac.uk
To: BNB <@nsfnet-relay.ac.uk:BNB@MATH.AMS.com>
Subject: RE: flash!!! notice that tex update is about to happen

b

Just one thing on the TeXbook: I mentioned this to you but have not 
sent it to Don.

In the table on page 447, it states that \sigma_2 is used in Rule 17.
Now, \sigma_2 is (page 441) \fontdimen2 of a font in family 2, but 
Rule 17 uses \fontdimen2 of the font from which the character comes 
(page 445).  This may seem a bit pedantic but it has misled more than 
one person in an area where a lot of misunderstandings are found (not 
caused by what is in the TeXbook)!

 [ dek: 
        $2.56
        Well it might be \sigma_2 or \xi_2 but I agree that the
        present chart is misleading.
        (TFM designers do need to make \sigma_2, \sigma_5, \xi_2,
        \xi_5 sensible for characters in those fonts used as Ord.)
 ]

\chi
-------
Date: Fri 10 Jan 92 22:41:54-EST
From: bbeeton <BNB@MATH.AMS.COM>
To: winkler@cs.stanford.edu
Subject: message for don re tex update

i just received this today.
i'm pretty sure this will be the last, as i'm leaving sunday
morning to go to los angeles for two weeks.
                ---------------

Date: Fri, 10 JAN 92 18:53:27 GMT
From: CA_ROWLEY@vax.acs.open.ac.uk
To: bnb <@nsfnet-relay.ac.uk:bnb@math.ams.com>
Subject: for Don, copy sent to CET1

Don and Barbara

As Joachim Schrod discovered (in implementing change-bar macros), when
the output routine is called with no intention to shipout anything,
then not only is it necessary to ensure that the inserts are left as
before (using \holdinginserts positive) but it is also sensible to
leave the values of the three marks as they were, especially if they
are null.

Frank and I have been discussing this and we now think that what is
needed for marks is a little more complex than for inserts, as
follows.

Even when \holdinginserts is positive, it is useful to have the three
marks set as normal: this is so that the output routine can use the
information in them but, in the case that \holdinginserts is
positive, after the end of the output routine they should be restored
to the values they had before the output routine was called.

This needs to be done at the Pascal level because it is not possible,
at the TeX level, to restore the pointer associated with a mark to
 [ dek:
        You can simulate the behavior without great difficulty; no
        need to `put back' a value into the real mark registers.
        Solutions to that are well known and the user can be disciplined
        in marks used.
 ]
have the value "null"; and it is also difficult to detect, in a
robust way, whether one of these pointers does have the value "null"
or to distinguish this value from a pointer to an empty string.

Therefore, at the Pascal level, the values of "top_mark", "bot_mark"
and "first_mark" should be saved before these pointers are updated (in
the procedure "fire_up"), and then the values of all three to be
restored to these saved values when the user's output routine
finishes.   This will also require appropriate changes to the parts of
the code which  call "delete_token_ref" and "add_token_ref".

 [ dek:
        Somehow my message about TeX being frozen is not getting through.
        If you feel extremely strong about this, please
         (a) send me a carefully written detailed report about why
             this is an important problem -- say 5-10 pages long and
             easy for me to understand -- and include rigorous proofs
             that TeX as it stands cannot possibly deal with a certain
             important application -- and include rigorous justification
             for assertion that an incompatible change such as you
             propose will not affect any present users adversely
         (b) send me complete list of necessary changes to the Pascal
             code after having tested them thoroughly, including
             extensive documentation about the tests you have made.
             Vague paragraphs about `save a pointer and make appropriate
             changes' are inadmissible.
         (c) Wait until I have time to look at TeX again.
 ]

Best wishes

Chris
Frank
-------

<<< end Chris Rowley, two items: \fontdimen2 and marks

************************************************************************

dek:

Barbara, while I've got all this whitespace I have room to tell you that
I have made several changes to Computer Modern fonts:
 (1) Arrowheads made thicker and wider
 (2) Calligraphic F, H, I, and T revised.
I have seen too many cases of `arrowhead burnoff' after weak
laserprinter and Xeroxing, hence (1).  I have begun to read lots of math
papers that use the \cal letters and I've had very negative personal
reaction to H and T, slightly to I, and very very slightly to F.  Thus I
have to try to change them so as to minimize the number of future times
my mind is taken from math to fonts ...

I realize it's a hassle changing fonts, so the upgrade will take awhile
before it spreads around (although with Rokicki's DVIPS it's very
simple, I just update the CM sources and delete the affected fonts, then
DVIPS will remake them automatically as needed).

The affected fonts are cmsy* (and cmbsy10) mainly, but the harpoons
change in math italic and the arrows change a bit in cmtt.
    ==> NO TFM FILES ARE AFFECTED <==
so there is no change to TeX's line breaks.  I will never change the
TFMs again (unless I discover a major glitch).

At the same time I'm installing a new cmbase with ideas by John Hobby;
it makes digitization better (_much_ better at low res and some better
even at 300dpi).  The new CM sources are available for alphatesting
on labrea ~ftp/alpha .

[ enclosed: proofs of arrows and modified \cal letters as noted above ]


########################################################################

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%  Character code reference
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
%                       Upper case letters: ABCDEFGHIJKLMNOPQRSTUVWXYZ
%                       Lower case letters: abcdefghijklmnopqrstuvwxyz
%                                   Digits: 0123456789
% Square, curly, angle braces, parentheses: [] {} <> ()
%           Backslash, slash, vertical bar: \ / |
%                              Punctuation: . ? ! , : ;
%          Underscore, hyphen, equals sign: _ - =
%                Quotes--right left double: ' ` "
%"at", "number" "dollar", "percent", "and": @ # $ % &
%           "hat", "star", "plus", "tilde": ^ * + ~
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

[ end of message 040 ]