path: root/dviware/beebe/updates/00mail.14

 2-Nov-87 11:28:47-MST,30752;000000000001
Date: Mon 2 Nov 87 11:28:45-MST
From: "Nelson H.F. Beebe" <Beebe@SCIENCE.UTAH.EDU>
Subject: DVI driver family update #14
To: "DVI mailing list": ;
cc: BEEBE@SCIENCE.UTAH.EDU
X-US-Mail: "Center for Scientific Computing, South Physics, University of Utah, Salt Lake City, UT 84112"
X-Telephone: (801) 581-5254
Message-ID: <12347446387.22.BEEBE@SCIENCE.UTAH.EDU>

                  DVI Driver Family Update #14
                           [01-Nov-87]

This issue announces Version 2.10 of the DVI driver family.

There has a been a long delay in filling orders for the
software, since I decided in mid-September to place new
requests on hold until 2.10 was complete.  It turned out to
be a much larger job than I expected, since very substantial
modifications were made in two drivers, DVIALW and DVIJEP.
The delay, I believe, has been justified by the substantial
improvements made in these two drivers, plus a large number
of minor editorial changes, plus two new drivers.  The
details are contained in the extracts from 00REVHST.TXT
given below.

In a nutshell:

** DVIJEP has completely eliminated the limit on the number of
   fonts per page (previously 16) and the number of fonts per
   document (previously 32); these limits are imposed by the
   HP LaserJet Plus printer itself, and remain unchanged in
   the Series II.

** DVIALW has a radically different internal organization.
   The output is nearly 10% more compact, and obeys a maximum
   line width.  It also is considerably more economical of
   printer virtual memory resources (which are grossly
   inadequate for downloaded-font applications), and it should
   be routinely possible to print 100-page documents on an
   Apple LaserWriter Plus non-stop.  Large characters no
   longer raise printer errors.

** LW78 incorporates changes to eliminate apparent Apple
   LaserWriter printer buffer overrun, which I hope will
   eliminate the "I/O error" conditions we have experienced
   for over 2 years.  Users who have prepared their own local
   PostScript drivers may find it useful to examine the new
   LW78.

** Two experimental drivers for the Epson and Epson clone
   9-pin printers, DVIE72 and DVIEPS, have been added.

** On VAX VMS, the drivers now incorporate dynamic
   determination of the VMS quota on the number of open files,
   preventing them from terminating when a font file open
   fails.

** On Unix, the drivers may now be used as filters, with the
   DVI file coming from stdout, and driver output to stdout,
   eliminating the need to wrap them in a shell script when
   used for printer spooling.

** New debug option 128 for display of text to be set.

** The -d (debug) option can now be specified more than once,
   so you need not add values.  Originally I thought it might
   be useful to have -d0 cancel a previously-typed debug
   request, but this appears less useful than the new
   alternative.

** The -z option is available for 4.xBSD Unix systems (or any
   that have an ioctl(fildes,TIOCSTI,*character) call) for all
   devices.  This requests the driver to type in a command to
   automatically spool the output.

** A Unix man page in troff format (dvi.1l) has been prepared,
   so full on-line documentation of the drivers is now
   available.  This complements the Emacs Info format
   documentation (dviman.texinfo), and the typeset manual-page
   format document (dviman.ltx).  There is insufficient
   commonality in these 3 formats to adopt only one, and
   inventing a suitable subset of troff or LaTeX (which is
   what the TeXinfo format attempts to do, with only limited
   success) would require a substantial effort well beyond the
   time I would be willing to devote to it.

** The drivers now support 256-character fonts, which is
   important for European additions to the Computer Modern
   alphabets, and for oriental fonts.

Finally, here are the details from 00REVHST.TXT:

[01-Nov-87]     Replaced references to (void)fprintf(..., plotfp)
                with  references   to   new   macros   OUTF(...),
                OUTF2(...),  and  OUTF3(...),  or  existing  ones
                OUTC() and  OUTS().   Replaced all  instances  of
                putc(*c++,plotfp)   by   "OUTC(*c);c++".    Since
                putc() is usually a  macro, there is a  potential
                incorrect side effect  if its  first argument  is
                evaluated more than once.  Most C implementations
                use it only in the two branches of a  conditional
                expression, which is safe.

[01-Nov-87]     Completed major overhaul  of dvijep.c.  The  font
                deletion mechanism  was  not  successful,  partly
                because the  font  number  was  not  subsequently
                reset in one place, but mostly because it  causes
                a page eject.  This  would make it impossible  to
                print some documents.  dvijep now keeps track  of
                both the number  of fonts in  use on the  current
                page,  and  the  total  number  in  use  in   the
                document.   When  either   of  these  limits   is
                reached, the characters in  those fonts are  sent
                as bitmaps, instead of  as downloaded fonts.   It
                is still possible to have a document which is too
                complex for the LaserJet to print (raising  error
                code  21  on  the  printer  front  panel   status
                display), but that is now a limit imposed by  the
                printer, rather than the  DVI driver, and  cannot
                be gotten around.

                There is  room for  further work  here,  however.
                Most documents  do  not  require  an  excessively
                large   number    of   fonts.     For    example,
                dvidriver.ltx uses 23  fonts, and weave.tex  uses
                15, but the TeXbook uses 54.

                When the DVI driver begins execution, through the
                call chain main() ->  dvifile() -> readpost()  ->
                getfntdf()  ->   readfont()  ->   reldfont()   ->
                read{pk|gf|pxl}() -> newfont(), it normally reads
                in  the  font  definitions  from  the  DVI   file
                postamble and assigns  font numbers  in order  of
                occurrence, and  builds  its  internal  font  and
                character tables.  Later, as pages are processed,
                font  changes  occasioned  by  DVI  fnt_num_0  ..
                fnt_num_63 and fnt1  .. fnt4  commands result  in
                setting the global  flag font_switched, which  in
                turn causes setchar() and setstr() to output font
                switch commands to the device.

                The implication of  this for dvijep  is that  all
                fonts  beyond  the   first  32   are  marked   as
                non-downloadable.  In the case of the TeXbook, it
                turns out  that  this includes  the  heavily-used
                cmr, cmsy, and cmmi fonts at  5, 6, 7, 8, 9,  and
                10 pt sizes, which means  that almost all of  the
                characters set on a page will be sent as bitmaps,
                instead of as  downloaded characters.   On my  HP
                LaserJet Plus,  every page  raised printer  error
                code 21.  This might not happen on an HP LaserJet
                Series II,  which  can  have  substantially  more
                memory, but  I haven't  one  locally to  try  the
                experiment.

                The solution  to this problem  is to  delay  font
                number assignment  until  the  font  is  actually
                referenced.  Since a considerably smaller  number
                of fonts  are needed  on each  page, it  is  then
                possible  to  download  most   of  them,  and   a
                font-rich  document  like  the  TeXbook  can   be
                printed successfully  in  sections.   The  change
                involves moving the  body of newfont()  to a  new
                function, makefont(), and  having newfont()  only
                initialize  the   font_number  with   a   special
                "undefined" value.   The  macro  SET_CURRENT_FONT
                has  become   big  enough   that  it   has   been
                transformed   into    a   function,    setfont().
                setfont()  now   checks   font_number   for   the
                undefined  value,  and  in  such  a  case,  calls
                makefont() to assign the font a number and create
                it in the printer.  With this change, the TeXbook
                can be printed successfully.

[30-Oct-87]     Changed  vaxvms.c,  gblprocs.h,  openfont.h,  and
                machdefs.h to  support dynamic  determination  of
                the limit on  the number  of open  files for  VAX
                VMS.  This is a VMS quota parameter whose dynamic
                value depends  on  the  number  of  subprocesses.
                Previously, a user with  too small a limit  would
                find the dvi drivers could fail because the  hard
                limit on the  number of open  font files was  too
                large.

[29-Oct-87]     In LW78.C,  removed typeout  of  final CR  LF  in
                Get_Msg(), and  inserted system  call to  dismiss
                the process until the  output buffer is empty  at
                the start of Put_Packet().

                LW78 had been experiencing I/O error returns from
                the Apple LaserWriter Plus, and when I  attempted
                to  copy  a  PostScript  file  directly  to   the
                printer,  the  same  thing  happened,  only  much
                earlier in the job.  This raised the suspicion of
                buffer overrun in the printer.  Three experiments
                using a  100-page document  with (1)  a  5-second
                sleep before outputting each buffer, (2)  process
                dismissal until the buffer was empty, and (3) the
                Kermit TRANSMIT command (which waits for a prompt
                from  the  LaserWriter   after  each  line)   all
                resulted  in  successful  printing,  and  I  have
                confidence that this change in LW78 will  provide
                superior performance.

                With the new LW78, I sent dvidriver.dvi-alw  with
                the modified BOP and EOP definitions given below,
                and printed the 100 pages at an average rate of 5
                pages  per  minute.   Virtual  memory  usage  was
                initially 2Kb to  4Kb per page  while fonts  were
                being downloaded, and in later pages, dropped  to
                a low of  1105 bytes/page.  Pages  with that  low
                figure had only  text set on  them, and no  fonts
                downloaded, with the text guarded by save/restore
                sequences.   That   virtual  memory   was   still
                consumed leads one  to wonder if  there is not  a
                bug in  the  PostScript  memory  allocation  code
                which causes  some  "seepage" loss.   If  one  of
                dvialw's macros were leaving unclaimable  garbage
                behind, one would expect that the garbage  amount
                would vary, but many different pages had the same
                1105  bytes/page   consumption.   Total   virtual
                memory consumed by the  job was 112Kb, with  78Kb
                remaining available at end-of-job.

[27-Oct-87]     Simplified beginning-of-page and end-of-page code
                in prtpage.h  by creating  bopact() and  eopact()
                routines in several drivers, and removed bopact.h
                and eopact.h (which were previously used only  by
                dvibit),  including   references   to   them   in
                makefile.*.

[27-Oct-87]     Numerous  small   editorial  changes   for   code
                consistency, and to reduce the number of warnings
                from the lint utility.

[27-Oct-87]     {Thanks to John Pavel (mcvax!nplpsg!jrp@uunet.uu.net)
                for urging this change}
                On Unix only, Version  2.10 now permits input  to
                come from  stdin, and  output  to go  to  stdout,
                allowing use  of the  drivers as  filters.   This
                choice is automatically made  when no file  names
                are specified on the command line.

                Use of  stdout  for  binary  files  is  generally
                impossible for  other  operating  systems,  which
                either make a distinction between text and binary
                file formats, or use line terminators other  than
                <LF>,   necessitating    translation   of    Unix
                end-of-line '\n' on input  and output.  With  the
                exception of  dvialw,  all  the  drivers  produce
                binary output files.

[27-Oct-87]     {Thanks to Bernhard Nebel (NEBEL@DB0TUI11.BITNET)
                for useful suggestions}
                Completed  major   overhaul   of   dvialw.c.    A
                fundamental  problem  with  existing   PostScript
                interpreters   is   that   they   lack    garbage
                collection,  and  the  PostScript  programmer  is
                expected  to  manage  storage  with  save/restore
                command  judiciously  wrapped  around  throw-away
                code.   The  problem   this  poses   for  a   TeX
                environment is that one does not wish to  discard
                downloaded  fonts,  since  they  will  likely  be
                needed again.

                In the previous version of dvialw, the -s and  -v
                options provided some user  control of this,  but
                such user control is only of marginal utility.

                It proved  interesting to  install the  following
                PostScript code  segment immediately  before  the
                BOJ (beginning of job) command in an output  file
                from dvialw  produced  from  Appendix  F  of  the
                TeXBook.   The   TeXbook  uses   54  fonts,   and
                therefore puts  a sizeable  burden on  a  printer
                that has only limited font memory.

% These provide alternate definitions of BOP and EOP in dvialw.ps
% which cause CPU time and virtual memory statistics to be printed
% at the bottom of each output page
/BOP
{
clear
Mtrx setmatrix
25 0 translate
vmstatus pop /VMUSED exch def pop
usertime
} bdf
/EOP
{
/#copies exch def
/USERTIMEIN exch def
save
/pt {Resolution 72 div mul} def
/Helvetica-BoldOblique findfont 12 pt scalefont setfont
/SP {12 pt 0 pt rmoveto} def
/PN {SP 10 vmstr cvrs show} def
/vmstr 15 string def
36 pt 36 pt moveto
([VMstatus:) show
vmstatus PN PN PN (]) show
SP SP ([VMpage:) show
vmstatus pop exch pop VMUSED sub PN
(]) show
SP SP ([Page time:) show
usertime USERTIMEIN sub PN ( msec]) show
restore
showpage
} bdf

                With this addition, each  page now  carries  both
                memory usage and (printer) CPU time statistics at
                the  bottom   of  each   page.   On   the   Apple
                LaserWriter with  paper  type  "note",  there  is
                about 250Kb of  virtual memory  available at  job
                start.   Here   is   a  table   of   the   memory
                requirements for successive  pages of Appendix  F
                of the TeXbook:

                          ----  -----------  -----------
                          Page  DVIALW 2.09  DVIALW 2.10
                          ----  -----------  -----------
                           431                   4953
                           432                  26057
                           433                   3713
                           434                   9693
                           435     27948        21101
                           436     18971        13421
                           437     15644        10109
                           438     21764        15324
                           439     27064        19596
                           440      8172         6628
                          ----  -----------  -----------

                Version  2.09  reached   a  VM  error   condition
                (virtual memory  exhausted)  on page  434,  while
                Version 2.10 was able  to proceed 4 pages  beyond
                that before hitting the same limit.  Each page in
                Version 2.10  requires  about  30%  less  virtual
                memory.

                Initially, I wrote  a short  filter program  that
                moved the text strings on each page to after  the
                font download sequences, wrapping  the text in  a
                save/restore pair to  recover the virtual  memory
                needed to store  them.  This proved  to be  quite
                straightforward, requiring only about 60 lines of
                C code.

                To implement the same  facility in dvialw  proved
                much more complex, because  I was not willing  to
                suffer the  performance penalty  of reading  each
                page of  the  DVI  file twice,  once  to  collect
                needed fonts, and a second  time to set text  and
                rules.  Instead,  a completely  new mechanism  is
                implemented whereby output is channelled  through
                several new routines, which ultimately reach  the
                low-level textchr() routine  which gets a  single
                character to process.   Based on  a global  flag,
                textchr() either outputs  this immediately  (e.g.
                for macros,  rules,  and  font  downloading),  or
                stores it in an  internal buffer (text  setting).
                On the basis of examination of .dvi-alw files for
                the DVI  driver  manual,  and  the  TeXbook,  the
                internal buffer is chosen  to be large enough  so
                that it will usually only need to be dumped  once
                per page.

                PostScript loop  tests on  the Apple  LaserWriter
                Plus (Version  38.0  PostScript)  give  execution
                times of about 0.25 msec for one iteration of  an
                empty loop, determined by executing

                usertime 10000 {} repeat usertime exch sub

                and 55 msec for one save/restore pair:

                usertime 1000 {save restore} repeat usertime exch sub

                Consequently,  one  does   NOT  want   to  do   a
                save/restore around every string, but only around
                large groups of strings.

                Because of  the use  of relative  coordinates  in
                most output typeset text (the T macro instead  of
                the S  macro), and  the  use of  remembered  rule
                widths (w and h in the B and Q macros), it is not
                trivial to correctly handle the case of more than
                one  bufferful   per  page   (since   intervening
                non-buffered output  might  change  the  font  or
                current point), and it took a considerable number
                of iterations to  get it working  satisfactorily,
                with the buffer size temporarily set artificially
                low.

                Having  the  output  being  filtered  through   a
                central location, textchr(), has made it possible
                to further economize on the output volume (e.g by
                collapsing    <letter><newline><open-paren>    to
                <letter><open-paren>,   and   to   more   closely
                approach  a  constant  line  width  (except   for
                PostScript from  a \special{}  command, which  is
                output verbatim).   Some operating  systems  have
                trouble with  long lines  in files,  and  Version
                2.10 should now largely remove this problem.

                The  line  breaking   algorithm  implemented   in
                textchr() and  textflush() is  far from  trivial,
                because PostScript does  not uniformly  recognize
                <backslash><newline> as  a continuation  sequence
                that can be discarded;  instead, it permits  this
                continuation  mechanism  only  in   parenthesized
                strings.    Otherwise,    comments   cannot    be
                continued,   angle-bracketed   strings   can   be
                continued by simply inserting a bare newline, and
                elsewhere, a  newline can  be  used any  place  a
                blank  is  used.   Some  of  the  comment   lines
                produced by dvialw are  long; they are now  split
                automatically by  placing a  hyphen in  the  last
                column of the line,  and continuing with "%-"  on
                the next line.  Any software that takes advantage
                of the %%PageTable at the end will therefore need
                adjusting (I  doubt  that  any  exists,  so  this
                change does not concern  me).  It is  regrettable
                that the PostScript developers did not give  line
                continuation more thought.

                Font  change  commands   are  now  defined   more
                compactly as  "F#" or  "F##",  where "#"  is  the
                sequential number of  the font  in the  document.
                This further helps to reduce output volume.   The
                initial font definitions (the NF macro) still use
                the full name, and carry a comment with the  font
                file name, so it is still possible to easily find
                out what font file was  used for any part of  the
                output.  The problem of corrupted font files  has
                come to my attention on several occasions, so  it
                is worthwhile  to  maintain  this  connection  in
                dvialw, which is  the only  driver that  produces
                human-readable printer output.

                On Appendix  F  of  the  TeXbook,  these  changes
                reduced the output file size by about 8% compared
                to Version 2.09, and about 9% when  PS_SHORTLINES
                was non-zero.   On  the  DVI  driver  manual  (99
                pages), the  file  size from  Version  2.10  with
                PS_SHORTLINES defined was  15% shorter than  that
                from   Version    2.09.    Since    a    non-zero
                PS_SHORTLINES produces marginally smaller output,
                it is now  the default  for all  implementations,
                but  it  can  be  overridden  by  a  compile-time
                redefinition.

[22-Oct-87]     {Thanks to Marcus Moehrmann (marcus%fkihh@unido.uucp)}
                Added drivers dvieps and  dvie72 for Epson  9-pin
                printers and compatibles.  dvieps  is for 240H  x
                216V  resolution,  and  dvie72  for  60H  x   72V
                resolution  (same   as  dviprx   for   Printronix
                printers).

                It was necessary to make  a number of changes  to
                the submitted code  to bring  it up-to-date  with
                the rest  of the  family, and  to conform  to  my
                coding practices.  Since I  do not have an  Epson
                printer  on  which  to  test  this  locally,  the
                version     number     carries     the     suffix
                "[experimental]".  The  same  suffix  appears  in
                dvica2.  These will be  removed when the  drivers
                are considered ready for final release, based  on
                positive user feedback.

[22-Oct-87]     Added  debug option 128 (DBG_SET_TEXT) to display
                each piece of  text set by  the driver,  together
                with its page coordinates  in dots, and its  font
                name.  This of course produces voluminous output,
                but is quite handy when you are trying to step  a
                long way into execution to track down a bug.

[22-Oct-87]     All debug options have been turned into symbolic
                references, e.g.

                if (DBGOPT(DBG_OFF_PAGE))

                instead of

                if (debug_code & 32)

                It was clearly a mistake not to have done this in
                the first place.   I also  wish that  I had  made
                these  options  sequential  numbers,  instead  of
                powers of two,  but it  is probably  too late  to
                change that now.

[12-Oct-87]     Edited dviman.texinfo to bring it up-to-date with
                dvi.1l and dviman.ltx.

[01-Oct-87]     Created Unix-style man page file, dvi.1l.

[01-Oct-87]     Added  support  for  a  "-z"  option  (typein  of
                "DVISPOOL dvifilename") for 4.xBSD Unix  systems,
                which  have  imitated  TOPS-20's  STI%  (Simulate
                Terminal Input) JSYS facility with  ioctl(fildes,
                TIOCSTI, *char).

[30-Sep-87]     {Thanks to Stefan Kronberg (ZEFFI@FINABO.BITNET)}
                Added support  of  font  files  with  up  to  256
                characters.  Normal TeX Computer Modern fonts use
                only 128 characters per font, but extensions  for
                European  alphabets,  and  Japanese  fonts,  will
                require more than  128.

                Unfortunately, this was not quite as trivial as I
                had  originally   intended,  where   the   values
                LASTPXLCHAR and NPXLCHARS in gendefs.h need  only
                be increased  from 127  and 128  to 255  and  256
                respectively.  The problem is that several  loops
                used code like

                BYTE k;
                ...
                for (k = FIRSTPXLCHAR; k <= LASTPXLCHAR; ++k)

                With LASTPXLCHAR = 255,  the last loop  iteration
                will increment k to 256,  but since it is a  byte
                value, this  wraps to  0,  and the  loop  becomes
                infinite on a  byte-oriented machine.  This  made
                it necessary to retype the offending loop indices
                as UNSIGN16.

                The context  diff listing  for these  changes  is
                rather long  (about  1200  lines), so  I  do  not
                include it here.

[30-Sep-87]     Changed -d option processing in option.h to allow
                multiple -d  options  to accumulate,  instead  of
                requiring them to be set with one -d option.  The
                change is simple: replace

                debug_code = (BYTE)atoi(optstr+2);

                by

                debug_code |= (BYTE)atoi(optstr+2);


[29-Sep-87]     Because  of  problems  on  record-oriented   file
                systems (VAX VMS being  the worst offender),  the
                code in dvialw.c does more when PS_SHORTLINES  is
                defined; it now keeps track of output widths, and
                will not produce output lines more than a  couple
                of  characters   longer  than   a  fixed   value,
                PS_MAXWIDTH.  However, no such limit is placed on
                comments, or  macros  retrieved  from  dvialw.ps;
                both of these are already of limited length.

                Changed machdefs.h to  allow PS_SHORTLINES to  be
                set at compile time.

                A single experiment on a 20-page DVI file  showed
                that the .dvi-alw output was about 3% longer when
                a version of  dvialw with  PS_SHORTLINES set  was
                used.

[29-Sep-87]     {Thanks to John Pavel (mcvax!nplpsg!jrp@seismo.css.gov)
                and Brendan McKay (munnari!anucsd.oz!dbm@seismo.css.gov)}

                The code in dvialw.c  in functions setchar()  and
                setstr() emits a save/restore pair around a large
                character  font   definition.    On   the   Apple
                LaserWriter Plus (which I only recently got as  a
                replacement for our stolen ALW), this results  in
                a "Fatal system  error @ 0X2e6bec".   The fix  is
                fortunately simple.  Change  those two  instances
                of

                OUTS("save");

                (but NOT the  one in special()) to

                OUTS("save /temporary_font NF 1 /temporary_font SF");

                This will put the  character in a temporary  font
                which is discarded  when the  restore command  is
                executed.  Here is simple test file using cminch;
                it produces a large (~400Kb) .dvi-alw file.

% test for dvialw downloading of large characters
\font\bigfont=cminch
This file tests a DVI driver's capability to support downloading
of large characters.  The following uninteresting tables of the
Roman alphabet are printed in font {\it cminch\/}.
\bigfont
\obeylines
ABC
DEF
GHI
JKL
MNO
PQR
STU
VWX
YZ
\vfill\eject\end


[29-Sep-87]     {Thanks to John Pavel (mcvax!nplpsg!jrp@seismo.css.gov)}
                Removed  obsolete   SHORTLINES  references   from
                machdefs.h; PS_SHORTLINES remains, and can now be
                set in machdefs.h, instead  of only in  dvialw.c.
                The default setting in machdefs.h now is

                #define PS_SHORTLINES 0

                and dvialw.c now has

                #ifndef PS_SHORTLINES
                #define PS_SHORTLINES   0  /* run with long output lines */
                #endif

-------