summaryrefslogtreecommitdiff
path: root/macros/generic/hyacc-cm/hyacc-cm.doc
diff options
context:
space:
mode:
authorNorbert Preining <norbert@preining.info>2019-09-02 13:46:59 +0900
committerNorbert Preining <norbert@preining.info>2019-09-02 13:46:59 +0900
commite0c6872cf40896c7be36b11dcc744620f10adf1d (patch)
tree60335e10d2f4354b0674ec22d7b53f0f8abee672 /macros/generic/hyacc-cm/hyacc-cm.doc
Initial commit
Diffstat (limited to 'macros/generic/hyacc-cm/hyacc-cm.doc')
-rw-r--r--macros/generic/hyacc-cm/hyacc-cm.doc411
1 files changed, 411 insertions, 0 deletions
diff --git a/macros/generic/hyacc-cm/hyacc-cm.doc b/macros/generic/hyacc-cm/hyacc-cm.doc
new file mode 100644
index 0000000000..1d3a057a11
--- /dev/null
+++ b/macros/generic/hyacc-cm/hyacc-cm.doc
@@ -0,0 +1,411 @@
+ %%% File HyAcc-CM.doc
+ %% Documentation for typographers and TeX programmers
+ %% For basic user instructions see the header
+ %% of HyAcc-CM.tex, the one example below
+ %% and the format compilation mechanism
+ %% fe-format-dumper-cm
+
+
+ OUR PERSPECTIVE ON THE HYPHENATION PROBLEM
+
+ Jacques Desarmenien's devices for obtaining respectable
+hyphenation in the presence of accents were never optimized nor
+internationalized --- conjecturally because it was expected
+that a version 3 of TeX would be based on MLTeX of M. Ferguson
+and rather simply offer optimal hyphenation with no change in
+the TeX environment.
+
+ This did not come to pass. When TeX version 3 came along,
+it instead offered something better inasmuch as the accented
+characters can be tailor-made under the new 8-bit regime
+(rather than composed of accent plus character). But, at the
+same time, the cost was unexpectedly high: new fonts, new
+drivers, new norms, and new formats. They are burdensome for
+many European TeX users, and perhaps out of reach of many
+non-European users.
+
+ It was then announced that TeX and Metafont will not be
+further changed by its author Don Knuth, nor by by anyone else
+so long as they bear their present copyrighted names.
+
+ Thus, rather unexpectedly, it has became desirable to have a
+good international version of modest but decent hyphenation in
+the presence of accents that works with absolutely all versions
+of TeX since 1982, in particularly with TeX and Computer Modern
+fonts as documented by Knuth.
+
+ If you use TeX in a version <3, prior to fall 1989, you
+should avoid recent hyphenation pattern files. Also, for better
+portability, always prefer pattern files in ASCII format.
+
+ Many 7-bit european hyphenation pattern tables are sufficiently
+compact and portable to circulate along with .tex manuscripts by
+electronic mail. For example, f7hyphen.tex the 7-bit French
+hyphenation file of Desarmenien is just 4Ko, while HyAcc-CM.tex is
+5Ko (without comments).
+
+
+
+A TYPICAL (FRENCH) USE OF HYACC-CM
+
+ Let us consider the typical case of a French visitor to the
+Anglo-Saxon world who, to process his French typescript, has to
+use a "mainframe" computer exploiting exclusively CM fonts. If he
+is fortunate enough to use Plain.tex, he can quite simply replace
+TeX by INITEX at the cost of adding as a header to his typescript
+the incantation:
+
+ \input plain.tex %% standard, using US hyphenation for language 0
+ \bgroup\catcode`@=11
+ \global\def\US{\lccode`\'=0\nofrenchspacing\language=0}
+ \newlanguage\l@French
+ \global\def\FR{\lccode`\'=`\'\frenchspacing\language=\l@French}
+ \egroup
+ \FR
+ \input f7hyphen.tex
+ \input hyacc-cm.tex
+
+His TeX typescript will be set by INITEX starting with French
+hyphenation; \US restores US English. This assumes the
+typescript inputs adequate macros for French punctuation etc.
+
+ Alternatively, he can make of this header a separate file
+concluded by \dump newformat.fmt. Then INITEX dumps a format,
+say newformat.fmt, which can in turn be used under INITEX
+by reducing the header to
+
+ &newformat.fmt
+
+This second procedure is the one that best generalizes to massive
+formats such as LaTeX. Some ready-made recipes for them can be
+found in fe-format-dumper-cm posted on ftp matups.matups.fr.
+(These include fe-punct.tex and gllmts.tex giving rudimentary
+macros for French versus English punctuation.) However, in
+hostile environments, the use of simple formulas as above is often
+the most cost-effective strategy.
+
+ In a worst possible scenario, INITEX (initex) is unavailable.
+Then one has to use the English hyphenation in conjunction with
+HyAcc-CM. For this, the following simpler header can be used on
+top of any standard English format:
+
+ \def\US{\lccode`\'=0 \nofrenchspacing}
+ \def\FR{\lccode`\'=`\' \frenchspacing}
+ \FR
+ \input hyacc-cm.tex
+ \pretolerance=800 %% discourages hyphenation
+
+Hyphenation points that occur have to be checked for correctness
+and replaced by explicit hypenations (via \-) when necessary.
+
+
+A HISTORY AND EXPLANATION OF THE MACROS
+
+ The background reference for all matters to be discussed is
+Appendix H of the TeXbook, and particularly pages 454-455.
+
+ The story begins with Jacques Desarmenien's thesis, Univ.
+Louis Pasteur, Strasbourg, written in the early 1980's partly at
+Stanford under Knuth's influence. The accessible reference is
+Jacques D'esarm'enien: How to run TeX in a French environment:
+hyphenation, fonts, typography, Tugboat, 5 (1984) 91-102. This
+reference discusses hyphenation through the use of fonts
+containing ready-made accented characters of the sort slowly
+coming into use now some 10 years later. This package puts to
+work devices for hyphenation which do NOT require new fonts, and
+which were called upon a few years later when distribution of
+new fonts was provisionally abandoned.
+
+ It is often repeated that hyphenation of a word under 1982
+TeX is blocked at points at and after the first accented
+character because of the explicit kerns that the \accent
+primitive of TeX introduces there.
+
+ The following formula appeared in an unsigned file
+feHyphen.tex that was available from Strasbourg from the mid
+1980's. According to Raymond Seroul, the author is J.
+Desarmanien. The resulting bilingual fePlain format had wide
+currency in France through Seroul's influence.
+
+ \newbox\accentb@x \newdimen\wdb@x
+ \newbox\accentb@@x
+ \let\aig@=\'%
+ \def\'#1{\if e#1\setbox\accentb@x=\hbox{\char"13}%
+ \setbox\accentb@@x=\hbox{e}%
+ \wdb@x=.5\wd\accentb@@x \advance\wdb@x by -.5\wd\accentb@x
+ \kern\wdb@x \char"13 \kern-\wd\accentb@x
+ \penalty\@M \hskip-\wdb@x e%
+ \else \aig@#1\fi}
+
+A verbal interpretation of this is as follows. Let the one
+argument macro \' for accute accents be redefined as follows in
+case the letter being accented is e. Build the accented
+character <e-acute> from the accute accent character "13 and the
+character e using boxes and kerns as common sense recommends,
+except for one detail. Let the last kern (which occurs just
+before the letter e) become an hskip. This causes TeX to
+consider that a new word has begun with a letter e and allows
+hyphenation in that (sub)word. Normally an hskip is an
+acceptable place for a linebreak, but this is prevented in this
+case by insertion of an explicit penalty of 10000.
+
+ Why was only the character \'e dealt with? Conjecturally
+because effort was made via Desarmenien's hyphenation patterns
+to assure that no incorrect hyphenation point resulted from this
+trick. This is a standard we cannot maintain since it is
+implausible that anything but a standard 8 bit hyphenation file
+will be available to users.
+
+ This means that the Desarmenien mechanism will occasionally
+allow an unacceptable hyphenation point. (We do not yet have
+statistics on this.) Some specialists, perhaps including Knuth,
+would frown on this. Other specialists believe that hyphenation
+should in any case be based more on phoenetics meaning and
+etymology than on spelling, and that sort is sometimes
+impossible to perfect with the Knuth-Liang approach. If
+perfection is impossible why insist on complete rigour?
+
+ In the late 1980's, in the Latex style file called
+German.sty H. Partl (and collaborators) used a version of this
+"penalty and hskip" trick which is so clear and simple that it
+allows one to deal with all accents and all letters on an equal
+footing. For example Partl gives
+
+ \gdef\allowhyphens{\penalty\@M \hskip\z@skip}
+ % {\nobreak \hskip 0pt plus 0pt\relax}
+
+ \gdef\highumlaut#1{\leavevmode\allowhyphens
+ \accent127 #1\allowhyphens}
+
+Here the one-argument macro \highumlaut#1 is a new version of
+\"#1 designed to allow hyphenation after the accented character.
+This is certainly shorter and faster than Desarmenien's
+formulation. But there are some peculiarities to discuss.
+
+ The subword following the accent does not include the
+character accented. This unfortunately supresses any kern
+between the character accented and a following character. Such
+kerns are respected by both Knuth's regime (\DKaccents) and
+(generalized) Desarmenien's (\JDaccents). (Note that kerning
+with *preceeding* material is not within the scope of either,
+but see below.)
+
+ Another effect is to keep hyphens further from the
+accented character, which is not in itself a good thing, but
+it may also tend to eliminate bad hyphenations caused by the
+fake word break. (No detailed evaluation has been made.)
+
+ The positive role of \leavevmode and the first
+\allowhyphens is unclear to us and we have let them drop.
+
+ One of us (D.F.) noticed that in case the accented letter
+occurs at the end of a word, linebreak becomes impossible there.
+This may not occur in German but if one were to adopt the German
+approach for accute accent, defining
+
+ \def\'#1{{\accent"13 #1\allowhyphens}}
+
+then linebreak would be prevented after all the innumerable
+French words terminating in e-acute! Here, surely, is the
+reason why \allowhyphens never gained currency in France.
+
+ This disaster does not occur with the Desarmenien approach
+because the construction of the accented character concludes
+with a character and not an hskip.
+
+ Examine now our own definition of \'; under our \JDaccents
+regime it is essentially
+
+ \newbox\hy@accbox
+ \def\'#1{{\accent"13 #1}\setbox\hy@accbox=\hbox{#1}
+ \penalty10000\hskip-\wd\hy@accbox #1}
+
+We chose to use the standard \accent macro and nevertheless use
+Desarmenien's "penalty-hskip-letter" approach, at the
+(negligible!) cost of setting two copies of the letter to be
+accented, one exactly on top of the other. Our approach is thus
+not quite J. Desarmenien's but rather a trick simplification. It
+would have been possible to pursue the Desarmenien approach in
+all cases, but the coding would have been messy for capital
+letters and miscellaneous accents. Incidentally, Desarmenien
+makes it quite clear that he was privy to the folklore of Knuth
+and his students and has made no explicit claim to this trick.
+He did first bring it to fruition, and so we use his initials in
+\JDaccents.
+
+ Granting that one chooses an approach making the letter to
+be accented part of the post-accent (sub)word, it might be
+desirable to make the pre-accent (sub)word terminate with the
+same letter. The two subwords then have that letter in common.
+This might give slightly more acceptable hyphenation points. And
+it would enable kerning with a preceeding letter, not just with
+a following letter. In principle the method extends to do this:
+
+ \def\'#1{{#1\setbox\hy@accbox=\hbox{#1}%
+ \penalty10000\hskip-\wd\hy@accbox
+ \accent"13 #1%
+ \penalty10000\hskip-\wd\hy@accbox #1}}
+
+THREE copies of the letter to be accented are then printed
+directly on top of one another.
+
+ In principle, with digital raster imaging, the result
+should be identical to printing once. However many driver/font
+combinations will at low resolution slightly misplace the first
+of the three copies --- in case the accent occurs at the end of
+a long string of ordinary letters. This is because character
+width errors accumulate over a long string, and thereafter the
+\accent macro makes a fresh calculation of absolute position.
+The Textures screen driver dramatically exhibits this behavior.
+Knowing no cure for this malady, we have abandoned this "three
+copy" approach. Other versions are afflicted to a lesser degree
+by this phenomenon, because the shift involved, although
+present, is between characters several pixels apart.
+
+ We believe that our "two-copy" or "double strike" approach
+will very seldom cause a problem.
+
+ Alas, with some combinations of low resolution drivers and
+fonts some sort of drift has indeed been observed on screen
+and/or paper. For these unfortunate cases we give another
+approach \GFaccents (GF for German-French) which directly
+salvages the German approach by use of an \if ...\fi clause
+involving \futurelet. If the token following the character
+accented is a letter (like a) then one inserts before it a
+penalty of 1000 followed by an hskip of 0 points in order to
+start a new word; but in other cases (notably when a space comes
+next) one inserts nothing. This allows linebreaks after words
+ending in an accented character and followed by a space.
+Expressed in formulae
+
+ \def\preLetterSkip@{\ifcat a\next\penalty\@M\hskip\z@skip\fi}
+ \def\@@@@Allowhyphens{\futurelet\next\preLetterSkip@}
+ \def\'#1{{\accent18#1}\@@@@Allowhyphens}
+
+is essentially our definition of the accute accent \' for the
+\GFaccents regime.
+
+ Unfortunately, this solution will suppress all kerns
+between the character accented and a following character. Also,
+it can be defeated by nonstandard typing; for example TeX braces
+around an accented character will block hyphenation beyond.
+
+ Further, 'futurelet' has occasional nasty side-effects in
+LaTeX. For example, disaster occurs when an accented character
+ends a 'tabular' environment as in:
+
+ -- \begin{tabular} ... Painlev\'e\end{tabular}
+ -- \author{J. Dieudonn\'e}
+
+It's easy to find work-arounds: just add a space after the
+terminal accented character (or put the whole word between curly
+braces) and your problems are over. Thus the modified versions
+
+ -- \begin{tabular} ... Painlev\'e \end{tabular}
+ -- \author{J. Dieudonn\'e } or \author{J. {\sc Dieudonn\'e}}
+
+will cause no problem. But all this shows that this solution is
+fragile.
+
+
+
+LEFTOVERS AND IMPERFECTIONS
+
+ 0) \ogonek #1 provides a provisional ogonek accent under character #1
+It is a translated copy of the left single quote. With this exception
+we treat only the accents in the TeXbook, page 52.
+
+ 1) This version makes no effort to eliminate the following
+deficiency of Plain TeX: \uppercase and \lowercase fail when
+applied to the miscellaneous characters \oe, \OE, \ae, \AE, \o,
+\O, \ss and also to \i. Our \MLcodes macro merely serves to
+allow some hyphenations in words involving these characters.
+This is an imperfection.
+
+ 2) If your computer and TeX have eight bit input capabilities,
+you may wish to type some accented characters in ready-made
+eight bit form using macros of the following sort. (Your TeX
+installation probably has a ready-made collection.)
+
+ \catcode <e-accute>=13
+ \def<e-accute>{\'e}
+
+The formulas are here in an informal but portable ASCII format,
+and you have to put your local e-accute in place of <e-accute>
+before they will work.
+
+ Next some changes that are common safety measures in various
+national macro packages and which we have included here only in
+the form of hints for the TeXpert, since we feel they are
+primarily the responsibility of other macro packages.
+
+ 3) Problems with \write. Only in the last few years have
+simple and fast "sanitized" macros come into use for the purpose
+of writing pieces of the .tex file to an external file. The
+naked \write command makes a mess of accents and of many other
+things, and even may choke. The basic reference seems to be R.
+Whitney, TUGboat, November 1990. Here is a still simpler method:
+
+ \newwrite\index \openout\index{myindex}
+ \def\iWrite#1#2{\toks0{#2}%
+ \immediate\write#1{\the\toks0}
+
+And a typical use is:
+
+ \iWrite\index{...}
+
+The material ... then goes out to the file myindex.tex more or
+less untouched.
+
+ This does not work well for non-immediate writes ---
+typically used to record page number of an index entry, but one
+reasonable remedy is to write the dangerous material with
+\iWrite to one file and the page numbers with \write to another
+file and then collate the two using some device like serial
+numbers or simply line numbers.
+
+ 4) Cleaner messages.
+
+ \message is of the same ilk as \write (and \edef). The above
+"\the\toks..." non-expansion trick can be applied to it to
+prevent mutilation or mishap.
+
+ However most messages come from the bowels of
+macro packages and it is not easy to be sure of their contents.
+Active punctuation as well as accent macros often creep in.
+Recall that character category (eg. 13=active) is fixed forever at TeX's
+lips when the character is input.
+
+ One can input the following redefinition while
+;:?! are active. It will help combat these problems.
+But first check your current meaning of \message.
+
+ \let\@oldmessage@=\message
+ \def\message#1{\begingroup
+ \def\'{'}\def\`{`}\def\"{"}%
+ \def\i{i}\def\oe{oe}\def\ae{ae}\def\c{}%
+ \def\.{}\def\u{}\def\v{}%
+ \def\H{}\def\={}\def\relax{}%
+ \edef~{\string~}%
+ \edef\^{\string^^5e}%
+ \edef:{\string:}%
+ \edef;{\string;}%
+ \edef?{\string?}%
+ \edef!{\string!}%
+ \@oldmessage@{#1}\endgroup}
+
+
+ -------------------------
+
+
+If you notice errors or oversights please contact us.
+
+
+
+ --- Daniel Flipo, Lille <flipo@frcitl81.bitnet>
+
+ --- Laurent Siebenmannn, Orsay <siebenmann@LALCLS.IN2P3.FR>
+
+ April, August, December 1992
+
+