path: root/Master/texmf-dist/doc/latex-dev/base/ltnews33.tex

% \iffalse meta-comment
%
% Copyright (C) 2019-2024
% The LaTeX Project and any individual authors listed elsewhere
% in this file.
%
% This file is part of the LaTeX base system.
% -------------------------------------------
%
% It may be distributed and/or modified under the
% conditions of the LaTeX Project Public License, either version 1.3c
% of this license or (at your option) any later version.
% The latest version of this license is in
%    https://www.latex-project.org/lppl.txt
% and version 1.3c or later is part of all distributions of LaTeX
% version 2008 or later.
%
% This file has the LPPL maintenance status "maintained".
%
% The list of all files belonging to the LaTeX base distribution is
% given in the file `manifest.txt'. See also `legal.txt' for additional
% information.
%
% The list of derived (unpacked) files belonging to the distribution
% and covered by LPPL is defined by the unpacking scripts (with
% extension .ins) which are part of the distribution.
%
% \fi
% Filename: ltnews33.tex
%
% This is issue 33 of LaTeX News.

\NeedsTeXFormat{LaTeX2e}[2020-02-02]

\documentclass{ltnews}

%%CCC  Temporary definitions:
\providecommand\Dash {\unskip ---}



%% NOTE:  Chris' preferred hyphens!
%%\showhyphens{parameters}
\hyphenation{because para-me-ters}

\usepackage[T1]{fontenc}

\usepackage{lmodern,url,hologo}

\usepackage{csquotes}
\usepackage{multicol}

\providecommand\hook[1]{\texttt{#1}}

\providecommand\meta[1]{$\langle$\textrm{\itshape#1}$\rangle$}
\providecommand\option[1]{\texttt{#1}}
\providecommand\env[1]{\texttt{#1}}
\providecommand\Arg[1]{\texttt\{\meta{#1}\texttt\}}


\providecommand\eTeX{\hologo{eTeX}}
\providecommand\XeTeX{\hologo{XeTeX}}
\providecommand\LuaTeX{\hologo{LuaTeX}}
\providecommand\pdfTeX{\hologo{pdfTeX}}
\providecommand\MiKTeX{\hologo{MiKTeX}}
\providecommand\CTAN{\textsc{ctan}}
\providecommand\TL{\TeX\,Live}
\providecommand\githubissue[2][]{\ifhmode\unskip\fi
     \quad\penalty500\strut\nobreak\hfill
     \mbox{\small\slshape(%
       \href{https://github.com/latex3/latex2e/issues/\getfirstgithubissue#2 \relax}%
          	    {github issue#1 #2}%
           )}%
     \par\smallskip}

% simple solution right now (just link to the first issue if there are more)
\def\getfirstgithubissue#1 #2\relax{#1}

\providecommand\sxissue[1]{\ifhmode\unskip\fi
     \quad\penalty500\strut\nobreak\hfill
     \mbox{\small\slshape(\url{https://tex.stackexchange.com/#1})}\par}

\providecommand\gnatsissue[2]{\ifhmode\unskip\fi
     \quad\penalty500\strut\nobreak\hfill
     \mbox{\small\slshape(%
       \href{https://www.latex-project.org/cgi-bin/ltxbugs2html?pr=#1\%2F\getfirstgithubissue#2 \relax}%
          	    {gnats issue #1/#2}%
           )}%
     \par}

\let\cls\pkg
\providecommand\env[1]{\texttt{#1}}
\providecommand\acro[1]{\textsc{#1}}

\vbadness=1400  % accept slightly empty columns


\makeatletter
% maybe not the greatest design but normally we wouldn't have subsubsections
\renewcommand{\subsubsection}{%
   \@startsection      {subsubsection}{2}{0pt}{1.5ex \@plus 1ex \@minus .2ex}%
      {-1em}{\@subheadingfont\colonize}%
}
\providecommand\colonize[1]{#1:}
\makeatother

\let\finalvspace\vspace          % for document layout fixes



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\providecommand\tubcommand[1]{}
\tubcommand{\input{tubltmac}}

\publicationmonth{June}
\publicationyear{2021}  %% --- Draft Version 3p

\publicationissue{33}

\begin{document}

\tubcommand{\addtolength\textheight{4.2pc}}   % only for TUB

\maketitle
{\hyphenpenalty=10000 \spaceskip=3.33pt \hbadness=10000 \tableofcontents}

\setlength\rightskip{0pt plus 3em}


\medskip


\section{Introduction}

The focus of the June 2021 release is to provide further important
building blocks for the future production of
reliable tagged PDF output
(see \cite{33:blueprint}); these enhancements
are discussed in the next two
sections. 

Subsequent sections describe quite a number of recent smaller
enhancements and fixes.  As usual, more detail on individual changes
can be found in the \file{changes.txt} files in the distribution
and, of course, in the documented sources~\cite{33:source2e}.



\section{Extending the hook concept to paragraphs}

Largely triggered by the need for better control of paragraph text
processing, in particular when producing tagged PDF output, we have
changed \LaTeX{} so that the kernel gains control both at the start
and at the end of each paragraph. This is done in a manner that is (or
should be) transparent to both packages and documents.

Besides the addition of internal control points for the exclusive use
of the \LaTeX{} kernel, we also implemented four public hooks that can
be used in packages or documents (via the normal hook management
declarations) to achieve special effects, etc.  Until now, such
enhancements required redefinitions of \cs{everypar} or \cs{par},
which led to the usual issues since such changes can easily conflict
with changes made by other packages.

The documentation of these new \enquote{paragraph hooks}, together
with a few examples, is in \file{ltpara-doc.pdf} and, for those who
want to study it, the (quite interesting) code can be found in
\file{ltpara-code.pdf}. Additionally, both of these files are included
as part of the full kernel documentation in \file{source2e.pdf}.



\section{Extending the hook concept to commands}

Up to now, hook management covered hooks for only a few core areas,
such as the hooks for the \cs{shipout} process or those in the
\env{document} environment, as well as some \enquote {generic} hooks,
both for file loading (helpful for patching such files) and for
arbitrary environments (the hooks executed within \cs{begin} and
\cs{end}).  This concept of \enquote{generic hooks} has now been
extended to provide \hook{/before} and \hook{/after} hooks for any
(document-level) command\Dash in theory at least.


In practice, these new generic \hook{cmd} hooks, especially the
\hook{cmd/.../after}, hooks may fail with commands that are too
complex to be automatically patched, breaking if the hook contains any
code.  These restrictions are documented in
\file{ltcmdhooks-doc.pdf}.
%   
However, given that these hooks are mainly meant for developers who
wish to provide better interoperability between different packages,
and between packages and the \LaTeX{} kernel, these restrictions are,
we hope, of minor importance.  Indeed, for commands where this
mechanism can't be applied, one is in the same situation as before;
and for all others there will be a noticeable improvement.

These hooks will be especially important for our current project to
provide accessible and tagged PDF output~\cite{33:blueprint} because
we will eventually have to patch many third-party packages, and this
must be done in controlled and standardized ways.




\section{Other hook business}


\subsection{Shipping out a page while bypassing hooks}

In the 2020 October release, several hooks were added to control the
process of constructing and shipping out a page box: these support,
for example, the addition of background or foreground material 
to some or all pages.

 
We have now added a command, called \cs{RawShipout}, which does not do
any rebuilding of the page box and so does not run most of these
hooks.  When using this new command, essential internal book-keeping
is still carried out, such as updating the \texttt{totalpages} counter
and adding \texttt{shipout/firstpage} or \texttt{shipout/lastpage}
material when appropriate.


\subsection{A new Lua callback in \pkg{ltshipout},
for custom attributes}

For use just before shipping out a page, there is now a \LuaTeX{}
callback \texttt{pre\_shipout\_filter} to contain final adjustments to
the box being shipped out.  This is particularly useful for
Lua\TeX\ packages which flag (using, for example, attributes or
properties) elements on a page in order to apply effects (such as the
insertion of \enquote{color commands}) to these elements at shipout.



\section{Improved handling of file names}

\subsection[File names with spaces, multiple dots or\\
            \acro{utf-8} characters]
           {File names with spaces, multiple dots or \acro{utf-8} characters}

In one of the recent \LaTeX{} releases we improved the interface
for specifying file names so that they can now safely contain spaces
(as is common these days),
more than one dot character, and also \acro{utf-8} characters
outside the \acro{ascii} range. 
In the past this was only possible by applying a special syntax
in the case of spaces, 
while file names with several dots often failed, 
as did most \acro{utf-8} characters.


\subsubsection{Consequences for file names in \cs{include}}

\TeX{} has a built-in rule saying that you can normally leave out the
extension if it is \texttt{.tex}.  Thus \verb=\input{file}= and
\verb=\input{file.tex}= both load \file{file.tex} (if it exists).
While this is convenient most of the time, it is a little awkward in
some scenarios (for example, when both \file{file} and \file{file.tex}
exist) and also when you manually try to implement the rule.

\LaTeX{} therefore had one special syntax for \cs{include} and
\cs{includeonly}: they always expected that 
their arguments contain a
file name\footnote{In the case of \cs{includeonly}, a comma-separated list of such names.} 
with no extension given,
  so that it had to be \texttt{.tex}.  Thus,
 when you mistakenly wrote
\verb=\include{mychap.tex}= (for example,
because you changed from \cs{input}
to \cs{include}),
\LaTeX{} went ahead and looked for the
file \file{mychap.tex.tex} for inclusion and tried to
use the file \file{mychap.tex.aux} for internal (auxiliary) information.  The reason was that
\cs{include} had to construct both
of these file names from the given
argument and it didn't bother to do
anything special
with the supplied 
extension \texttt{.tex}.

With the new implementation this has
changed:
the extension \texttt{.tex}
now gets removed/ignored if it was
supplied.
Thus \verb=\include{mychap.tex}= now 
no longer looks for \file{mychap.tex.tex} 
but loads
\file{mychap.tex} 
and uses \file{mychap.aux}.
%
\githubissue{486}



\subsection{Normalization of robust commands in file names}

The handling of file names has been modified so that \verb|\string| is
applied to normalize robust commands within the file name.
Previously, for example, \verb|\input{\sqrt{2}}| would cause
\LaTeX\ to loop indefinitely whereas with 
the new normalization
it looks for the file named \verb|sqrt {2}.tex|
(and therefore very likely reports ``file not found'').
%
\githubissue{481}




\subsection{Fix for \env{filecontents} with \acro{utf-8} 
  chars in the file name}

Since a few releases back, the \env{filecontents} environment has
allowed \acro{utf-8} characters in the file name.  There was, however,
a bug that would not allow \emph{over}writing a file with \acro{utf-8}
characters in its name.  This has been fixed and now
\env{filecontents} allows any characters in the file name.
%
\githubissue{415}



\section{Updates to the font selection scheme}


\subsection{A new hook in \cs{selectfont}}

After \cs{selectfont} has changed the font, we now run a hook (\hook{selectfont})
so that packages can make final \mbox{adjustments}. This functionality was
originally provided by the \pkg{everysel} package but our
implementation is slightly different and uses the standard hook
management.
%
\githubissue{444}


\subsection{Change of font series/shape delayed until \cs{selectfont}}

With the NFSS extensions introduced in 2020, the font series and shape
settings can be influenced by changes to the font family.  The
settings of these two are now therefore delayed until \cs{selectfont}
is executed; this avoids unnecessary or incorrect substitutions that
may otherwise happen due to the order of declarations.
%
\githubissue{444}



\section{Glyphs, characters \& encodings}

\subsection{Improved copy\,\&\,paste for \pdfTeX{} documents}

When compiling with \pdfTeX{}, additional information
(from the file \file{glyphtounicode.tex}) is now added automatically
to the PDF file in order to improve copying from, and searching in,
text.

In particular, this allows the most common ligatures to be copied as
intended from all generated PDF files without the need to explicitly
load the package \pkg{cmap}.  
%
\githubissue{465}





\subsection{Support for more Unicode characters}


\LaTeX\ is quite capable of typesetting characters such as
\enquote{\d{m}}, but until now it could not access some Unicode
characters from the Latin Extended Additional block.  This meant that,
for example, there were no Unicode mappings for some characters that
are used to write Sanskrit words in Latin transliteration (as seen in
books about yoga, Buddhist philosophy, etc.).
%
These characters have now been added so that they can be entered
directly instead of using \verb=\d{m}=, etc.
%
\githubissue{484}





\subsection{More ``dashes'' in encodings \texttt{OT1},
  \texttt{T1} and \texttt{TU}}

When pasting in text from external sources, one can encounter these
three Unicode characters
%
\texttt{"2011} (non-breaking hyphen),
\texttt{"2012} (figure dash) and
\texttt{"2015} (horizontal bar),
%
in addition to the more common 
%
\texttt{"2013} (en-dash) and \texttt{"2014} (em-dash).
%
In the past, these first three produced an error message when used
with \pdfTeX{} (since they are not available in \texttt{OT1} or
\texttt{T1} encoded fonts).  They now typeset an approximation to the
glyph: e.g., the ``figure dash'' is approximated by an en-dash.

With Unicode engines they either work (when the glyph is contained in
the selected Unicode font) or they typeset nothing, producing a
``Missing character'' warning in the log file.

With all engines these characters can also now be accessed using the
command names \cs{textnonbreakinghyphen}, \cs{textfiguredash} and
\cs{texthorizontalbar}, respectively.
%
\githubissue{404}


\subsection{Poor man's \cs{textasteriskcentered}}

The \cs{textasteriskcentered} symbol, used as part of the set of
footnote symbols in \LaTeX{}, is assumed to be implemented by every
font with the \texttt{TS1} encoding (when \pdfTeX{} is used) or with
the \texttt{TU} encoding for the Unicode engines.  That assumption is
unfortunately not correct for all fonts since, for example, the
\texttt{stix2} fonts don't provide this glyph.  A result is that one
gets missing glyph messages when using \cs{thanks}, etc.

Therefore \cs{textasteriskcentered} now checks whether there is such a
glyph and, if not, uses a normal \enquote{*}, but slightly enlarged
and lowered.  This may not be perfect in all cases, but it is
certainly better than no glyph showing up.
%
\githubissue{502}

\subsection {The characters from \pkg{textcomp} are in the kernel}

A couple of releases back, the functionality of the \pkg{textcomp}
package was integrated into the \LaTeX{} kernel.  Thus it is no longer
necessary to load this package in order to access glyphs such as
\cs{textcopyright}, \cs{texteuro} or \cs{textyen}.
 
 
At this time the opportunity was also taken to bring some order to the
chaos surrounding the question: \enquote{which glyphs from the
  \texttt{TS1} encoding are available in a given font?}.  This was
done using an approach based on font families and collections, with
the differing glyph coverage of the \enquote*{text symbols} being
indicated by assigning to a font family or collection a ``sub-encoding
number'' that indicates which glyphs from the \texttt{TS1} encoding are
guaranteed to be available when using a font from that family or
collection. This assignment ensures that \LaTeX{} always errs on the
side of caution, possibly claiming that a glyph is not available even
when it in fact is.

\iffalse  %%FMi but drop that

The documented code for this can be found now in the file 
\file{lttextcomp.dtx} but we hope to publish a full explanation of 
the approach very soon now. 

\fi  %% FMi potential drop



\subsubsection
   [A note on the history of ``text symbols'']
   {A note on the history of ``text symbols'' and
     the  \texttt{TS1} encoding}


The \enquote{text symbol encoding} (\texttt{TS1}) was originally
designed at the Cork Conference as a companion to the \texttt{T1}
encoding. In it various symbols that are not subject to hyphenation
got assembled and the \pkg{textcomp} package was developed to make
them accessible. Unfortunately the \TeX{} community was a bit too
enthusiastic and included several symbols only available in a few
\TeX{} fonts and some, such as the capital accents, not available at
all but developed as part of the reference font implementation.

In hindsight that was a very bad idea because it meant that other
existing fonts (at the time) and later new fonts that got developed
were unable to provide the full set of glyphs that made up the
\texttt{TS1} encoding. For existing free PostScript fonts people 
took the extra effort and produced virtual fonts that faked (some) of
the missing glyphs. But this was and is a time-consuming effort so it
was done for only a few basic fonts. But even then, only some fonts
included all glyphs from \texttt{TS1} so the \pkg{textcomp} already
back then contained a long list, dividing fonts into 5 categories
according to which glyphs were implemented and which were missing.


When we recently integrated the functionality of the \pkg{textcomp}
into the \LaTeX{} kernel
many new free fonts had appeared and
unfortunately the chaos around the question \enquote{which glyphs of
  the \texttt{TS1} encoding are implemented by which font} had
increased with it. Not only did one find many new holes, it was next to
impossible to order the set of fonts into a reasonable set of
sub-encodings that are contained in each other in a single sequence.

In the end we decided on nine or ten sub-encodings with a reasonable
number of fonts in each so that all fonts implemented all glyphs of the
sub-encoding they got mapped to. Thus when typesetting with a font one
could be sure that a command like \cs{textcopyleft} would either
typeset the requested character (if the glyph was part of the
sub-encoding the font belonged to) or it would raise an error, saying
that the glyph is unavailable in that font.  The mapping would ensure
that \LaTeX{} always errs on the side of caution, because it might
claim a glyph is unavailable even though in fact it is.\looseness-1

For example, the old \texttt{pcr} (PostScript Courier) font (as well
as most other older PS fonts) is mapped to sub-encoding 5 and
therefore claims that \cs{textasciigrave} is unavailable even though
in fact for Courier this is not true. If one uses such a font and this
becomes an issue then there are a couple (suboptimal) possibilities.
For one, one can alter the mapping of Courier and pretend that belongs
to a fuller sub-encoding, e.g.
\begin{verbatim}
  \DeclareEncodingSubset{TS1}{pcr}{2}
\end{verbatim}
The downside is, that \LaTeX{} then believes other glyphs that are in fact
unavailable are also there, so that it is important to check that the
final document doesn't have some missing glyphs.

An alternative is to pretend that \cs{textasciigrave} can always be
taken from the \texttt{TS1} encoding (no questions asked):
\begin{verbatim}
  \DeclareTextSymbolDefault{\textasciigrave}{TS1}
\end{verbatim}
Again there is a danger that this is not true when it is used with a
different font and would then generate a missing glyph.

Finally, and possibly the best solution, if not impossible for other
reasons, is to simply use a different font, for example, to use the
\TeX{} Gyre Cursor font (a reimplementation of Courier with a
much more complete glyph set).




\section{New or improved commands}

\subsection{Adjusting \env{itemize} labels with \cs{labelitemfont}}

The command \cs{labelitemfont} was introduced already with the
\LaTeX\ release 2020-02-02, but back then we forgot to describe it, so
we do this now. Its purpose is to resolve some bad formatting issues
with the \env{itemize} environment and also to make it easier to
adjust the layout when necessary. What could happen in the past was
that the \env{itemize} labels (e.g., the \textbullet{}) would
sometimes react to surrounding font changes and could then suddenly
change shape, for example to \textit{\textbullet}.

This new command \cs{labelitemfont}, which defaults to \cs{normalfont}, 
can be used to provide additional control in the typesetting of
each label.  Thus by choosing
different settings other effects can be achieved.  Here are two
examples:
\begin{verbatim}
  \renewcommand\labelitemfont
     {\normalfont\fontfamily{lmss}\selectfont}
  \renewcommand\labelitemfont
     {\rmfamily\normalshape}
\end{verbatim}
The first definition will take the symbols from the font Latin Modern
Sans, so that you get
%
\def\myfont#1{{\let\labelitemfont\empty\fontfamily{lmss}\selectfont#1}}%
%
\myfont\labelitemi, \myfont\labelitemii, \myfont\labelitemiii\ and
\!\!\myfont\labelitemiv\,; while the second variant freezes the font family
and shape, but leaves the series as a variable quantity, so that an
\env{itemize} in a bold context would show bolder symbols. Making
\cs{labelitemfont} empty would give you back the buggy old behavior.
%
\githubissue{497}


\subsection{Producing several marks for one footnote}

It is sometimes necessary to reference the same footnote several
times: i.e., to produce several footnote marks using the same number
or symbol. This is now easily possible by placing a \cs{label} within
the referenced \cs{footnote} and referencing this label by using the
new command \cs{footref}.  This means that footnote marks can be
generated to refer to arbitrary footnotes (including those in
\env{minipage}s).

This \cs{footref} command has previously been available, but only when
using certain classes or the \pkg{footmisc} package.
%
\githubissue{482}


\subsection{Allow \cs{nocite} in the preamble}

A natural place for \verb=\nocite{*}= would be the preamble of the
document, but for historical reasons \LaTeX{} issued an error message
if it was placed there.  This command is now allowed in the preamble.
%
\githubissue{424}


\subsection{Made \cs{\textbackslash} generally robust}

In 2018 most \LaTeX{} user-level commands were made robust, including
the \cs{\textbackslash} command.  However, \cs{\textbackslash} gets
redefined in various environments and not all these cases were caught:
such as, in particular, its use as the row delimiter in \env{tabular}
structures.  This has been corrected so that \cs{\textbackslash}
should now be robust in all circumstances.


This change also fixed one anomaly present in the past:
in a tabular preamble of the form
\finalvspace*{-.3\baselineskip}
\begin{quote}
  \hspace*{-.15em}\verb={l=\texttt{\string>}\verb={\raggedright}p{10cm}r}=    % stupid class
\end{quote}
\finalvspace*{-.3\baselineskip}
a \cs{\textbackslash} in the second column would have the definition
used within \cs{raggedright} and so it would not indicate the
(premature) end of the \env{tabular}.  Thus, for example,%
\finalvspace*{-.3\baselineskip}
\begin{quote}
   \verb=a & b1 \\ b2 & c \\=
\end{quote}
\finalvspace*{-.3\baselineskip}
was interpreted as a single row of the \env{tabular} (as intended),
whereas
\finalvspace*{-.3\baselineskip}
\begin{quote}
   \verb=a &    \\ b2 & c \\=
\end{quote}
\finalvspace*{-.3\baselineskip}
resulted in two rows!  This happened because the \cs{\textbackslash}
directly following the \verb=&= got interpreted while it still had the
\enquote{end the row} meaning and not yet the \enquote{start a new
  line within the second column} meaning.

With \cs{\textbackslash} now being robust, the special scanning mode
initiated by the \verb=&= ends immediately when this command is seen:
the second column is therefore then started, which results in the
\cs{\textbackslash} being interpreted as being within that column and
hence as having its expected, within-column, meaning.

We have restored consistency here: now both of the above lines
produce a single \env{tabular} row.
%
As before, you can 
put \cs{raggedright}\cs{arraybackslash} in the \env{tabular}'s
preamble for a column to ensure that \cs{\textbackslash} is always
interpreted as a tabular row separator when used in that column. And
you can use \cs{tabularnewline} to explicitly ask for a new table row,
even when \cs{\textbackslash} has a different meaning within the
current column.
%
\githubissue{548}




\subsection{Allow extra space between name and address in \pkg{letter} class}

The \cs{opening} command in the \pkg{letter} class expects the name
and address to be separated by \verb=\\=, but it didn't allow the use
of an optional argument to add some extra space after the name. The
code has now been slightly altered to allow this.
%
\githubissue{427}



\subsection{Additions to \cs{tracingall}}

In July 2020 David Jones suggested an extension to \TeX{} engines,
that added the possibility to set \cs{tracinglostchars}\texttt{=3} in
order to generate an error message in case some character is missing
from a font.  In previous years, a warning about a missing character
was silently printed to the \texttt{.log} file\linebreak
(if $\cs{tracinglostchars}>0$) and to the terminal\linebreak
 (if ${}>1$).  This extension was added for \TL{} and \MiKTeX{} 
(except in Knuth's \TeX, of course), 
so that with $\cs{tracinglostchars}>2$ you now also get an
error message for each missing glyph.

Later, in January 2021, Petr Olšák suggested yet another extension:
a new primitive
\mbox{parameter}
%
\cs{tracingstacklevels} that, when both it and \cs{tracingmacros} are
positive, will add to the \mbox{tracing} information for each 
macro a visual indication (using dots) of
its nesting level in the macro expansion stack.

These changes have both now been added to \LaTeX's debugging macros
\cs{tracingall} and \cs{tracingnone}, so that these two new extensions
are activated/deactivated as appropriate, so long as the \TeX{} engine
supports them.  An example document demonstrating these parameters is
in the linked GitHub issue.
%
\githubissue{524}




\section{Code improvements}

\subsection{Execute \cs{par} at the end of \cs{marginpar}}

Previously, \LaTeX{} ended a \cs{marginpar} without ever explicitly
calling \cs{par}.  This command is now explicitly added because it is
essential to the correct working of the paragraph hooks.

Another case where this issue caused problems was the \pkg{lineno}
package, where the last line was not numbered if the \cs{marginpar}
ended without an explicit \cs{par}.
%
\githubissue{489}



\subsection{Execute \cs{AtEndDocument} hook in vertical mode}

Until now \verb=\end{document}= executed the code from the
\cs{AtEndDocument} hook as its first action.  This meant that this
hook was executed in horizontal mode if the user left no empty line
after the last paragraph.  As a result, one could get a spurious space
added when, for example, that code contained a \cs{write}
statement. This was fixed and now \cs{enddocument} first issues a
\cs{par} to ensure that it always goes into vertical mode.
%
\githubissue{385}


\subsection{Color groups made permanent}

The use of color in certain \LaTeX{} constructs, especially boxes,
needs an extra layer of grouping to ensure that the color setting does
not \emph{escape} and continue outside the box when it shouldn't.
%
To support this, the \LaTeX{} kernel defines a number of commands,
e.g., \cs{color@begingroup} to be used in such places.


Until now, these commands were initially set as no-ops and only the
color packages redefined them to become real groups; this methodology
complicates the coding as one has to account for a group being present
or not (\mbox{depending} on what is loaded in the document).
%
The kernel therefore now permanently adds these \enquote{color groups}.
%
\githubissue{488}


\subsection{Provide the raw option list to key/value option handlers}

Before any further processing of the option list, the original
(un-normalized, \enquote{raw} and unchanged) list of package or class
options is now saved, as \cs{@raw@opt@...}; this list is not used by
the standard option processing code but it is now available for use by
extended class/package processing systems.  Note that, for
compatibility reasons, the standard option processing code has not
been changed.



One
aspect of this
change does affect the standard \mbox{processing}: any tokens to the
right of an \texttt{=}
sign are \mbox{removed} 
from consideration
when constructing 
the \enquote{\mbox{unused} option list}.
For example, in 
this release \texttt{clip=true} and
\texttt{clip=false} both contribute \texttt{clip} to the list of
options that have been used.
%
\githubissue{85}


\subsection{New for \pkg{latexrelease}\,: \cs{NewModuleRelease}}

To explain the need for this new feature, we shall consider the
following example: in the 2020-10-01 release, \LaTeX's new hook
management system was added to the kernel (see \cite{33:ltnews32})
and, as with all changes to the kernel, it was added to
\pkg{latexrelease}; this made it possible to roll back to a date where
this module didn't yet exist, or to roll forward from an older
\LaTeX{} release to get the hook management system (by loading the
\pkg{latexrelease} package).
%
However, this method of rolling back from a later release to the
2020-10-01 release didn't quite work because it would try to define
all the commands from \pkg{lthooks} again; and this would of course
result in the expected errors from commands defined with
\cs{newcommand} or (as in \pkg{lthooks}) \cs{cs\_new:Npn}.

To solve such issues, we now provide \cs{NewModuleRelease} so that
completely new modules can be defined using the facilities of
\pkg{latexrelease} in such a way that, when rolling back or forward,
the system will know whether the code of the new module has to be read
or completely ignored.  More details on this can be found in the
\pkg{latexrelease} documentation (get this with
\verb|texdoc latexrelease|).
%
\githubissue{479}


\subsection{Small fix for rolling back prior to 2020-02-02}

Whereas the \pkg{latexrelease} package can usually emulate an older
\LaTeX{} kernel without much problem, rolling back to before the
2020-02-02 release didn't work properly: this is because the
management of the \cs{ExplSyntaxOn/Off} status for packages (after an
\pkg{expl3}-based package is loaded) cannot be removed by the rollback
without messing up the catcodes.  This has been fixed so that rollback
is now more careful not to leave \cs{ExplSyntaxOn} after a package
ends.
%
\githubissue{504}



\section{Changes to packages in the \pkg{graphics} category}


\subsection{Removed warning when loading graphics files}

A previous release sometimes mistakenly caused a (false) warning
message to appear when \mbox{using} a generic graphics rule to find
and load a graphics file with an unknown \mbox{extension}.
%%CCC removing hyphenation here makes this one line longer.
This warning would incorrectly say that the file was not found,
whereas the file would in fact be correctly loaded.  The warning now
doesn't show up in that case.
%
\githubissue{516}


\subsection{Fixed loading of \texttt{gzip}ped
  PostScript files}

A previous release mistakenly changed the file searching mechanism so
that compressed PostScript graphics files would raise an error when
being loaded with \cs{includegraphics}.  This has been fixed so that
\texttt{gzip}ped graphics files now load correctly.
%
\githubissue{519}




\section{Changes to packages in the \pkg{tools} category}

\subsection{\pkg{layout}: Added language options}

This package now recognizes \option{japanese} and \option{romanian} as
language options.
%
\githubissue[s]{353 and 529}


\subsection{\pkg{array} and \pkg{longtable}: Make \cs{\textbackslash} generally robust}

The fix for this issue was also applied to these packages; see above.
%
\githubissue{548}


\subsection{\pkg{longtable}: General bug fix update}

This is a minor update to the \pkg{longtable} package that fixes several
reported bugs: notably the possibility of incorrect page breaks when
floats appear on the page where a \env{longtable} starts.  As this may
affect page breaking in existing documents, a rollback to
\pkg{longtable 4.13} (\file{longtable-2020-01-07.sty}) is supported.
%
\gnatsissue{tools}{2914 3396 3512}
\githubissue{133 137 183 464 561}


\subsection{\pkg{trace}: Additions to \cs{traceon}}

The \cs{tracingstacklevels} and \cs{tracinglostchars} extensions to
\cs{tracingall} (see above) were also added to \cs{traceon} in the
\pkg{trace} package, so its users can also benefit from these new
debugging possibilities.
%
\githubissue{524}


\subsection{\pkg{bm}: Better support for commands with optional arguments}

Some uses of optional arguments in \cs{bm} stopped being supported (in
2004) when \cs{kernel@ifnextchar} was used internally by the format
instead of \cs{@ifnextchar}. This update handles both versions of this
command and restores the original \mbox{behavior}.

In addition, package options for guiding the use of \enquote{poor
  man's bold} in fallback situations were added.
%
\githubissue{554}



\section{Changes to packages in the \pkg{amsmath} category}

The fix for issue 548 was also applied in \pkg{amsmath}; see above. 
%
\githubissue{548}




\medskip

\begin{thebibliography}{9}

\fontsize{9.3}{11.3}\selectfont

\bibitem{33:blueprint} Frank Mittelbach and Chris Rowley:
  \emph{\LaTeX{} Tagged PDF \Dash A blueprint for a large project}.\\
  \url{https://latex-project.org/publications/indexbyyear/2020/}

\bibitem{33:source2e}
  \emph{\LaTeX{} documentation on the \LaTeX{} Project Website}.\\
  \url{https://latex-project.org/help/documentation/}

\bibitem{33:ltnews32} \LaTeX{} Project Team:
  \emph{\LaTeXe{} news 32}.\\
  \url{https://latex-project.org/news/latex2e-news/ltnews32.pdf}

\end{thebibliography}



\end{document}