diff options
Diffstat (limited to 'Master/texmf-dist/source/latex/babel/babel.dtx')
-rw-r--r-- | Master/texmf-dist/source/latex/babel/babel.dtx | 1229 |
1 files changed, 990 insertions, 239 deletions
diff --git a/Master/texmf-dist/source/latex/babel/babel.dtx b/Master/texmf-dist/source/latex/babel/babel.dtx index 75cea56e4d5..d76895d5856 100644 --- a/Master/texmf-dist/source/latex/babel/babel.dtx +++ b/Master/texmf-dist/source/latex/babel/babel.dtx @@ -36,7 +36,7 @@ %\iffalse %% File 'babel.dtx' %\fi -%%\ProvidesFile{babel.dtx}[2017/05/19 v3.10 The Babel package] +%%\ProvidesFile{babel.dtx}[2017/06/26 v3.11 The Babel package] %\iffalse % % Babel DOCUMENT-STYLE option for LaTeX version 2.09 or plain TeX; @@ -208,8 +208,8 @@ \vspace{2cm}\hspace{1.5cm} \begin{minipage}{5cm} \Large -Version 3.10\newline -2017/05/19 +Version 3.11\newline +2017/06/26 \vspace{1cm} \textit{Original author}\newline @@ -238,10 +238,9 @@ Javier Bezos in a language other than US English, or in more than one language or script. - However, no attempt has been done to take full advantage of the - features provided by the latter, which would require a completely - new core (as for example \file{polyglossia} or as part of \LaTeX3). - \vspace{20cm} + Current development is focused on Unicode engines (Xe\TeX{} and + Lua\TeX). New features related to font selection, bidi writing and + the like will be added incrementally. \vspace{20cm} \end{minipage} \end{tabular} \end{minipage} @@ -254,23 +253,24 @@ Javier Bezos \section{The user interface}\label{U-I} -The basic user interface of this package is quite simple. It consists -of a set of commands that switch from one language to another, and a -set of commands that deal with shorthands. It is also possible to find -out what the current language is. +\subsection{Monolingual documents} In most cases, a single language is required, and then all you need in \LaTeX{} is to load the package using its standand mechanism for this purpose, namely, passing that language as an optional argument. In -addition, you may want to set the font and input encodings. Here is a -simple full example: +addition, you may want to set the font and input encodings. + +\begin{example} + Here is a simple full example. The packages |fontenc| and |inputenc| + do not belong to \babel, but they are included in the example + because typically you will need them: \begin{verbatim}[escapechar=\&] \documentclass{article} \usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} -\usepackage[frenchb]{babel} +\usepackage[french]{babel} \begin{document} @@ -278,45 +278,78 @@ Plus &\c c&a change, plus c'est la m&\^e&me chose! \end{document} \end{verbatim} - +\end{example} \begin{warning} A common source of trouble is a wrong setting of the input encoding. Make sure you set the encoding actually used by your editor. \end{warning} -In multilingual documents, just use several option. So, in \LaTeX2e\ -the preamble of the document: +Another approach is making \Lopt{french} a global +option in order to let other packages detect and use it: + +\begin{verbatim} +\documentclass[french]{article} +\usepackage{babel} +\usepackage{varioref} +\end{verbatim} + +In this last example, the package \texttt{varioref} will also see +the option and will be able to use it. + +\begin{note} + Because of the way \babel{} has evolved, ``language'' can refer to + (1) a set of hyphenation patterns as preloaded into the format, (2) + a package option, (3) an |ldf| file, and (4) a name used in the + document to select a language or dialect. So, a package option + refers to a language in a generic way -- sometimes it is the actual + language name used to select it, sometimes it is a file name loading + a language with a different name, sometimes it is a file name + loading several languages. Please, read the documentation for + specific languages for further info. +\end{note} + +\subsection{Multilingual documents} + +In multilingual documents, just use several options. The last one is +considered the main language, activated by default. Sometimes, the +main language changes the document layout (eg., |spanish| and |french|). + +\begin{example} + In \LaTeX, the preamble of the document: \begin{verbatim} \documentclass{article} \usepackage[dutch,english]{babel} \end{verbatim} -would tell \LaTeX\ that the document would be written in two -languages, Dutch and English, and that English would be the first -language in use, and the main one. You can also set the main language -explicitly: + would tell \LaTeX\ that the document would be written in two + languages, Dutch and English, and that English would be the first + language in use, and the main one. +\end{example} + +You can also set the main language explicitly: \begin{verbatim} \documentclass{article} \usepackage[main=english,dutch]{babel} \end{verbatim} -Another approach is making \Lopt{dutch} and \Lopt{english} global -options in order to let other packages detect and use them: -\begin{verbatim} -\documentclass[dutch,english]{article} -\usepackage{babel} -\usepackage{varioref} -\end{verbatim} - -In this last example, the package \texttt{varioref} will also see -the options and will be able to use them. - -Languages may be set as global and as package option at the same -time, but in such a case you should set explicitly the main language -with the package option |main|: +\begin{warning} + Languages may be set as global and as package option at the same + time, but in such a case you should set explicitly the main language + with the package option |main|: \begin{verbatim} \documentclass[italian]{book} \usepackage[ngerman,main=italian]{babel} \end{verbatim} +\end{warning} + +\begin{warning} + In the preamble the main language has \textit{not} been selected, + except hyphenation patterns and the name assigned to |\languagename| + (in particular, shorthands, captions and date are not activated). If + you need to define boxes and the like in the preamble, you might + want to use some of the language selectors described below. +\end{warning} + +\subsection{Modifiers} \New{3.9c} The basic behaviour of some languages can be modified when loading \babel{} by means of \textit{modifiers}. They are set after @@ -334,22 +367,11 @@ Attributes (described below) are considered modifiers, ie, you can set an attribute by including it in the list of modifiers. However, modifiers is a more general mechanism. -\begin{note} - Because of the way \babel{} has evolved, ``language'' can refer to - (1) a set of hyphenation patterns as preloaded into the format, (2) - a package option, (3) an |ldf| file, and (4) a name used in the - document to select a language or dialect. So, a package option - refers to a language in a generic way -- sometimes it is the actual - language name used to select it, sometimes it is a file name loading - a language with a different name, sometimes it is a file name - loading several languages. Please, read the documentation for - specific languages for further info. -\end{note} - +\subsection{Notes} Loading directly |sty| files in \LaTeX{} (ie, |\usepackage|\marg{language}) is deprecated and you will get the -error:\footnote{In former versions the error read ``You have used an +error:\footnote{In old versions the error read ``You have used an old interface to call babel'', not very helpful.} \begin{verbatim} ! Package babel Error: You are loading directly a language style. @@ -358,7 +380,7 @@ old interface to call babel'', not very helpful.} \end{verbatim} Another typical error when using \babel{} is the -following:\footnote{In former versions the error read ``You haven't +following:\footnote{In old versions the error read ``You haven't loaded the language LANG yet''.} \begin{verbatim} ! Package babel Error: Unknown language `LANG'. Either you have misspelled @@ -372,6 +394,8 @@ all, and therefore you removed it from the option list). In most cases, the error vanishes when the document is typeset again, but in more severe ones you will need to remove the |aux| file. +\subsection{Plain} + In Plain, load languages styles with |\input| and then use |\begindocument| (the latter is defined by \babel): \begin{verbatim}[morekeywords={[0]{\input,\begindocument}}] @@ -383,17 +407,16 @@ not compatible with Plain.\footnote{Even in the \babel{} kernel there were some macros not compatible with plain. Hopefully these issues will be fixed soon.} -\subsection{Selecting languages} +\subsection{Basic language selectors} This section describes the commands to be used in the document to -switch the language in multilingual document. +switch the language in multilingual document. In most cases, only the +two basic macros |\selectlanguage| and |\foreignlanguage| are +necessary. The environments |otherlanguage|, |otherlanguage*| and +|hyphenrules| are auxiliary, and described in the next section. The main language is selected automatically when the |document| -environment begins. In the preamble it has \textit{not} been selected, -except hyphenation patterns and the name assigned to |\languagename| -(in particular, shorthands, captions and date are not activated). If -you need to define boxes and the like in the preamble, you might want -to use some of the following commands. +environment begins. \Describe\selectlanguage{\marg{language}} When a user wants to switch from one language to another he can @@ -401,24 +424,43 @@ do so using the macro |\selectlanguage|. This macro takes the language, defined previously by a language definition file, as its argument. It calls several macros that should be defined in the language definition files to activate the special definitions -for the language chosen. For ``historical reasons'', a macro name is -converted to a language name without the leading |\|; in other words, -the two following declarations are equivalent: +for the language chosen: \begin{verbatim} \selectlanguage{german} -\selectlanguage{\german} \end{verbatim} -Using a macro instead of a ``real'' name is deprecated. -If used inside braces there might be some non-local changes, as this -would be roughly equivalent to: + +This command can be used as environment, too. + +\begin{note} + For ``historical reasons'', a macro name is converted to a language + name without the leading |\|; in other words, + |\selectlanguage{\german}| is equivalent to |\selectlanguage{german}|. + Using a macro instead of a ``real'' name is deprecated. +\end{note} + +\begin{warning} + If used inside braces there might be some non-local changes, as this + would be roughly equivalent to: \begin{verbatim} {\selectlanguage{<inner-language>} ...}\selectlanguage{<outer-language>} \end{verbatim} -If you want a change which is really local, you must enclose this -code with an additional grouping level. + If you want a change which is really local, you must enclose this + code with an additional grouping level. +\end{warning} -This command can be used as environment, too. +\Describe\foreignlanguage{\marg{language}\marg{text}} +The command |\foreignlanguage| takes two arguments; the second +argument is a phrase to be typeset according to the rules of the +language named in its first one. This command (1) only switches the +extra definitions and the hyphenation rules for the language, +\emph{not} the names and dates, (2) does not send information about +the language to auxiliary files (i.e., the surrounding language is +still in force), and (3) it works even if the language has not been +set as package option (but in such a case it only sets the hyphenation +patterns and a warning is shown). + +\subsection{Auxiliary language selectors} \Describe{\begin\menv{otherlanguage}}{\marg{language}\Eenv{otherlanguage}} @@ -440,27 +482,17 @@ environment with an additional grouping, like braces |{}|. Spaces after the environment are ignored. -\Describe\foreignlanguage{\marg{language}\marg{text}} -The command |\foreignlanguage| takes two arguments; the second -argument is a phrase to be typeset according to the rules of the -language named in its first one. This command (1) only switches the -extra definitions and the hyphenation rules for the language, -\emph{not} the names and dates, (2) does not send information about -the language to auxiliary files (i.e., the surrounding language is -still in force), and (3) it works even if the language has not been -set as package option (but in such a case it only sets the hyphenation -patterns and a warning is shown). - \Describe{\begin\menv{otherlanguage*}}% {\marg{language}\Eenv{otherlanguage*}} Same as |\foreignlanguage| but as environment. Spaces after the environment are \textit{not} ignored. -This environment was also intended for intermixing left-to-right +This environment was originally intended for intermixing left-to-right typesetting with right-to-left typesetting in engines not supporting a -change in the writing direction inside a line. However, its behaviour -is not always the expected one. +change in the writing direction inside a line. However, by default it never +complied with the documented behaviour and it was just a version as +environment of |\foreignlanguage|. \Describe{\begin\menv{hyphenrules}}{\marg{language}\Eenv{hyphenrules}} @@ -473,9 +505,9 @@ deactivates language shorthands, too (but not user shorthands). Except for these simple uses, |hyphenrules| is discouraged and |otherlanguage*| (the starred version) is preferred, as the former -does not take into account possible changes in encodings or characters +does not take into account possible changes in encodings of characters like, say, |'| done by some languages (eg, \textsf{italian}, -\textsf{frenchb}, \textsf{ukraineb}). To set hyphenation exceptions, +\textsf{french}, \textsf{ukraineb}). To set hyphenation exceptions, use |\babelhyphenation| (see below). \subsection{More on selection} @@ -483,14 +515,15 @@ use |\babelhyphenation| (see below). \Describe\babeltags{\char`\{\m{tag1} \texttt{=} \m{language1}, \m{tag2} \texttt{=} \m{language2}, \dots\char`\}} -\New{3.9i} In multilingual documents with many language switches the commands -above can be cumbersome. With this tool shorter names can be -defined. It adds nothing really new -- it is just syntactical sugar. +\New{3.9i} In multilingual documents with many language switches +the commands above can be cumbersome. With this tool shorter names can +be defined. It adds nothing really new -- it is just syntactical +sugar. It defines |\text|\m{tag1}\marg{text} to be |\foreignlanguage|\marg{language1}\marg{text}, and |\begin|\marg{tag1} to be |\begin{otherlanguage*}|\marg{language1}, and so on. Note -|\|\m{tag1} is also allowed, but remember set it locally inside a +|\|\m{tag1} is also allowed, but remember to set it locally inside a group. So, with \begin{verbatim} \babeltags{de = german} @@ -543,9 +576,13 @@ the language, not global macros (eg, |\TeX| of |\dag|). \Describe\languagename{} The control sequence |\languagename| contains the name of the -current language. However, due to some internal inconsistencies in -catcodes it should \textit{not} be used to test its value (use -\textsf{iflang}, by Heiko Oberdiek). +current language. + +\begin{warning} + Due to some internal inconsistencies in catcodes it should + \textit{not} be used to test its value. Use \textsf{iflang}, by + Heiko Oberdiek. +\end{warning} \Describe\iflanguage{\marg{language}\marg{true}\marg{false}} @@ -556,8 +593,13 @@ sense, as a set of hyphenation patterns, and \textit{not} as its \textsf{babel} name. This macro takes three arguments. The first argument is the name of a language; the second and third arguments are the actions to take if the result of the test is true or false -respectively. The advice about |\languagename| also applies here -- -use \textsf{iflang} instead of |\iflanguage| if possible. +respectively. + +\begin{warning} + The advice about |\languagename| also applies here -- use + \textsf{iflang} instead of |\iflanguage| if possible. +\end{warning} + \subsection{Selecting scripts} @@ -573,6 +615,10 @@ contains glyphs for Modern Greek does not mean it includes them for Classic Greek. As to directionality, it poses special challenges because it also affects individual characters and layout elements.} +The same applies for bidi texts -- writing direction is intrinsic to +each writing system and therefore it is best set by the language +(which could be a dummy one). + Some languages sharing the same script define macros to switch it (eg, |\textcyrillic|), but be aware they may also set the language to a certain default. Even the \babel{} core defined |\textlatin|, but is @@ -601,7 +647,7 @@ but if you load |LY1,T2A| it is set to |T2A|. The symbol encodings used for ``ordinary'' text. The foregoing rules (which are applied ``at begin document'') cover -most of cases. Note no asumption is made on characters above +most of cases. No asumption is made on characters above 127, which may not follow the LICR conventions -- the goal is just to ensure most of the ASCII letters and symbols are the right ones. @@ -667,7 +713,7 @@ sets the |\catcode| to active (13). Both commands only work on `known' shorthand characters. If a character is not known to be a shorthand character its category code will be left unchanged. -\New{3.9a} Note however, |\shorthandoff| does not behave as +\New{3.9a} However, |\shorthandoff| does not behave as you would expect with characters like |~| or |^|, because they usually are not ``other''. For them |\shorthandoff*| is provided, so that with @@ -687,7 +733,7 @@ The command |\useshorthands| initiates the definition of user-defined shorthand sequences. It has one argument, the character that starts these personal shorthands. -\New{3.9a} However, user shorthands are not always alive, as they may +\New{3.9a} User shorthands are not always alive, as they may be deactivated by languages (for example, if you use |"| for your user shorthands and switch from \textsf{german} to \textsf{french}, they stop working). Therefore, a starred version @@ -715,30 +761,31 @@ User shorthands override language ones, which in turn override system shorthands. Language-dependent user shorthands (new in 3.9) take precedence over ``normal'' user shorthands. -As an example of their applications, let's assume you want a -unified set of shorthand for discretionaries (languages do not -define shorthands consistently, and |"-|, |\-|, |"=| have -different meanings). You could start with, say: +\begin{example} + Let's assume you want a unified set of shorthand for discretionaries + (languages do not define shorthands consistently, and |"-|, |\-|, + |"=| have different meanings). You could start with, say: \begin{verbatim} \useshorthands*{"} \defineshorthand{"*}{\babelhyphen{soft}} \defineshorthand{"-}{\babelhyphen{hard}} \end{verbatim} -However, behaviour of hyphens is language dependent. For example, in -languages like Polish and Portugese, a hard hyphen inside compound -words are repeated at the beginning of the next line. You could then -set: + However, behaviour of hyphens is language dependent. For example, in + languages like Polish and Portugese, a hard hyphen inside compound + words are repeated at the beginning of the next line. You could then + set: \begin{verbatim}[morekeywords={[1]{polish,portugese}}] \defineshorthand[*polish,*portugese]{"-}{\babelhyphen{repeat}} \end{verbatim} -Here, options with |*| set a language-dependent user shorthand, which -means the generic one above only applies for the rest of languages; -without |*| they would (re)define the language shorthands instead, -which are overriden by user ones. + Here, options with |*| set a language-dependent user shorthand, + which means the generic one above only applies for the rest of + languages; without |*| they would (re)define the language shorthands + instead, which are overriden by user ones. -Now, you have a single unified shorthand (|"-|), with a content-based -meaning (`compound word hyphen') whose visual behavior is that -expected in each context. + Now, you have a single unified shorthand (|"-|), with a + content-based meaning (`compound word hyphen') whose visual behavior + is that expected in each context. +\end{example} \Describe\aliasshorthand{\marg{original}\marg{alias}} @@ -794,8 +841,8 @@ off with |\shorthandoff| or (3) deactivated with the internal \verb|\babelshorthand{:}|. (You can conveniently define your own macros, or even you own user shorthands provided they do not ovelap.) -For your records, here is a list of shorthands, but you must check -them, as they may change:\footnote{Thanks to Enrico Gregorio} +For your records, here is a list of shorthands, but you must double +check them, as they may change:\footnote{Thanks to Enrico Gregorio} \begin{description} \itemsep=-\parskip @@ -850,7 +897,7 @@ $\string|$ \texttt{off}} The only language shorthands activated are those given, like, eg: \begin{verbatim} -\usepackage[esperanto,frenchb,shorthands=:;!?]{babel} +\usepackage[esperanto,french,shorthands=:;!?]{babel} \end{verbatim} If \verb|'| is included, \texttt{activeacute} is set; if \verb|`| is included, \texttt{activegrave} is set. Active characters (like @@ -922,7 +969,8 @@ traditional \TeX, LICR and ASCII strings), |unicode| (for engines like encodings). Other allowed values are font encoding codes (|T1|, |T2A|, |LGR|, |L7X|...), but only in languages supporting them. Be aware with |encoded| captions are protected, but they work in |\MakeUppercase| -and the like. +and the like (this feature misuses some internal \LaTeX\ tools, so use +it only as a last resort). \Describe{hyphenmap=}{\texttt{off} $\string|$ \texttt{main} $\string|$ \texttt{select} $\string|$ \texttt{other} $\string|$ @@ -972,9 +1020,9 @@ This command is currently the only provided by |base|. Executes \meta{code} when the file loaded by the corresponding package option is finished (at |\ldf@finish|). The setting is global. So \begin{verbatim} -\AfterBabelLanguage{frenchb}{...} +\AfterBabelLanguage{french}{...} \end{verbatim} -does ... at the end of |frenchb.ldf|. It can be used in |ldf| files, +does ... at the end of |french.ldf|. It can be used in |ldf| files, too, but in such a case the code is executed only if \meta{option-name} is the same as |\CurrentOption| (which could not be the same as the option name as set in |\usepackage|!). @@ -1002,7 +1050,7 @@ Defines the internal structure of the language with some defaults: the hyphen rules, if not available, are set to the current ones, left and right hyphen mins are set to 2 and 3, but captions and date are not defined. Conveniently, \babel{} warns you about what to do. Very -likely you will find warnings like that in the |log| file: +likely you will find alerts like that in the |log| file: \begin{verbatim} Package babel Warning: \mylangchaptername not set. Please, define (babel) it in the preamble with something like: @@ -1054,7 +1102,7 @@ subsequent ones are silently ignored). It is mostly useful with for example: \begin{verbatim} \babelprovide[hyphenrules=+]{neo} -\babelpatterns[neo]{a1 e2 i1 o1 u1} +\babelpatterns[neo]{a1 e1 i1 o1 u1} \end{verbatim} In other engines it just supresses hyphenation (because the pattern list is empty). @@ -1272,7 +1320,7 @@ and whether the attribute(s) are known for this language. Very often, using a \textit{modifier} in a package option is better. Several language definition files use their own methods to set -options. For example, \textsf{frenchb} uses |\frenchbsetup|, +options. For example, \textsf{french} uses |\frenchsetup|, \textsf{magyar} (1.5) uses |\magyarOptions|; modifiers provided by |spanish| have no attribute counterparts. Macros settting options are also used (eg, |\ProsodicMarksOn| in \textsf{latin}). @@ -1437,15 +1485,18 @@ some languages (kerning and spacing). Ligatures can be disabled. Useful additions would be, for example, time, currency, addresses and personal names.\footnote{See for example POSIX, ISO 14652 and the -Unicode Common Locale Data Repository (CLDR).}. But that is the easy +Unicode Common Locale Data Repository (CLDR). Those system, however, +have limited application to \TeX\ because their aim is just to display +information and not fine typesetting.}. But that is the easy part, because they don't require modifying the \LaTeX{} internals. -More interesting are differences in the sentence structure or related to -it. For example, in Basque the number precedes the name (including -chapters), in Hungarian ``from (1)'' is ``(1)-b\H{o}l'', but ``from (3)'' -is ``(3)-b\'{o}l'', in Spanish an item labelled -``3.$^{\textrm{\scriptsize o}}$'' may -be referred to as either ``\'{\i}tem 3.$^{\textrm{\scriptsize o}}$'' or ``3.$^{\textrm{\scriptsize er}}$ \'{\i}tem'', and so on. +More interesting are differences in the sentence structure or related +to it. For example, in Basque the number precedes the name (including +chapters), in Hungarian ``from (1)'' is ``(1)-b\H{o}l'', but ``from +(3)'' is ``(3)-b\'{o}l'', in Spanish an item labelled +``3.$^{\textrm{\scriptsize o}}$'' may be referred to as either +``\'{\i}tem 3.$^{\textrm{\scriptsize o}}$'' or +``3.$^{\textrm{\scriptsize er}}$ \'{\i}tem'', and so on. Even more interesting is right-to-left, vertical and bidi typesetting. Babel provided a basic support for bidi text as part of @@ -1453,15 +1504,16 @@ the style for Hebrew, but it is somewhat unsatisfactory and internally replaces some hardwired commands by other hardwired commands (generic changes would be much better). -Handling of ``Unicode'' fonts is also problematic. There is +\subsection{Tentative and experimental code} + +Handling of ``Unicode'' fonts is problematic. There is \textsf{fontspec}, but special macros are required (not only the NFSS ones) and it doesn't provide ``orthogonal axis'' for features, including those related to the language (mainly language and script). A couple of tentative macros, which solve the two main cases, are provided by \babel{} ($\ge$3.9g) with a partial solution (only \xetex{} and \luatex, for obvious reasons), but use them at your own -risk, as they might be removed in the future. For this very reason, -they are described here: +risk, as they might be removed in the future. \begin{itemize} \item |\babelFSstore|\marg{babel-language} sets the current three basic families (rm, sf, tt) as the default for the language @@ -1481,8 +1533,32 @@ So, for example: \babelFSfeatures{turkish}{Language=Turkish} \end{verbatim} Note you can set any feature required for the language -- not only -|Language|, but also |Script| or a local |.fea|. This makes those macros -a bit more verbose, but also more powerful. +|Language|, but also |Script| and even raw features. This makes those +macros a bit more verbose, but also more powerful. + +Bidi writing is taking its \textit{first steps}. Here is a simple example: +\begin{verbatim} +\documentclass{article} + +\usepackage[english]{babel} +\babelprovide{arabic} % declare a new empty language + +\usepackage{fontenc} +\setmainfont[Script=Arabic,Language=Arabic]{Traditional Arabic} +\babelFSstore[Arabic]{arabic} + +\begin{document} +English \foreignlanguage{arabic}{Arabic} English +\end{document} +\end{verbatim} +\textit{First steps} means exactly that. For example, in \luatex{} +digits and short Latin texts must me marked up explicitly in RL +mode. On the other hand, \xetex{} poses quite different +challenges. The bidi mechanism is activated when an RL script is +passed as the new optional argument of |\babelFSstore|. + +See the code section for |\foreignlanguage*| (a new starred version of +|\foreignlanguage|). \section{Loading languages with \file{language.dat}} @@ -2174,62 +2250,48 @@ This macro is not intended to fix wrong mappings done by Unicode assignment is wrong, fix it directly. -\section{Compatibility and changes} - -\subsection{Compatibility with \file{german.sty}}\label{l-h} +\section{Changes} -The file \file{german.sty} has been one of the sources of inspiration -for the \babel{} system. Because of this I wanted to include -\file{german.sty} in the \babel{} system. To be able to do that I had -to allow for one incompatibility: in the definition of the macro -|\selectlanguage| in \file{german.sty} the argument is used as the -{$\langle \it number \rangle$} for an |\ifcase|. So in this case a -call to |\selectlanguage| might look like |\selectlanguage{\german}|. +\subsection{Changes in \babel\ version 3.9} -In the definition of the macro |\selectlanguage| in \file{babel.def} -the argument is used as a part of other macronames, so a call to -|\selectlanguage| now looks like |\selectlanguage{german}|. Notice -the absence of the escape character. As of version~3.1a of \babel{} -both syntaxes are allowed. +Most of changes in version 3.9 are related to bugs, either to fix them +(there were lots), or to provide some alternatives. Even new features +like |\babelhyphen| are intended to solve a certain problem (in this +case, the lacking of a uniform syntax and behaviour for shorthands +across languages). These changes are described in this manual in the +corresponding place. A selective list follows: +\begin{itemize} +\item |\select@language| did not set |\languagename|. This meant the + language in force when auxiliary files were loaded was the one used + in, for example, shorthands -- if the language was |german|, a + |\select@language{spanish}| had no effect. -All other features of the original \file{german.sty} have been copied -into a new file, called \file{germanb.sty}\footnote{The `b' is added -to the name to distinguish the file from Partls' file.}. +\item |\foreignlanguage| and |otherlanguage*| messed up + |\extras<language>|. Scripts, encodings and many other things were + not switched correctly. -Although the \babel{} system was developed to be used with \LaTeX, -some of the features implemented in the language definition files -might be needed by plain \TeX\ users. Care has been taken that all -files in the system can be processed by plain \TeX. +\item The |:ENC| mechanism for hyphenation patterns used the encoding + of the \textit{previous} language, not that of the language being + selected. -\subsection{Compatibility with \file{ngerman.sty}} +\item |'| (with |activeacute|) had the original value when writing to an + auxiliary file, and things like an infinite loop could happen. It + worked incorrectly with |^| (if activated) and also if deactivated. -When used with the options \Lopt{ngerman} or \Lopt{naustrian}, -\babel{} will provide all features of the package \pkg{ngerman}. -There is however one exception: The commands for special hyphenation -of double consonants (|"ff| etc.) and ck (|"ck|), which are no longer -required with the new German orthography, are undefined. With the -\pkg{ngerman} package, however, these commands will generate -appropriate warning messages only. +\item Active chars where not reset at the end of language options, and + that lead to incompatibilities between languages. -\subsection{Compatibility with the \pkg{french} package} +\item |\textormath| raised and error with a conditional. -It has been reported to me that the package \pkg{french} by Bernard -Gaulle (\texttt{gaulle@idris.fr}) works together with \babel. On the -other hand, it seems \emph{not} to work well together with a lot of -other packages. Therefore I have decided to no longer load -\file{french.ldf} by default. Instead, when you want to use the -package by Bernard Gaulle, you will have to request it specifically, -by passing either \Lopt{frenchle} or \Lopt{frenchpro} as an option to -\babel. +\item |\aliasshorthand| didn't work (or only in a few and very specific + cases). -\subsection{Changes in \babel\ version 3.9} +\item |\l@english| was defined incorrectly (using |\let| instead of + |\chardef|). -Most of changes in version 3.9 are related to bugs, either to fix them -(there were lots), or to provide some alternatives. Even new features -like |\babelhyphen| are intended to solve a certain problem (in this -case, the lacking of a uniform syntax and behaviour for shorthands -across languages). These changes are described in this manual in the -correspondin place. +\item |ldf| files not bundled with babel were not recognized when + called as global options. +\end{itemize} \subsection{Changes in \babel\ version 3.7} @@ -2304,7 +2366,7 @@ have occurred: an active character. The advantage is that |'{}a| works as expected for languages that have the |'| active. \item Support for typesetting french texts is much enhanced; the - file \file{francais.ldf} is now replaced by \file{frenchb.ldf} + file \file{francais.ldf} is now replaced by \file{french.ldf} which is maintained by Daniel Flipo. \item Support for typesetting the russian language is again available. The language definition file was originally @@ -2438,7 +2500,6 @@ help from Bernd Raichle, for which I am grateful. \end{thebibliography} \end{document} %</filedriver> -%\fi % % \changes{babel~3.9a}{2012/05/16}{Now using \cs{ldf@quit} for the test} % @@ -2479,6 +2540,13 @@ help from Bernd Raichle, for which I am grateful. % % \part{The code} % +% \babel{} is being developed incrementtally, which means parts of the +% code are under development and therefore incomplete. Only documented +% features are considered complete. In other words, use \babel{} only +% as documented (except, of course, if you want to explore and test +% them -- you can post suggestions about multilingual issues to +% |kadingira@tug.org| on |http://tug.org/mailman/listinfo/kadingira|). +% % \section{Identification and loading of required files} % % \textit{Code documentation is still under revision.} @@ -2506,8 +2574,8 @@ help from Bernd Raichle, for which I am grateful. % little bit of literate programming. % % \begin{macrocode} -%<<version=3.10>> -%<<date=2017/05/19>> +%<<version=3.11>> +%<<date=2017/06/26>> % \end{macrocode} % % \section{Tools} @@ -2763,20 +2831,7 @@ help from Bernd Raichle, for which I am grateful. %<</Make sure ProvidesFile is defined>> % \end{macrocode} % -% The following code is used in \file{babel.sty} and \file{babel.def}, -% and makes sure the current version of |switch.ldf| is used, if -% different from that in the format. -% -% \begin{macrocode} -%<<*Load switch if newer>> -\def\bbl@tempa{<@version@>}% -\ifx\bbl@version\bbl@tempa\else - \input switch.def\relax -\fi -%<</Load switch if newer>> -% \end{macrocode} -% -% The following code is also used in \file{babel.sty} and +% The following code is used in \file{babel.sty} and % \file{babel.def}, and loads (only once) the data in |language.dat|. % % \begin{macrocode} @@ -2898,7 +2953,7 @@ help from Bernd Raichle, for which I am grateful. % \changes{babel~3.9a}{2013/01/14}{Added the debug option} % \changes{babel~3.9a}{2013/02/05}{Added \cs{bbl@add}} % -% \section{The Package File (\LaTeX)} +% \section{The Package File (\LaTeX, \texttt{babel.sty})} % % In order to make use of the features of \LaTeXe, the \babel\ % system contains a package file, \file{babel.sty}. This file is @@ -2937,10 +2992,9 @@ help from Bernd Raichle, for which I am grateful. \NeedsTeXFormat{LaTeX2e}[2005/12/01] \ProvidesPackage{babel}[<@date@> <@version@> The Babel package] \@ifpackagewith{babel}{debug} - {\let\bbl@debug\@firstofone - \input switch.def\relax} - {\let\bbl@debug\@gobble - <@Load switch if newer@>} + {\let\bbl@debug\@firstofone} + {\let\bbl@debug\@gobble} +\input switch.def\relax <@Load patterns in luatex@> <@Basic macros@> \def\AfterBabelLanguage#1{% @@ -3412,7 +3466,7 @@ help from Bernd Raichle, for which I am grateful. %</package> % \end{macrocode} % -% \section{The kernel of Babel (common)} +% \section{The kernel of Babel (\texttt{babel.def}, common)} % % The kernel of the \babel\ system is stored in either % \file{hyphen.cfg} or \file{switch.def} and \file{babel.def}. The @@ -3469,7 +3523,7 @@ help from Bernd Raichle, for which I am grateful. \let\bbl@afterlang\relax \let\bbl@language@opts\@empty \fi -<@Load switch if newer@> +\input switch.def\relax \ifx\bbl@languages\@undefined \ifx\directlua\@undefined \openin1 = language.def @@ -3653,7 +3707,7 @@ help from Bernd Raichle, for which I am grateful. everylanguage=1,loadkernel=1,loadpatterns=1,loadexceptions=1,% adddialect=2,patterns=2,defaultcommands=0,encodedcommands=2,write=0,% beforeextras=0,afterextras=0,stopcommands=0,stringprocess=0,% - hyphenation=2,initiateactive=3,afterreset=0} + hyphenation=2,initiateactive=3,afterreset=0,foreign=0,foreign*=0} % \end{macrocode} % % \begin{macro}{\babelensure} @@ -6111,7 +6165,7 @@ help from Bernd Raichle, for which I am grateful. % % \begin{macro}{\lower@umlaut} % The command |\lower@umlaut| is used to position the |\"| closer -% the the letter. +% to the letter. % % We want the umlaut character lowered, nearer to the letter. To do % this we need an extra \meta{dimen} register. @@ -6205,7 +6259,7 @@ help from Bernd Raichle, for which I am grateful. \fi % \end{macrocode} % -% \section{The kernel of Babel (only \LaTeX)} +% \section{The kernel of Babel (\texttt{babel.def}, only \LaTeX)} % % \subsection{The redefinition of the style commands} % @@ -6231,7 +6285,7 @@ help from Bernd Raichle, for which I am grateful. \fi} % \end{macrocode} % -% \section{Creating languages} +% \subsection{Creating languages} % % |\babelprovide| is a general purpose tool for creating % languages. Currently it just creates the language infrastructure, @@ -6257,9 +6311,8 @@ help from Bernd Raichle, for which I am grateful. {If you want to modify `#2' you must tell how in\\% the optional argument. Currently there are two\\% options: captions=lang-tag, hyphenrules=lang-list}% - {Use this macro as documented} -}% - {\bbl@provide@renew{#2}}}% + {Use this macro as documented}}% + {\bbl@provide@renew{#2}}}% \babelensure{#2}% \let\languagename\bbl@savelangname} % \end{macrocode} @@ -6306,7 +6359,7 @@ help from Bernd Raichle, for which I am grateful. \fi} % \end{macrocode} % -% The |hyphenrules| options is handled with an auxiliary macro. +% The |hyphenrules| option is handled with an auxiliary macro. % % \begin{macrocode} \def\bbl@provide@hyphens#1{% @@ -6382,7 +6435,7 @@ help from Bernd Raichle, for which I am grateful. % Key-value pairs are treated differently depending on the section in % the |ini| file. The following macros are the readers for % |captions| (with Unicode) or |captions.licr| (for 8-bit) and for -% |identification|. But first, an auxiliary macros. +% |identification|. But first, an auxiliary macro. % % \begin{macrocode} \def\bbl@exportkey#1#2#3#4{% 1:bbl id,2:ini name,3:ini key,4:ini val @@ -6417,6 +6470,41 @@ help from Bernd Raichle, for which I am grateful. \bbl@exportkey{sotf}{script.tag.opentype}{#1}{#2}} % \end{macrocode} % +% Dates will require some macros for the basic formatting. They may be +% redefined by language, so ``semi-public'' names (camel case) are +% used. +% +% \begin{macrocode} +\newcommand\BabelDateSpace{\nobreakspace{}} +\newcommand\BabelDateDot{.\@} +\newcommand\BabelDated[1]{{\number#1}} +\newcommand\BabelDatedd[1]{{\ifnum#1<10 0\fi\number#1}} +\newcommand\BabelDateM[1]{{\number#1}} +\newcommand\BabelDateMM[1]{{\ifnum#1<10 0\fi\number#1}} +\newcommand\BabelDateMMMM[1]{{% + \csname month\romannumeral\month name\endcsname}}% +\newcommand\BabelDatey[1]{{\number#1}}% +\newcommand\BabelDateyy[1]{{% + \ifnum#1<10 0\number#1 % + \else\ifnum#1<100 \number#1 % + \else\ifnum#1<1000 \expandafter\@gobble\number#1 % + \else\ifnum#1<10000 \expandafter\@gobbletwo\number#1 % + \else\ERRORd % TODO - real error + \fi\fi\fi\fi}} +\newcommand\BabelDateyyyy[1]{{\number#1}} +\def\bbl@TG@@date{% + \bbl@replace\bbl@toreplace{[ ]}{\BabelDateSpace{}}% + \bbl@replace\bbl@toreplace{[.]}{\BabelDateDot{}}% + \bbl@replace\bbl@toreplace{[d]}{\BabelDated{####3}}% + \bbl@replace\bbl@toreplace{[dd]}{\BabelDatedd{####3}}% + \bbl@replace\bbl@toreplace{[M]}{\BabelDateM{####2}}% + \bbl@replace\bbl@toreplace{[MM]}{\BabelDateMM{####2}}% + \bbl@replace\bbl@toreplace{[MMMM]}{\BabelDateMMMM{####2}}% + \bbl@replace\bbl@toreplace{[y]}{\BabelDatey{####1}}% + \bbl@replace\bbl@toreplace{[yy]}{\BabelDateyy{####1}}% + \bbl@replace\bbl@toreplace{[yyyy]}{\BabelDateyyyy{####1}}} +% \end{macrocode} +% % \subsection{Cross referencing macros} % % The \LaTeX\ book states: @@ -6846,7 +6934,7 @@ help from Bernd Raichle, for which I am grateful. % Sometimes a document writer wants to create a special effect % depending on the page a certain fragment of text appears on. This % can be achieved by the following piece of code: -% \begin{verbatim} +%\begin{verbatim} % \ifthenelse{\isodd{\pageref{some:label}}} % {code for odd pages} % {code for even pages} @@ -7097,7 +7185,7 @@ help from Bernd Raichle, for which I am grateful. % % \changes{babel~3.9i}{2014/02/14}{Macro added, to replace % \cs{textlatin} and friends} -% \changes{babel~3.9j}{2014/03/17}{Moved mispaced code - it should be +% \changes{babel~3.9j}{2014/03/17}{Moved misplaced code - it should be % executed only with LaTeX} % % \begin{macrocode} @@ -7214,6 +7302,142 @@ help from Bernd Raichle, for which I am grateful. % \end{macrocode} % % \end{macro} +% +% \subsection{Basic bidi support} +% +% \textbf{Work in progress.} This code is currently placed here for +% practical reasons. +% +% \begin{itemize} +% \item pdftex provides a minimal support for bidi text, and it +% must be done by hand. Vertical typesetting is not possible. +% \item XeTeX is somewhat better, thanks to the font +% engine. However, very little is done at the paragraph level. +% \item LuaTeX could provide the most complete solution, as we can +% manipulate almost freely the node list, the generated lines, +% and so on, but unlike XeTeX bidi text does not work out of the +% box and some development is necessary. It also provides tools +% to properly set left-to-right and right-to-left page +% layouts. As LuaTeX-ja shows, vertical typesetting is posible. +% \end{itemize} +% +% \begin{macrocode} +\def\bbl@trtscripts{% + ,Arabic,Imperial Aramaic,Avestan,Cypriot,Hatran,Hebrew,% + Old Hungarian,Old Hungarian,Lydian,Mandaean,Manichaean,% + Manichaean,Meroitic Cursive,Meroitic,Old North Arabian,% + Nabataean,N'Ko,Orkhon,Palmyrene,Inscriptional Pahlavi,% + Psalter Pahlavi,Phoenician,Inscriptional Parthian,Samaritan,% + Old South Arabian,Syriac,Thaana,}% +\def\bbl@ensuredir{% + \@expandtwoargs + \in@{\csname bbl@script@\languagename\endcsname}{\bbl@trtscripts}% + \ifin@ + \bbl@setdirs\@ne + \else + \bbl@setdirs\z@ + \fi} +\def\bbl@setdirs#1{% TODO - math + \ifcase\bbl@select@type % TODO - strictly, not the right test + \bbl@pagedir{#1}% + \bbl@bodydir{#1}% + \bbl@pardir{#1}% + \fi + \bbl@textdir{#1}} +\ifcase\bbl@engine +\or + \AddBabelHook{babel-bidi}{afterextras}{\bbl@ensuredir} + \DisableBabelHook{babel-bidi} + \def\bbl@getluadir#1{% + \directlua{ + if tex.#1dir == 'TLT' then + tex.sprint('0') + elseif tex.#1dir == 'TRT' then + tex.sprint('1') + end}} + \def\bbl@setdir#1#2#3{% 1=text/par.. 2=\textdir.. 3=0 lr/1 rl + \ifcase#3\relax + \ifcase\bbl@getluadir{#1}\relax\else + #2 TLT\relax + \fi + \else + \ifcase\bbl@getluadir{#1}\relax + #2 TRT\relax + \fi + \fi} + \def\bbl@textdir{\bbl@setdir{text}\textdir}% TODO - ?\linedir + \def\bbl@pardir{\bbl@setdir{par}\pardir} + \def\bbl@bodydir{\bbl@setdir{body}\bodydir} + \def\bbl@pagedir{\bbl@setdir{page}\pagedir} + \def\bbl@dirparastext{\pardir\the\textdir\relax}% %%%% +\or + \AddBabelHook{babel-bidi}{afterextras}{\bbl@ensuredir} + \DisableBabelHook{babel-bidi} + \newcount\bbl@dirlevel + \chardef\bbl@thetextdir\z@ + \chardef\bbl@thepardir\z@ + \def\bbl@textdir#1{% + \ifcase#1\relax + \chardef\bbl@thetextdir\z@ + \bbl@textdir@i\beginL\endL + \else + \chardef\bbl@thetextdir\@ne + \bbl@textdir@i\beginR\endR + \fi} + \def\bbl@textdir@i#1#2{% + \ifhmode + \ifnum\currentgrouplevel>\z@ + \ifnum\currentgrouplevel=\bbl@dirlevel + \bbl@error{Multiple bidi settings inside a group}% + {I'll insert a new group, but expect wrong results.}% + \bgroup\aftergroup#2\aftergroup\egroup + \else + \ifcase\currentgrouptype\or % 0 bottom + \aftergroup#2% 1 simple {} + \or + \bgroup\aftergroup#2\aftergroup\egroup % 2 hbox + \or + \bgroup\aftergroup#2\aftergroup\egroup % 3 adj hbox + \or\or\or % vbox vtop align + \or + \bgroup\aftergroup#2\aftergroup\egroup % 7 noalign + \or\or\or\or\or\or % output math disc insert vcent mathchoice + \or + \aftergroup#2% 14 \begingroup + \else + \bgroup\aftergroup#2\aftergroup\egroup % 15 adj + \fi + \fi + \bbl@dirlevel\currentgrouplevel + \fi + #1% + \fi} + \def\bbl@pardir#1{\chardef\bbl@thepardir#1\relax} + \let\bbl@bodydir\@gobble + \let\bbl@pagedir\@gobble + \def\bbl@dirparastext{\chardef\bbl@thepardir\bbl@thetextdir} +% \end{macrocode} +% +% The following command is executed only if there is a right-to-left +% script (once). It activates the |\everypar| hack for \xetex, to +% properly handle the par direction. Note text and par dirs are +% decoupled. +% +% \begin{macrocode} + \def\bbl@xebidipar{% + \let\bbl@xebidipar\relax + \TeXXeTstate\@ne + \def\bbl@xeeverypar{% + \ifcase\bbl@thepardir\else + {\setbox\z@\lastbox\beginR\box\z@}% + \fi + \ifcase\bbl@thetextdir\else\beginR\fi}% + \let\bbl@severypar\everypar + \newtoks\everypar + \everypar=\bbl@severypar + \bbl@severypar{\bbl@xeeverypar\the\everypar}} +\fi +% \end{macrocode} % % \subsection{Local Language Configuration} % @@ -7261,7 +7485,7 @@ help from Bernd Raichle, for which I am grateful. % \end{macrocode}% % \end{macro} % -% \section{Multiple languages} +% \section{Multiple languages (\texttt{switch.def)}} % % Plain \TeX\ version~3.0 provides the primitive |\language| that % is used to store the current language. When used with a pre-3.0 @@ -7655,11 +7879,8 @@ help from Bernd Raichle, for which I am grateful. % left-to-right and right-to-left typesetting you have to use this % environment in order to let things work as you expect them to. % -% The first thing this environment does is store the name of the -% language in |\languagename|; it then calls -% \verb*=\selectlanguage = to switch on everything that is needed for -% this language The |\ignorespaces| command is necessary to hide -% the environment when it is entered in horizontal mode. +% The |\ignorespaces| command is necessary to hide the environment +% when it is entered in horizontal mode. % % \changes{babel~3.9a}{2012/07/31}{Removed \cs{originalTeX}} % @@ -7715,6 +7936,32 @@ help from Bernd Raichle, for which I am grateful. % any |\global| changes. The coding is very similar to part of % |\selectlanguage|. % +% |\bbl@beforeforeign| is a trick to fix a bug in bidi +% texts. |\foreignlanguage| is supposed to be a `text' command, and +% therefore it must emit a |\leavevmode|, but it does not, and +% therefore the indent is placed on the opposite margin. For +% backward compatibility, however, it is done only if a +% right-to-left script is requested; otherwise, it is no-op. +% +% (3.11) |\foreignlanguage*| is a temporary, experimental macro for +% a few lines with a different script direction, while preserving +% the paragraph format (thank the braces around |\par|, things like +% |\hangindent| are not reset). Do not use it in production, +% because its semantics and its syntax may change (and very likely +% will, or even it could be removed altogether). Currently it +% enters in vmode and then selects the language (which in turn sets the +% paragraph direction). +% +% (3.11) Also experimental are the hook |foreign| and +% |foreign*|. With them you can redefine |\BabelText| which by +% default does nothing. Its +% behaviour is not well defined yet. So, use it in +% horizontal mode only if you do not want surprises. +% +% In other words, at the beginning of a paragraph |\foreignlanguage| +% enters into hmode with the surrounding lang, and with +% |\foreignlanguage*| with the new lang. +% % \changes{babel~3.9a}{2012/07/30}{Removed unnecesary \cs{noextras} % just before closing the group} % \changes{babel~3.9a}{2012/07/31}{Moved \cs{originalTeX} to @@ -7722,15 +7969,33 @@ help from Bernd Raichle, for which I am grateful. % \texttt{otherlanguage*}} % \changes{babel~3.9a}{2012/12/24}{\cs{foreignlanguage} defined % similarly to \cs{selectlanguage}, protecting the whole macro} +% \changes{babel~3.11}{2017/03/04}{\cs{foreignlanguage*}, +% \cs{bbl@beforeforeign} and hooks} % % \begin{macrocode} +\let\bbl@beforeforeign\@empty \edef\foreignlanguage{% \noexpand\protect \expandafter\noexpand\csname foreignlanguage \endcsname} -\expandafter\def\csname foreignlanguage \endcsname#1#2{% +\expandafter\def\csname foreignlanguage \endcsname{% + \@ifstar\bbl@foreign@s\bbl@foreign@x} +\def\bbl@foreign@x#1#2{% \begingroup + \let\BabelText\@firstofone + \bbl@beforeforeign \foreign@language{#1}% - #2% + \bbl@usehooks{foreign}{}% + \BabelText{#2}% Now in horizontal mode! + \endgroup} +\def\bbl@foreign@s#1#2{% TODO - \shapemode, \@setpar, ?\@@par + \begingroup + {\par}% + \let\BabelText\@firstofone + \foreign@language{#1}% + \bbl@usehooks{foreign*}{}% + \bbl@dirparastext + \BabelText{#2}% Still in vertical mode! + {\par}% \endgroup} % \end{macrocode} % @@ -7958,7 +8223,7 @@ help from Bernd Raichle, for which I am grateful. % \changes{babel~3.9s}{2017/04/13}{Reserved macro names for `locale'} % % \begin{macrocode} -\newcommand\setlocale{% +\providecommand\setlocale{% \bbl@error {Not yet available}% {Find an armchair, sit down and wait}} @@ -8474,7 +8739,7 @@ help from Bernd Raichle, for which I am grateful. % % Also remove some macros from memory and raise an error % if |\toks@| is not empty. Finally load \file{switch.def}, but the -% letter is not required and the line inputting it may be commented +% latter is not required and the line inputting it may be commented % out. % % \changes{babel~3.9a}{2012/12/11}{Raise error if there are synonyms @@ -8498,6 +8763,508 @@ help from Bernd Raichle, for which I am grateful. % Here the code for ini\TeX\ ends. % \end{macro} % +% \section{Tentative font handling with fontspec} +% +% A general solution is far from trivial: +% \begin{itemize} +% \item |\addfontfeature| only sets it for the current family and it's +% not very efficient, and +% \item |\defaultfontfeatures| requires to redefine the font (and the +% options are not ``orthogonal''). +% \end{itemize} +% +% \begin{macrocode} +%<<*Font selection>> +\newcommand\babelFSstore[2][]{% + \bbl@ifblank{#1}% + {\bbl@csarg\def{script@#2}{Latin}}% + {\bbl@csarg\def{script@#2}{#1}}% + \@expandtwoargs % TODO should go to the ini loaders + \in@{\csname bbl@script@#2\endcsname}{\bbl@trtscripts}% + \ifin@ + \let\bbl@beforeforeign\leavevmode + \EnableBabelHook{babel-bidi}% + \@nameuse{bbl@xebidipar}% + \fi + \bbl@foreach{#2}{% + \bbl@FSstore{##1}{rm}\rmdefault\bbl@save@rmdefault + \bbl@FSstore{##1}{sf}\sfdefault\bbl@save@sfdefault + \bbl@FSstore{##1}{tt}\ttdefault\bbl@save@ttdefault}} +\def\bbl@FSstore#1#2#3#4{% + \bbl@csarg\edef{#2default#1}{#3}% + \expandafter\addto\csname extras#1\endcsname{% + \let#4#3% + \ifx#3\f@family + \edef#3{\csname bbl@#2default#1\endcsname}% + \fontfamily{#3}\selectfont + \else + \edef#3{\csname bbl@#2default#1\endcsname}% + \fi}% + \expandafter\addto\csname noextras#1\endcsname{% + \ifx#3\f@family + \fontfamily{#4}\selectfont + \fi + \let#3#4}} +\let\bbl@langfeatures\@empty +\def\babelFSfeatures{% make sure \fontspec is redefined once + \let\bbl@ori@fontspec\fontspec + \renewcommand\fontspec[1][]{% + \bbl@ori@fontspec[\bbl@langfeatures##1]} + \let\babelFSfeatures\bbl@FSfeatures + \babelFSfeatures} +\def\bbl@FSfeatures#1#2{% + \expandafter\addto\csname extras#1\endcsname{% + \babel@save\bbl@langfeatures + \edef\bbl@langfeatures{#2,}}} +%<</Font selection>> +% \end{macrocode} +% +% +% \section{Hooks for XeTeX and LuaTeX} +% +% \subsection{XeTeX} +% +% Unfortunately, the current encoding cannot be retrieved and +% therefore it is reset always to |utf8|, which seems a sensible +% default. +% +% \LaTeX{} sets many ``codes'' just before loading +% \verb|hyphen.cfg|. That is not a problem in luatex, but in xetex +% they must be reset to the proper value. Most of the work is done in +% \textsf{xe(la)tex.ini}, so here we just ``undo'' some of the +% changes done by \LaTeX. Anyway, for consistency Lua\TeX{} also +% resets the catcodes. +% +% \changes{bbunicode~1.0c}{2014/03/10}{Reset ``codes'' set by \cs{LaTeX} +% to what xetex expects. Used also in luatex.} +% \changes{bbunicode~1.0f}{2015/12/06}{This block was assigned to +% xetex, even in luatex. Fixed here and below.} +% +% \begin{macrocode} +%<<*Restore Unicode catcodes before loading patterns>> + \begingroup + % Reset chars "80-"C0 to category "other", no case mapping: + \catcode`\@=11 \count@=128 + \loop\ifnum\count@<192 + \global\uccode\count@=0 \global\lccode\count@=0 + \global\catcode\count@=12 \global\sfcode\count@=1000 + \advance\count@ by 1 \repeat + % Other: + \def\O ##1 {% + \global\uccode"##1=0 \global\lccode"##1=0 + \global\catcode"##1=12 \global\sfcode"##1=1000 }% + % Letter: + \def\L ##1 ##2 ##3 {\global\catcode"##1=11 + \global\uccode"##1="##2 + \global\lccode"##1="##3 + % Uppercase letters have sfcode=999: + \ifnum"##1="##3 \else \global\sfcode"##1=999 \fi }% + % Letter without case mappings: + \def\l ##1 {\L ##1 ##1 ##1 }% + \l 00AA + \L 00B5 039C 00B5 + \l 00BA + \O 00D7 + \l 00DF + \O 00F7 + \L 00FF 0178 00FF + \endgroup + \input #1\relax +%<</Restore Unicode catcodes before loading patterns>> +% \end{macrocode} +% +% Now, the code. +% +% \begin{macrocode} +%<*xetex> +\def\BabelStringsDefault{unicode} +\let\xebbl@stop\relax +\AddBabelHook{xetex}{encodedcommands}{% + \def\bbl@tempa{#1}% + \ifx\bbl@tempa\@empty + \XeTeXinputencoding"bytes"% + \else + \XeTeXinputencoding"#1"% + \fi + \def\xebbl@stop{\XeTeXinputencoding"utf8"}} +\AddBabelHook{xetex}{stopcommands}{% + \xebbl@stop + \let\xebbl@stop\relax} +\AddBabelHook{xetex}{loadkernel}{% +<@Restore Unicode catcodes before loading patterns@>} +<@Font selection@> +%</xetex> +% \end{macrocode} +% +% \subsection{LuaTeX} +% +% The new loader for luatex is based solely on |language.dat|, which +% is read on the fly. The code shouldn't be executed when the format +% is build, so we check if |\AddBabelHook| is defined. Then comes a +% modified version of the loader in |hyphen.cfg| (without the +% hyphenmins stuff, which is under the direct control of \babel). +% +% The names |\l@<language>| are defined and take some value from the +% beginning because all \texttt{ldf} files assume this for the +% corresponding language to be considered valid, but patterns are not +% loaded (except the first one). This is done later, when the language +% is first selected (which usually means when the \texttt{ldf} +% finishes). If a language has been loaded, |\bbl@hyphendata@<num>| +% exists (with the names of the files read). +% +% The default setup preloads the first language into the format. This +% is intended mainly for `english', so that it's available without +% further intervention from the user. To avoid duplicating it, the +% following rule applies: if the ``0th'' language and the first +% language in |language.dat| have the same name then just ignore the +% latter. If there are new synonymous, the are added, but note if the +% language patterns have not been preloaded they won't at run time. +% +% Other preloaded languages could be read twice, if they has been +% preloaded into the format. This is not optimal, but it shouldn't +% happen very often -- with \luatex{} patterns are best loaded when +% the document is typeset, and the ``0th'' language is preloaded just +% for backwards compatibility. +% +% As of 1.1b, lua(e)tex is taken into account. Formerly, loading of +% patterns on the fly didn't work in this format, but with the new +% loader it does. Unfortunately, the format is not based on \babel, +% and data could be duplicated, because languages are reassigned above +% those in the format (nothing serious, anyway). Note even with this +% format |language.dat| is used (under the principle of a single +% source), instead of |language.def|. +% +% Of course, there is room for improvements, like tools to read and +% reassign languages, which would require modifying the language list, +% and better error handling. +% +% We need catcode tables, but no format (targeted by \babel) provide a +% command to allocate them (although there are packages like +% \textsf{ctablestack}). For the moment, a dangerous approach is used +% -- just allocate a high random number and cross the fingers. To +% complicate things, \textsf{etex.sty} changes the way languages are +% allocated. +% +% \changes{bbunicode~1.0b}{2013/04/22}{luatex-hyphen is loaded with +% require. Changes supplied by \'{E}lie Roux.} +% \changes{bbunicode~1.0c}{2014/03/10}{Defined hook for +% `initiateactive', to fetch the next token and continue only if +% letter or other.} +% \changes{bbunicode~1.0d}{2014/03/21}{Removed the `misfeature' for +% `initiateactive'.} +% \changes{bbunicode~1.0e}{2015/05/10}{Use brackets instead of +% \cs{luaescapestring}.} +% \changes{bbunicode~1.0e}{2015/07/26}{Added function addpattern +% and modified the patterns hook.} +% \changes{bbunicode~1.1a}{2016/01/26}{New hyphenation loader for +% luatex.} +% \changes{bbunicode~1.1b}{2016/02/05}{Also lua(e)tex.} +% \changes{bbunicode~1.1c}{2016/02/08}{Base reading of patterns on +% number, not in name.} +% \changes{bbunicode~1.1c}{2016/02/08}{Some hacks for polyglossia. To +% be improved.} +% \changes{bbunicode~1.1c}{2016/02/23}{Thoroughly revised.} +% \changes{bbunicode~1.1d}{2016/4/22}{Lua: Fixed a line break at +% \cs{foreignlanguage} with unloaded patterns. Added +% \cs{babelcatcodetablenum}, just in case.} +% +% \begin{macrocode} +%<*luatex> +\ifx\AddBabelHook\@undefined +\begingroup + \toks@{} + \count@\z@ % 0=start, 1=0th, 2=normal + \def\bbl@process@line#1#2 #3 #4 {% + \ifx=#1% + \bbl@process@synonym{#2}% + \else + \bbl@process@language{#1#2}{#3}{#4}% + \fi + \ignorespaces} + \def\bbl@manylang{% + \ifnum\bbl@last>\@ne + \bbl@info{Non-standard hyphenation setup}% + \fi + \let\bbl@manylang\relax} + \def\bbl@process@language#1#2#3{% + \ifcase\count@ + \@ifundefined{zth@#1}{\count@\tw@}{\count@\@ne}% + \or + \count@\tw@ + \fi + \ifnum\count@=\tw@ + \expandafter\addlanguage\csname l@#1\endcsname + \language\allocationnumber + \chardef\bbl@last\allocationnumber + \bbl@manylang + \let\bbl@elt\relax + \xdef\bbl@languages{% + \bbl@languages\bbl@elt{#1}{\the\language}{#2}{#3}}% + \fi + \the\toks@ + \toks@{}} + \def\bbl@process@synonym@aux#1#2{% + \global\expandafter\chardef\csname l@#1\endcsname#2\relax + \let\bbl@elt\relax + \xdef\bbl@languages{% + \bbl@languages\bbl@elt{#1}{#2}{}{}}}% + \def\bbl@process@synonym#1{% + \ifcase\count@ + \toks@\expandafter{\the\toks@\relax\bbl@process@synonym{#1}}% + \or + \@ifundefined{zth@#1}{\bbl@process@synonym@aux{#1}{0}}{}% + \else + \bbl@process@synonym@aux{#1}{\the\bbl@last}% + \fi} + \ifx\bbl@languages\@undefined % Just a (sensible?) guess + \chardef\l@english\z@ + \chardef\l@USenglish\z@ + \chardef\bbl@last\z@ + \global\@namedef{bbl@hyphendata@0}{{hyphen.tex}{}} + \gdef\bbl@languages{% + \bbl@elt{english}{0}{hyphen.tex}{}% + \bbl@elt{USenglish}{0}{}{}} + \else + \global\let\bbl@languages@format\bbl@languages + \def\bbl@elt#1#2#3#4{% Remove all except language 0 + \ifnum#2>\z@\else + \noexpand\bbl@elt{#1}{#2}{#3}{#4}% + \fi}% + \xdef\bbl@languages{\bbl@languages}% + \fi + \def\bbl@elt#1#2#3#4{\@namedef{zth@#1}{}} % Define flags + \bbl@languages + \openin1=language.dat + \ifeof1 + \bbl@warning{I couldn't find language.dat. No additional\\% + patterns loaded. Reported}% + \else + \loop + \endlinechar\m@ne + \read1 to \bbl@line + \endlinechar`\^^M + \if T\ifeof1F\fi T\relax + \ifx\bbl@line\@empty\else + \edef\bbl@line{\bbl@line\space\space\space}% + \expandafter\bbl@process@line\bbl@line\relax + \fi + \repeat + \fi +\endgroup +\def\bbl@get@enc#1:#2:#3\@@@{\def\bbl@hyph@enc{#2}} +\ifx\babelcatcodetablenum\@undefined + \def\babelcatcodetablenum{5211} +\fi +\def\bbl@luapatterns#1#2{% + \bbl@get@enc#1::\@@@ + \setbox\z@\hbox\bgroup + \begingroup + \ifx\catcodetable\@undefined + \let\savecatcodetable\luatexsavecatcodetable + \let\initcatcodetable\luatexinitcatcodetable + \let\catcodetable\luatexcatcodetable + \fi + \savecatcodetable\babelcatcodetablenum\relax + \initcatcodetable\numexpr\babelcatcodetablenum+1\relax + \catcodetable\numexpr\babelcatcodetablenum+1\relax + \catcode`\#=6 \catcode`\$=3 \catcode`\&=4 \catcode`\^=7 + \catcode`\_=8 \catcode`\{=1 \catcode`\}=2 \catcode`\~=13 + \catcode`\@=11 \catcode`\^^I=10 \catcode`\^^J=12 + \catcode`\<=12 \catcode`\>=12 \catcode`\*=12 \catcode`\.=12 + \catcode`\-=12 \catcode`\/=12 \catcode`\[=12 \catcode`\]=12 + \catcode`\`=12 \catcode`\'=12 \catcode`\"=12 + \input #1\relax + \catcodetable\babelcatcodetablenum\relax + \endgroup + \def\bbl@tempa{#2}% + \ifx\bbl@tempa\@empty\else + \input #2\relax + \fi + \egroup}% +\def\bbl@patterns@lua#1{% + \language=\expandafter\ifx\csname l@#1:\f@encoding\endcsname\relax + \csname l@#1\endcsname + \edef\bbl@tempa{#1}% + \else + \csname l@#1:\f@encoding\endcsname + \edef\bbl@tempa{#1:\f@encoding}% + \fi\relax + \@namedef{lu@texhyphen@loaded@\the\language}{}% Temp + \@ifundefined{bbl@hyphendata@\the\language}% + {\def\bbl@elt##1##2##3##4{% + \ifnum##2=\csname l@\bbl@tempa\endcsname % #2=spanish, dutch:OT1... + \def\bbl@tempb{##3}% + \ifx\bbl@tempb\@empty\else % if not a synonymous + \def\bbl@tempc{{##3}{##4}}% + \fi + \bbl@csarg\xdef{hyphendata@##2}{\bbl@tempc}% + \fi}% + \bbl@languages + \@ifundefined{bbl@hyphendata@\the\language}% + {\bbl@info{No hyphenation patterns were set for\\% + language ‘\bbl@tempa’. Reported}}% + {\expandafter\expandafter\expandafter\bbl@luapatterns + \csname bbl@hyphendata@\the\language\endcsname}}{}} +\endinput\fi +\begingroup +\catcode`\%=12 +\catcode`\'=12 +\catcode`\"=12 +\catcode`\:=12 +\directlua{ + Babel = {} + function Babel.bytes(line) + return line:gsub("(.)", + function (chr) return unicode.utf8.char(string.byte(chr)) end) + end + function Babel.begin_process_input() + if luatexbase and luatexbase.add_to_callback then + luatexbase.add_to_callback('process_input_buffer', + Babel.bytes,'Babel.bytes') + else + Babel.callback = callback.find('process_input_buffer') + callback.register('process_input_buffer',Babel.bytes) + end + end + function Babel.end_process_input () + if luatexbase and luatexbase.remove_from_callback then + luatexbase.remove_from_callback('process_input_buffer','Babel.bytes') + else + callback.register('process_input_buffer',Babel.callback) + end + end + function Babel.addpatterns(pp, lg) + local lg = lang.new(lg) + local pats = lang.patterns(lg) or '' + lang.clear_patterns(lg) + for p in pp:gmatch('[^%s]+') do + ss = '' + for i in string.utfcharacters(p:gsub('%d', '')) do + ss = ss .. '%d?' .. i + end + ss = ss:gsub('^%%d%?%.', '%%.') .. '%d?' + ss = ss:gsub('%.%%d%?$', '%%.') + pats, n = pats:gsub('%s' .. ss .. '%s', ' ' .. p .. ' ') + if n == 0 then + tex.sprint( + [[\string\csname\space bbl@info\endcsname{New pattern: ]] + .. p .. [[}]]) + pats = pats .. ' ' .. p + else + tex.sprint( + [[\string\csname\space bbl@info\endcsname{Renew pattern: ]] + .. p .. [[}]]) + end + end + lang.patterns(lg, pats) + end +} +\endgroup +\def\BabelStringsDefault{unicode} +\let\luabbl@stop\relax +\AddBabelHook{luatex}{encodedcommands}{% + \def\bbl@tempa{utf8}\def\bbl@tempb{#1}% + \ifx\bbl@tempa\bbl@tempb\else + \directlua{Babel.begin_process_input()}% + \def\luabbl@stop{% + \directlua{Babel.end_process_input()}}% + \fi}% +\AddBabelHook{luatex}{stopcommands}{% + \luabbl@stop + \let\luabbl@stop\relax} +\AddBabelHook{luatex}{patterns}{% + \@ifundefined{bbl@hyphendata@\the\language}% + {\def\bbl@elt##1##2##3##4{% + \ifnum##2=\csname l@#2\endcsname % #2=spanish, dutch:OT1... + \def\bbl@tempb{##3}% + \ifx\bbl@tempb\@empty\else % if not a synonymous + \def\bbl@tempc{{##3}{##4}}% + \fi + \bbl@csarg\xdef{hyphendata@##2}{\bbl@tempc}% + \fi}% + \bbl@languages + \@ifundefined{bbl@hyphendata@\the\language}% + {\bbl@info{No hyphenation patterns were set for\\% + language ‘#2’. Reported}}% + {\expandafter\expandafter\expandafter\bbl@luapatterns + \csname bbl@hyphendata@\the\language\endcsname}}{}% + \@ifundefined{bbl@patterns@}{}{% + \begingroup + \@expandtwoargs\in@{,\number\language,}{,\bbl@pttnlist}% + \ifin@\else + \ifx\bbl@patterns@\@empty\else + \directlua{ Babel.addpatterns( + [[\bbl@patterns@]], \number\language) }% + \fi + \@ifundefined{bbl@patterns@#1}% + \@empty + {\directlua{ Babel.addpatterns( + [[\space\csname bbl@patterns@#1\endcsname]], + \number\language) }}% + \xdef\bbl@pttnlist{\bbl@pttnlist\number\language,}% + \fi + \endgroup}} +\AddBabelHook{luatex}{everylanguage}{% + \def\process@language##1##2##3{% + \def\process@line####1####2 ####3 ####4 {}}} +\AddBabelHook{luatex}{loadpatterns}{% + \input #1\relax + \expandafter\gdef\csname bbl@hyphendata@\the\language\endcsname + {{#1}{}}} +\AddBabelHook{luatex}{loadexceptions}{% + \input #1\relax + \def\bbl@tempb##1##2{{##1}{#1}}% + \expandafter\xdef\csname bbl@hyphendata@\the\language\endcsname + {\expandafter\expandafter\expandafter\bbl@tempb + \csname bbl@hyphendata@\the\language\endcsname}} +% \end{macrocode} +% +% \begin{macro}{\babelpatterns} +% +% This macro adds patterns. Two macros are used to store them: +% |\bbl@patterns@| for the global ones and |\bbl@patterns@<lang>| +% for language ones. We make sure there is a space between words +% when multiple commands are used. +% \changes{bbunicode~1.0e}{2015/07/26}{Macro \cs{babelpatterns} added} +% +% \begin{macrocode} +\@onlypreamble\babelpatterns +\AtEndOfPackage{% + \newcommand\babelpatterns[2][\@empty]{% + \ifx\bbl@patterns@\relax + \let\bbl@patterns@\@empty + \fi + \ifx\bbl@pttnlist\@empty\else + \bbl@warning{% + You must not intermingle \string\selectlanguage\space and\\% + \string\babelpatterns\space or some patterns will not\\% + be taken into account. Reported}% + \fi + \ifx\@empty#1% + \protected@edef\bbl@patterns@{\bbl@patterns@\space#2}% + \else + \edef\bbl@tempb{\zap@space#1 \@empty}% + \bbl@for\bbl@tempa\bbl@tempb{% + \bbl@fixname\bbl@tempa + \bbl@iflanguage\bbl@tempa{% + \bbl@csarg\protected@edef{patterns@\bbl@tempa}{% + \@ifundefined{bbl@patterns@\bbl@tempa}% + \@empty + {\csname bbl@patterns@\bbl@tempa\endcsname\space}% + #2}}}% + \fi}} +% \end{macrocode} +% \end{macro} +% +% Common stuff. +% +% \begin{macrocode} +\AddBabelHook{luatex}{loadkernel}{% +<@Restore Unicode catcodes before loading patterns@>} +<@Font selection@> +%</luatex> +% \end{macrocode} +% % \section{The `nil' language} % % This `language' does nothing, except setting the hyphenation patterns to @@ -8560,7 +9327,7 @@ help from Bernd Raichle, for which I am grateful. % \end{macrocode} % % -% \section{Support for Plain \TeX} +% \section{Support for Plain \TeX\ (\texttt{plain.def})} % % \subsection{Not renaming \file{hyphen.tex}} % As Don Knuth has declared that the filename \file{hyphen.tex} may @@ -9180,22 +9947,6 @@ help from Bernd Raichle, for which I am grateful. % % \Finale % -%% -%% \CharacterTable -%% {Upper-case \A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z -%% Lower-case \a\b\c\d\e\f\g\h\i\j\k\l\m\n\o\p\q\r\s\t\u\v\w\x\y\z -%% Digits \0\1\2\3\4\5\6\7\8\9 -%% Exclamation \! Double quote \" Hash (number) \# -%% Dollar \$ Percent \% Ampersand \& -%% Acute accent \' Left paren \( Right paren \) -%% Asterisk \* Plus \+ Comma \, -%% Minus \- Point \. Solidus \/ -%% Colon \: Semicolon \; Less than \< -%% Equals \= Greater than \> Question mark \? -%% Commercial at \@ Left bracket \[ Backslash \\ -%% Right bracket \] Circumflex \^ Underscore \_ -%% Grave accent \` Left brace \{ Vertical bar \| -%% Right brace \} Tilde \~} \endinput % % Local Variables: |