%#!make ptex-guide-en.pdf \documentclass[a4paper,11pt]{article} \usepackage[textwidth=42zw,lines=40,truedimen,centering]{geometry} %%%%%%%%%%%%%%%% % additional packages \usepackage{amsmath} \usepackage{array}\usepackage[all]{xy} \SelectTips{cm}{} %\usepackage[dvipdfmx]{graphicx} \usepackage[T1]{fontenc} \usepackage{booktabs,enumitem,multicol} \usepackage[defaultsups]{newpxtext} \usepackage[zerostyle=c,straightquotes]{newtxtt} \usepackage{newpxmath} \usepackage[dvipdfmx,hyperfootnotes=false]{hyperref} \usepackage{pxjahyper} \usepackage{hologo} \usepackage{makeidx}\makeindex % common \usepackage{ptex-manual} \makeatletter \newlist{simplelist}{description}1 \setlist[simplelist]{% itemsep=0pt, listparindent=1zw, itemindent=10pt, font=\normalfont\mdseries, leftmargin=2zw, before=\advance\@listdepth\@ne, after=\advance\@listdepth\m@ne } \makeatother \def\code#1{\texttt{#1}} %%%%%%%%%%%%%%%% \makeatletter \setlist{leftmargin=2zw} \setlist[description]{labelwidth=2zw,labelindent=1zw,topsep=\medskipamount} \def\>{\ifhmode\hskip\xkanjiskip\fi} \def\tsp{_{\mbox{\fontsize\sf@size\z@\ttfamily \char32}}} \def\tpar{_{\mbox{\fontsize\sf@size\z@\ttfamily \string\par}}} \def\tign{_{\mbox{\fontsize\sf@size\z@\selectfont --}}} \usepackage{shortvrb} \MakeShortVerb*{|} %%%%%%%%%%%%%%%% % logos \def\epTeX{$\varepsilon$-\pTeX}\def\eTeX{$\varepsilon$-\TeX} \def\eupTeX{$\varepsilon$-\upTeX}\def\upTeX{u\pTeX} \def\pTeX{p\kern-.10em\TeX} \def\pLaTeX{p\LaTeX}\def\upLaTeX{u\pLaTeX} \def\pdfTeX{pdf\TeX} \def\OMEGA{$\Omega$} \def\TL{\TeX\ Live\ } \def\_{\leavevmode\vrule width .45em height -.2ex depth .3ex\relax} \frenchspacing \begin{document} \catcode`\<=13 \title{\emph{Guide of \pTeX\ for developers not interested in Japanese}} \author{Japanese \TeX\ Development Community\null \thanks{\url{https://texjp.org},\ e-mail: \texttt{issue(at)texjp.org}}} \date{version p\the\ptexversion.\the\ptexminorversion\ptexrevision, \today} \maketitle \pTeX\ and its variants, \upTeX, \epTeX\ and \eupTeX, are all \TeX\ engines with native Japanese support. Its output is always a DVI file, which can be processed by several DVI drivers with Japanese support including {\em dvips} and {\em dvipdfmx}. Formats based on \LaTeX\ is called \pLaTeX\ when running on \pTeX/\epTeX, and called \upLaTeX\ when running on \upTeX/\eupTeX. %%% Target readers of this document: 日本語機能でないパッケージ作者に絞る \section*{Purpose of this document} This document is written for developers of \TeX/\LaTeX, who aim to support \pTeX/\pLaTeX\ and its variants \upTeX/\upLaTeX. Knowledge of the followings are assumed: \begin{itemize} \item Basic knowledge of Western \TeX\ (Knuthian \TeX, \eTeX\ and \pdfTeX), \item ... and its programming conventions. \end{itemize} %%% 日本語(の組版・文字コード)の知識は要らない。その話はできる限り避ける Any knowledge of Japanese (characters, encodings, typesetting conventions etc.) is not assumed; some explanations are provided in this document when needed. We hope that this document helps authors of packages or classes to proceed with supporting \pTeX\ family smoothly. \begin{quotation} Note: This English edition (\jobname.pdf) is {\em not} meant to be a complete translation of Japanese edition (ptex-manual.pdf). For example, this document does not cover the following aspects of \pTeX: \begin{itemize} \item Typesetting conventions of Japanese characters \item Details of vertical writing \end{itemize} For beginners of writing Japanese texts, please refer to the Japanese edition. \end{quotation} \tableofcontents \newpage %%%%% \part{Brief introduction}% 概論 %%% pTeX とその仲間 \section{\pTeX\ and its variants} % [TODO] \pTeX, \upTeX, \epTeX, \eupTeX を簡単に紹介 %%% e-(u)pTeX の話がメイン There is no advantage to choose \pTeX/\upTeX\ over \epTeX/\eupTeX, so we focus mainly on \epTeX/\eupTeX. % [TODO] pLaTeX/upLaTeX も e- で走っていて,これで十分であると付け加える % [TODO] 既に pTeX/upTeX が暗に e-pTeX/e-upTeX を指す文献もあることに言及 %%% 欧文 TeX との互換性 \section{Compatibility with Western \TeX} %%% pTeX/upTeX は Knuthian TeX に対してほぼ上位互換 \pTeX/\upTeX\ are almost uppercompatible with Knuthian \TeX, however, they do not pass TRIP test. %%% 入力の8bitの扱いが異なる。フォントの8bitはそのまま In \pTeX/\upTeX, input handling is different from Knuthian \TeX; if a pair of two or more 8-bit codes matches Japanese character code, it is regarded as one Japanese character. There is no difference in handling 8-bit TFM font. %%% e-pTeX/e-upTeX は e-TeX に対してほぼ上位互換 \epTeX/\eupTeX\ are almost uppercompatible with \eTeX, however, input handling is similar to \pTeX/\upTeX. It does not pass e-TRIP test. %%% だけど e-TeX はもうなく,pdfTeX の DVI モードがあるだけ That said, please note that ``raw \eTeX'' is unavailable anymore in \TeX\ Live and derived distributions; they provide a command |etex| only as ``DVI mode of \pdfTeX.'' %%% e-pTeX/e-upTeX は pdfTeX の DVI モードに対して上位互換ではない We should note that \epTeX/\eupTeX\ are {\em not} uppercompatible with DVI mode of \pdfTeX, which will be discussed later in section \ref{dvi-pdftex}. %%% LaTeX ムニャムニャ \section{\LaTeX\ on \pTeX/\upTeX\ --- \pLaTeX/\upLaTeX} %%% pLaTeX と upLaTeX ムニャムニャ Formats based on \LaTeX\ is called \pLaTeX\ when running on \pTeX/\epTeX, and called \upLaTeX\ when running on \upTeX/\eupTeX. In recent versions (around 2011) of \TeX\ Live and its derivatives, the default engines of \pLaTeX\ and \upLaTeX\ are \epTeX\ and \eupTeX. That is, the command |platex| starts \epTeX\ (not \pTeX) with preloaded format |platex.fmt|. %%% カーネルが拡張されている In the kernel level (|platex.ltx| and |uplatex.ltx|), \pLaTeX\ and \upLaTeX\ adds some additional commands related to the followings: \begin{itemize} \item Selection of Japanese fonts \item Crop marks (called ``tombow'') for printings \item Adjustment for mixing horizontal and vertical texts \end{itemize} %%% author レベルでは LaTeX とほぼ互換,ただし例外あり For authors, \pLaTeX/\upLaTeX\ is almost uppercompatible with original \LaTeX, except for the followings: \begin{itemize} \item Order of float objects; in \pLaTeX/\upLaTeX, is placed above . That is, the complete order is $\rightarrow$ $\rightarrow$ $\rightarrow$ . % [TODO] 他にもあるか? \end{itemize} %%% developer レベルでは pdfTeX 拡張や pLaTeX カーネルでムニャムニャ For developers, additional cares may be needed, for changes in the kernel macros and/or absence of \pdfTeX\ features. %%% 際立った pTeX 系列の特徴 \section{Eminent characteristics of \pTeX/\upTeX} The most important characteristics of \pTeX/\upTeX\ can be summarized as follows: %%% 欧文と和文が別個に存在する・縦組がある \begin{itemize} \item Japanese characters are interpret and handled completely apart from Western characters. \item Texts can be aligned vertically, called ``tate-gumi'' (縦組). The horizontal alignment of texts is called ``yoko-gumi'' (横組), and both ``tate-gumi'' and ``yoko-gumi'' can be mixed even within a single document. \end{itemize} \newpage %%%%% \part{Details}% 各論 %%% 出力フォーマット \section{Output format --- DVI} %%% DVI だけ The output of \pTeX/\upTeX\ is always a DVI file. %%% pTeX の DVI は欧文の横組みだけなら普通。和文が入ると特殊,縦組ならIDも変化 Its DVI format is completely compatible with Knuthian \TeX, as long as the following conditions are met: \begin{itemize} \item No Japanese characters are typeset. \item There is no portion of vertical text alignment. \end{itemize} \subsection{Extensions of DVI format} In \pTeX/\upTeX, some additional DVI commands, which are defined in the standard \cite{dvistd0} but never used in \TeX82, are used. \begin{itemize} \item |set2| (129), |put2| (134): Appears in both \pTeX\ and \upTeX\ DVI. Used to typeset a Japanese character with 2-byte code. \item |set3| (130), |put3| (135): Appears in only \upTeX\ DVI. Used to typeset a Japanese character with 3-byte code. \end{itemize} When \pTeX\ is going to typeset a Japanese character into DVI, it is encoded in JIS, which is always a 2-byte code. For this purpose, |set2| or |put2| are used. When \upTeX\ is going to output a Japanese character into DVI, it is encoded in UTF-32. If the code is equal to or less than |U+FFFF|, the lower 16-bit is used with |set2| or |put2|. If the code is equal to or greater than |U+10000|, the lower 24-bit is used with |set3| or |put3|. In addition, \pTeX/\upTeX\ defines one additional DVI command. \begin{itemize} \item |dir| (255): Used to change directions of text alignment. \end{itemize} The DVI format in the preamble is always set to 2, as with \TeX82. On the other hand, the DVI ID in the postamble can be special. Normally it is set to 2, as with \TeX82; however, when |dir| (255) appears at least once in a single \pTeX/\upTeX\ DVI, the |post_post| table of postamble contains $\mathrm{ID} = 3$. %%% 日本の DVI ドライバの状況 \subsection{DVI drivers with Japanese support} There is some DVI drivers with Japanese support. The most eminent drivers are {\em dvips} and {\em dvipdfmx}. Nowadays most of casual Japanese users are using {\em dvipdfmx} as a DVI driver. On the other hand, users of {\em dvips} are unignorable, especially those working in publishing industry. \subsubsection{Using {\em dvipdfmx}} A DVI file which is output by \pTeX\ can be converted directly to a PDF file using dvipdfmx. % [TODO] Mention kanji-config-updmap for font setup {\em Note for casual \LaTeX\ users} --- when you choose to process the resulting DVI file with dvipdfmx after running \LaTeX\ (command |platex| or |uplatex|), you need to pass a proper driver option |[dvipdfmx]| for all driver-dependent packages, such as |graphicx| and |color|. This is because the default for such packages is set to |dvips| mode as with the original \LaTeX\ in DVI mode (command |latex|). For simplicity, we recommend a global driver option |[dvipdfmx]| as in the following example: \begin{verbatim} \documentclass[dvipdfmx,...]{article} \usepackage{graphicx} \usepackage{color} \end{verbatim} \subsubsection{Using {\em dvips}} A DVI file which is output by \pTeX\ can be converted to a PostScript file using dvips. % [TODO] Mention kanji-config-updmap again The resulting PostScript file can then be converted to a PDF file using Ghostscript (ps2pdf) or Adobe Distiller. When using Ghostscript, a proper setup of Japanese font must be done before converting PostScript into PDF. An easy solution for the setup is a script |cjk-gs-integrate| developed by Japanese \TeX\ Development Community. \section{Programming on \pTeX\ family} We focus on programming aspects of \pTeX\ and its variants. %%% レジスタの数 \subsection{Number of registers} \pTeX\ and \upTeX\ have exactly the same number ($=256$) of registers as Knuthian \TeX. \epTeX\ and \eupTeX\ in extended mode have more registers; there are 65536, which is twice as many as 32768 of \eTeX. % [TODO] まとめて 65536 個と書いて良い? レジスタの種類毎に書くべき? %%% 拡張プリミティブ \subsection{Additional primitives} % tex -ini: 322 multiletter control sequences % ptex -ini: 366 multiletter control sequences % uptex -ini: 374 multiletter control sequences % eptex -ini: 379 multiletter control sequences % eptex -ini -etex: 469 multiletter control sequences % euptex -ini: 387 multiletter control sequences % euptex -ini -etex: 477 multiletter control sequences % etex (pdftex) -ini: 470 multiletter control sequences % etex (pdftex) -ini -etex: 539 multiletter control sequences Here we provide only complete lists of additional primitives of \pTeX\ family in alphabetical order. Feature of each primitive can be found in Japanese edition. % [TODO] 抜けがないか? % [TODO] アルファベット順に正しく並んでいるか? % [TODO] 追加されたバージョン情報は正しいか? \def\New#1{--- New primitive since #1} \subsubsection{Sync\TeX\ additions (available in \pTeX, \upTeX, \epTeX, \eupTeX)} In the standard build of \TL, Sync\TeX\ extension is unavailable in Knuthian \TeX; however, it is enabled in \pTeX\ family. \begin{simplelist} \csitem[\.{synctex}] \end{simplelist} %%% pTeX のやつ \subsubsection{\pTeX\ additions (available in \pTeX, \upTeX, \epTeX, \eupTeX)} \begin{simplelist} \csitem[\.{autospacing}] \csitem[\.{autoxspacing}] \csitem[\.{disinhibitglue} \New{p3.8.2 (\TL2019)}] \csitem[\.{dtou}] \csitem[\.{euc}] \csitem[\.{ifdbox} \New{p3.2 (\TL2011)}] \csitem[\.{ifddir} \New{p3.2 (\TL2011)}] \csitem[\.{ifmbox} \New{p3.7.1 (\TL2017)}] \csitem[\.{ifmdir}] \csitem[\.{iftbox}] \csitem[\.{iftdir}] \csitem[\.{ifybox}] \csitem[\.{ifydir}] \csitem[\.{inhibitglue}] \csitem[\.{inhibitxspcode}] \csitem[\.{jcharwidowpenalty}] \csitem[\.{jfam}] \csitem[\.{jfont}] \csitem[\.{jis}] \csitem[\.{kanjiskip}] \csitem[\.{kansuji}] \csitem[\.{kansujichar}] \csitem[\.{kcatcode}] \csitem[\.{kuten}] \csitem[\.{noautospacing}] \csitem[\.{noautoxspacing}] \csitem[\.{postbreakpenalty}] \csitem[\.{prebreakpenalty}] \csitem[\.{ptexminorversion} \New{p3.8.0 (\TL2018)}] \csitem[\.{ptexrevision} \New{p3.8.0 (\TL2018)}] \csitem[\.{ptexversion} \New{p3.8.0 (\TL2018)}] \csitem[\.{scriptbaselineshiftfactor} \New{p3.7 (\TL2016)}] \csitem[\.{scriptscriptbaselineshiftfactor} \New{p3.7 (\TL2016)}] \csitem[\.{showmode}] \csitem[\.{sjis}] \csitem[\.{tate}] \csitem[\.{tbaselineshift}] \csitem[\.{textbaselineshiftfactor} \New{p3.7 (\TL2016)}] \csitem[\.{tfont}] \csitem[\.{xkanjiskip}] \csitem[\.{xspcode}] \csitem[\.{ybaselineshift}] \csitem[\.{yoko}] \csitem[\texttt{H}\index{H=\texttt{H}}] \csitem[\texttt{Q}\index{Q=\texttt{Q}}] \csitem[\texttt{zh}\index{zh=\texttt{zh}}] \csitem[\texttt{zw}\index{zw=\texttt{zw}}] \end{simplelist} %%% upTeX のやつ \subsubsection{\upTeX\ additions (available in \upTeX, \eupTeX)} \begin{simplelist} \csitem[\.{disablecjktoken}] \csitem[\.{enablecjktoken}] \csitem[\.{forcecjktoken}] \csitem[\.{kchar}] \csitem[\.{kchardef}] \csitem[\.{ucs}] \csitem[\.{uptexrevision} \New{u1.23 (\TL2018)}] \csitem[\.{uptexversion} \New{u1.23 (\TL2018)}] \end{simplelist} %%% e-pTeX/e-upTeX の pdfTeX 由来 %%% e-pTeX/e-upTeX の Omega 由来 %%% その他の独自拡張 \subsubsection{\epTeX\ additions (available in \epTeX, \eupTeX)} \begin{simplelist} \csitem[\.{epTeXinputencoding} \New{160201 (\TL2016)}] \csitem[\.{epTeXversion} \New{180121 (\TL2018)}] \csitem[\.{expanded} \New{180518 (\TL2019)}] \csitem[\.{hfi}] \csitem[\.{ifincsname} \New{190709 (\TL2020)}] \csitem[\.{ifpdfprimitive} \New{150805 (\TL2016)}] \csitem[\.{lastnodechar} \New{141108 (\TL2015)}] \csitem[\.{lastnodesubtype} \New{180226 (\TL2018)}] \csitem[\.{odelcode}] \csitem[\.{odelimiter}] \csitem[\.{omathaccent}] \csitem[\.{omathchar}] \csitem[\.{omathchardef}] \csitem[\.{omathcode}] \csitem[\.{oradical}] \csitem[\.{pagefistretch}] \csitem[\.{pdfcreationdate} \New{130605 (\TL2014)}] \csitem[\.{pdfelapsedtime} \New{161114 (\TL2017)}] \csitem[\.{pdffiledump} \New{140506 (\TL2015)}] \csitem[\.{pdffilemoddate} \New{130605 (\TL2014)}] \csitem[\.{pdffilesize} \New{130605 (\TL2014)}] \csitem[\.{pdflastxpos}] \csitem[\.{pdflastypos}] \csitem[\.{pdfmdfivesum} \New{150702 (\TL2016)}] \csitem[\.{pdfnormaldeviate} \New{161114 (\TL2017)}] \csitem[\.{pdfpageheight}] \csitem[\.{pdfpagewidth}] \csitem[\.{pdfprimitive} \New{150805 (\TL2016)}] \csitem[\.{pdfrandomseed} \New{161114 (\TL2017)}] \csitem[\.{pdfresettimer} \New{161114 (\TL2017)}] \csitem[\.{pdfsavepos}] \csitem[\.{pdfsetrandomseed} \New{161114 (\TL2017)}] \csitem[\.{pdfshellescape} \New{141108 (\TL2015)}] \csitem[\.{pdfstrcmp}] \csitem[\.{pdfuniformdeviate} \New{161114 (\TL2017)}] \csitem[\.{readpapersizespecial} \New{180901 (\TL2019)}] \csitem[\.{vfi}] \csitem[\texttt{fi}\index{fi=\texttt{fi}}] \end{simplelist} % [TODO] 引数は何で返り値は何か,expandable? %%% (e-)TeX にあるが (e-)upTeX にないもの \subsection{Omitted primitives and unsupported features} Compared to Knuthian \TeX\ and \eTeX, some primitives are omitted due to conflict with Japanese handling. %%% encTeX 拡張など One is enc\TeX\ extension, such as |\mubyte|. The ML\TeX\ extension, such as |\charsubdef|, is included but not well-tested. %%% コマンドラインオプションの話 % [TODO] ptex.man1.pdf も参照 \subsection{Behavior of Western \TeX\ primitives} Here we provide some notes on behavior of Knuthian \TeX\ and \eTeX\ primitives when used within \pTeX\ family. \subsubsection{Primitives with limitations in handling Japanese} Each of the following primitives allows only character codes 0--255; other codes will give an error ``! Bad character code.'' \begin{quote} |\catcode|, |\sfcode|, |\mathcode|, |\delcode|, |\mathchardef|, |\lccode|, |\uccode|. \end{quote} Each of the following primivies has |\...char| in its name, however, the effective values are restricted to 0--255. \begin{quote} |\endlinechar|, |\newlinechar|, |\escapechar|, |\defaulthyphenchar|, |\defaultskewchar|. \end{quote} \subsubsection{Primitives capable of handling Japanese} The following primitives are extended to support Japanese characters: \begin{cslist} \csitem[\.{char} , \.{chardef} =] In addition to 0--255, internal codes of Japanese characters are allowed. For putting Japanese characters, a Japanese font is chosen. \csitem[\.{font}, \.{fontname}, \.{fontdimen}] % [TODO] \csitem[\.{accent} =] % [TODO] \csitem[\.{if} , \.{ifcat} ] Japanese character token is also allowed. In that case, \begin{itemize} \item |\if| tests the internal character code of the Japanese character. \item |\ifcat| tests the |\kcatcode| of the Japanese character. \end{itemize} \end{cslist} \begin{dangerous} \TeX book describes the behavior of |\if| and |\ifcat| as follows; \begin{quote} If either token is a control sequence, \TeX\ considers it to have character code 256 and category code 16, unless the current equivalent of that control sequence has been |\let| equal to a non-active character token. \end{quote} However, this includes a lie; in the real implementation of tex.web, a control sequence is considered to have a category code 0. \end{dangerous} \subsection{Case study} Based on the above knowledge, we provide some code examples which may be useful for package developers. %%% pTeX かどうかの判定 \subsubsection{Detecting \pTeX} Since the primitive |\ptexversion| is rather new (added in 2018), the safer solution for detecting \pTeX\ is to test if a primitive |\kanjiskip| is defined. \begin{verbatim} \ifx\kanjiskip\undefined % not pTeX, upTeX, e-pTeX and e-upTeX \else % pTeX, upTeX, e-pTeX and e-upTeX \fi \end{verbatim} % [TODO] upTeX 系列は \enablecjktoken か \ucs か? % [TODO] \forcecjktoken は比較的新しい? % [TODO] 内部コードが uptex かどうかの判定 %%% pTeX のバージョン判定 %%% pdfTeX と違うところ \subsection{Difference from \pdfTeX\ in DVI mode}\label{dvi-pdftex} As stated above, \epTeX/\eupTeX\ are {\em not} uppercompatible with DVI mode of \pdfTeX. %%% pdfTeX の DVI モードにあって e-(u)pTeX にないプリミティブ %%% その他ムニャムニャ %%% ファイルの文字コードはどうするか \subsection{Recommendation for file encoding} %%% 全部アスキーにしてしまえ %%% UTF-8 にしたい場合はムニャムニャ %%% \epTeXinputencoding を使う話 %%% e-upTeX の入力解析:euc/sjis を語りたくないので utf8.uptex 前提 \subsection{Input handling} For simplicity, first we introduce of input handling of \eupTeX. %%% 文字の和文扱い・欧文扱い %%% 和文カテゴリーコード %%% \...cjktoken %%% ちなみに e-pTeX では:euc/sjis があることにだけ触れる %%% e-upTeX の和文トークンの話:和文トークンが混ざっているので注意 \subsection{Japanese tokens} %%% 和文組版の厳選トピック \section{Basic introduction to Japanese typesetting} %%% 和文をやり過ごすために注意すべきこと This section does not aim to explain Japanese typesetting completely; here we provide a minimum requirement for ``getting away'' with Japanese. %%% 空白・ペナルティ挿入:勝手に入ってくる! \subsection{Automatic insertion of glues and penalties} Sometimes \pTeX\ family automatically inserts glues and penalties between characters. % [TODO] もう少しだけ詳しく %%% 和文フォント \subsection{Japanese fonts} For typesetting Japanese characters, a JFM (Japanese \TeX\ font metric) format is used. It is a modified version of \TeX\ TFM. %%% 欧文とは別個 %%% \nullfont しても全部消えない %%% 縦組は諦めよう \newpage \begin{thebibliography}{99} \bibitem{dvistd0} TUG DVI Standards Working Group, \textit{The DVI Driver Standard, Level 0}.\\ \url{https://ctan.org/pkg/dvistd} \end{thebibliography} \newpage \printindex \end{document}