diff options
author | Karl Berry <karl@freefriends.org> | 2021-07-18 17:16:29 +0000 |
---|---|---|
committer | Karl Berry <karl@freefriends.org> | 2021-07-18 17:16:29 +0000 |
commit | 78591567fc12bcecdf39f8d632d39c12568401ea (patch) | |
tree | bb339f362e3a36d653efa2f7fbd7e219b12abde7 | |
parent | 24ff0d4c91e4e1d9c2d149465d25d78e24825325 (diff) |
checking fmt sharability across architectures
git-svn-id: svn://tug.org/texlive/trunk@59973 c570f23f-e606-0410-a88d-b1316a301751
-rw-r--r-- | Build/source/texk/web2c/doc/web2c.info | 242 | ||||
-rw-r--r-- | Build/source/texk/web2c/doc/web2c.texi | 51 | ||||
-rwxr-xr-x | Master/tlpkg/bin/tl-check-fmtshare | 268 | ||||
-rw-r--r-- | Master/tlpkg/doc/releng.txt | 23 |
4 files changed, 445 insertions, 139 deletions
diff --git a/Build/source/texk/web2c/doc/web2c.info b/Build/source/texk/web2c/doc/web2c.info index 35325f15ba9..f02ba7d31f3 100644 --- a/Build/source/texk/web2c/doc/web2c.info +++ b/Build/source/texk/web2c/doc/web2c.info @@ -1,4 +1,4 @@ -This is web2c.info, produced by makeinfo version 6.7 from web2c.texi. +This is web2c.info, produced by makeinfo version 6.8 from web2c.texi. This file documents the installation and use of the programs in Web2c, an implementation of Donald Knuth's TeX system. @@ -58,7 +58,7 @@ Web2c This document describes how to install and use the programs in the Web2c implementation of the TeX system, especially for Unix systems. It -corresponds to Web2c version 2021, released in May 2021. +corresponds to Web2c version 2021, released in July 2021. * Menu: @@ -83,7 +83,7 @@ File: web2c.info, Node: Introduction, Next: Installation, Prev: Top, Up: Top 1 Introduction ************** -This manual corresponds to version 2021 of Web2c, released in May 2021. +This manual corresponds to version 2021 of Web2c, released in July 2021. "Web2c" is the name of a TeX implementation, originally for Unix, but now also running under various operating systems. By "TeX @@ -774,33 +774,38 @@ File: web2c.info, Node: Hardware and memory dumps, Prev: Determining the memor 4.3.3 Hardware and memory dumps ------------------------------- -By default, memory dump files are generally sharable between -architectures of different types; specifically, on machines of different -endianness (*note (libc)Byte order::). (This is a feature of the Web2c -implementation, and is not true of all TeX implementations.) If you -specify '--disable-dump-share' to 'configure', however, memory dumps -will be endian-dependent. - - The reason to do this is speed. To achieve endian-independence, the -reading of memory dumps on LittleEndian architectures, such as PC's and -DEC architectures, is somewhat slowed (all the multibyte values have to -be swapped). Usually, this is not noticeable, and the advantage of -being able to share memory dumps across all platforms at a site far -outweighs the speed loss. But if you're installing Web2c for use on -LittleEndian machines only, perhaps on a PC being used only by you, you -may wish to get maximum speed. +By default, memory dump files are sharable between architectures of +different types; specifically, on machines of different endianness +(*note (libc)Byte order::) and with different word sizes (4-byte 'long' +vs. 8-byte 'long'). This is a feature of the Web2c implementation, and +is not true of all TeX implementations. + + The script 'tl-check-fmtshare' in the TeX Live source tree +('Master/tlpkg/bin') provides a relatively easy way to test that a +'.fmt' built on the local host can be loaded by a TeX engine built on +some remote host. + + If you specify '--disable-dump-share' to 'configure', however, memory +dumps will be endian-dependent. The reason to do this is speed. To +achieve endian-independence, the reading of memory dumps on LittleEndian +architectures, such as PC's and DEC architectures, is somewhat slowed +(all the multibyte values have to be swapped). Usually, this is not +noticeable, and the advantage of being able to share memory dumps across +all platforms at a site far outweighs the speed loss. But if you're +trying to squeeze out every possible bit of performance, you may wish to +do this. TeXnically, even without '--disable-dump-share', sharing of '.fmt' files cannot be guaranteed to work. Floating-point values are always written in native format, and hence will generally not be readable across platforms. Fortunately, TeX uses floating point only to -represent glue ratios, and all common formats (plain, LaTeX, AMSTeX, -...) do not do any glue setting at '.fmt'-creation time. Metafont does +represent glue ratios, and none of the common formats (plain, LaTeX, +AMSTeX, ...) do any glue setting at '.fmt'-creation time. Metafont does not use floating point in any dumped value at all. Incidentally, different memory dump files will never compare equal -byte-for-byte, because the program always dumps the current date and -time. So don't be alarmed by just a few bytes difference. +byte-for-byte, because the programs dump the current date and time. So +don't be alarmed by a few bytes difference. If you don't know what endianness your machine is, and you're curious, here is a little C program to tell you. (The 'configure' @@ -827,6 +832,9 @@ References::). exit (u.c[sizeof (long) - 1] == 1); } + You can add 'printf("long %d\n", sizeof(long));' to see the size of +the 'long' data type. + File: web2c.info, Node: Editor invocation, Next: \input filenames, Prev: Memory dumps, Up: Three programs @@ -3502,7 +3510,7 @@ Index * - starts option names: Option conventions. (line 11) * -- starts option names: Option conventions. (line 11) * --disable-dump-share configure option: Hardware and memory dumps. - (line 6) + (line 17) * --disable-mf-nowin: Online Metafont graphics. (line 65) * --enable-epsfwin: Online Metafont graphics. @@ -3534,7 +3542,7 @@ Index * -base=BASE: Determining the memory dump to use. (line 15) * -base=DUMPNAME: Common options. (line 39) -* -change=CHFILE: mft invocation. (line 63) +* -change=CHFILE: mft invocation. (line 62) * -charcode-format=TYPE: tftopl invocation. (line 27) * -charcode-format=TYPE <1>: vftovp invocation. (line 30) * -cnf-line: Common options. (line 25) @@ -3596,7 +3604,7 @@ Index * -shell-restricted: tex invocation. (line 122) * -show-opcodes: dvitype invocation. (line 52) * -strict: tangle invocation. (line 46) -* -style=MFTFILE: mft invocation. (line 67) +* -style=MFTFILE: mft invocation. (line 66) * -T: mpost invocation. (line 111) * -terse: bibtex invocation. (line 28) * -tex=TEXPROGRAM: mpost invocation. (line 114) @@ -3629,6 +3637,10 @@ Index * .Xresources: Online Metafont graphics. (line 69) * 2602gf: mf invocation. (line 48) +* 32-bit architecture: Hardware and memory dumps. + (line 6) +* 64-bit architecture: Hardware and memory dumps. + (line 6) * 8 bit clean: Common options. (line 110) * 8 bit clean output, specifying: Common options. (line 110) * 8-bit characters: TCX files. (line 6) @@ -3738,11 +3750,11 @@ Index * byte position: pktype invocation. (line 40) * byte position <1>: gftype invocation. (line 76) * byte swapping: Hardware and memory dumps. - (line 13) + (line 17) * c-sources Makefile target: Additional targets. (line 12) * caveats for \input filenames: \input filename caveats. (line 6) -* change files, and MFT: mft invocation. (line 63) +* change files, and MFT: mft invocation. (line 62) * change files, and Tangle: tangle invocation. (line 11) * change files, and Weave: weave invocation. (line 14) * changing error messages style: Common options. (line 31) @@ -3762,7 +3774,7 @@ Index (line 69) * cm.base: Initial Metafont. (line 26) * cmbase.mf: Initial Metafont. (line 26) -* cmbase.mft: mft invocation. (line 73) +* cmbase.mft: mft invocation. (line 72) * cmmf.base not recommended: Initial Metafont. (line 26) * color, in DVItoMP: dvitomp invocation. (line 15) * comments, in TCX files: TCX files. (line 70) @@ -3775,7 +3787,7 @@ Index (line 6) * Computer Modern fonts, and Troff: mpost invocation. (line 61) * Computer Modern macros: Initial Metafont. (line 26) -* Computer Modern Typefaces, production of: mft invocation. (line 73) +* Computer Modern Typefaces, production of: mft invocation. (line 72) * configuration: Installation. (line 6) * configuration file reading: Path searching. (line 6) * configuration file values: Runtime options. (line 6) @@ -3805,7 +3817,7 @@ Index * Cweb: WEB. (line 12) * CWEB: WEB. (line 12) * date and time, in memory dumps: Hardware and memory dumps. - (line 30) + (line 35) * debugging DVI utilities: dvitype invocation. (line 52) * debugging flags, specifying: Common options. (line 64) * decimal character codes, in TCX files: TCX files. (line 77) @@ -3847,7 +3859,7 @@ Index * e response at error prompt: Editor invocation. (line 6) * e-circumflex: \charsubdef. (line 20) * e-TeX: TeX extensions. (line 10) -* e.mft: mft invocation. (line 73) +* e.mft: mft invocation. (line 72) * EC fonts: tex invocation. (line 41) * EC fonts <1>: mf invocation. (line 24) * editing of bitmap fonts: Font utilities available elsewhere. @@ -3896,7 +3908,7 @@ Index * floating-point arithmetic: Compile-time options. (line 21) * floating-point values: Hardware and memory dumps. - (line 22) + (line 27) * fmt file, determining: Determining the memory dump to use. (line 6) * fmt files: Initial TeX. (line 6) @@ -3944,7 +3956,7 @@ Index * gftype GF validation: gftype invocation. (line 6) * gftype.web: Font utilities. (line 11) * glue ratio representations: Hardware and memory dumps. - (line 22) + (line 27) * glue, memory for: Runtime options. (line 20) * glyph substitutions: MLTeX. (line 6) * gray font: gftodvi invocation. (line 29) @@ -3954,7 +3966,7 @@ Index * gsftopk: Font utilities available elsewhere. (line 35) * Harbison, Samuel P.: Hardware and memory dumps. - (line 34) + (line 39) * hardware and memory dumps: Hardware and memory dumps. (line 6) * headerbyte information: tftopl invocation. (line 95) @@ -4041,6 +4053,8 @@ Index (line 6) * location of texmf.cnf in TeX Live: Runtime options. (line 56) * log file, BibTeX: bibtex invocation. (line 14) +* long data type, size of: Hardware and memory dumps. + (line 6) * Lua: TeX extensions. (line 30) * LuaTeX: TeX extensions. (line 30) * machine dependencies: Hardware and memory dumps. @@ -4070,7 +4084,7 @@ Index * memory dumps and hardware: Hardware and memory dumps. (line 6) * memory dumps, contain date and time: Hardware and memory dumps. - (line 30) + (line 35) * memory dumps, creating: Creating memory dumps. (line 6) * mems Make target: Additional targets. (line 34) @@ -4095,7 +4109,7 @@ Index (line 15) * MetaPost input files: mpost invocation. (line 26) * MetaPost invocation: mpost invocation. (line 6) -* MetaPost source, prettyprinting: mft invocation. (line 79) +* MetaPost source, prettyprinting: mft invocation. (line 77) * MetaPost, initial: Initial MetaPost. (line 6) * MetaPost, TeX, and Metafont: Three programs. (line 6) * metatype1: Font utilities available elsewhere. @@ -4212,7 +4226,7 @@ Index * plain.bst: Basic BibTeX style files. (line 11) * plain.fmt: Initial TeX. (line 13) -* plain.mft: mft invocation. (line 67) +* plain.mft: mft invocation. (line 66) * pltotf: pltotf invocation. (line 6) * pool file, writing: tangle invocation. (line 16) * Poole, Simon: Online Metafont graphics. @@ -4314,14 +4328,14 @@ Index * starting page: dvicopy invocation. (line 28) * starting page <1>: dvitype invocation. (line 46) * Steele Jr., Guy L.: Hardware and memory dumps. - (line 34) + (line 39) * stopping at the first error: Common options. (line 46) * strategy, overall: Introduction. (line 16) * string numbers, displaying: pooltype invocation. (line 6) * string pool, writing: tangle invocation. (line 16) * string representation: pooltype invocation. (line 30) * style design, for BibTeX: bibtex invocation. (line 49) -* style files: mft invocation. (line 67) +* style files: mft invocation. (line 66) * substitutions of font glyphs: MLTeX. (line 6) * sun: Online Metafont graphics. (line 42) @@ -4333,7 +4347,7 @@ Index (line 43) * swap space, as array limit: Runtime options. (line 45) * swapping bytes: Hardware and memory dumps. - (line 13) + (line 17) * syntax of TCX files: TCX files. (line 65) * system C library function: Shell escapes. (line 6) * system command: Shell escapes. (line 6) @@ -4394,8 +4408,10 @@ Index * tftopl: tftopl invocation. (line 6) * three programs: Three programs. (line 6) * time and date, in memory dumps: Hardware and memory dumps. - (line 30) + (line 35) * title font: gftodvi invocation. (line 33) +* tl-check-fmtshare: Hardware and memory dumps. + (line 12) * tokenization, and \input filenames: \input filename caveats. (line 6) * toolkits, X: Online Metafont graphics. @@ -4505,78 +4521,78 @@ Index Tag Table: Node: Top2759 -Node: Introduction3829 -Node: Installation7165 -Node: configure options9209 -Node: Compile-time options11231 -Node: Additional targets12333 -Node: Triptrap13610 -Node: Commonalities15113 -Node: Option conventions15646 -Node: Common options16869 -Node: Path searching21487 -Node: Output file location22459 -Node: Three programs23761 -Node: Runtime options24468 -Node: Initial and virgin27609 -Node: Memory dumps28558 -Node: Creating memory dumps29010 -Node: Determining the memory dump to use30217 -Node: Hardware and memory dumps31794 -Node: Editor invocation34240 -Node: \input filenames35103 -Node: \input quoted filename36654 -Node: \input braced filename37431 -Node: \input filename caveats38361 -Node: TeX39875 -Node: tex invocation41043 -Node: Initial TeX48029 -Node: Formats49393 -Node: Languages and hyphenation52210 -Node: MLTeX52640 -Node: \charsubdef54268 -Node: \tracingcharsubdef56591 -Node: TCX files57166 -Node: patgen invocation62606 -Node: Shell escapes63313 -Node: IPC and TeX66893 -Node: TeX extensions67463 -Node: Metafont69492 -Node: mf invocation70780 -Node: Initial Metafont74617 -Node: Modes76247 -Node: Online Metafont graphics78497 -Node: gftodvi invocation81922 -Node: mft invocation84742 -Node: MetaPost88740 -Node: mpost invocation89576 -Node: Initial MetaPost94612 -Node: dvitomp invocation95534 -Node: BibTeX96199 -Node: bibtex invocation96560 -Node: Basic BibTeX style files99034 -Node: WEB100364 -Node: tangle invocation101573 -Node: weave invocation103697 -Node: pooltype invocation105104 -Node: DVI utilities106234 -Node: dvicopy invocation107168 -Node: dvitype invocation108453 -Node: dvitype output example110784 -Node: Font utilities113837 -Node: Font file formats115017 -Node: gftopk invocation118284 -Node: pktogf invocation119475 -Node: pktype invocation120641 -Node: gftype invocation123464 -Node: tftopl invocation127967 -Node: pltotf invocation132560 -Node: vftovp invocation133611 -Node: vptovf invocation135846 -Node: Font utilities available elsewhere136873 -Node: Legalisms139078 -Node: References140525 -Node: Index144658 +Node: Introduction3830 +Node: Installation7167 +Node: configure options9211 +Node: Compile-time options11233 +Node: Additional targets12335 +Node: Triptrap13612 +Node: Commonalities15115 +Node: Option conventions15648 +Node: Common options16871 +Node: Path searching21489 +Node: Output file location22461 +Node: Three programs23763 +Node: Runtime options24470 +Node: Initial and virgin27611 +Node: Memory dumps28560 +Node: Creating memory dumps29012 +Node: Determining the memory dump to use30219 +Node: Hardware and memory dumps31796 +Node: Editor invocation34546 +Node: \input filenames35409 +Node: \input quoted filename36960 +Node: \input braced filename37737 +Node: \input filename caveats38667 +Node: TeX40181 +Node: tex invocation41349 +Node: Initial TeX48335 +Node: Formats49699 +Node: Languages and hyphenation52516 +Node: MLTeX52946 +Node: \charsubdef54574 +Node: \tracingcharsubdef56897 +Node: TCX files57472 +Node: patgen invocation62912 +Node: Shell escapes63619 +Node: IPC and TeX67199 +Node: TeX extensions67769 +Node: Metafont69798 +Node: mf invocation71086 +Node: Initial Metafont74923 +Node: Modes76553 +Node: Online Metafont graphics78803 +Node: gftodvi invocation82228 +Node: mft invocation85048 +Node: MetaPost89046 +Node: mpost invocation89882 +Node: Initial MetaPost94918 +Node: dvitomp invocation95840 +Node: BibTeX96505 +Node: bibtex invocation96866 +Node: Basic BibTeX style files99340 +Node: WEB100670 +Node: tangle invocation101879 +Node: weave invocation104003 +Node: pooltype invocation105410 +Node: DVI utilities106540 +Node: dvicopy invocation107474 +Node: dvitype invocation108759 +Node: dvitype output example111090 +Node: Font utilities114143 +Node: Font file formats115323 +Node: gftopk invocation118590 +Node: pktogf invocation119781 +Node: pktype invocation120947 +Node: gftype invocation123770 +Node: tftopl invocation128273 +Node: pltotf invocation132866 +Node: vftovp invocation133917 +Node: vptovf invocation136152 +Node: Font utilities available elsewhere137179 +Node: Legalisms139384 +Node: References140831 +Node: Index144964 End Tag Table diff --git a/Build/source/texk/web2c/doc/web2c.texi b/Build/source/texk/web2c/doc/web2c.texi index 447c19f3e62..46ef2741455 100644 --- a/Build/source/texk/web2c/doc/web2c.texi +++ b/Build/source/texk/web2c/doc/web2c.texi @@ -12,7 +12,7 @@ The latest version is available from ftp://ftp.tug.org/tex/texinfo.tex.} @end tex @set version 2021 -@set month-year May 2021 +@set month-year July 2021 @c Define new indices for commands in auxiliary files, filenames, and options. @defcodeindex cm @@ -789,24 +789,34 @@ will be used. @cindex machine dependencies @cindex architecture dependencies @cindex dependencies, hardware -@opindex --disable-dump-share configure @r{option} -By default, memory dump files are generally sharable between -architectures of different types; specifically, on machines of different -endianness (@pxref{Byte order,,,libc,GNU C Library}). (This is a +@cindex 32-bit architecture +@cindex 64-bit architecture +@cindex @code{long} data type, size of +By default, memory dump files are sharable between architectures of +different types; specifically, on machines of different endianness +(@pxref{Byte order,,, libc, GNU C Library}) and with different word +sizes (4-byte @code{long} vs.@: 8-byte @code{long}). This is a feature of the Web2c implementation, and is not true of all @TeX{} -implementations.) If you specify @samp{--disable-dump-share} to -@code{configure}, however, memory dumps will be endian-dependent. +implementations. + +@pindex tl-check-fmtshare +The script @code{tl-check-fmtshare} in the @TeX{} Live source tree +(@file{Master/tlpkg/bin}) provides a relatively easy way to test that +a @file{.fmt} built on the local host can be loaded by a @TeX{} engine +built on some remote host. @cindex byte swapping @cindex swapping bytes -The reason to do this is speed. To achieve endian-independence, the -reading of memory dumps on LittleEndian architectures, such as PC's and -DEC architectures, is somewhat slowed (all the multibyte values have to -be swapped). Usually, this is not noticeable, and the advantage of -being able to share memory dumps across all platforms at a site far -outweighs the speed loss. But if you're installing Web2c for use on -LittleEndian machines only, perhaps on a PC being used only by you, you -may wish to get maximum speed. +@opindex --disable-dump-share configure @r{option} +If you specify @samp{--disable-dump-share} to @code{configure}, +however, memory dumps will be endian-dependent. The reason to do this +is speed. To achieve endian-independence, the reading of memory dumps +on LittleEndian architectures, such as PC's and DEC architectures, is +somewhat slowed (all the multibyte values have to be swapped). +Usually, this is not noticeable, and the advantage of being able to +share memory dumps across all platforms at a site far outweighs the +speed loss. But if you're trying to squeeze out every possible bit of +performance, you may wish to do this. @cindex floating-point values @cindex glue ratio representations @@ -814,8 +824,8 @@ may wish to get maximum speed. @file{.fmt} files cannot be guaranteed to work. Floating-point values are always written in native format, and hence will generally not be readable across platforms. Fortunately, @TeX{} uses floating point -only to represent glue ratios, and all common formats (plain, -@LaTeX{}, AMS@TeX{}, @dots{}) do not do any glue setting at +only to represent glue ratios, and none of the common formats (plain, +@LaTeX{}, AMS@TeX{}, @dots{}) do any glue setting at @file{.fmt}-creation time. Metafont does not use floating point in any dumped value at all. @@ -823,8 +833,8 @@ any dumped value at all. @cindex time and date, in memory dumps @cindex memory dumps, contain date and time Incidentally, different memory dump files will never compare equal -byte-for-byte, because the program always dumps the current date and -time. So don't be alarmed by just a few bytes difference. +byte-for-byte, because the programs dump the current date and time. So +don't be alarmed by a few bytes difference. @cindex Harbison, Samuel P. @cindex Steele Jr., Guy L. @@ -855,6 +865,9 @@ main () @} @end example +You can add @code{printf("long %d\n", sizeof(long));} to see the size +of the @code{long} data type. + @node Editor invocation @section Editor invocation diff --git a/Master/tlpkg/bin/tl-check-fmtshare b/Master/tlpkg/bin/tl-check-fmtshare new file mode 100755 index 00000000000..08969ff6a0f --- /dev/null +++ b/Master/tlpkg/bin/tl-check-fmtshare @@ -0,0 +1,268 @@ +#!/bin/sh +# Public domain. Originally written by Karl Berry, 2021. +# Test that a .fmt file built on one system can be used on another. + +version='$Id$' +renice 20 $$ >/dev/null 2>&1 +unset CDPATH +LC_ALL=C; export LC_ALL +umask 077 # since we write in a tmp dir (unsafe still, but oh well) + +fmtutil_path=`cd /tmp && which fmtutil-sys 2>/dev/null` +if test -z "$fmtutil_path"; then + echo "$0: no fmtutil-sys in PATH, goodbye." >&2 + exit 1 +fi +bindir=`dirname "$fmtutil_path"` +mydir=`cd \`dirname $0\` && pwd` # Master/tlpkg/bin +#echo "fmtutil_path=$fmtutil_path, bindir=$bindir, mydir=$mydir" + +Master=`cd "$bindir"/../.. && pwd` +enginedir=`cd "$mydir"/../../../Build/source/Work/texk/web2c 2>/dev/null&& pwd` +outdir=/tmp/fmtshare.`id -u` +default_fmts="tex.fmt pdftex.fmt pdflatex.fmt xetex.fmt xelatex.fmt + luatex.fmt optex.fmt euptex.fmt" # not lualatex, see below +fmts= +remote= + +usage="Usage: $0 [OPTION]... RHOST:RDIR +Build TeX .fmt files locally, then copy them to RHOST:RDIR and try to +load them with the TeX engines there. + +The idea is to support testing for .fmts being sharable among different +system architectures, as intended; for instance, 4-byte long vs. 8-byte +long and BigEndian vs. LittleEndian. + +The further idea is to be able to test new binaries in the Build +directory (since that's where the compiled sources change), while using +support files from a Master directory (since that's where they are +updated). + +fmtutil-sys is run locally (found using PATH) to discern the engine and +full command line used for building the given .fmts. + +The remote execution is done with \"ssh -n RHOST && cd RDIR && ...\", +so ssh has to be working. + +The default for the Build directory is to be relative to where this +script is executed. The default for the Master directory is to be +relative to where fmtutil-sys is found in PATH. These may or may not be +the same. + +The output directory is not cleaned before starting, in case older logs +are helpful in debugging new failures. + +Special case: unfortunately, lualatex.fmt cannot be easily tested, and +it is omitted by default. Loading the fmt requires finding many Lua +files, which are unlikely to be available on the remote machine. The +script does not try to handle this. + +Options (must use --, must use space to separate option from value, sorry): +--Master MDIR use MDIR for local support files + [$Master] +--enginedir EDIR use EDIR for local engine binaries + [$enginedir] +--outdir ODIR build locally in ODIR [$outdir] +--fmt FMT build FMT; can be given more than once + [$default_fmts] + +--help display this help and exit +--version display version information and exit + +Doc: https://tug.org/texinfohtml/web2c.html#Hardware-and-memory-dumps +Bug reports, discussion: tex-live@tug.org +Version: $version" + +while test $# -gt 0; do + case $1 in + --Master) shift; Master=$1;; + --enginedir) shift; enginedir=$1;; + --fmt) shift; fmts="$fmts $1";; + --outdir) shift; outdir=$1;; + # + --help) echo "$usage"; exit 0;; + --version) echo "$version"; exit 0;; + -*) echo "$0: goodbye, unrecognized option: $1" >&2; exit 1;; + # + *) if test -n "$remote"; then + echo "$0: remote argument already seen: $remote" >&2 + echo "$0: now have second: $1" >&2 + exit 1 + fi + remote=$1;; + esac + shift +done + +if test -z "$remote"; then + echo "$0: no remote argument given, goodbye." >&2 + exit 1 +elif echo "$remote" | grep -v : >/dev/null; then + echo "$0: remote argument must have form HOST:DIR, not like: $remote" >&2 + exit 1 +elif test `echo "$remote" | tr -cd : | wc -c` != 1 >/dev/null; then + # not worth the fancier parsing until we need it. + echo "$0: remote argument must have only one colon: $remote" >&2 + exit 1 +fi +remotesys=`echo "$remote" | sed 's/:.*//'` +remotedir=`echo "$remote" | sed 's/.*://'` +# +if test ! -d "$Master/texmf-dist/web2c"; then + echo "$0: no texmf-dist/web2c subdir of Master: $Master" >&2 + ls "$Master" >&2 + exit 1 +fi +# +if test ! -d "$enginedir"; then + echo "$0: enginedir not a directory: $enginedir" >&2 + ls "$enginedir" >&2 + exit 1 +fi + +# create output directory if needed. +test -d "$outdir" || mkdir "$outdir" || exit 1 + +# default fmt list. +test -z "$fmts" && fmts=$default_fmts + +# Our function to create a format locally: +# mkfmt FMT ENGINEDIR MASTERDIR OUTDIR +# Generates FMT using the binary from ENGINEDIR, +# support files from MASTERDIR, and leaving FMT in OUTDIR. +# +# Leaves current directory as OUTDIR. +# +# On success, returns zero and outputs two lines to stdout: +# 1) the full path to the newly-generated fmt file; and +# 2) the basename of the engine binary that was used to build it. +# +# On failure, returns nonzero and outputs nothing to stdout; issues +# diagnostics to stderr. +# +mkfmt () +{ + fmt=$1 + enginedir=$2 + Master=$3 + outdir=$4 + + # Our goal is to build a .fmt using a binary from the Build tree + # while still using the support files from the Master tree. + # The idea being that after changing the source and rebuilding the + # binary, we want to easily test whether the new binary is ok. + # + # First, we run fmtutil-sys (assumed to be in PATH) without generating + # anything (--dry-run), to garner the command line needed to build # FMT. + # + fcmd="fmtutil-sys --dry-run --no-engine-subdir --fmtdir $outdir --byfmt $fmt" + ffot=$outdir/`basename $fmt .fmt`-fmtutil.fot + # + echo "$0: running: $fcmd" >"$ffot" + if $fcmd >>"$ffot" 2>&1; then :; + echo "$0: could not get cmdline to build $fmt" >&2 + echo "$0: fmtutil command failed: $fcmd" >&2 + echo "$0: see transcript: $ffot" >&2 + return 1 + fi + + # Extract the build command from the fmtutil output, which has a line like: + # fmtutil: running `pdftex -ini [more options] *pdfetex.ini' ... + # We want what's between the quotes. Format names and options can't + # contain quote or other special characters. + lq='`' + rq="'" + ecmd=`sed -n "s/^fmtutil.*running $lq\(.*\)$rq.*/\1/p" "$ffot"` + if test -z "$ecmd"; then + echo "$0: could not extract cmdline to build $fmt" >&2 + echo "$0: from fmtutil output; see: $ffot" >&2 + return 1 + fi + + # Generate everything in OUTDIR. + cd "$outdir" || return 1 + + # Set environment variables so the given Master tree is used, and + # prepend the given engine directory to PATH, and run the command. + env="env PATH=$enginedir:$PATH \ + TEXMFCNF=$Master/texmf-dist/web2c \ + TEXMFROOT=$Master" + efot=$outdir/`basename $fmt .fmt`-engine.fot + echo "$0: running: $ecmd" >"$efot" + if $env $ecmd >>"$efot" 2>&1 && test -s "$outdir/$fmt"; then + sed -n '2p;q' "$efot" + echo "$outdir/$fmt" + echo "$ecmd" | awk '{print $1}' # engine that was used + else + echo "$0: could not build $fmt" >&2 + echo "$0: engine command failed: $ecmd" >&2 + echo "$0: see transcript: $efot" >&2 + return 1 + fi +} + +# Loop over all given fmts. +# +for fmt in $fmts; do + if echo "$fmt" | grep '\.fmt$' >/dev/null; then :; else + echo "$0: fmt does not end with .fmt, goodbye: $fmt" >&2 + exit 1 + fi + + # first build fmt on local machine. + set - `mkfmt "$fmt" "$enginedir" "$Master" "$outdir"` + fmtfile=$1 + engine=$2 + # + if test -z "$fmtfile"; then + # error messages already given, but give another just to be sure. + echo "$0: mkfmt failed: $fmt " \ + "(enginedir $enginedir) (Master $Master) (outdir $outdir)" >&2 + exit 1 + elif test -z "$engine"; then + echo "$0: should not happen, mkfmt returned fmtfile: $fmtfile" >&2 + echo "$0: but no engine value?!" >&2 + exit 1 + elif test ! -s "$fmtfile"; then + echo "$0: should not happen, returned fmtfile is empty: $fmtfile" >&2 + exit 1 + fi + echo "$0: (`date`) built fmtfile: `ls -l $fmtfile`" + + # The TeX \command to exit a job immediately. Assume a LaTeX fmt + # if "latex" is in the name, else plain (enough). + if echo "$fmt" | grep latex >/dev/null; then + endcmd='\stop' + else + endcmd='\end' + fi + + # copy to remote machine. + echo "$0: copying $fmtfile to $remotesys:$remotedir..." + scp -pq "$fmtfile" "$remotesys:$remotedir" || exit 1 + + # load on remote machine. + echo "$0: running $engine in $remotesys:$remotedir..." + # + # On the remote side, we need to find a texmf.cnf or pdftex gets the + # mysterious "Must increase the hyph_size"; assume the + # TL source directory structure to find kpathsea/texmf.cnf. + # + # Do not try to dynamically create the .fmt. + # + remoteenv="env MKTEXFMT=0 TEXMFCNF=../../../texk/kpathsea" + remotecmd="cd $remotedir && $remoteenv ./$engine -fmt=./$fmt '$endcmd'" + # + rfot=$outdir/`basename $fmt .fmt`-$remotesys.fot + echo "$0: running on $remotesys: $remotecmd" >$rfot + ssh -n $remotesys "$remotecmd" </dev/null >>$rfot 2>&1 + if test $? -ne 0; then + echo "$0: fmt load failed on $remotesys: $fmt" >&2 + echo "$0: see transcript: $rfot" >&2 + exit 1 + else + echo "$0: fmt load ok on $remotesys: $fmt" + fi +done + +exit 0 diff --git a/Master/tlpkg/doc/releng.txt b/Master/tlpkg/doc/releng.txt index 563ed61953c..a567d28d19b 100644 --- a/Master/tlpkg/doc/releng.txt +++ b/Master/tlpkg/doc/releng.txt @@ -273,11 +273,20 @@ M. As pretest proceeds, update the doc: tl-update-txi Don't forget to commit everything (texmf-dist/doc, Build/source, ...). -N. Really test during pretest (really! don't leave it for Knuth to do!): - .fmt's sharable across endian-different platforms (including luatex). - interactive mf '\drawdot (50,50); showit;' % and from * prompt - -O. Two days before the final pretest (i.e., to-be-published) build: +N. Really test during and at end of pretest that + (really! don't leave it for post-release failure) + .fmt's sharable are across endian/word-different platforms, + including luatex: + tlpkg/bin/tl-check-fmtshare RHOST:RDIR + where RHOST is a host you can ssh to with different specifications + than the local host, and RDIR is a TL:Build/source/Work/texk/web2c/ + build directory. (Must build TL there first before running, naturally.) + Nelson Beebe has a wide variety of hosts available. + +O. Test that basic interactive Metafont is still working: + mf '\drawdot (50,50); showit;' % and from * prompt + +W. Two days before the final pretest (i.e., to-be-published) build: Put date into NEWS and ChangeLog files above. Update Build/source/tardate.ac && reautoconf. tlmgr version number in texmf-dist/scripts/texlive/NEWS. @@ -291,13 +300,13 @@ O. Two days before the final pretest (i.e., to-be-published) build: Tell developers no more commits: tldev + hirata, hironori, luigi, mojca, takuji, -P. The day before the final pretest: +X. The day before the final pretest: In cron.tl, catalogue_compare=false to avoid skew. In tl-update-tlnet, option adjustrepo 1, else install-tl-unx won't go to ctan Update Master/source (if not earlier): xz wget w32tex-src texlive*source mactexdoc install-tl-unx.tar.gz -Q. The day of the final tlnet-from-branch build, which may or may not +Y. The day of the final tlnet-from-branch build, which may or may not also be the day of the final tlpretest-from-trunk build: Set frozen=>1 in branch TLConfig.pm. Freeze of tlnet should happen at that night's rebuild. |