path: root/Build/source/README.2building

(This file was generated by makeinfo and splitinfo.gawk.)
(Released under the old-style GNU documentation license;
 see sources or other output files for full text.)

4 Building
**********

The top-level 'Build' script is intended to simplify building the
binaries distributed with TeX Live itself--we call this the "native" TL
build.  It runs 'configure && make world', which builds everything in a
subdirectory of the main source tree (default 'Work/'), installs
everything in another subdirectory (default 'inst/'), and finally runs
'make check'.  The exact directory and command names can be specified
via environment variables and a few leading options.  All remaining
arguments (assignments or options) are passed to the 'configure' script.
Please take a look at the './Build' source file for more information; it
is a straightforward shell script.

   An alternative, and the one we will mainly discuss here, is to run
'configure' and 'make' in a suitable empty subdirectory.  Building in
the source directory itself is not supported (sorry).

4.1 Build iteration
===================

Running the top-level 'configure' script configures the top level and
the subdirectories 'libs', 'utils', and 'texk'.  Running 'make' at the
top level first iterates over the TeX-specific libraries, and then runs
'make' in 'libs', 'utils', and 'texk' to iterate over the generic
libraries, utility programs, and TeX-specific programs, respectively.
These iterations consist of two steps:

  1. For each library or program module not yet configured, run
     'configure', adding the configure option '--disable-build' if the
     module need not be built, otherwise running 'make all'.

  2. For each library or program module that must be built, run 'make'
     for the selected target(s): 'default' or 'all' to (re-)build,
     'check' to run tests, 'install', etc.

   Running the top-level 'make' a second time iterates again over all
the library and program modules, and finds (should find) nothing to be
done.

4.2 Build in parallel
=====================

The TL build system carefully formulates dependencies as well as 'make'
rules when a tool (such as 'tangle', 'ctangle', and 'convert') creates
several output files.  This allows for parallel builds ('make -j N' with
N>1 or even 'make -j') that can considerably speed up the TL build.

   Incidentally, a noticeable speed-up can be independently gained by
using a configure cache file, i.e., specifying the 'configure' option
'-C' (recommended).

4.3 Build distribution
======================

Running 'make dist' at the top level creates a tarball
'tex-live-YYYY-MM-DD.tar.xz' from the TL source tree.  Running 'make
distcheck' also verifies that this tarball suffices to build and install
all of TL.

   This is useful for checking consistency of the source tree and
Makefiles, but the result is not a complete or even usable TeX system,
since all the support files are lacking; *note Installing::.  We do not
actually distribute any such tarball, and have no plans to do so.

4.4 Build one package
=====================

To build one package, the basic idea is to use the 'configure' option
'--disable-all-pkgs' (*note --disable-all-pkgs::).  Then all program and
library modules are configured but none are made.  However, the
'Makefile's still contain all build rules and dependencies and can be
invoked to build an individual program or library, first building any
required libraries.

   Here is an example from start to finish for working on 'dvipdfm-x'.
Unfortunately, this does not suffice for building the TeX engines; see
the next section.

     mkdir mydir && cd mydir  # new working directory

     # Get sources (<https://tug.org/texlive/svn>), e.g.:
     rsync -a --delete --exclude=.svn --exclude=Work \
           tug.org::tldevsrc/Build/source/ .

     # Create build directory:
     mkdir Work && cd Work

     # Do the configure:
     ../configure --disable-all-pkgs --enable-dvipdfm-x \
       -C CFLAGS=-g CXXFLAGS=-g >&outc || echo fail

     # Do the make:
     make >&outm || echo fail

     # Run the tests:
     cd texk/dvipdfm-x
     make check

     # Run the new binary in the buil tree, finding support files
     # in a separate tree for a TeX Live release YYYY
     # (Bourne shell syntax):
     TEXMFROOT=/usr/local/texlive/YYYY \
     TEXMFCNF=$TEXMFROOT/texmf-dist/web2c \
     ./xdvipdfmx ...

   Then you can modify source files in 'mydir/texk/dvipdfm-x' and rerun
'make' in 'mydir/Work/texk/dvipdfm-x' to rebuild; that build directory
is where the binary ends up and where you can run a debugger, etc.

   The second line of the 'configure' invocation shows examples of extra
things you likely want to specify if you intend to hack the sources (and
not just build binaries): the '-C' speeds 'configure' by enabling a
cache file, and the 'CFLAGS' and 'CXXFLAGS' settings eliminate compiler
optimization for debugging purposes.

   Of course, you need to actually look at the output and check that
things are working.  There are many 'configure' options you can tweak as
desired; check the output from 'configure --help'.  It is also a good
idea to run 'make check' after making any changes, to ensure that
whatever tests have been written still pass.

Reducing source download size
.............................

The above retrieves the entire TL source tree (several hundred
megabytes).  It is natural to ask if this is really necessary.  Strictly
speaking, the answer is no, but it is vastly more convenient to do so.
If you cut down the source tree, you must also give additional
'configure' flags to individually disable using system versions of
libraries, or the intricacies of the dependencies (such as 'teckit'
requiring 'zlib') will have undesired side effects.  For an example of
this approach, see the 'build-pdftex.sh' script in the 'pdftex'
development source (details at <http://pdftex.org>), which is indeed
such a cut-down TL source tree.

GCC used by default
...................

By default, the 'gcc' compilers will be used if present; otherwise,
individual packages may use something different.  You can explicitly
specify the compilers to be used with the environment variables 'CC',
'CXX', and 'OBJCXX'.

Removing C+11 dependency
........................

Some libraries and programs require C++11; one such is XeTeX. If you
want to build with an older compiler lacking such support, you need to
(re)move those source directories; unfortunately, specifying '--disable'
for them does not suffice.  It's also necessary to specify
'--disable-xetex' explicitly.  Specifically, before running 'configure
--disable-xetex ...':

     rm -rf libs/icu libs/graphite2 texk/dvisvgm texk/web2c/xetexdir

   Also, even with '--disable-all-pkgs', dependencies are (currently)
checked.  One notable case: if a (non-MacOS) system does not have
'fontconfig', XeTeX cannot be built (*note Prerequisites::), and
'configure' will terminate even with '--disable-xetex'.  To proceed
without such dependencies, specify '--enable-missing' also.

   As of 2022, HarfBuzz also requires C++11.  Therefore even more would
have to be disabled and removed, notably including 'luahbtex', the
standard engine used for LuaLaTeX.  Removing that would not be
acceptable for builds intended for distribution; but perhaps for testing
the above information could still be useful.

   In general, the TL 'configure' will run in all directories.
Therefore a general workaround for build problems is to remove failing
directories from the tree, and also specify the relevant '--disable-...'
option(s).

   Patches to improve all this would be most welcome.

4.5 Build one engine
====================

Unfortunately, there is one common case where the steps in the preceding
section to build one package (*note Build one package::) do not suffice:
wanting to build one, or a subset, of the TeX engines (or other Web2c
programs).

   The simplest way to do this is to disable everything and then
explicitly specify what to make.  For example, to build only the
original TeX:

     cd Work        # top build directory
     ../configure --without-x --disable-shared --disable-all-pkgs \
                  --enable-tex --disable-synctex --disable-xetex \
                  --enable-missing -C CFLAGS=-g CXXFLAGS=-g
     make
     cd texk/web2c  # cd engine build directory
     make tex       # must specify target

   The first 'make' run will configure everything, and even build the
libraries, even though the packages are disabled.

   The source tree can be cut down to just what is needed for the given
engine (the separate pdfTeX and LuaTeX source repositories do this, for
example), but see caveats in previous section.  When the
'--disable-xetex' and '--enable-missing' options are needed is also
explained in the previous section.

   If you want to debug an X-related program or shared library setup, or
other variants, change the 'configure' options accordingly.  Either
'../Build' or '../configure' can be run.

   Then it is necessary to again specify the target engine ('tex', in
the above) in the 'make'.

   All these complications are rather unfortunate.  Patches are welcome.

Testing one engine
..................

To run only the tests for a given engine, say 'hitex':

     make -C $ww check SUBDIRS=. TESTS='$(hitex_tests)'

where '$ww' is the web2c build directory, that is,
'ww=/wherever/Build/source/Work/texk/web2c'.

   It's also possible to run individual tests the same way, using the
test name exactly as specified in the '.am' file:

     make -C $ww check SUBDIRS=. TESTS=hitexdir/tests/hello.test

   If you get tired of looking at the 'Entering'/'Leaving directory'
lines, you can add the (GNU) make option '--no-print-dir'.

   You may find it useful to put lengthy incantations like this into a
trivial shell script with a short name (say, 'hitst').  Then you just
run 'hitst' and edit the file when necessary to change things around.

4.6 Cross compilation
=====================

In a cross compilation a "build" system is used to create binaries to be
executed on a "host" system with different hardware and/or operating
system.

   In simple cases, the build system can execute binaries for the host
system.  This typically occurs for bi-arch systems where, e.g.,
'i386-linux' binaries can run on 'x86_64-linux' systems and 'win32'
binaries can run on 'win64' systems.  Although sometimes called "native
cross", technically this is not cross compilation at all.  In most such
cases it suffices to specify suitable compiler flags.  It might be
useful to add the configure option '--build=HOST' to get the correct
canonical host name, but note that this should _not_ be '--host=HOST'
(*note (autoconf)Hosts and Cross-Compilation::).

   In order to build, e.g., 32-bit binaries with 'clang' on a 64-bit
MacOS system one could use:

     TL_BUILD_ENV="CC='clang -arch i386' \
       CXX='clang++ -arch i386' \
       OBJCXX='clang++ -arch i386'" \
       ./Build --build=i386-apple-darwin

4.6.1 Cross configuring
-----------------------

In a standard cross compilation, binaries for the host system cannot
execute on the build system and it is necessary to specify the configure
options '--host=HOST' and '--build=BUILD' with two different values.

   Building binaries requires suitable "cross" tools, e.g., compiler,
linker, and archiver, and perhaps a "cross" version of 'pkg-config' and
similar to locate host system libraries.  Autoconf expects that these
cross tools are given by their usual variables or found under their
usual name prefixed with 'HOST-'.  Here a list of such tools and
corresponding variables:

     ar                AR
     freetype-config   FT2_CONFIG
     g++               CXX
     gcc               CC
     icu-config        ICU_CONFIG
     objdump           OBJDUMP
     pkg-config        PKG_CONFIG
     ranlib            RANLIB
     strip             STRIP

In order to, e.g., build 'mingw32' binaries on 'x86_64-linux' with a
cross compiler found as 'i386-pc-mingw32-gcc' one would specify

     --host=i386-pc-mingw32 --build=x86_64-linux-gnu

or perhaps

     --host=mingw32 --build=x86_64-linux CC=i386-pc-mingw32-gcc

but this latter, especially, might require adding 'CXX' and others.

   Configure arguments such as 'CFLAGS=...' refer to the cross compiler.
If necessary, you can specify compilers and flags for the few auxiliary
C and C++ programs required for the build process as configure arguments

     BUILDCC=...
     BUILDCPPFLAGS=...
     BUILDCFLAGS=...
     BUILDCXX=...
     BUILDCXXFLAGS=...
     BUILDLDFLAGS=...

4.6.2 Cross problems
--------------------

The fact that binaries for the host system cannot be executed on the
build system causes some problems.

   One problem is that configure tests using 'AC_RUN_IFELSE' can compile
and link the test program but cannot execute it.  Such tests should be
avoided if possible and otherwise must supply a pessimistic test result.

   Another problem arises if the build process must execute some
(auxiliary or installable) programs.  Auxiliary programs can be placed
into a subdirectory that is configured natively as is done for
'texk/web2c/web2c', 'texk/dvipsk/squeeze', and 'texk/xdvik/squeeze'.
The module 'libs/freetype2' uses the value of 'CC_BUILD', 'BUILD-gcc',
'gcc', or 'cc' as the compiler for the auxiliary program.

   The situation for installable programs needed by the build process is
somewhat different.  A rather expensive possibility, chosen for the ICU
libraries in module 'libs/icu', is to first compile natively for the
build system and in a second step to use these (uninstalled) programs
during the cross compilation.

   This approach would also be possible for the tools such as 'tangle'
used in the module 'texk/web2c' to build the WEB programs, but that
would require first building a native 'kpathsea' library.  To avoid this
complication, cross compilation of programs written in (C)WEB requires
sufficiently recent installed versions of 'tangle', 'ctangle',
'otangle', and 'tie'.

   Building 'xindy' requires running the host system 'clisp' binary,
thus cross compilation is painful, but possible.