Build system
************

The TL build system was redesigned in 2009, consistently using Autoconf, Automake, and Libtool.  Thus
   configure &&
make && make check && make install
or the basically-equivalent top-level Build script suffice to build and install the TL programs.  The make check clause performs various tests of the generated programs---not strictly required but strongly recommended. Running configure --help will display a comprehensive list of all configure options.

The main components of the TL build system are:

texk/LIB
     TeX-specific libraries in subdirectories, notably LIB=kpathsea.  (The other one is texk/ptexenc.)

texk/PROG
     TeX-specific programs (that use Kpathsea).

utils/PROG
     Other programs (that don't use Kpathsea).

libs/LIB
     Generic libraries.

The primary design goal of the build system is modularity.  Each program and library module (or package) specifies its own requirements and properties, such as required libraries, whether an installed (system) version of a library can be used, configure options to be seen at the top-level, and more.  An explicit list of all available modules is kept in only one, central, place (m4/kpse-pkgs.m4).

A second, related goal is to configure and build each library before configuring any other (program or library) module which uses that library.  This allows checking for properties and features of a library built as part of the TL tree in much the same way as for a system version of that library.

All generic libraries and several programs are maintained independently.  The corresponding modules use (most of) the distributed source tree and document any modifications of that source tree.

All this is for the sake of simplifying both upgrading of modules maintained independently and integrating new modules into the TL build system.  (Not to say that either task has become trivial.)

Prerequisites for building TL
=============================

Overall, building the TL programs, when using all libraries from the TL source tree, requires only C and C++ compilers and GNU make.  (If make from your PATH is not GNU make, you can set MAKE in the environment to whatever is necessary.)

Indeed, GNU make is required only because of some third-party libraries, notably FreeType; all the TL-maintained directories (and Automake/Autoconf output in general) should work with any reasonable make.

However, a few programs in the tree have additional requirements:

biber
     requires perl for autodetection of the TL platform name.

xdvik
xpdfopen
     require X11 headers and libraries, often a ``development'' package that is not installed by default.

xetex
     requires fontconfig (again both headers and library).

xindy
     requires GNU clisp and in addition perl, latex, and pdflatex to build the rules and/or documentation.

Lacking the required tools, building these programs must avoided, e.g., configure --disable-biber --without-x --disable-xetex --disable-xindy.

Modifying source files also induces more requirements:

  * Modification of any .y or .l source files requires bison or flex to updatete the corresponding C sources.

  * Modification of the sources for .info files requires makeinfo.

  * Modification of any part of the build system (M4 macros, configure.ac, Makefile.am, or their fragments) requires GNU M4, GNU Autoconf, GNU Automake, and GNU Libtool to update the generated files.  See Build system tools.

Building
========

The top-level Build script is intended to simplify building the binaries distributed with TL itself---we call this the ``native'' TL build.  It configures and makes everything in a subdirectory of the main build tree (default Work/), installs everything in an other subdirectory (default inst/), and finally runs make check.  The exact directory and command names can be specified via environment variables and a few leading options.  All remaining arguments (assignments or options) are passed to the configure script.  Please take a look at the script itself for more information; it is not complicated.

An alternative, and the one we will mainly discuss here, is to run configure and make oneself in a suitable empty subdirectory.  Building in the source directory itself is not supported (sorry).

Running the top-level configure script configures the top level and the subdirectories libs, utils, and texk. Running make at the top-level first iterates over all TeX-specific libraries, and then runs make in libs, utils, and texk to iterate over all generic libraries, utility programs, and TeX-specific programs.  These iterations consist of two steps:

 1. For each library or program module not yet configured, run configure, adding the configure option --disable-build if the module need not be built, otherwise running make all.

 2. For each library or program module that must be built, run make for the selected target(s): default or all to (re-)build, check to run tests, install, etc.

Running the top-level make a second time iterates again over all the library and program modules, but finds (should find) nothing to be done unless some source files have been modified.

If configuring or building a module fails, you could either (a) find and fix the problem, or (b) remove the subdirectory for that module from the build tree, and rerun the top-level make (or Build with --no-clean as its first argument).

With the configure option --disable-all-pkgs, all program and library modules are configured but none of them is built.  The Makefile for each such module contains all build rules and dependencies and can be invoked to build an individual program or library and causes to first build any required libraries.  This ``build-on-demand'' procedure is used, e.g., in the luatex repository to build LuaTeX, essentially from a subset of the complete TL tree.  Similarly, when, e.g., building the original e-TeX has been disabled (as it is by default), one can run make etex (or make etex.exe) in texk/web2c/ to build e-TeX (although there is no comparably simple way to install e-TeX).

The TL build system carefully formulates dependencies as well as make rules when a tool (such as tangle, ctangle, or convert) creates several output files.  This allows for parallel builds (make -j N with N>1 or even make -j) that can considerably speed up the TL build. Independently, speed-up can also be achieved by using a configure cache file, i.e., with the option -C (recommended).

Running make dist at the top-level creates a tarball tex-live-YYYY-MM-DD.tar.xz from the TL source tree.  Running make dist-check also verifies that this tarball suffices to build and install all of TL.  (This is useful for checking consistency of the source tree and Makefiles, but the result is not a complete or even usable TeX system, since all the support files are lacking; see the next section.)

Installing
==========

This section discusses the results of make install in the source tree.

The main consideration is that this is not enough to make a usable TeX installation.  Beyond the binaries that are built from the sources, (thousands of) support files are needed; just as a first example plain.tex is not in the source tree.

These support files are maintained completely independently and are not present in the source tree.  The best basis for dealing with them is the TL (plain text) database in Master/tlpkg/texlive.tlpdb, and/or the TL installer, install-tl.  More information is under Master/tlpkg and at http://tug.org/texlive/distro.html.

Installation directories
------------------------

Running make install (or make install-strip) installs executables in BINDIR, libraries in LIBDIR, headers in INCLUDEDIR, general data (including ``linked scripts'', see Linked scripts) in DATAROOTDIR/texmf-dist, man pages in MANDIR, and Info files in INFODIR.

The values of these directories are determined by configure and can be specified explictly as options such as --prefix=PREFIX or --bindir=BINDIR; otherwise, they are given by their usual Autoconf defaults:

     PREFIX                  /usr/local
     EXEC_PREFIX             PREFIX
     BINDIR                  EXEC_PREFIX/bin
     LIBDIR                  EXEC_PREFIX/lib
     INCLUDEDIR              PREFIX/include
     DATAROOTDIR             PREFIX/share
     MANDIR                  DATAROOTDIR/man
     INFODIR                 DATAROOTDIR/info

... except possibly modified as follows:

  * If the option --enable-multiplatform is given, /PLATFORM (i.e., the canonical platform name) is appended to BINDIR and LIBDIR.  This is implied for a native TL build.

  * In a native TL build, DATAROOTDIR is set to PREFIX, INFODIR is set to PREFIX/texmf-dist/doc/info, and MANDIR to PREFIX/texmf-dist/doc/man, corresponding to the directories used in the TL distribution.

The top-level configure script displays all these installation paths.

For the native TL build, the Build script leaves the binaries in ./inst/bin/STD-PLATFORM-NAME.  The new binaries are not directly usable from that location; they need to be copied to Master/bin/TL-PLATFORM.  The other files and directories that end up in ./inst/ are ignored.

Linked scripts
--------------

Quite a few executables are architecture-independent shell, Perl, or other scripts, rather than compiled binaries.  A few are maintained as part of the TL source tree, but most are maintained elsewhere with copies under texk/texlive/linked_scripts.  These are installed under DATAROOTDIR/texmf-dist/scripts; for Unix-like systems a symbolic link is made in BINDIR.  For example, a symlink points from BINDIR/ps2eps to DATAROOTDIR/texmf-dist/scripts/ps2eps/ps2eps.pl.

For Windows, a standard wrapper binary (e.g., BINDIR/ps2eps.exe) serves the same purpose.  (The source for it is in texk/texlive/w32_wrapper.)

One reason for all this is to avoid having many copies for the same script, but most importantly this allows invoking the same script on all operating systems.  Furthermore, we want the BINDIR resulting from the build to be exactly what is in the TL distribution; any deviations cause considerable extra work.

Distro builds
-------------

Although they use the same code base, building for the native TL distribution as shipped by the TL user groups is typically quite different from a ``distro'' build needed by, e.g., a full GNU/Linux or BSD operating system distribution.

The native TL distribution uses shared libraries only when absolutely necessary (libc, libm, X11 libraries, and libfontconfig).  However, a distro typically wants to use as many shared libraries as possible from elsewhere on the system, including TeX-specific libraries such as libkpathsea (even though Kpathsea has never officially been released as a shared library, but we digress).  In addition, the installation paths will, in general, be completely different.

Here are the configure options that distro builds are likely to find most relevant:

--disable-native-texlive-build
     This must be specified to avoid interference from the many tweaks we do for the native TL build.

--with-banner-add=/SOMEDISTRO
     This isn't technically required, but is strongly recommended, so your build and your distro can be distinguished from others.

--enable-shared
     Build shared versions of the TeX-specific libraries (uses libtool).

--disable-static
     Do not build the static versions of the TeX-specific libraries.

--with-system-LIB
     Use system versions for as many libraries LIB as possible. configure --help will give you the list of possibilities.

--with-LIB-includes=DIR
--with-LIB-libdir=DIR
     If needed, allows you to specify where the headers/code are for the given library LIB.

--prefix=/usr
--prefix=/opt/TeXLive
     Or whatever your convention is.  The default is /usr/local and you shouldn't install there for a distro.

--libdir=\${exec_prefix}/lib64
     May be needed for 64-bit bi-architecture (GNU/Linux) systems.

You will need to take care of the support files mentioned above (see Installing), and many other issues, such as font maps, languages, and formats, independently of the build.  Norbert Preining has written a detailed article on adapting TL for distros: http://tug.org/TUGboat/tb34-3/tb108preining-distro.pdf.  (If it needs updating at some point, perhaps we will merge it into this document.)

Infrastructure and layout
=========================

The TL source tree is the subtree rooted at Build/source of the complete TL distribution and contains the sources for all executables distributed by TL, as well as configure scripts and make rules to build and install them together with some of their support files.

Build system tools
------------------

As mentioned above (see Prerequisites), a normal build requires very little.  On the other hand, if you want to modify the TL infrastructure sources, such as configure.ac or Makefile.am files, you will need to have several additional tools installed.

In general, the TL build system uses the latest released versions of the GNU build tools, installed directly from the original GNU releases (e.g., by building them with configure --prefix=/usr/local/gnu and having PATH start with /usr/local/gnu/bin).  We have found that trying to use the versions of these tools packaged for distros causes many extra hassles, so don't do that, tempting as it may be.

Currently the versions we use are:

          [file build-tools.txt not included]

These versions should be used to update the generated files (e.g., configure or Makefile.in) in all or parts of the TL tree after their dependencies have been changed.  This can be done explicitly with the top-level reautoconf script or implicitly by using the configure option --enable-maintainer-mode.

The files in the Subversion repository (see http://tug.org/texlive/svn) are all up to date, but unfortunately this may not be reflected by their timestamps.  (For starters, be sure to set use-commit-times=yes in ~/.subversion/config or the equivalent.)

To avoid unnecessary runs of bison, flex, or makeinfo it may be necessary to touch the generated (.c, .h, or .info) files.  With --enable-maintainer-mode it may also be necessary to touch first aclocal.m4, then configure and config.h.in (or c-auto.in), and finally all Makefile.in files.  Perhaps make -t will help.

Top-level directories
---------------------

Here is a brief description of the top-level directories in the TL source tree.

As mentioned at the beginning of the chapter (see Build system), the main source directories are texk/ (TeX-specific programs and libraries), utils/ (additional programs), and libs (generic libraries).

The top-level directories am/ and m4/ contain Makefile.am fragments and Autoconf macros, respectively, used in many places.  Specifically, the file m4/kpse-pkgs.m4 contains lists of all program and library modules; missing modules are silently ignored.  (This helps in creating cut-down source trees.)

Each module contributes fragments (in separate files) defining its capabilities and requirements to the configure.ac scripts at the top-level and in the subdirectories libs, utils, and texk.  The fragments from program modules supply configure options to disable or enable building them; those from library modules specify if an installed (system) version of that library can be used.  This ultimately determines which modules need to be built---although all modules must be configured for the benefit of make targets such as dist or distcheck.

The top-level build-aux/ directory contains the common files compile, config.guess, config.sub, depcomp, etc. for most packages, pulled from the GNU Gnulib sources (http://www.gnu.org/software/gnulib), which in turn synchronizes with the appropriate ultimate upstream repository.  There are, however, independent copies in, e.g., libs/freetype2/freetype-*/builds/unix/, and similar places. The reautoconf script does not touch those, but a TL cron job keeps them in sync (nightly).

The directory extra/ contains things which are not part of the TL build, but are present for convenience, e.g., epstopdf developed here and xz/ required by the TL installer.

Autoconf macros
---------------

Here we describe some of the Autoconf macros used in several modules--not a complete list, by any means.  These general macros are supplemented by module-specific macros in directories such as texk/dvipng/m4/; some of those are described in Library modules and see Program modules.

General setup macros
....................

The TL sources use two general setup macros:

 -- Macro: KPSE_BASIC (NAME, [more-options])
     Initialize the basic TL infrastructure for module NAME:

     AM_INIT_AUTOMAKE([foreign MORE-OPTIONS])

     AM_MAINTAINER_MODE
        KPSE_COMPILER_WARNINGS

     and make sure the C compiler understands function prototypes.  This is used for all generic library and program modules.

 -- Macro: KPSE_COMMON (NAME, [more-options])
     Like KPSE_BASIC but add:
        LT_PREREQ([2.2.6])

      LT_INIT([win32-dll])
        AC_SYS_LARGEFILE

     AC_FUNC_FSEEKO
     and check for frequently used functions, headers, types, and structures.  This is used for TeX-specific modules.

Macros for programs
...................

Macros for program checks:

 -- Macro: KPSE_CHECK_LATEX
     Set LATEX to the name of the first of latex, elatex, or lambda which exists in PATH, or to no if none of them exists.  Call AC_SUBST for LATEX.  The result of this test can be overridden by setting the LATEX environment variable or the cache variable ac_cv_prog_LATEX.

 -- Macro: KPSE_CHECK_PDFLATEX
     Check for pdflatex in PATH and set PDFLATEX.

 -- Macro: KPSE_CHECK_PERL
     Check for perl or perl5 in PATH and set PERL.

 -- Macro: KPSE_PROG_LEX
     Call AC_PROG_LEX and add the flag -l for flex.

Macros for compilers
....................

Macros for compiler-related checks:

 -- Macro: KPSE_COMPILER_WARNINGS
     When using the (Objective) C/C++ compiler, set WARNING_[OBJ]C[XX]FLAGS to suitable warning flags (depending on the value given to or implied for --enable-compiler-warnings). Call AC_SUBST for them.  At the moment this only works for GNU compilers, but could be extended to others if necessary.

     This macro caches its results in the kpse_cv_warning_cflags, ... variables.

 -- Macro: KPSE_COMPILER_VISIBILITY
     When using the C or C++ compiler, try to set VISIBILITY_C[XX]FLAGS to flags to hide external symbols.  Call AC_SUBST for this variable.  At the moment this only tests for the compiler option -fvisibility=hidden, but that could be extended with more checks if necessary.

     This macro caches its results in the kpse_cv_visibility_cflags or kpse_cv_visibility_cxxflags variable.

 -- Macro: KPSE_CXX_HACK
     Provide the configure option --enable-cxx-runtime-hack.  If enabled and when using g++, try to statically link with libstdc++, somewhat improving portability of the resulting binary.

     This macro caches its result in the kpse_cv_cxx_hack variable.

Macros for libraries
....................

One macro for a library check:

 -- Macro: KPSE_LARGEFILE (VARIABLE, [extra-define])
     Call AC_SYS_LARGEFILE and AC_FUNC_FSEEKO and append suitable -D flags (optionally including -DEXTRA-DEFINE) to VARIABLE.

Macros for library and header flags
...................................

Each library module libs/LIB or texk/LIB is supplemented by a macro KPSE_LIB_FLAGS (all uppercase) that provides make variables for that library.  E.g., for libs/libpng:

 -- Macro: KPSE_LIBPNG_FLAGS
     Provide the configure option --with-system-libpng.  Set and AC_SUBST make variables for modules using this library (either an installed version or from the TL tree): LIBPNG_INCLUDES for use in CPPFLAGS, LIBPNG_LIBS for use in LDADD, LIBPNG_DEPEND for use as dependency, and LIBPNG_RULE defining make rules to rebuild the library.

 -- Macro: KPSE_ADD_FLAGS (LIBNAME)
     Temporarily extend CPPFLAGS and LIBS with the values required for the library module NAME.

 -- Macro: KPSE_RESTORE_FLAGS
     Restore CPPFLAGS and LIBS to their original values.

As an example, the configure.ac file for a hypothetical program utils/foo using libpng, and hence zlib, would contain

     KPSE_ZLIB_FLAGS
     KPSE_LIBPNG_FLAGS

and its Makefile.am would be along these lines:

     bin_PROGRAMS = foo
     AM_CPPFLAGS = ${ZLIB_INCLUDES} ${LIBPNG_INCLUDES}
     foo_LDADD = ${ZLIB_LIBS} ${LIBPNG_LIBS}
     foo_DEPENDENCIES = ${ZLIB_DEPEND} ${LIBPNG_DEPEND}
     ## Rebuild libz
     @ZLIB_RULE@
     ## Rebuild libpng
     @LIBPNG_RULE@

If it was necessary to examine whether certain zlib or libpng features were available, configure.ac should be continued this way:

     KPSE_ADD_FLAGS([zlib])
     ... # tests for zlib features, if any
     KPSE_ADD_FLAGS([libpng])
     ... # tests for libpng features
     KPSE_RESTORE_FLAGS # restore CPPFLAGS and LIBS

Macros for Windows
..................

Windows differs in several aspects from Unix-like systems, many of them due to the lack of symbolic links.

 -- Macro: KPSE_CHECK_WIN32
     Check if compiling for a Windows system.  The result is no for Unix-like systems (including Cygwin), mingw32 for Windows with GCC, or native for Windows with MSVC.  The result is cached in the kpse_cv_have_win32 variable.

 -- Macro: KPSE_COND_WIN32
     Call KPSE_CHECK_WIN32 and define the Automake conditional WIN32 (true if the value of kpse_cv_have_win32 is not no).

 -- Macro: KPSE_COND_MINGW32
     Call KPSE_COND_WIN32 and define the Automake conditional MINGW32 (true if the value of kpse_cv_have_win32 is mingw32).

 -- Macro: KPSE_COND_WIN32_WRAP
     Call KPSE_COND_WIN32 and define the Automake conditional WIN32_WRAP (true if the standard Windows wrapper (texk/texlive/w32_wrapper/runscript.exe) exists.  This wrapper is used on Windows instead of symlinks for the ``linked scripts'' (see Linked scripts).

 -- Macro: KPSE_WIN32_CALL
     Call KPSE_COND_WIN32, check if the file texk/texlive/w32_wrapper/callexe.c exists; if it does, create a symlink in the build tree.  Compiling callexe.c with -DEXEPROG='"FOO.exe"' and installing callexe.exe as BAR.exe is used on Windows instead of a symlink BAR->FOO for Unix-like systems.

Library modules
---------------

Here we discuss some specifics for a few of the libraries in TL, both for the details themselves, and as a way of illuminating the general structure and variation.

The png library in libs/libpng
..............................

This generic library uses the source tree in, e.g., the subdirectory libpng-1.5.17 with all modifications for TL recorded in libpng-1.5.17-PATCHES/*.  The configure.ac fragment ac/withenable.ac contains

     KPSE_WITH_LIB([libpng], [zlib])

specifying the module name, and indicating the dependency on zlib.  A third literal argument tree would specify that the library from the TL tree cannot be replaced by a system version.  That not being the case here, a second fragment ac/libpng.ac contains

     KPSE_TRY_LIB([libpng],
                  [#include <png.h>],
                  [png_structp png; png_voidp io; png_rw_ptr fn;
     png_set_read_fn(png, io, fn);])

thus providing the simple C code

     #include <png.h>
     int main ()
     { png_structp png; png_voidp io; png_rw_ptr fn;
       png_set_read_fn(png, io, fn);
       return 0; }

which Autoconf uses to verify the usability of a system version with C code.  The analogous macro KPSE_TRY_LIBXX would check using C++ code.  These fragments are included by configure.ac at the top level.

For this library, among many other modules, a proxy build system for TL is used (configure.ac, Makefile.am, and include/Makefile.am), ignoring the distributed one. Consequently, a few generated files and auxiliary scripts are removed from the distributed source tree.  The public headers png.h, pngconf.h, and pnglibconf.h are ``installed'' (as symlinks) under include/ in the build tree exactly as they are for a system version under, e.g., /usr/include/.

The module is supplemented by the file m4/kpse-libpng-flags.m4 that defines the M4 macro KPSE_LIBPNG_FLAGS used by all modules depending on this library in their configure.ac to generate the make variables LIBPNG_INCLUDES for use in CPPFLAGS, LIBPNG_LIBS for use in LDADD, LIBPNG_DEPEND for use as dependencies, and LIBPNG_RULE defining make rules to rebuild the library.

m4/kpse-libpng-flags.m4 also supplies the configure option --with-system-libpng and uses pkg-config to determine the flags required for the system library.

The zlib library in libs/zlib
.............................

This generic library is very much analogous to libpng, but without the dependency on any other library.  The file m4/kpse-zlib-flags.m4 supplies the configure option --with-system-zlib, as well as --with-zlib-includes and --with-zlib-libdir to specify non-standard locations of the zlib headers and/or library.

The freetype library in libs/freetype2
......................................

This module uses a wrapper build system with an almost trivial configure.ac and with a Makefile.am that invokes configure and make for the distributed source, followed by make install with the build tree as destination.  The flags required for the system library are obtained through freetype-config.

The kpathsea library in texk/kpathsea
.....................................

This is one of the TeX-specific libraries that are maintained as part of TL (see Kpathsea).  Despite being a core part of the TeX system, it is not a terribly special case in the infrastructure.  The TeX libraries are Libtool libraries (static and/or shared) and are installed by make install together with the programs.  They are, however, not part of the TL DVD as distributed by TeX user groups, and have never been officially released for standalone use.

It is possible, and possibly even useful for distro builds (see Distro builds, to specify the configure option --with-system-kpathsea in order to use a system version of the library and it may then be necessary to specify --with-kpathsea-includes and/or --with-kpathsea-libdir.

In addition to ac/withenable.ac and ac/kpathsea.ac there is a third fragment ac/mktex.ac included by both ac/withenable.ac and configure.ac that supplies configure options such as --enable-mktextfm-default, which determine the compile time default of whether or not to run mktextfm to generate a missing .tfm file.  In any case, however, the command line options -mktex=tfm or -no-mktex=tfm for the TeX-like engines override this default.

Program modules
---------------

As with libraries (see Library modules), here we discuss the details for a few of the programs in TL.

The t1utils package in utils/t1utils
....................................

Once again we use the distributed source tree t1utils-1.38 with modifications documented in t1utils-1.38-PATCHES/* and a proxy build system consisting of configure.ac and Makefile.am. The fragment ac/withenable.ac contains

     KPSE_ENABLE_PROG([t1utils])

specifying the module name without any dependencies, and supplies the configure option --disable-t1utils.

The xindy package in utils/xindy
................................

This module uses the distributed source tree xindy-2.4 with modifications documented in xindy-2.4-PATCHES/*, a proxy configure.ac, and a wrapper Makefile.am that descends into xindy-2.4.  The xindy build requires that the distributed Makefiles allow a VPATH build, can handle all targets, and do not refer to ${top_srcdir} or ${top_builddir}.  The fragment ac/withenable.ac contains

     KPSE_ENABLE_PROG([xindy], , [disable native])
     m4_include(kpse_TL[utils/xindy/ac/xindy.ac])
     m4_include(kpse_TL[utils/xindy/ac/clisp.ac])

where disable in the third argument indicates that xindy is only built if explicitly enabled by the user with configure --enable-xindy (the need for clisp is too painful to require by default), and native disallows cross compilation.  The additional fragments ac/xindy.ac and ac/clisp.ac specify more configure options to be seen at the top level with ac/xindy.ac also included by configure.ac.

The xdvik package in texk/xdvik
...............................

This package is maintained as part of the TL tree with sources in its top level directory and the subdirectory gui.  The fragment ac/withenable.ac contains

     dnl extra_dirs = texk/xdvik/squeeze
     KPSE_ENABLE_PROG([xdvik], [kpathsea freetype2], [x])
     m4_include(kpse_TL[texk/xdvik/ac/xdvik.ac])

thus specifying the dependency on the kpathsea, freetype, and X11 libraries.  The M4 comment (following dnl) signals the subsidiary squeeze/configure.ac.  This is needed because the main executable xdvi-bin (to be installed as, e.g., xdvi-xaw) is for the host system whereas the auxiliary program squeeze/squeeze has to run on the build system and in a cross compilation they differ.

The additional fragment ac/xdvik.ac is also included by configure.ac and supplies the configure option --with-xdvi-x-toolkit also seen at the top-level.

The subdirectory utils/asymptote
................................

This subdirectory contains the sources for asy and xasy but due to its complexity and prerequisites (e.g., OpenGL) it is not part of the TL build system.  These programs must be built and installed independently, but are included on the TL DVD together with their support files.

Extending TeX Live
------------------

This section outlines the basic process for adding new packages to the TL build system.

Adding a new program module
...........................

A TeX-specific program module in a subdirectory texk/PROG may use the TeX-specific libraries and is included by adding its name PROG to the M4 list kpse_texk_pkgs defined in m4/kpse-pkgs.m4.

A generic program module in a subdirectory utils/PROG must not use the TeX-specific libraries and is included by adding its name PROG to the M4 list kpse_utils_pkgs in m4/kpse-pkgs.m4.

In either case, apart from the program sources and build system (configure.ac and Makefile.am), the subdirectory texk/PROG or utils/PROG must provide a fragment ac/withenable.ac that contains the M4 macro KPSE_ENABLE_PROG defined in m4/kpse-setup.m4 with PROG as the mandatory first argument and three optional arguments:

 1. a list of required libraries from the TL tree;

 2. a list of options (disable if this module is not to be built without the configure option --enable-PROG, native if cross compilation is not possible, x if the program requires X11 libraries);

 3. and a comment added to the help text for the configure option --enable-PROG or --disable-PROG.

If the module requires specific configure options to be seen at the top-level, they should be defined in an additional fragment ac/PROG.ac included from ac/withenable.ac and configure.ac.

Adding a new generic library module
...................................

A generic library module in a subdirectory libs/LIB must not depend on TeX-specific libraries, by definition.  It is included by adding its name LIB to the M4 list kpse_libs_pkgs in m4/kpse-pkgs.m4---before any other libraries from the TL tree on which it depends.

As with program modules, the subdirectory libs/LIB must contain the sources and build system for the library (and any installable support programs) and a fragment ac/withenable.ac that contains the M4 macro KPSE_WITH_LIB defined in m4/kpse-setup.m4 with LIB as the mandatory first argument and two optional arguments: a list of required libraries from the TL tree, and a list of options (currently there is only one: specify tree if this library cannot be replaced by a system version).

If a system version can be used, a second fragment ac/LIB.ac is needed, containing the M4 macro KPSE_TRY_LIB (or KPSE_TRY_LIBXX) with LIB as the mandatory first argument and two additional arguments for the Autoconf macro AC_LANG_PROGRAM used to compile and link a small C (or C++) program as sanity check for using the system library.

In addition a file m4/kpse-LIB-flags (at the top level) must define the M4 macro KPSE_LIB_FLAGS (all uppercase) setting up the make variables LIB_INCLUDES, LIB_LIBS, LIB_DEPEND, and LIB_RULE with the values required for CPPFLAGS, LDADD, dependencies, and a (multi-line) make rule to rebuild the library when necessary.  All of that is needed for the library from the TL tree and, if supported, for a system version.

If a system library is allowed, KPSE_LIB_FLAGS also provides the configure option --with-system-LIB and uses the additional M4 macro KPSE_LIB_SYSTEM_FLAGS to generate the make variables for a system library.  Furthermore the definition of the M4 macro KPSE_ALL_SYSTEM_FLAGS in m4/kpse-pkgs.m4 must be extended by the line:

AC_REQUIRE([KPSE_LIB_SYSTEM_FLAGS])

Adding a new TeX-specific library module
........................................

A TeX-specific library module in a subdirectory texk/LIB may depend on other TeX-specific libraries but must not depend on any generic library from the TL tree.  It is included in the same general ways as a generic library (see the previous section), with these modifications:

  * The library name LIB is added to the M4 list kpse_texlibs_pkgs also in m4/kpse-pkgs.m4.

  * The fragment ac/withenable.ac must use KPSE_WITH_TEXLIB.

Configure options
=================

Corresponding to the large number of program and library modules there are a large number configure options, most of which are described here.  The command
   configure --help
at the top level gives an exhaustive list of all global options and a few important module-specific ones, whereas, e.g.,

texk/lcdf-typetools/configure --help
 also displays the lcdf-typetools specific options, which are not shown by the top-level --help.

The help text also mentions several influential environment variables, but for TL it is better to specify them as assigments on the command line.

The ./Build script used to make the binaries shipped with TL invokes the top-level configure with a few additional options (see Building).  The defaults discussed below are those for the actual configure script; invoking configure via ./Build may yield different results.

Defaults for most options are set at the top level and propagated explicitly to all subdirectories.  Options specified on the command line are checked for consistency but never modified.

Global configure options
------------------------

Here are the global configure options.

--disable-native-texlive-build
..............................

If enabled (the default), build for a TL binary distribution as shipped by the TeX user groups.  This requires GNU make and implies --enable-multiplatform and --enable-cxx-runtime-hack (unless they are explicitly disabled), and enforces --disable-shared.

If building TL for a GNU/Linux or other distribution, this should be disabled and system versions of most libraries would be used (see Distro builds).  This may fail without GNU make, but will be tried regardless.

A related option, --enable-texlive-build, is automatically passed to all subdirectories (and cannot be disabled).  Subdirectories that can also be built independently from the TL tree (such as utils/xindy and texk/dvipng) can use this option, e.g., to choose TL-specific installation paths.

--prefix, --bindir, ...
.......................

These standard Autoconf options specify various installation directories as usual.  For the complete list, see Installation directories.

Also as usual, all values are prefixed by the value of DESTDIR, if set, on the make command line (see Installation in a temporary location (GNU Automake)).

--disable-largefile
...................

Omit large file support (LFS), needed on most 32-bit Unix systems for files with 2GB or more.  Regardless of this, the size of DVI and GF files must always be <2GB, due to the file format specifications.

With LFS, there is no fixed limit on the size of PDF files created by pdftex or PostScript files created by dvips.  The size of PDF images included by pdftex must, however, be <4GB when using xpdf and <2GB when using older versions of poppler (even on 64-bit systems with LFS), whereas poppler versions 0.23 and later impose no such limit.

--disable-missing
.................

Immediately terminate the build process if a requested program or feature must be disabled, e.g., due to missing libraries.  This can help when figuring out a specific (sub)set of modules to enable.

--enable-compiler-warnings=LEVEL
................................

Enable various levels of compiler warnings for (Objective) C and C++: the LEVEL value can be one of: no min yes max all. The default is yes in maintainer-mode (see below) and min otherwise.  This option defines WARNING_[OBJ]C[XX]FLAGS but these flags are not used in all library and program modules.  Using them should help to resolve portability problems.

At present, these warning flags are only defined for the GNU compilers but flags for other compilers could be added when needed.

--enable-cxx-runtime-hack
.........................

If enabled (as it is for the native TL build) and when using g++, try to statically link with libstdc++, somewhat improving portability of the resulting binary.  See Macros for compilers.

--enable-maintainer-mode
........................

Enable make rules and dependencies not useful (and sometimes confusing) to the casual user.  This requires current versions of the GNU build tools (see Build system tools), as it automatically rebuilds infrastructure files as needed.  See missing and AM_MAINTAINER_MODE (GNU Automake).

--enable-multiplatform
......................

If enabled, install executables and libraries in per-platform subdirectories of EPREFIX/bin and EPREFIX/lib where EPREFIX is the value given or implied for exec_prefix.  This can be overridden by explicitly --bindir=DIR or --libdir=DIR.  In any case, the values for bindir and libdir are automatically propagated to all subdirectories.

--enable-shared
...............

Build shared versions of the TeX-specific libraries such as libkpathsea.  This is not allowed for a native TL build (i.e., --disable-native-texlive-build must also be specified).

--enable-silent-rules
.....................

Enable the use of less verbose build rules.  When using GNU make (or another make implementation supporting nested variable expansions), you can specify V=1 on the make command line to get more verbosity, or V=0 to get less, regardless of this option.

--without-ln-s
..............

Required when using a system without a working ln -s to build binaries for a Unix-like system.  However, make install will not create anything useful and might even fail.

--without-x
...........

Disable all programs using the X Window System.

Program-specific configure options
----------------------------------

Here are (some of) the program-specific configure options.

--enable-PROG, --disable-PROG
.............................

Do or do not build and install the program(s) of the module PROG.

--disable-all-pkgs
..................

Do not build any program modules by default---only those explicitly enabled.  Without this option, all modules are built except those that are explicitly disabled or specify disable in their ac/withenable.ac fragment.

Configure options for texk/web2c
................................

--with-banner-add=STR
Add STR to the default version string (TeX Live YEAR or Web2C YEAR) appended to banner lines.  This is ignored for a native TL build, but distro builds should specify, e.g., /SOMEDISTRO.

--with-editor=CMD
Specify the command CMD to invoke from the e option of TeX-like engines, replacing the default vi +%d '%s' for Unix or texworks --position=%d "%s" for Windows.

--with-fontconfig-includes=DIR, --with-fontconfig-libdir=DIR
Building XeTeX on non-Mac systems requires the fontconfig library headers and code.  If one or both of these options are given, the required flags are derived from them; otherwise, they are determined via pkg-config (if present).

--with-mf-x-toolkit
Use the X toolkit (libXt) for Metafont (the default is to use the lowest-level Xlib support; it seems this has the best chance of working across X installations nowadays).

--disable-dump-share
Make the fmt/base dump files architecture dependent (somewhat faster on LittleEndian architectures).

--disable-ipc
Disable TeX's --ipc option.

--disable-mf-nowin
Do not build a separate non-graphically-capable Metafont (mf-nowin).

--disable-omfonts
Build the WEB versions of the Omega font utilities ofm2opl, opl2ofm, ovf2ovp, and ovp2ovf instead of the C version omfonts. The WEB and C versions should be roughly equivalent.

--disable-tex, --enable-etex, ...
Do not or do build the various TeX, Metafont, and MetaPost engines (defaults are defined in the fragment texk/web2c/ac/web2c.ac).

--disable-web-progs
Do not build the core WEB programs bibtex, ..., weave.  Useful if, e.g., you only want to (re)build some engines.

--enable-auto-core
This option causes TeX and Metafont to produce a core dump when a particular hacky filename is encountered, for use in creating preloaded binaries.  This is rarely done nowadays.

--enable-libtool-hack

If enabled (which is the default for all platforms), prevents libtool from linking explicitly with dependencies of libfontconfig such as libexpat.

--enable-*win
Include various types of other window support for Metafont (EPSF output, mftalk, old terminals, ...).

--enable-tex-synctex, --disable-etex-synctex, ...
Build the TeX engines with or without SyncTeX support; ignored for a native TL build, defaults are again defined in texk/web2c/ac/web2c.ac.

Configure options for texk/bibtex-x
...................................

The former programs bibtex8 and bibtexu have been merged into the module bibtex-x (extended BibTeX).

--disable-bibtex8
Do not build the bibtex8 program.

--disable-bibtexu --disable-bibtexu
Do not build the bibtexu program (building bibtexu requires ICU libraries).

Configure options for texk/dvipdfm-x
....................................

The former modules dvipdfmx (extended DVI to PDF converter) and xdvipdfmx (the same, as used by XeTeX) have been merged into dvipdfm-x.

--disable-dvipdfmx
Do not build the dvipdfmx program.

--disable-xdvipdfmx
Do not build the xdvipdfmx program (building xdvipdfmx requires the freetype library).

Configure options for texk/dvisvgm
..................................

--with-system-libgs
Build dvisvgm using installed Ghostscript (gs) headers and library (not allowed for a native TL build).  The default is to load the gs library at runtime if possible, or otherwise disable support for PostScript specials.

--without-libgs
Build dvisvgm without PostScript support at all.  Because the dynamic loading just mention defeats all attempts at static linking, the result can crash due to library incompatibilities, e.g., on CentOS 5.

--with-libgs-includes=DIR, --with-libgs-libdir=DIR
Specify non-standard locations of the Ghostscript headers and library.

Configure options for texk/texlive
..................................

--disable-linked-scripts
Do not install the ``linked scripts'' (see Linked scripts), except for the TL scripts required to run texlinks.

Configure options for texk/xdvik
................................

--with-gs=FILENAME
Hardwire the location of Ghostscript (gs).

--with-xdvi-x-toolkit=KIT
Use toolkit KIT for xdvik, one of: motif xaw xaw3d neXtaw.  The default is motif if available, else xaw.

Configure options for utils/xindy
.................................

--enable-xindy-rules
Build and install xindy rules (default: yes, except for a native TL build).

--enable-xindy-docs
Build and install xindy documentation (default: yes, except for a native TL build).

--with-clisp-runtime=FILENAME
Specifies the Full path for the CLISP runtime file (lisp.run or lisp.exe) to be installed.  When specified as default (the default for a native TL build) the path is determined by the CLISP executable; the value system (not allowed for a native TL build, but the default for a non-native one) indicates that xindy will use the installed version of clisp (which must be identical to the one used to build xindy).

--with-recode
Use recode instead of iconv to build the xindy rules and documentation, required for some systems where iconv is missing or broken.

Library-specific configure options
----------------------------------

Here are (some of) the library-specific configure options, starting with this generic one:

--with-system-LIB

Use an installed (system) version of the library LIB; this option exists for most libraries, but is not allowed for a native TL build.  Using a system version implies also using the system versions of all libraries (if any) that LIB depends on.

For many libraries --with-LIB-includes=DIR and --with-LIB-libdir=DIR to specify non-standard search locations; others use pkg-config or similar to determine the required flags.

The top-level configure script performs a consistency check for all required system libraries and bails out if tests fail.

Configure options for kpathsea
..............................

--enable-CMD-default, --disable-CMD-default
Determine the compile time default whether or not to run CMD, one of:

mkocp
     (Omega compiled translation process file)

mkofm
     (Omega font metrics file)

mktexfmt
     (format/base dump file)

mktexmf
     (Metafont source)

mktexpk
     (PK bitmap font)

mktextex
     (TeX source)

mktextfm
     (TFM file)

to generate the specified type of file dynamically. The default can be overridden by the user in any case.

Configure options for system poppler
....................................

Building LuaTeX and XeTeX requires poppler, either from the TL tree or system headers and library.  Building pdfTeX requires either xpdf from the TL tree or system poppler headers and library.

--with-system-poppler
Use a system version (0.18 or newer) of poppler for LuaTeX and XeTeX, and use pkg-config to obtain the required flags.

--with-system-xpdf
Use a system version (0.12 or better) of poppler (and pkg-config) for pdfTeX instead of xpdf from the TL tree.  See --disable-largefile.

Variables for configure
-----------------------

The values for these variables can be specified as configure arguments of the form VAR=VALUE.  They can also be defined in the environment, but that might not work for cross compilations.

CC
CXX
CPPFLAGS
     And plenty more.  As usual with Autoconf, these variables specify the name (or full path) of compilers, preprocessor flags, and similar. See autoconf (GNU Autoconf).

CLISP
     Name (or full path) of the clisp executable, used to build xindy.

FT2_CONFIG
ICU_CONFIG
PKG_CONFIG
     These specify the name (or path) for the freetype-config, icu-config, and pkg-config commands used to determine the flags required for system versions of libfreetype, the ICU libraries, or many other libraries.

KPSEWHICH
     Name (or path) of an installed kpsewhich binary, used by make check to determine the location of, e.g., cmbx10.tfm.

MAKE
SED
     And more. Name (or path) of the make, sed, and similar programs; used at the top level and propagated to all subdirectories.

PERL
LATEX
PDFLATEX
     Name (or full path) for the perl, latex, and pdflatex commands used, e.g., to build the xindy documentation.

TL_PLATFORM
     The utility program biber consists of many Perl modules bundled into an executable by the Par::Packer mechanism.  Therefore it is not feasible to build biber as part of TL.

     However, in order for make install to create all executables in bindir, the biber module in TL contains pre-made binaries (by biber contributors, released on the original biber site) for the platforms for which they are available.  TL merely checks if an executable is present for the current platform, and if so, installs it.

     The complication is hidden in the phrase ``current platform''.  TL has its own ideas about platform names, and the mapping from the canonical system name determined by config.guess or config.sub to the TL platform name is not trivial.  So, a value given for TL_PLATFORM is used as the TL platform name.  Otherwise, the build uses copies of the Perl modules TeXLive/TLUtils.pm and TeXLive/TLConfig.pm, to avoid duplicating the platform-detection logic.

Cross compilation
=================

In a cross compilation a build system is used to create binaries to be executed on a host system with different hardware and/or operating system.

In simple cases, the build system can execute binaries for the host system.  This typically occurs for bi-arch systems where, e.g., i386-linux binaries can run on x86_64-linux systems and win32 binaries can run on win64 systems.  Although sometimes called ``native cross'', technically this is not cross compilation at all.  In most such cases it suffices to specify suitable compiler flags.  It might be useful to add the configure option --build=HOST to get the correct canonical host name, but note that this should not be --host=HOST (see Hosts and Cross-Compilation (GNU Autoconf)).

In order to build, e.g., 32-bit binaries with clang on a 64-bit OSX system one could use
    ./Build --build=i386-apple-darwin
CC='clang -arch i386' \

CXX='clang++ -arch i386' OBJCXX='clang++ -arch i386'

Cross configuring
-----------------

In a standard cross compilation, binaries for the host system cannot execute on the build system and it is necessary to specify the configure options --host=HOST and --build=BUILD with two different values.

Building binaries requires suitable ``cross'' tools, e.g., compiler, linker, and archiver, and perhaps a ``cross'' version of pkg-config and similar to locate host system libraries. Autoconf expects that these cross tools are given by their usual variables or found under their usual name prefixed with HOST-.  Here a list of such tools and corresponding variables:

     ar                AR
     freetype-config   FT2_CONFIG
     g++               CXX
     gcc               CC
     icu-config        ICU_CONFIG
     objdump           OBJDUMP
     pkg-config        PKG_CONFIG
     ranlib            RANLIB
     strip             STRIP

In order to, e.g., build mingw32 binaries on x86_64-linux with a cross compiler found as i386-pc-mingw32-gcc one would specify

     --host=i386-pc-mingw32 --build=x86_64-linux-gnu

or perhaps

     --host=mingw32 --build=x86_64-linux CC=i386-pc-mingw32-gcc

but this latter, especially, might require adding CXX and others.

Configure arguments such as CFLAGS=... refer to the cross compiler.  If necessary, you can specify compilers and flags for the few auxiliary C and C++ programs required for the build process as configure arguments

     BUILDCC=...
     BUILDCPPFLAGS=...
     BUILDCFLAGS=...
     BUILDCXX=...
     BUILDCXXFLAGS=...
     BUILDLDFLAGS=...

Cross problems
--------------

The fact that binaries for the host system cannot be executed on the build system causes some problems.

One problem is that configure tests using AC_RUN_IFELSE can compile and link the test program but cannot execute it.  Such tests should be avoided if possible and otherwise must supply a pessimistic test result.

Another problem arises if the build process must execute some (auxiliary or installable) programs.  Auxiliary programs can be placed into a subdirectory that is configured natively as is done for texk/web2c/web2c, texk/dvipsk/squeeze, and texk/xdvik/squeeze.  The module libs/freetype uses the value of CC_BUILD, BUILD-gcc, gcc, or cc as compiler for the auxiliary program.

The situation for installable programs needed by the build process is somewhat different.  A quite expensive possibility, chosen for the ICU libraries in module libs/icu, is to first compile natively for the build system and in a second step to use these (uninstalled) programs during the cross compilation.

This approach would also be possible for the tools such as tangle used in the module texk/web2c to build the WEB programs, but that would require first building a native kpathsea library.  To avoid this complication, cross compilation of the WEB or CWEB programs requires sufficiently recent installed versions of tangle, ctangle, otangle, and tie.

Building xindy requires running the host system clisp binary, thus cross compilation is not possible.

Coding conventions
==================

Ideally, building all of TL with --enable-compiler-warnings=max should produce no (GCC) compiler warnings at all.  In spite of considerable efforts into that direction we are still far from that goal and there are reasons that we may never fully reach it.  Below are some rules about declarations of functions or variables and the use of const. These rules should be applied to all parts of the TL tree, except some of those maintained independently.

Declarations and definitions
----------------------------

C standards
...........

The TL build system no longer supports pre-ANSI C compilers.  Thus all function prototypes and definitions must conform to the ANSI C standard (including void in the declaration of C functions with no parameters).  On the other hand, TL is built for a wide variety of systems, not all of which support the C99 standard.  Therefore using C99 features should be avoided if that can easily be done.  In particular C code must not contain declarations after statements or C++-style comments.

If some C99 (or later) constructs must be used, the module should verify that they are available and otherwise provide an alternative. For example, the module texk/chktex uses the C99 function stpcpy() that may or may not be available on a particular system.  It uses AC_CHECK_DECLS([stpcpy]) in configure.ac to test this, and provides the perhaps slightly less efficient alternative

     #if !(defined HAVE_DECL_STPCPY && HAVE_DECL_STPCPY)
     #static inline char *stpcpy(char *dest, const char *src)
     {
       return strcpy(dest, src) + strlen(src);
     }
     #endif

in the file Utility.h.

Static functions
................

Functions used in only one file should be declared static; they require no prototype except as forward declaration.

Extern functions
................

Functions not declared static, usually because they are used in several files, require an (extern) prototype in exactly one header file, which is included in the file defining the function and in all files using that function---this is the only way to guarantee consistency between definition and use.  There should be no extern declarations sprinkled throughout the C code (with or without comments as to where that function is defined).

Variable declarations
.....................

The declaration of global variables follows analogous rules: they are either declared static if used in only one file or declared extern in exactly one header and instantiated in exactly one file.

Const
-----

The const feature of C is valuable, but easy to mis-use.

Function parameters
...................

Ideally, a function parameter not modified by the function should be declared as const.  This is important in particular for strings (char*) because the actual arguments are often string literals. It is perfectly legitimate and safe to use a type char* value for a type const char* variable (in an assignment, as initializer, as function argument, or as return value).  It is equally safe to use a type char** value for a type const char*const* variable, but not for a type const char** variable since that might cause modification of a quantity supposed to be constant.

Getting all const qualifiers right can get quite involved but can almost always be done.  There are only a couple notable exceptions: the X11 headers are full of declarations that ought to use const but do not, and the same is true to some extent for libfreetype (but, thankfully, not for zlib nowadays).

What must be avoided with const
...............................

The GCC compiler warnings ``assignment discards qualifiers...'' and analogous warnings for ``initialization'', ``passing arg'', or ``return'' must be strenously avoided in our own code.  The only exception is when they are caused by X11 headers or macros or other third party code.

What should be avoided with const
.................................

A type cast, e.g., from const char* to char* does not solve any problems; depending on warning options, it may only hide them.  Therefore such casts should be avoided whenever possible and otherwise must be carefully analyzed to make sure that they cannot cause the modification of quantities supposed to be constant.