$Id$ (This document is public domain.) How to set up your own TeX Live Packages repository =================================================== TeX Live 2008 and later contain the TeX Live Manager, which allows updates over the network. By specifying different installation sources the user can in theory install packages using tlmgr from a different source as the default TeX Live repository. This document describes how to set up a repository in a way that tlmgr can use it. Possible applications include distribution of company-internal packages or non-free stuff not included in TeX Live. The current document is based on the status of the TeX Live repository scripts as of 2018-02-13. Please write tex-live@tug.org if questions or suggestions. Contents -------- 0. Preliminaries 1. Prerequisites 2. Preparing the "repository" 3. tlpsrc files 4. Creating the TeX Live Database 5. Adjusting the revision numbers 6. Making the web distribution 7. Testing your distribution 8. Signing repositories 9. Example update script 0. Preliminaries ---------------- First of all, before you start with providing your own packages, please be reminded that you should take good care not to - overwrite files which are in a different package The current system does NOT check for files that got overwritten - take care not to hose the rest of the TeX Live system, imagine you install a package that removes/deletes tlmgr -- the user will hardly be able to revert to any working state! - test, test, and again TEST before shipping your stuff - might be good to alert us at tex-live@tug.org so that we can take a look and comment, or check for other problems. Also, if we change something in the infrastructure you have a better chance in getting notifications and better treatment ;-) In principle what you have to do is to set up some directory in a layout paralleling the one in our repository under Master. See http://www.tug.org/texlive/svn/ for details on accessing our repo. That means, with ROOT as the root of your local "repository": ROOT/tlpkg/ infrastructure stuff, modules, scripts ROOT/texmf-dist/ stuff where you should put your TeX code, fonts etc After that you call the very same scripts as we do when building the distribution database in ROOT/tlpkg/texlive.tlpdb Finally you create/update your web distribution from that ROOT. The repository can then be used either on a one-time basis with tlmgr --repository URL-of-your-repo or by permanently adding it to the used repositories with tlmgr repository add URL-of-your-repo Please see the documentation of tlmgr (tlmgr -help or man tlmgr) for details. 1. Prerequisites ---------------- You need from the TeX Live Repository (see http://www.tug.org/texlive/svn/) the contents of at least Master/tlpkg/TeXLive the perl modules Master/tlpkg/bin the scripts for managing These files can be checked out via svn, or downloaded from the net. We recommend svn, so that you can easily update them regularly. Note that it is not necessary to have the above files and the files you are distributing in the same file hierarchy. It is completely valid to have a full or partial checkout of TeX Live (svn or git) and use the scripts from there. See Section 9 for an example management script. 2. Preparing your repository ---------------------------- You will have to deal with two different folders on your system: * The preparation place, where you collect the files of the packages and create the necessary configuration files. We will refer to this folder as ROOT In TeX Live this would be the subversion repository. * The generated repository which will be synced to some web server and accessed via tlmgr. This folder will be generated and maintained by the scripts mentioned in this document. We will refer to this folder as NETROOT In TeX Live this would be the 'tlnet' folder. Note that ROOT and NETROOT should be different! Put all of YOUR files in a TDS compliant way into the texmf-dist folder: ROOT/texmf-dist Usage of 'texmf' as tree is strongly discouraged but still possible. IMPORTANT: Create the directory ROOT/texmf-dist/web2c (in case of a git repository put in a dummy .dont-delete-me to make sure the directory is present). tl-update-tlpdb and tl-update-containers rely on this directory existence to determine the type of repository (files based or container based). If you really want to replace packages with files in texmf, please think twice, and maybe contact us before. 3. tlpsrc files --------------- Now you have to create several TeX Live Package Source files, .tlpsrc files and place them into ROOT/tlpkg/tlpsrc You have to have two .tlpsrc file for each package you want to ship, and furthermore three configuration files for the overall installation: 00texlive.config.tlpsrc 00texlive.installation.tlpsrc These files can be copied from the main TeX Live repository, and only 00texlive.config.tlpsrc needs adjustment. This package defines some basic configurations for the tlpdb, like how it announces itself, whether separate src and doc containers are build, and the release years it supports. In the end the generate "package" 00texlive.config should contain (at least) the following lines: category TLCore depend container_format/xz depend release/NNNN-YOUR_ID and might optionally contain: depend container_split_src_files/1 depend container_split_doc_files/1 depend minrelease/MMMM-YOUR_ID The depend container_format/xz is *obligatory* because currently we do NOT support any other format. The "container_split_src_files" and "container_split_doc_files" are options, they specify whether at a later stage the container script will generate separate .tar.xz archives for source and doc files, or put all of them into one .tar.xz. It is up to you. TeX Live uses splitting of src and doc files. For users there is hardly any difference. The default is splitting source and doc files out. Finally, the depend release/NNNN-YOUR_ID and (optionally) depend minrelease/MMMM-YOUR_ID determine(s) which release(s) of TL your repository works with. If minrelease is present, the year specified in the local (user's) installation's $TLConfig::ReleaseYear must be in the range [minrelease,release]. If minrelease is not present, the release/NNNN value must exactly match the TL release number NNNN. For example, you might have: depend release/2010-myorg and if your packages work with both TL'09 and TL'10, you could add: depend minrelease/2009-myorg For the rest of your package you should create .tlpsrc files describing their content. Either copy one of our .tlpsrc files, or (much better) PLEASE read the documentation of TLPSRC.pm (in POD format). For local packages or those that are not in the TeX catalogue it is nice to provide descriptions in the two fields shortdesc longdesc again, see the TLPSRC.pm documentation. If you want to use the auto-generated patterns we are using in TeX Live you have to copy our 00texlive.autopatterns.tlpsrc into your tlpkg/tlpsrc directory. Some guidelines: For simple packages, say a LaTeX package foobar, put all the files in ROOT/texmf-dist/tex/latex/foobar/ ROOT/texmf-dist/source/latex/foobar/ ROOT/texmf-dist/doc/latex/foobar/ then create a two-line .tlpsrc file foobar.tlpsrc containing: category Package (that is all). The rest will be automatically done by our scripts. If you want to understand that, read the section on auto-generated patterns in the TLPSRC.pm documentation (and don't forget to copy 00texlive.autopatterns.tlpsrc). For more documentation concerning patterns and autopatterns, see the POD documentation in TLPSRC.pm. 4. Creating the TeX Live Database --------------------------------- If you have set up all your local files, and the tlpsrc files, you can create the TeX Live Database in ROOT/tlpkg/texlive.tlpdb by calling perl PATH-TO/tl-update-tlpdb OPTIONS from within ROOT. Here PATH-TO points to the directory Master/tlpkg/bin in your TeX Live checkout, or where you have saved the files from TeXLive and tlpkg directories. How revision numbers are computed and which files are used can be controlled by one of the several -from-XXXX options. By default Subversion is used. Other options, if your repository is not subversion based, are -from-files use find to get list of files, revisions are all 1 -from-git use git and compute the history (number of commits) -from-gitsvn use git over svn repo By default tl-update-tlpdb uses all tlpsrc files present in ROOT/tlpkg/tlpsrc/. Presumably this won't be very many files. But if you like, you can specify individual package names together with -tlpsrc-from-cmdline. You can add the option -with-w32-pattern-warning which will warn on missing files for Windows; some of those warnings are by default suppressed. Another option that might be of interest is -no-reverse-revision-check Normally if there is an old tlpdb the packages in the new one are compared with the packages in the old one and checked that the revisions are increasing. Passing in -master ROOT should allow you to run the script from an arbitrary directory, but we recommend running it from within the ROOT folder. As usual, please read the documentation of tl-update-tlpdb (again in POD) for further information. That process will probably show you some warnings; read them carefully. After it runs successfully, you should have a file ROOT/tlpkg/texlive.tlpdb Your next job is to check that your new texlive.tlpdb has the files you expected in the corresponding packages that you expected. Go through the entire file (each package starts with a line "name foobar") and the file list.. If you are not content, work on your patterns to fix that. There are some scripts in tlpkg/bin/ starting with "check-" which you might use for doing some checks on your repository. 5. Adjusting the revision numbers --------------------------------- If you used -from-files above all the revision numbers in the generated texlive.tlpdb will be set to 0: revision 0 The revision is used for checking which package is newer. So if you have different "releases" of a package you should increase the revision number from one step to the next. If you want to change that, there are several options: - do it by hand in the texlive.tlpdb ... if you can remember what numbers you already shipped - do it automatically with some script which takes into account the already shipped version of the packages, or the version of the package in the main TeX Live repository (if you replace a TeX Live package) An example for doing that take a look at our repository tlpkg/etc/tlrepo-from-minimals.pl - use either subversion or git for your repository and get automatically increasing numbers. 6. Making the web distribution ------------------------------ Now that the texlive.tlpdb has been set up, revision numbers adjusted, you can create or update your web distribution with perl PATH-TO/tl-update-containers OPTIONS OPTIONS: -location NETROOT The -location gives the place where your former distribution was or where it should be created. In the former case only those containers with larger revision numbers are recreated. In the latter case all containers are created. -master ROOT The option -master defines the location of *your* files, that is ROOT in this document. -all Update all containers -no-sign -gpgcmd See below for information about signing repositories. For other options, please see the documentation (in POD) of tl-update-containers. 7. Testing your distribution ---------------------------- Having completed all these steps, you can test your distribution by calling the TeX Live Manager with the location argument. Continuing the above example we use NETROOT: tlmgr -location NETROOT update --list or tlmgr -location NETROOT update --all or tlmgr -location NETROOT list etc. Please see the documentation of tlmgr (POD again) for details. 8. Signing repositories ----------------------- Since 2016 tlmgr is able to check cryptographic signatures and thus verify that the downloaded packages are what is distributed by the author/maintainer (you reading this document). If you don't want to sign, please pass -no-sign to tl-update-containers above. If you want to sign you should provide a signing script that accepts one file name and creates a detached signature for that file. Note that in this case you should publish your public key, and ask users to import your public key with tlmgr key add ... (see documentation of tlmgr for details). You can use the signing script used in TeX Live by setting the following environment variable: TL_GNUPGOPTS="--local-user 0xYOUR_ID" 9. Example update script ------------------------ For the TeX Live Contrib repository which is managed in git, I have the following files: ROOT/bin/... /texmf-dist /web2c/.dont-delete-this-dir /.... all the other files are also under texmf-dist /tlpkg /tlpsrc /00texlive.autopatterns.tlpsrc /00texlive.config.tlpsrc /00texlive.installation.tlpsrc /...tlpsrc all the other .tlpsrc files, mostly empty /texlive.tlpdb /update.sh I am using the following script update.sh (more or less, for details see the git repo of tlcontrib): TLCHECKOUT=${TLCHECKOUT:-$HOME/Development/TeX/texlive.git} TLNETDEST=${TLNETDEST:-$HOME/Domains/server/texlive.info/contrib/2017} TLCATALOGUE=${TLCATALOGUE:-$HOME/Development/TeX/texcatalogue-svn} # signing options TL_GNUPGOPTS="--local-user 0xYOUR_ID" export TL_GNUPGOPTS $TLCHECKOUT/Master/tlpkg/bin/tl-update-tlpdb \ -with-w32-pattern-warning \ -from-git \ -catalogue=$TLCATALOGUE \ -master=`pwd` $TLCHECKOUT/Master/tlpkg/bin/tl-update-containers \ -master `pwd` \ -location $TLNETDEST \ -all # sometimes we need -recreate =================================================================== Closing remark: We cannot repeat it often enough, please don't play around with files not belonging to your packages. That is dangerous! If you have further comments, questions, additions to this file please let us know at tex-live@tug.org. Last changed 2018-03-08 by np