update repository-setup.txt to current status

git-svn-id: svn://tug.org/texlive/trunk@46607 c570f23f-e606-0410-a88d-b1316a301751

1 files changed, 161 insertions, 57 deletions

diff --git a/Master/tlpkg/doc/repository-setup.txt b/Master/tlpkg/doc/repository-setup.txt
index e51454bc9fc..f72af9c8f44 100644
--- a/Master/tlpkg/doc/repository-setup.txt
+++ b/Master/tlpkg/doc/repository-setup.txt
@@ -12,8 +12,10 @@ This document describes how to set up a repository in a way that tlmgr
 can use it. Possible applications include distribution of company-internal
 packages or non-free stuff not included in TeX Live.
 
-Multiple repository support is very much work in progress.
-Write tex-live@tug.org if questions or suggestions.
+The current document is based on the status of the TeX Live repository
+scripts as of 2018-02-12.
+
+Please write tex-live@tug.org if questions or suggestions.
 
 Contents
 --------
@@ -25,7 +27,8 @@ Contents
 5. Adjusting the revision numbers
 6. Making the web distribution
 7. Testing your distribution
-8. Final notes (and unsupported stuff)
+8. Signing repositories
+9. Example update script
 
 
 0. Preliminaries
@@ -51,9 +54,6 @@ That means, with ROOT as the root of your local "repository":
 	ROOT/tlpkg/		infrastructure stuff, modules, scripts
 	ROOT/texmf-dist/	stuff where you should put your 
 				TeX code, fonts etc
-	ROOT/texmf		if you have the need for that please read
-				first about their usage in the 
-				TeX Live Guide.
 
 After that you call the very same scripts as we do when building the
 distribution database in
@@ -71,22 +71,28 @@ the contents of at least
 These files can be checked out via svn, or downloaded from the net.
 We recommend svn, so that you can easily update them regularly.
 
+Note that it is not necessary to have the above files and the files
+you are distributing in the same file hierarchy. It is completely valid
+to have a full checkout of TeX Live (svn or git) and use the scripts
+from there. See Section 9 for an example management script.
+
+
 
 2. Preparing your repository
 ----------------------------
 Create a root folder which will mirror our directory layout. We will call
 that root folder ROOT from now on.
 
-Put the above files into 
-	ROOT/tlpkg/TeXLive
-	ROOT/tlpkg/bin
-After that put all of YOUR files in a TDS compliant way in one of the texmf
-trees, texmf-dist, texmf, texmf-doc.
-	ROOT/texmf{,-dist}
-We recommend putting all the files in 
+Put all of YOUR files in a TDS compliant way into the texmf-dist folder:
 	ROOT/texmf-dist
-or, if you are replacing a package which is already present in TeX Live,
-in the very same places as in TeX Live.
+Usage of 'texmf' as tree is strongly discouraged but still possible.
+
+IMPORTANT: Create the directory
+  ROOT/texmf-dist/web2c
+(in case of a git repository put in a dummy .dont-delete-me to make sure
+the directory is present). tl-update-tlpdb and tl-update-containers rely
+on this directory existence to determine the type of repository
+(files based or container based).
 
 If you really want to replace packages with files in texmf, please think
 twice, and maybe contact us before.
@@ -98,16 +104,15 @@ Now you have to create several TeX Live Package Source files, .tlpsrc files
 and place them into
 	ROOT/tlpkg/tlpsrc
 
-You have to have ONE .tlpsrc file for each package you want to ship, and
-furthermore ONE for your overall installation, called
-00texlive.config.tlpsrc. This package defines some basic configurations
-for the tlpdb, like how it announces itself, whether separate src and
-doc containers are build, and the release years it supports.
-
-By default these values are taken from %TeXLive::TLConfig::TLPDBConfigs
-where you can make changes in your local TLConfig.pm file, or you add
-these settings directly in the tlpsrc file and call tl-update-tlpdb
-later on with the option -tlpdb-options-from-tlpsrc.
+You have to have two .tlpsrc file for each package you want to ship, and
+furthermore three configuration files for the overall installation:
+  00texlive.config.tlpsrc
+  00texlive.installation.tlpsrc
+These files can be copied from the main TeX Live repository, and only
+00texlive.config.tlpsrc needs adjustment. This package defines some 
+basic configurations for the tlpdb, like how it announces itself,
+whether separate src and doc containers are build, and the release 
+years it supports.
 
 In the end the generate "package" 00texlive.config should contain
 (at least) the following lines:
@@ -170,23 +175,30 @@ the TLPSRC.pm documentation (and don't forget to copy
 ---------------------------------
 If you have set up all your local files, and the tlpsrc files, you can 
 create the TeX Live Database in ROOT/tlpkg/texlive.tlpdb by calling
-	perl ROOT/tlpkg/bin/tl-update-tlpdb -from-files
-If you have set your TLPDB configuration options directly in the .tlpsrc
-file you have to add
-		-tlpdb-options-from-tlpsrc
-The -from-files does use a find command to find all the files in ROOT 
-instead of using "svn status" what is done normally. By default
-tl-update-tlpdb uses all tlpsrc files present in ROOT/tlpkg/tlpsrc/.
-Presumably this won't be very many files.  But if you like, you can
-specify individual package names together with -tlpsrc-from-cmdline.
+	perl PATH-TO/tl-update-tlpdb OPTIONS
+Here PATH-TO points to the directory Master/tlpkg/bin in your TeX Live
+checkout, or where you have saved the files from TeXLive and tlpkg 
+directories.
+
+How revision numbers are computed and which files are used can be controlled
+by one of the several -from-XXXX options. By default Subversion is used.
+Other options, if your repository is not subversion based, are
+  -from-files          use find to get list of files, revisions are all 1
+  -from-git            use git and compute the history (number of commits)
+  -from-gitsvn         use git over svn repo
+
+By default tl-update-tlpdb uses all tlpsrc files present in 
+ROOT/tlpkg/tlpsrc/. Presumably this won't be very many files.
+But if you like, you can specify individual package names together 
+with -tlpsrc-from-cmdline.
 
 You can add the option
-	-with-win-pattern-warning
+	-with-w32-pattern-warning
 which will warn on missing files for Windows; some of those warnings are
 by default suppressed.
 
 Another option that might be of interest is
-	-no-revision-check
+	-no-reverse-revision-check
 Normally if there is an old tlpdb the packages in the new one are compared
 with the packages in the old one and checked that the revisions are
 increasing.
@@ -212,10 +224,10 @@ use for doing some checks on your repository.
 
 5. Adjusting the revision numbers
 ---------------------------------
-While reading through the texlive.tlpdb you might have seen that all the 
-packages will have 
+If you used -from-files above all the revision numbers in the generated
+texlive.tlpdb will be set to 0:
 	revision 0
-in there. The revision is used for checking which one is newer. So if you 
+The revision is used for checking which package is newer. So if you 
 have different "releases" of a package you should increase the revision
 number from one step to the next.
 
@@ -229,23 +241,36 @@ If you want to change that, there are several options:
   An example for doing that take a look at our repository
   tlpkg/etc/tlrepo-from-minimals.pl
 
+- use either subversion or git for your repository and get automatically
+  increasing numbers.
+
 
 6. Making the web distribution
 ------------------------------
 Now that the texlive.tlpdb has been set up, revision numbers adjusted, you
 can create or update your web distribution with
-	perl ROOT/tlpkg/bin/tl-update-containers -location DEST -no-setup
+	perl PATH-TO/tl-update-containers OPTIONS
 
+OPTIONS:
+-location DEST
 The -location gives the place where your former distribution was or where
 it should be created. In the former case only those containers with larger
 revision numbers are recreated. In the latter case all containers are
 created.
 
-The option -no-setup does not call the built-in routines for determining the
-location of the lzma and tar binary, and calls simply "lzma" and "tar". So
-you should have them in your PATH. If you do not give this option you also
-have to check out some more files, like config.guess, and tlpkg/installer
-where the binaries for lzma for the architectures we ship are present.
+-master `pwd`
+The option -master defines the location of *your* files, that is ROOT
+in this document.
+
+-all
+Update all containers
+
+-no-sign
+-gpgcmd
+See below for information about signing repositories.
+
+For other options, please see the documentation (in POD) of
+tl-update-containers.
 
 
 7. Testing your distribution
@@ -261,23 +286,102 @@ or
 etc.  Please see the documentation of tlmgr (POD again) for details.
 
 
-8. Final notes (and unsupported stuff)
---------------------------------------
+8. Signing repositories
+-----------------------
+Since 2016 tlmgr is able to check cryptographic signatures and thus
+verify that the downloaded packages are what is distributed by the
+author/maintainer (you reading this document).
+
+If you don't to sign, please pass
+  -no-sign
+to tl-update-containers above.
+
+If you want to sign you should provide a signing script that accepts
+one file name and creates a detached signature for that file.
+Note that in this case you should publish your public key, and ask users
+to import your public key with 
+  tlmgr key add ...
+(see documentation of tlmgr for details).
+
+The signing script can be as simple as
+  gpg --armor --detach-sign --local-user 0xYOUR_ID "$1"
+See below for a bit more complicated script.
+
+
+
+9. Example update script
+------------------------
+For the TeX Live Contrib repository which is managed in git, I have the 
+following files:
+ROOT/bin/...
+    /texmf-dist
+               /web2c/.dont-delete-this-dir
+               /....   all the other files are also under texmf-dist
+    /tlpkg
+          /tlpsrc
+                 /00texlive.autopatterns.tlpsrc
+                 /00texlive.config.tlpsrc
+                 /00texlive.installation.tlpsrc
+                 /...tlpsrc   all the other .tlpsrc files, mostly empty
+          /texlive.tlpdb 
+    /tl-sign-file
+    /update.sh
+    
+I am using the following script update.sh (more or less, for details
+see the git repo of tlcontrib):
+
+TLCHECKOUT=${TLCHECKOUT:-$HOME/Development/TeX/texlive.git}
+TLNETDEST=${TLNETDEST:-$HOME/Domains/server/texlive.info/contrib/2017}
+TLCATALOGUE=${TLCATALOGUE:-$HOME/Development/TeX/texcatalogue-svn}
+
+$TLCHECKOUT/Master/tlpkg/bin/tl-update-tlpdb \
+    -with-w32-pattern-warning \
+    -from-git \
+    -catalogue=$TLCATALOGUE \
+    -master=`pwd`
+
+$TLCHECKOUT/Master/tlpkg/bin/tl-update-containers \
+    -master `pwd` \
+    -location $TLNETDEST    \
+    -gpgcmd `pwd`/tl-sign-file \
+    -all # sometimes we need -recreate
+
+and the tl-sign-file is very similar to the TeX Live version, only
+that the --local-user is changed and I need to enter the passphrase
+manually.
+
+tl-sign-file:
+
+if test $# -ne 1; then
+  echo "$0: Exactly one argument must be given, the file to sign." >&2
+  exit 1
+fi
+
+# remove previous signature
+rm -f "$1.asc"
+
+prg=gpg
+if ! [ "x$TL_GNUPG" = "x" ] ; then
+  # use the environment variable TL_GNUPG
+  prg=$TL_GNUPG
+fi
+
+# sign
+$prg --armor \
+    --detach-sign \
+    --local-user 0xEC00B8DAD32266AA \
+    "$1"
+
+
+===================================================================
+
+Closing remark:
 We cannot repeat it often enough, please don't play around with files not
 belonging to your packages. That is dangerous!
 
-Finally, besides the usual post installation stuff (running mktexlsr,
-fmtutil and updmap) you have no way to change the post installation and
-removal procedure at this time. Currently all the Post actions are hard
-coded into tlpkg/TeXLive/TLPostActions.pm as it is on the installed
-system (not the one in your repo, so changing something there will NOT
-help). You might ask us for including something there (but we need GOOD
-reason for that), or you have to wait until we extended the
-infrastructure such that post actions can be specified freely in the
-.tlpsrc file.
 
 If you have further comments, questions, additions to this file please
 let us know at tex-live@tug.org.
 
 
-Last changed 2009-09-10 by np, based on comments of Dohyun Kim (kotex)
+Last changed 2018-02-12 by np