From a360890a6c2a4befab6b48084d0731ad09c46631 Mon Sep 17 00:00:00 2001 From: Karl Berry Date: Wed, 24 Feb 2021 18:27:49 +0000 Subject: 2021 pretest setup git-svn-id: svn://tug.org/texlive/trunk@57875 c570f23f-e606-0410-a88d-b1316a301751 --- .../graphics/texlive-installer-graphics-horiz.png | Bin 16590 -> 16623 bytes .../graphics/texlive-installer-graphics-vert.png | Bin 17026 -> 17233 bytes .../doc/graphics/texlive-installer-graphics.pdf | Bin 129433 -> 129518 bytes Master/tlpkg/doc/releng.txt | 167 +++++++++------------ 4 files changed, 71 insertions(+), 96 deletions(-) (limited to 'Master/tlpkg/doc') diff --git a/Master/tlpkg/doc/graphics/texlive-installer-graphics-horiz.png b/Master/tlpkg/doc/graphics/texlive-installer-graphics-horiz.png index e02c89936c2..67f2a66f961 100644 Binary files a/Master/tlpkg/doc/graphics/texlive-installer-graphics-horiz.png and b/Master/tlpkg/doc/graphics/texlive-installer-graphics-horiz.png differ diff --git a/Master/tlpkg/doc/graphics/texlive-installer-graphics-vert.png b/Master/tlpkg/doc/graphics/texlive-installer-graphics-vert.png index 8cfe304b75a..e338f78736f 100644 Binary files a/Master/tlpkg/doc/graphics/texlive-installer-graphics-vert.png and b/Master/tlpkg/doc/graphics/texlive-installer-graphics-vert.png differ diff --git a/Master/tlpkg/doc/graphics/texlive-installer-graphics.pdf b/Master/tlpkg/doc/graphics/texlive-installer-graphics.pdf index cb05422882b..b41dd43f3b4 100644 Binary files a/Master/tlpkg/doc/graphics/texlive-installer-graphics.pdf and b/Master/tlpkg/doc/graphics/texlive-installer-graphics.pdf differ diff --git a/Master/tlpkg/doc/releng.txt b/Master/tlpkg/doc/releng.txt index 75caaea63b4..b73229f534e 100644 --- a/Master/tlpkg/doc/releng.txt +++ b/Master/tlpkg/doc/releng.txt @@ -19,17 +19,20 @@ To stop updating of certain packages before the general freeze: To run test builds without affecting production dirs, see: tlpkg/bin/tltestnet - I. Preparing, can be done before pretest starts. + Part I. Preparing, can/should be done before pretest starts. -1. doc: write updates for pretest.html, leave commented out until start. - - check NEWS/CHANGES/etc. files for all engines and major programs. +1. doc: write updates for pretest.html, but leave commented out until start. + - check NEWS/CHANGES/etc. files for all engines and major programs; + ask developers as needed. - diff new and old bindirs to see if there are interesting new programs: - \ls $lb >/tmp/now + \ls /usr/local/texlive-rel/bin/x86_64-linux >/tmp/now \ls $ib >/tmp/new comm -3 /tmp/now /tmp/new >/tmp/cm + - in general, writing these news items takes considerable time, but + it's important, because this summary of changes is what users want to know. - if new kpathsea file types (kpsewhich --help-formats), update kpathsea.texi. - see also doc item below for more necessary updates after pretest starts. - + 2. Ask tldistro for patches. 2. No /dev suffix: Build/source/{version.ac,texk/{kpathsea,ptexenc}/version.ac @@ -61,29 +64,45 @@ To run test builds without affecting production dirs, see: run ~karl/bin/tlversions to see+compare engine/etc. versions commit updates. +6. During the pretest, we want to keep updating tlnet. But we cannot do +that from the trunk, since the trunk will have the new binaries and +other changes. Therefore, we set up a branch for release YYYY where we +will continue to commit normal package updates and update tlnet, while +the trunk is updating tlpretest for YYYY+1. + +The best time to create the branch is right before the first +pretest-specific commit (the next item, in Part II). +cd /home/texlive/branch +rm -rf * .svn # clean out completely; trunk should also be clean +rev=... # svn info in trunk +tlyear=`date +%Y -d "last year"` # presumably +svn copy -r $rev \ + -m"branch$tlyear branch based on r$rev" \ + svn://tug.org/texlive/trunk svn://tug.org/texlive/branches/branch$tlyear - II. Building pretest instead of regular release. +# If you created the branch earlier for testing or whatever, no problem, +# just merge from trunk before starting the double commits. Add @REV +# after trunk if needed. Probably have to M(erge) some not-really-conflicts. +nice -19 svn merge svn://tug.org/texlive/trunk >&/tmp/merge -0. On the day of the last tlnet update, - Set frozen=>1 in TLConfig.pm. - Freeze should happen at that night's rebuild. +# Alternatively, if you forgot to create the branch at the time, also +# no problem, just choose the right $rev and run the svn copy above. +# For TL2021, we made the branch independent at r57871 (created earlier). -1. Then, the next day, start building tlpretest: reset - frozen=>0 in TLConfig.pm -and - option adjustrepo 0 in tl-update-tlnet -(since pretest users shouldn't go to CTAN for updates). -1a. Then preserve final tlnet of current release YYYY: -cd /home/ftp/texlive/tlnet -cp -ar ../tlnet /home/ftp/historic/systems/texlive/YYYY/tlnet-final -# if and when post-freeze updates are needed, see below. + Part II. Building pretest from trunk (and tlnet from branch). -1b. Also preserve Catalogue as of the freeze, again for the sake of - post-freeze updates: -rsync -ar /home/httpd/html/catalogue /home/texlive/catalogue-freeze +0. Ensure that /home/texlive/branch from before last pretest-specific +commit is created, clean, and up to date: see item 6 just above. + +0b. In crontab, disable trunk->tlnet entries. -1c. Set up tlpretest: +1. When ready to start building tlpretest: on trunk, set + option adjustrepo 0 +in tl-update-tlnet since pretest users shouldn't go to CTAN for updates. +This change should not go to the branch. + +1b. Set up tlpretest: Update /home/ftp/texlive/tlpretest/README. Empty tlpretest except for that README and 00_TIME.txt (used for mirmon). @@ -104,7 +123,7 @@ Master/tlpkg/bin/tl-update-messages Master/tlpkg/bin/tl-update-tlnet Master/tlpkg/installer/texlive.png (make -C Master/tlpkg/doc/graphics -W texlive-installer-graphics.tex texlive.png) - then move into place. + then mv into place. Master/texmf-dist/web2c/texmfcnf.lua Master/texmf-dist/web2c/texmf.cnf # from Build/.../kpathsea @@ -132,7 +151,7 @@ for p in *conv hbf2gf; do cp /home/ftp/tex-archive/language/chinese/CJK/cjk-X.Y. force_rebuild=true cron.tl This will fail because the pretest dir is not completely set up, but that's ok; the idea is to check the output and ensure all is basically ok. -It's likely that .tlpsrc files will need to be adjusted. +It's likely that .tlpsrc files will need to be adjusted for new binaries. 8a. svn commit all the above. @@ -140,7 +159,7 @@ It's likely that .tlpsrc files will need to be adjusted. cd /home/ftp/texlive/tlpretest cp -ar ../tlnet/[a-u]* . # omit tlnet/README* env force_rebuild=true cron.tl -We use cp because we want to start with exactly the last tlnet, +We use cp -a because we want to start with exactly the last tlnet, and not recreate all packages, so that a file with a given name is never rewritten. See recreate comments in tl-update-tlpdb and tl-update-containers. @@ -153,11 +172,11 @@ This was a one-time event, but left here as an example for the unknown future. 10. Web pages: make new pretest.html live; mention pretest has started on texlive/index.html and tug's home page (at least). -11. Life and daily updates go on. In cron.tl, set: - net_frozen=false - catalogue_compare=true # maybe not every day? +11. Life and daily updates go on. In cron.tl, set net_frozen=false. + In the crontab, enable the pretest entries. + -A. At some point, after tlnet freeze: +A. At some point during the pretest: Update gpg expiration, see tlpkg/gpg/tl-key-extension.txt; do not fail to do this, or the key will expire and cause lots of hassle. Run tl-update-linked-scripts to ensure consistency. @@ -190,64 +209,7 @@ C. Build a test ISO image for initial TeX Collection testing: After successful image build, email tldistro@tug.org for info/review. Can disable image building again after the initial build for a while. - -L. It may be that we need to update the "frozen" release while the - pretest is going on. To do that, the idea is to make a tag+branch for - the last update before switching to the pretest, check that out, - and use it for such critical updates. To do that, - a) look at https://tug.org/svn/texlive/tags/ to choose the right - tag name. For TL19 (during the TL20 pretest), it was texlive-2019.4, - since .0-3 were used during the release. - b) look at https://tug.org/pipermail/tex-live-commits/ to determine - the right revision number: probably the "autoupdate" during the - last build before the pretest setup. For TL19, that was 53984. - c) run the svn commands given below to create the tag and branch. - d) make working checkout of the new branch (at this point we should - be fine to wipe out the old branch checkout, if there was one). -mkdir /home/texlive/branch -cd !$ -tlyear=2019 -subnum=4 -nice -19 svn co -q svn://tug.org/texlive/branches/branch$tlyear.$subnum . - - e) make the required update, probably most easily by copying the - affected files from the trunk. Running ctan2tl (or c2l) inside the - branch checkout should also work, if no other changes have intervened. - - f) remake tlnet (not tlpretest); cron.tl does too much overhead for - this delicate situation, so run the commands by hand. - 1) update tlpdb so we'll get the new revision number; - must run from branch, not trunk, or all will be wrong! -cd /home/texlive/branch/Master/tlpkg/bin # must be in branch directory - Use Catalogue snapshot saved above (at "preserve Catalogue") - to avoid myriad Catalogue changes. - If files have been removed without any other changes to a given - package (hopefully not), add --fix-reverse-revisions. -catdir=/home/texlive/catalogue-freeze -./tl-update-tlpdb --catalogue=$catdir - Check that only expected changes are made. - Might be easier to run this, get the new revision number(s), - then revert and manually edit if there are additional unneeded - changes. Don't worry about 00texlive.*, they aren't distributed. - - 2) for safety, make a backup of tlnet: -tar -cf /tmp/tlnet.tar -C /home/ftp/texlive/tlnet . # remove when done - - 3) update tlnet, again from the branch, not trunk. Double check! - we don't want to update the installer packages, as they have - already been getting updated from trunk, so are newer. -cd /home/texlive/branch/Master/tlpkg/bin -./tl-update-tlnet --no-install-pkgs - It will take about an hour to run. hopefully only the intended - packages will get updated. if not, that's why we made the tar backup. - - e) Update the tlnet-final directory in historic: -tlnet_live=/home/ftp/texlive/tlnet -tlnet_historic=/home/ftp/historic/systems/texlive/2019/tlnet-final -rsync -av -n $tlnet_live/ $tlnet_historic/ # trailing /'s needed -# remove the -n after checking that only expected files are updated. - -M. As work proceeds, update the doc: +M. As pretest proceeds, update the doc: MetaPost (in texmf-dist), other engine (in source) NEWS. Ensure all engine manuals updated in texmf-dist. Update texlive-en.tex (+ .pdf, .html), tell tldoc to translate. @@ -295,29 +257,42 @@ O. Two days before the final pretest (i.e., to-be-published) build: tlmgr version number in texmf-dist/scripts/texlive/NEWS. In cron.tl, set catalogue_compare=true for final syncs. Rebuild test images so we get the new source. - Tell developers no more commits: tldev + - hironobu, luigi, mojca, takuji. + Tell developers no more commits: tldev + + hirata, hironori, luigi, mojca, takuji, P. The day before the final pretest: option adjustrepo 1 in tl-update-tlnet, else install-tl-unx won't go to ctan Update Master/source (if not earlier): xz wget w32tex-src texlive*source mactexdoc install-tl-unx.tar.gz -Q. After final build, to do total freeze: +Q. The day of the final tlnet-from-branch build, which may or may not + also be the day of the final tlpretest-from-trunk build: + Set frozen=>1 in branch TLConfig.pm. + Freeze of tlnet should happen at that night's rebuild. + +R. After final build, to do total freeze: In cron.tl, set *_frozen=true. Also, set catalogue_compare=false now or at some point earlier. - Don't disable the cron entry, since we want to see if any commits - are mistakenly made. Nothing more than svn update will happen - with everything frozen. + For double protection, disable the nightly cron jobs. Also, in /l/bin/mirror-ctan, disable Catalogue updates, to avoid unnecessary changes if we have to rebuild the final image. -R. Ask for torrents to be created. In 2019, +S. Ask for torrents to be created. In 2020, Tim Stewart made the one that worked. +T. Preserve final tlnet of current release YYYY: +cd /home/ftp/texlive/tlnet +cp -ar ../tlnet /home/ftp/historic/systems/texlive/YYYY/tlnet-final +# if post-freeze updates are required (let's hope not), +# see tlpkg/archive/tlnet-by-hand.txt. + +T2. Also preserve Catalogue as of the freeze, again for the sake of + post-freeze updates: +rsync -ar /home/httpd/html/catalogue /home/texlive/catalogue-freeze + - III. Making the big release: + Part III. Making the big release: - unmount live install of last year's: umount /home/ftp/texlive/Contents/live @@ -449,7 +424,7 @@ svn copy -r $rev \ svn copy -r $rev \ -m"branch$tlyear branch based on r$rev" \ svn://tug.org/texlive/trunk svn://tug.org/texlive/branches/branch$tlyear - # add .$subnum to branch name (and in comment) if making + # add .$subnum to branch name (and in comment) if not making the first. It's the same command, just different destinations. http://svnbook.red-bean.com/en/1.8/svn.branchmerge.html -- cgit v1.2.3