diff options
author | Norbert Preining <norbert@preining.info> | 2019-09-02 13:46:59 +0900 |
---|---|---|
committer | Norbert Preining <norbert@preining.info> | 2019-09-02 13:46:59 +0900 |
commit | e0c6872cf40896c7be36b11dcc744620f10adf1d (patch) | |
tree | 60335e10d2f4354b0674ec22d7b53f0f8abee672 /usergrps/uktug/baskervi/4_4/ctanuk.tex |
Initial commit
Diffstat (limited to 'usergrps/uktug/baskervi/4_4/ctanuk.tex')
-rw-r--r-- | usergrps/uktug/baskervi/4_4/ctanuk.tex | 256 |
1 files changed, 256 insertions, 0 deletions
diff --git a/usergrps/uktug/baskervi/4_4/ctanuk.tex b/usergrps/uktug/baskervi/4_4/ctanuk.tex new file mode 100644 index 0000000000..234f9db60d --- /dev/null +++ b/usergrps/uktug/baskervi/4_4/ctanuk.tex @@ -0,0 +1,256 @@ +\title{Moving the UK CTAN} +\author[Martyn Johnson and Robin Fairbairns]{Martyn Johnson (\texttt{maj@cl.cam.ac.uk})\\ + and\\ + Robin Fairbairns (\texttt{rf@cl.cam.ac.uk})} +\begin{Article} +\section{The background (RF)} + +The UK node of the Comprehensive \TeX{} Archive Network (\CTAN) has a +long and honourable history, which starts long ago in the recognition +(by Peter Abbott at Aston~--- see \cite{abbott:exeter}) of the need to +provide an archive of \TeX-related material within the UK. At the +time that the Aston archive was created, \TeX-related material was +mostly made available \emph{ad hoc} by its originators~--- there was +no site with ambitions to provide a \emph{complete} set of systems, +macros, and so on. Furthermore, access from within the UK to overseas +material was less than straightforward (access to ftp, using an +account at UCL, was severely restricted). + +The Aston archive was originally a VMS-based facility, offering +connection via the Janet coloured-book protocols to machines that were +part of Aston's centrally-provided service. Later, the archive group +were given a second-hand VAX, and later still a parallel version of +the archive was established on a SparcStation that sat on Peter +Abbott's desk. This machine (with the net name |ftp.tex.ac.uk|) +eventually became part of \CTAN, offering access via anonymous +ftp to all and sundry throughout the world. + +At the beginning of this year, Peter Abbott told your committee that +he would be retiring (early) at the end of July; the implication was +that it would be unlikely that we could count on Aston's willingness +to offer a home to the archive beyond that date. + +The committee discussed whether it was reasonable even to consider +maintaining a CTAN node in the UK (covering the world with two sites +in Europe and one in the USA is hardly a convincing approach); and if +the node was to be retained, the implications of so doing. After much +soul-searching, we decided that it wasn't reasonable to spend \UKTUG{} +funds on a new home for the archive; we concluded that our membership +is such a small proportion of the \TeX{} community within the +UK\footnote{In contrast to the situation, say, in Germany, where +\DANTE{} owns the archive machine.}, that group funds really +\emph{shouldn't} be expected to cough up for support of the community +at large. + +In parallel with these discussions, the committee investigated +alternative new sites for the archive; candidates were the +Universities of Warwick, Sussex and Cambridge, the National +Typesetting service (at Oxford) and the Hensa\footnote{Higher +Education National Software Archive.}. While the typesetting archive +looked promising at first, they eventually +suggested that we should approach Hensa (which had already been on our +list of possible candidates). Hensa in fact maintains two archives, +one for micros and one for Unix; since \CTAN{} is a cross-architecture +service, offering support for micros, Unix machines and others, it was +obvious that Hensa couldn't maintain a \CTAN{} node. Sebastian Rahtz, +who's the only volunteer effort that the UK \CTAN{} node has, was +unwilling to maintain an archive that wasn't \CTAN, so we decided to +look elsewhere. + +Warwick (in the person of Malcolm Clark~--- he of the +\emph{Gleanings}) concluded that they would need to be provided with +some extra disk space before they could offer to host the service. +Sussex would very much like to host the service, but were unwilling to +do so until their connection to SuperJanet went live. Cambridge +expressed an early interest, whereafter very little (that was visible +to the committee) happened for some time while Martyn Johnson +established what was necessary and acquired agreement from the head of +the department and from the rest of the Computer Laboratory's systems +group. + +\section{The Archive Operational Requirement} + +Towards the end of June Sebastian Rahtz wrote an `operational +requirement', which described the facilities provided by the Aston +archive, and explained which were essential and which might, at a +pinch, be dropped. + +The complexity of it all was quite a surprise. The primary service is +an ftp server, but it needs to be able to perform many automatic +transformations, such as packing a directory into a |zip| image. +Gopher and WWW services were also provided at Aston. Behind the +scenes, there was a mechanism which kept the main \CTAN{} nodes in +step with each other whilst maintaining a peer relationship between +them. The archive maintainers needed remote access to the machine and +the ability to manipulate the archive. Finally, there were several +well-publicised mail addresses at |ftp.tex.ac.uk|. + +\section{Meeting the Operational Requirement (MAJ)} + +Whilst it was clear that we had the resources available to run the +archive, there were a number of awkward decisions to be made. The main +question was ``which machine?''; we certainly didn't have a machine +available to dedicate to the task. My initial assumption was that I +would put the service on one of our main fileserver machines, a +Digital Alpha/AXP 3000/500S, since that machine had plenty of disc +space available. But I was worried about security~--- we don't even +allow our own users to log in to our fileservers. Then I realised +that one of our most public machines (another Alpha) had a 1Gbyte disc +which could be made available, and the decision was made. A welcome +side effect of this choice was that it made it easier for me to +provide external access to the archive through NFS without +compromising the security of the internal services. + +Transferring the data from Aston to Cambridge was the least of the +problems. Not very long ago, moving such a large amount of data over +the network would have been unthinkable. I just did it, using a +standard ``mirror'' script running in the background. It took about a +day to pull the archive across, and a few minutes each day thereafter +to keep it up to date. + +It was obvious that the standard ftp daemon supplied with OSF/1 +wasn't up to the job of running the archive. The only sensible choice +was \texttt{wu-ftpd}, as used at the other \CTAN{} nodes. This +compiled easily enough for the Alpha, and initially appeared to work. +However, none of the document conversion scripts would function at +all, and the bug\footnote{Overwriting the daemon's own argument and + environment strings for cosmetic reasons, so that subprocesses start + up with a corrupt environment which the OSF/1 shell takes umbrage + at.} was only found after a great deal of detective work. Once found, +the problem was easy to fix, but even then, one of the document +conversion scripts still didn't work\footnote{But it didn't work at + Aston either, and has been mended.}. + +After those and other bugs\footnote{Notably the bug that made + \texttt{zoo}'s `portability library' not port to 64-bit machines.} +had been mended, there were still the Gopher and WWW services to +consider. We had not previously run a Gopher server, and I was not +very keen on doing it, but Sebastian assured me that the service was +well used and would be missed if we didn't provide it. In the end the +Gopher service turned out to be straightforward to make available, +though we decided to not to attempt to set up WAIS indexing initially. + +WWW was even easier, since we already ran a suitable server on another +machine, and we merely had to copy the data into it. Or did we? All +of the Aston archive's services were published as being available from +the machine |ftp.tex.ac.uk|, and this could obviously not be made an +alias for two different machines. I really did not want to run a +second WWW server, so we decided that the world would just have to +change. Each service offered by the archive now has its own name: +|ftp.tex.ac.uk|, |gopher.tex.ac.uk|, |nfs.tex.ac.uk| and +|www.tex.ac.uk|. While we were at it, we thought that a better name +for the mail domain would be simply |tex.ac.uk|, though we intend to +continue to accept mail using the old addresses for some time. + +Of course we cannot force people to use the correct name. In fact only +the mail and WWW services are provided on machines other than +|ftp.tex.ac.uk|. Mail is not a problem since there are +well-established mechanisms for redirecting it to a site hub. In order +that the WWW service should not seem to have disappeared to people +following old links, I wrote a tombstone service. No +matter what you ask for, it always returns a fixed page, which +explains that the server has moved, and offers a link to the new +place. + +The mail facilities were provided by configuring a logically separate +mail system within our mail hub, which runs the PP mailer. We did not +want to do anything which would make a further move of the archive +more difficult, so we were very keen to ensure that users could not +confuse the mail domains |tex.ac.uk| and |cl.cam.ac.uk|. We cannot +prevent them being confused, but at least we can ensure that they will +be told that they are confused by having incorrectly addressed mail +returned to them. + +It was always part of the agreement that the remote management of the +archive by Sebastian Rahtz, David Osborne and others should continue. +It is easy enough to give those users login access to the archive +machine, but they needed to be able to manipulate the archive without +needing ``|root|'' privilege. The archive is owned by a pseudo-user +``|ctan|'', and the archivists do their work using a local command +which allows them to pretend to be |ctan|, provided that they can +quote the password. Much of the maintenance of the archive is handled +automatically, by passing mail around and running periodic jobs. All +we had to do was arrange to deliver the mail~--- Sebastian did the rest! + +All of a sudden, it seemed that we were ready to go live. We had +already taken over control of the |tex.ac.uk| domain of the DNS +(a saga in itself) and simply had to flip over the addresses and watch +the traffic come in. And that is exactly what happened. + +Three days before he retired, Peter Abbott sent a message to the +committee mailing list saying that Aston did indeed want to reallocate +the Archive machine from the day that Peter left; the move was `just +in time'! + +\section{The first six weeks (MAJ)} + +We received remarkably little mail concerning the changeover, so it +must be considered a success. A handful of people had bound to the +old address and needed to be told the new one, but it seems that the +vast majority of clients simply did not notice any change. One person +complained that the Gopher service was not quite what it was, and this +is indeed an area which needs some more work. So far, after nearly 6 +weeks of operation, the new archive has shipped some 11Gbytes of stuff +via ftp. Equally encouragingly, there have been no complaints from +local users of the machine, for whom it is a compute server, that the +archive activities are having any impact on their work. In fact the +Alpha is a very fast machine, and provided that there is enough +memory, a few ftp sessions are hardly noticeable. + +By a curious twist of fate, the service offered in the first few weeks +has not been as good as we might have hoped. One of the reasons we +chose to use the Alpha machine we did was that in practice its +hardware and software have been highly reliable. Unfortunately there +have been three serious problems since we transferred the archive. + +The first was a spectacular thunderstorm in the Cambridge area which +caused widespread disruption to our equipment. The server itself +recovered with little difficulty, but the network problems were more +serious and our whole department was cut off from the outside world +for some time. + +The next weekend, the archive machine itself failed. The front +panel lights were dim and flickery, but this `obvious' power supply fault +was actually being provoked by a faulty system disc, so a replacement +had to be sent for, and the system had then to be restored from backup +tapes. This meant that the service was down for most of the weekend +and the following Monday. + +The third failure again started during a weekend. The machine simply +froze, and when rebooted, did exactly the same thing again. The +problem was tracked down to the NFS server, which was being fed +``poison packets'' by a machine in Norway. The machine's +administrator didn't respond to mail, but back-door intervention on +the archive machine had the required effect. We're told that the bug +in OSF/1 which made it crash on receiving the bad packet is mended in +the next release. + +Fortunately the user community has been very tolerant of these early +problems, and we very much hope that the next few weeks will be better. + +\section{Conclusion (RF)} + +The archive has grown from humble beginnings as a side service on a +VAX/VMS machine using protocols only (significantly) available within +the UK, to become part of an internationally-coordinated group of +archive sites offering services to anyone who chooses to use it in the +world at large. As a community, we have many reasons to thank Peter +Abbott for his foresight, and Aston University for hosting the archive +for many years. + +For the time being, Cambridge University has taken over the torch; let +us hope that the \CTAN{} network can continue to serve as a beacon +leading progress in the use of \TeX{} world-wide. + +\begin{thebibliography}{\bf{Abbott}90} + +\bibitem[Abbott 1990]{abbott:exeter} +Peter Abbott. +\newblock {\em {UK\TeX} and the Aston Archive}. +\newblock In Malcolm Clark, editor, {\em {\TeX} Applications, Uses, Methods}, + pages 109--114. Ellis Horwood, 1990. +\newblock Proceedings of {\TeX}eter (1988). + +\end{thebibliography} + +\end{Article} |