summaryrefslogtreecommitdiff
path: root/usergrps/uktug/baskervi/4_4/ctanuk.tex
diff options
context:
space:
mode:
Diffstat (limited to 'usergrps/uktug/baskervi/4_4/ctanuk.tex')
-rw-r--r--usergrps/uktug/baskervi/4_4/ctanuk.tex256
1 files changed, 256 insertions, 0 deletions
diff --git a/usergrps/uktug/baskervi/4_4/ctanuk.tex b/usergrps/uktug/baskervi/4_4/ctanuk.tex
new file mode 100644
index 0000000000..234f9db60d
--- /dev/null
+++ b/usergrps/uktug/baskervi/4_4/ctanuk.tex
@@ -0,0 +1,256 @@
+\title{Moving the UK CTAN}
+\author[Martyn Johnson and Robin Fairbairns]{Martyn Johnson (\texttt{maj@cl.cam.ac.uk})\\
+ and\\
+ Robin Fairbairns (\texttt{rf@cl.cam.ac.uk})}
+\begin{Article}
+\section{The background (RF)}
+
+The UK node of the Comprehensive \TeX{} Archive Network (\CTAN) has a
+long and honourable history, which starts long ago in the recognition
+(by Peter Abbott at Aston~--- see \cite{abbott:exeter}) of the need to
+provide an archive of \TeX-related material within the UK. At the
+time that the Aston archive was created, \TeX-related material was
+mostly made available \emph{ad hoc} by its originators~--- there was
+no site with ambitions to provide a \emph{complete} set of systems,
+macros, and so on. Furthermore, access from within the UK to overseas
+material was less than straightforward (access to ftp, using an
+account at UCL, was severely restricted).
+
+The Aston archive was originally a VMS-based facility, offering
+connection via the Janet coloured-book protocols to machines that were
+part of Aston's centrally-provided service. Later, the archive group
+were given a second-hand VAX, and later still a parallel version of
+the archive was established on a SparcStation that sat on Peter
+Abbott's desk. This machine (with the net name |ftp.tex.ac.uk|)
+eventually became part of \CTAN, offering access via anonymous
+ftp to all and sundry throughout the world.
+
+At the beginning of this year, Peter Abbott told your committee that
+he would be retiring (early) at the end of July; the implication was
+that it would be unlikely that we could count on Aston's willingness
+to offer a home to the archive beyond that date.
+
+The committee discussed whether it was reasonable even to consider
+maintaining a CTAN node in the UK (covering the world with two sites
+in Europe and one in the USA is hardly a convincing approach); and if
+the node was to be retained, the implications of so doing. After much
+soul-searching, we decided that it wasn't reasonable to spend \UKTUG{}
+funds on a new home for the archive; we concluded that our membership
+is such a small proportion of the \TeX{} community within the
+UK\footnote{In contrast to the situation, say, in Germany, where
+\DANTE{} owns the archive machine.}, that group funds really
+\emph{shouldn't} be expected to cough up for support of the community
+at large.
+
+In parallel with these discussions, the committee investigated
+alternative new sites for the archive; candidates were the
+Universities of Warwick, Sussex and Cambridge, the National
+Typesetting service (at Oxford) and the Hensa\footnote{Higher
+Education National Software Archive.}. While the typesetting archive
+looked promising at first, they eventually
+suggested that we should approach Hensa (which had already been on our
+list of possible candidates). Hensa in fact maintains two archives,
+one for micros and one for Unix; since \CTAN{} is a cross-architecture
+service, offering support for micros, Unix machines and others, it was
+obvious that Hensa couldn't maintain a \CTAN{} node. Sebastian Rahtz,
+who's the only volunteer effort that the UK \CTAN{} node has, was
+unwilling to maintain an archive that wasn't \CTAN, so we decided to
+look elsewhere.
+
+Warwick (in the person of Malcolm Clark~--- he of the
+\emph{Gleanings}) concluded that they would need to be provided with
+some extra disk space before they could offer to host the service.
+Sussex would very much like to host the service, but were unwilling to
+do so until their connection to SuperJanet went live. Cambridge
+expressed an early interest, whereafter very little (that was visible
+to the committee) happened for some time while Martyn Johnson
+established what was necessary and acquired agreement from the head of
+the department and from the rest of the Computer Laboratory's systems
+group.
+
+\section{The Archive Operational Requirement}
+
+Towards the end of June Sebastian Rahtz wrote an `operational
+requirement', which described the facilities provided by the Aston
+archive, and explained which were essential and which might, at a
+pinch, be dropped.
+
+The complexity of it all was quite a surprise. The primary service is
+an ftp server, but it needs to be able to perform many automatic
+transformations, such as packing a directory into a |zip| image.
+Gopher and WWW services were also provided at Aston. Behind the
+scenes, there was a mechanism which kept the main \CTAN{} nodes in
+step with each other whilst maintaining a peer relationship between
+them. The archive maintainers needed remote access to the machine and
+the ability to manipulate the archive. Finally, there were several
+well-publicised mail addresses at |ftp.tex.ac.uk|.
+
+\section{Meeting the Operational Requirement (MAJ)}
+
+Whilst it was clear that we had the resources available to run the
+archive, there were a number of awkward decisions to be made. The main
+question was ``which machine?''; we certainly didn't have a machine
+available to dedicate to the task. My initial assumption was that I
+would put the service on one of our main fileserver machines, a
+Digital Alpha/AXP 3000/500S, since that machine had plenty of disc
+space available. But I was worried about security~--- we don't even
+allow our own users to log in to our fileservers. Then I realised
+that one of our most public machines (another Alpha) had a 1Gbyte disc
+which could be made available, and the decision was made. A welcome
+side effect of this choice was that it made it easier for me to
+provide external access to the archive through NFS without
+compromising the security of the internal services.
+
+Transferring the data from Aston to Cambridge was the least of the
+problems. Not very long ago, moving such a large amount of data over
+the network would have been unthinkable. I just did it, using a
+standard ``mirror'' script running in the background. It took about a
+day to pull the archive across, and a few minutes each day thereafter
+to keep it up to date.
+
+It was obvious that the standard ftp daemon supplied with OSF/1
+wasn't up to the job of running the archive. The only sensible choice
+was \texttt{wu-ftpd}, as used at the other \CTAN{} nodes. This
+compiled easily enough for the Alpha, and initially appeared to work.
+However, none of the document conversion scripts would function at
+all, and the bug\footnote{Overwriting the daemon's own argument and
+ environment strings for cosmetic reasons, so that subprocesses start
+ up with a corrupt environment which the OSF/1 shell takes umbrage
+ at.} was only found after a great deal of detective work. Once found,
+the problem was easy to fix, but even then, one of the document
+conversion scripts still didn't work\footnote{But it didn't work at
+ Aston either, and has been mended.}.
+
+After those and other bugs\footnote{Notably the bug that made
+ \texttt{zoo}'s `portability library' not port to 64-bit machines.}
+had been mended, there were still the Gopher and WWW services to
+consider. We had not previously run a Gopher server, and I was not
+very keen on doing it, but Sebastian assured me that the service was
+well used and would be missed if we didn't provide it. In the end the
+Gopher service turned out to be straightforward to make available,
+though we decided to not to attempt to set up WAIS indexing initially.
+
+WWW was even easier, since we already ran a suitable server on another
+machine, and we merely had to copy the data into it. Or did we? All
+of the Aston archive's services were published as being available from
+the machine |ftp.tex.ac.uk|, and this could obviously not be made an
+alias for two different machines. I really did not want to run a
+second WWW server, so we decided that the world would just have to
+change. Each service offered by the archive now has its own name:
+|ftp.tex.ac.uk|, |gopher.tex.ac.uk|, |nfs.tex.ac.uk| and
+|www.tex.ac.uk|. While we were at it, we thought that a better name
+for the mail domain would be simply |tex.ac.uk|, though we intend to
+continue to accept mail using the old addresses for some time.
+
+Of course we cannot force people to use the correct name. In fact only
+the mail and WWW services are provided on machines other than
+|ftp.tex.ac.uk|. Mail is not a problem since there are
+well-established mechanisms for redirecting it to a site hub. In order
+that the WWW service should not seem to have disappeared to people
+following old links, I wrote a tombstone service. No
+matter what you ask for, it always returns a fixed page, which
+explains that the server has moved, and offers a link to the new
+place.
+
+The mail facilities were provided by configuring a logically separate
+mail system within our mail hub, which runs the PP mailer. We did not
+want to do anything which would make a further move of the archive
+more difficult, so we were very keen to ensure that users could not
+confuse the mail domains |tex.ac.uk| and |cl.cam.ac.uk|. We cannot
+prevent them being confused, but at least we can ensure that they will
+be told that they are confused by having incorrectly addressed mail
+returned to them.
+
+It was always part of the agreement that the remote management of the
+archive by Sebastian Rahtz, David Osborne and others should continue.
+It is easy enough to give those users login access to the archive
+machine, but they needed to be able to manipulate the archive without
+needing ``|root|'' privilege. The archive is owned by a pseudo-user
+``|ctan|'', and the archivists do their work using a local command
+which allows them to pretend to be |ctan|, provided that they can
+quote the password. Much of the maintenance of the archive is handled
+automatically, by passing mail around and running periodic jobs. All
+we had to do was arrange to deliver the mail~--- Sebastian did the rest!
+
+All of a sudden, it seemed that we were ready to go live. We had
+already taken over control of the |tex.ac.uk| domain of the DNS
+(a saga in itself) and simply had to flip over the addresses and watch
+the traffic come in. And that is exactly what happened.
+
+Three days before he retired, Peter Abbott sent a message to the
+committee mailing list saying that Aston did indeed want to reallocate
+the Archive machine from the day that Peter left; the move was `just
+in time'!
+
+\section{The first six weeks (MAJ)}
+
+We received remarkably little mail concerning the changeover, so it
+must be considered a success. A handful of people had bound to the
+old address and needed to be told the new one, but it seems that the
+vast majority of clients simply did not notice any change. One person
+complained that the Gopher service was not quite what it was, and this
+is indeed an area which needs some more work. So far, after nearly 6
+weeks of operation, the new archive has shipped some 11Gbytes of stuff
+via ftp. Equally encouragingly, there have been no complaints from
+local users of the machine, for whom it is a compute server, that the
+archive activities are having any impact on their work. In fact the
+Alpha is a very fast machine, and provided that there is enough
+memory, a few ftp sessions are hardly noticeable.
+
+By a curious twist of fate, the service offered in the first few weeks
+has not been as good as we might have hoped. One of the reasons we
+chose to use the Alpha machine we did was that in practice its
+hardware and software have been highly reliable. Unfortunately there
+have been three serious problems since we transferred the archive.
+
+The first was a spectacular thunderstorm in the Cambridge area which
+caused widespread disruption to our equipment. The server itself
+recovered with little difficulty, but the network problems were more
+serious and our whole department was cut off from the outside world
+for some time.
+
+The next weekend, the archive machine itself failed. The front
+panel lights were dim and flickery, but this `obvious' power supply fault
+was actually being provoked by a faulty system disc, so a replacement
+had to be sent for, and the system had then to be restored from backup
+tapes. This meant that the service was down for most of the weekend
+and the following Monday.
+
+The third failure again started during a weekend. The machine simply
+froze, and when rebooted, did exactly the same thing again. The
+problem was tracked down to the NFS server, which was being fed
+``poison packets'' by a machine in Norway. The machine's
+administrator didn't respond to mail, but back-door intervention on
+the archive machine had the required effect. We're told that the bug
+in OSF/1 which made it crash on receiving the bad packet is mended in
+the next release.
+
+Fortunately the user community has been very tolerant of these early
+problems, and we very much hope that the next few weeks will be better.
+
+\section{Conclusion (RF)}
+
+The archive has grown from humble beginnings as a side service on a
+VAX/VMS machine using protocols only (significantly) available within
+the UK, to become part of an internationally-coordinated group of
+archive sites offering services to anyone who chooses to use it in the
+world at large. As a community, we have many reasons to thank Peter
+Abbott for his foresight, and Aston University for hosting the archive
+for many years.
+
+For the time being, Cambridge University has taken over the torch; let
+us hope that the \CTAN{} network can continue to serve as a beacon
+leading progress in the use of \TeX{} world-wide.
+
+\begin{thebibliography}{\bf{Abbott}90}
+
+\bibitem[Abbott 1990]{abbott:exeter}
+Peter Abbott.
+\newblock {\em {UK\TeX} and the Aston Archive}.
+\newblock In Malcolm Clark, editor, {\em {\TeX} Applications, Uses, Methods},
+ pages 109--114. Ellis Horwood, 1990.
+\newblock Proceedings of {\TeX}eter (1988).
+
+\end{thebibliography}
+
+\end{Article}