summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--Master/texmf/doc/info/wget.info4451
1 files changed, 0 insertions, 4451 deletions
diff --git a/Master/texmf/doc/info/wget.info b/Master/texmf/doc/info/wget.info
deleted file mode 100644
index 5c50f0e6e64..00000000000
--- a/Master/texmf/doc/info/wget.info
+++ /dev/null
@@ -1,4451 +0,0 @@
-This is wget.info, produced by makeinfo version 4.8 from ./wget.texi.
-
-INFO-DIR-SECTION Network Applications
-START-INFO-DIR-ENTRY
-* Wget: (wget). The non-interactive network downloader.
-END-INFO-DIR-ENTRY
-
- This file documents the the GNU Wget utility for downloading network
-data.
-
- Copyright (C) 1996-2005 Free Software Foundation, Inc.
-
- Permission is granted to make and distribute verbatim copies of this
-manual provided the copyright notice and this permission notice are
-preserved on all copies.
-
- Permission is granted to copy, distribute and/or modify this document
-under the terms of the GNU Free Documentation License, Version 1.2 or
-any later version published by the Free Software Foundation; with the
-Invariant Sections being "GNU General Public License" and "GNU Free
-Documentation License", with no Front-Cover Texts, and with no
-Back-Cover Texts. A copy of the license is included in the section
-entitled "GNU Free Documentation License".
-
-
-File: wget.info, Node: Top, Next: Overview, Up: (dir)
-
-Wget 1.10.2
-***********
-
-This manual documents version 1.10.2 of GNU Wget, the freely available
-utility for network downloads.
-
- Copyright (C) 1996-2005 Free Software Foundation, Inc.
-
-* Menu:
-
-* Overview:: Features of Wget.
-* Invoking:: Wget command-line arguments.
-* Recursive Download:: Downloading interlinked pages.
-* Following Links:: The available methods of chasing links.
-* Time-Stamping:: Mirroring according to time-stamps.
-* Startup File:: Wget's initialization file.
-* Examples:: Examples of usage.
-* Various:: The stuff that doesn't fit anywhere else.
-* Appendices:: Some useful references.
-* Copying:: You may give out copies of Wget and of this manual.
-* Concept Index:: Topics covered by this manual.
-
-
-File: wget.info, Node: Overview, Next: Invoking, Prev: Top, Up: Top
-
-1 Overview
-**********
-
-GNU Wget is a free utility for non-interactive download of files from
-the Web. It supports HTTP, HTTPS, and FTP protocols, as well as
-retrieval through HTTP proxies.
-
- This chapter is a partial overview of Wget's features.
-
- * Wget is non-interactive, meaning that it can work in the
- background, while the user is not logged on. This allows you to
- start a retrieval and disconnect from the system, letting Wget
- finish the work. By contrast, most of the Web browsers require
- constant user's presence, which can be a great hindrance when
- transferring a lot of data.
-
- * Wget can follow links in HTML and XHTML pages and create local
- versions of remote web sites, fully recreating the directory
- structure of the original site. This is sometimes referred to as
- "recursive downloading." While doing that, Wget respects the
- Robot Exclusion Standard (`/robots.txt'). Wget can be instructed
- to convert the links in downloaded HTML files to the local files
- for offline viewing.
-
- * File name wildcard matching and recursive mirroring of directories
- are available when retrieving via FTP. Wget can read the
- time-stamp information given by both HTTP and FTP servers, and
- store it locally. Thus Wget can see if the remote file has
- changed since last retrieval, and automatically retrieve the new
- version if it has. This makes Wget suitable for mirroring of FTP
- sites, as well as home pages.
-
- * Wget has been designed for robustness over slow or unstable network
- connections; if a download fails due to a network problem, it will
- keep retrying until the whole file has been retrieved. If the
- server supports regetting, it will instruct the server to continue
- the download from where it left off.
-
- * Wget supports proxy servers, which can lighten the network load,
- speed up retrieval and provide access behind firewalls. However,
- if you are behind a firewall that requires that you use a socks
- style gateway, you can get the socks library and build Wget with
- support for socks. Wget uses the passive FTP downloading by
- default, active FTP being an option.
-
- * Wget supports IP version 6, the next generation of IP. IPv6 is
- autodetected at compile-time, and can be disabled at either build
- or run time. Binaries built with IPv6 support work well in both
- IPv4-only and dual family environments.
-
- * Built-in features offer mechanisms to tune which links you wish to
- follow (*note Following Links::).
-
- * The progress of individual downloads is traced using a progress
- gauge. Interactive downloads are tracked using a
- "thermometer"-style gauge, whereas non-interactive ones are traced
- with dots, each dot representing a fixed amount of data received
- (1KB by default). Either gauge can be customized to your
- preferences.
-
- * Most of the features are fully configurable, either through
- command line options, or via the initialization file `.wgetrc'
- (*note Startup File::). Wget allows you to define "global"
- startup files (`/usr/local/etc/wgetrc' by default) for site
- settings.
-
- * Finally, GNU Wget is free software. This means that everyone may
- use it, redistribute it and/or modify it under the terms of the
- GNU General Public License, as published by the Free Software
- Foundation (*note Copying::).
-
-
-File: wget.info, Node: Invoking, Next: Recursive Download, Prev: Overview, Up: Top
-
-2 Invoking
-**********
-
-By default, Wget is very simple to invoke. The basic syntax is:
-
- wget [OPTION]... [URL]...
-
- Wget will simply download all the URLs specified on the command
-line. URL is a "Uniform Resource Locator", as defined below.
-
- However, you may wish to change some of the default parameters of
-Wget. You can do it two ways: permanently, adding the appropriate
-command to `.wgetrc' (*note Startup File::), or specifying it on the
-command line.
-
-* Menu:
-
-* URL Format::
-* Option Syntax::
-* Basic Startup Options::
-* Logging and Input File Options::
-* Download Options::
-* Directory Options::
-* HTTP Options::
-* HTTPS (SSL/TLS) Options::
-* FTP Options::
-* Recursive Retrieval Options::
-* Recursive Accept/Reject Options::
-
-
-File: wget.info, Node: URL Format, Next: Option Syntax, Up: Invoking
-
-2.1 URL Format
-==============
-
-"URL" is an acronym for Uniform Resource Locator. A uniform resource
-locator is a compact string representation for a resource available via
-the Internet. Wget recognizes the URL syntax as per RFC1738. This is
-the most widely used form (square brackets denote optional parts):
-
- http://host[:port]/directory/file
- ftp://host[:port]/directory/file
-
- You can also encode your username and password within a URL:
-
- ftp://user:password@host/path
- http://user:password@host/path
-
- Either USER or PASSWORD, or both, may be left out. If you leave out
-either the HTTP username or password, no authentication will be sent.
-If you leave out the FTP username, `anonymous' will be used. If you
-leave out the FTP password, your email address will be supplied as a
-default password.(1)
-
- *Important Note*: if you specify a password-containing URL on the
-command line, the username and password will be plainly visible to all
-users on the system, by way of `ps'. On multi-user systems, this is a
-big security risk. To work around it, use `wget -i -' and feed the
-URLs to Wget's standard input, each on a separate line, terminated by
-`C-d'.
-
- You can encode unsafe characters in a URL as `%xy', `xy' being the
-hexadecimal representation of the character's ASCII value. Some common
-unsafe characters include `%' (quoted as `%25'), `:' (quoted as `%3A'),
-and `@' (quoted as `%40'). Refer to RFC1738 for a comprehensive list
-of unsafe characters.
-
- Wget also supports the `type' feature for FTP URLs. By default, FTP
-documents are retrieved in the binary mode (type `i'), which means that
-they are downloaded unchanged. Another useful mode is the `a'
-("ASCII") mode, which converts the line delimiters between the
-different operating systems, and is thus useful for text files. Here
-is an example:
-
- ftp://host/directory/file;type=a
-
- Two alternative variants of URL specification are also supported,
-because of historical (hysterical?) reasons and their widespreaded use.
-
- FTP-only syntax (supported by `NcFTP'):
- host:/dir/file
-
- HTTP-only syntax (introduced by `Netscape'):
- host[:port]/dir/file
-
- These two alternative forms are deprecated, and may cease being
-supported in the future.
-
- If you do not understand the difference between these notations, or
-do not know which one to use, just use the plain ordinary format you use
-with your favorite browser, like `Lynx' or `Netscape'.
-
- ---------- Footnotes ----------
-
- (1) If you have a `.netrc' file in your home directory, password
-will also be searched for there.
-
-
-File: wget.info, Node: Option Syntax, Next: Basic Startup Options, Prev: URL Format, Up: Invoking
-
-2.2 Option Syntax
-=================
-
-Since Wget uses GNU getopt to process command-line arguments, every
-option has a long form along with the short one. Long options are more
-convenient to remember, but take time to type. You may freely mix
-different option styles, or specify options after the command-line
-arguments. Thus you may write:
-
- wget -r --tries=10 http://fly.srk.fer.hr/ -o log
-
- The space between the option accepting an argument and the argument
-may be omitted. Instead `-o log' you can write `-olog'.
-
- You may put several options that do not require arguments together,
-like:
-
- wget -drc URL
-
- This is a complete equivalent of:
-
- wget -d -r -c URL
-
- Since the options can be specified after the arguments, you may
-terminate them with `--'. So the following will try to download URL
-`-x', reporting failure to `log':
-
- wget -o log -- -x
-
- The options that accept comma-separated lists all respect the
-convention that specifying an empty list clears its value. This can be
-useful to clear the `.wgetrc' settings. For instance, if your `.wgetrc'
-sets `exclude_directories' to `/cgi-bin', the following example will
-first reset it, and then set it to exclude `/~nobody' and `/~somebody'.
-You can also clear the lists in `.wgetrc' (*note Wgetrc Syntax::).
-
- wget -X '' -X /~nobody,/~somebody
-
- Most options that do not accept arguments are "boolean" options, so
-named because their state can be captured with a yes-or-no ("boolean")
-variable. For example, `--follow-ftp' tells Wget to follow FTP links
-from HTML files and, on the other hand, `--no-glob' tells it not to
-perform file globbing on FTP URLs. A boolean option is either
-"affirmative" or "negative" (beginning with `--no'). All such options
-share several properties.
-
- Unless stated otherwise, it is assumed that the default behavior is
-the opposite of what the option accomplishes. For example, the
-documented existence of `--follow-ftp' assumes that the default is to
-_not_ follow FTP links from HTML pages.
-
- Affirmative options can be negated by prepending the `--no-' to the
-option name; negative options can be negated by omitting the `--no-'
-prefix. This might seem superfluous--if the default for an affirmative
-option is to not do something, then why provide a way to explicitly
-turn it off? But the startup file may in fact change the default. For
-instance, using `follow_ftp = off' in `.wgetrc' makes Wget _not_ follow
-FTP links by default, and using `--no-follow-ftp' is the only way to
-restore the factory default from the command line.
-
-
-File: wget.info, Node: Basic Startup Options, Next: Logging and Input File Options, Prev: Option Syntax, Up: Invoking
-
-2.3 Basic Startup Options
-=========================
-
-`-V'
-`--version'
- Display the version of Wget.
-
-`-h'
-`--help'
- Print a help message describing all of Wget's command-line options.
-
-`-b'
-`--background'
- Go to background immediately after startup. If no output file is
- specified via the `-o', output is redirected to `wget-log'.
-
-`-e COMMAND'
-`--execute COMMAND'
- Execute COMMAND as if it were a part of `.wgetrc' (*note Startup
- File::). A command thus invoked will be executed _after_ the
- commands in `.wgetrc', thus taking precedence over them. If you
- need to specify more than one wgetrc command, use multiple
- instances of `-e'.
-
-
-
-File: wget.info, Node: Logging and Input File Options, Next: Download Options, Prev: Basic Startup Options, Up: Invoking
-
-2.4 Logging and Input File Options
-==================================
-
-`-o LOGFILE'
-`--output-file=LOGFILE'
- Log all messages to LOGFILE. The messages are normally reported
- to standard error.
-
-`-a LOGFILE'
-`--append-output=LOGFILE'
- Append to LOGFILE. This is the same as `-o', only it appends to
- LOGFILE instead of overwriting the old log file. If LOGFILE does
- not exist, a new file is created.
-
-`-d'
-`--debug'
- Turn on debug output, meaning various information important to the
- developers of Wget if it does not work properly. Your system
- administrator may have chosen to compile Wget without debug
- support, in which case `-d' will not work. Please note that
- compiling with debug support is always safe--Wget compiled with
- the debug support will _not_ print any debug info unless requested
- with `-d'. *Note Reporting Bugs::, for more information on how to
- use `-d' for sending bug reports.
-
-`-q'
-`--quiet'
- Turn off Wget's output.
-
-`-v'
-`--verbose'
- Turn on verbose output, with all the available data. The default
- output is verbose.
-
-`-nv'
-`--no-verbose'
- Turn off verbose without being completely quiet (use `-q' for
- that), which means that error messages and basic information still
- get printed.
-
-`-i FILE'
-`--input-file=FILE'
- Read URLs from FILE. If `-' is specified as FILE, URLs are read
- from the standard input. (Use `./-' to read from a file literally
- named `-'.)
-
- If this function is used, no URLs need be present on the command
- line. If there are URLs both on the command line and in an input
- file, those on the command lines will be the first ones to be
- retrieved. The FILE need not be an HTML document (but no harm if
- it is)--it is enough if the URLs are just listed sequentially.
-
- However, if you specify `--force-html', the document will be
- regarded as `html'. In that case you may have problems with
- relative links, which you can solve either by adding `<base
- href="URL">' to the documents or by specifying `--base=URL' on the
- command line.
-
-`-F'
-`--force-html'
- When input is read from a file, force it to be treated as an HTML
- file. This enables you to retrieve relative links from existing
- HTML files on your local disk, by adding `<base href="URL">' to
- HTML, or using the `--base' command-line option.
-
-`-B URL'
-`--base=URL'
- Prepends URL to relative links read from the file specified with
- the `-i' option.
-
-
-File: wget.info, Node: Download Options, Next: Directory Options, Prev: Logging and Input File Options, Up: Invoking
-
-2.5 Download Options
-====================
-
-`--bind-address=ADDRESS'
- When making client TCP/IP connections, bind to ADDRESS on the
- local machine. ADDRESS may be specified as a hostname or IP
- address. This option can be useful if your machine is bound to
- multiple IPs.
-
-`-t NUMBER'
-`--tries=NUMBER'
- Set number of retries to NUMBER. Specify 0 or `inf' for infinite
- retrying. The default is to retry 20 times, with the exception of
- fatal errors like "connection refused" or "not found" (404), which
- are not retried.
-
-`-O FILE'
-`--output-document=FILE'
- The documents will not be written to the appropriate files, but all
- will be concatenated together and written to FILE. If `-' is used
- as FILE, documents will be printed to standard output, disabling
- link conversion. (Use `./-' to print to a file literally named
- `-'.)
-
- Note that a combination with `-k' is only well-defined for
- downloading a single document.
-
-`-nc'
-`--no-clobber'
- If a file is downloaded more than once in the same directory,
- Wget's behavior depends on a few options, including `-nc'. In
- certain cases, the local file will be "clobbered", or overwritten,
- upon repeated download. In other cases it will be preserved.
-
- When running Wget without `-N', `-nc', or `-r', downloading the
- same file in the same directory will result in the original copy
- of FILE being preserved and the second copy being named `FILE.1'.
- If that file is downloaded yet again, the third copy will be named
- `FILE.2', and so on. When `-nc' is specified, this behavior is
- suppressed, and Wget will refuse to download newer copies of
- `FILE'. Therefore, "`no-clobber'" is actually a misnomer in this
- mode--it's not clobbering that's prevented (as the numeric
- suffixes were already preventing clobbering), but rather the
- multiple version saving that's prevented.
-
- When running Wget with `-r', but without `-N' or `-nc',
- re-downloading a file will result in the new copy simply
- overwriting the old. Adding `-nc' will prevent this behavior,
- instead causing the original version to be preserved and any newer
- copies on the server to be ignored.
-
- When running Wget with `-N', with or without `-r', the decision as
- to whether or not to download a newer copy of a file depends on
- the local and remote timestamp and size of the file (*note
- Time-Stamping::). `-nc' may not be specified at the same time as
- `-N'.
-
- Note that when `-nc' is specified, files with the suffixes `.html'
- or `.htm' will be loaded from the local disk and parsed as if they
- had been retrieved from the Web.
-
-`-c'
-`--continue'
- Continue getting a partially-downloaded file. This is useful when
- you want to finish up a download started by a previous instance of
- Wget, or by another program. For instance:
-
- wget -c ftp://sunsite.doc.ic.ac.uk/ls-lR.Z
-
- If there is a file named `ls-lR.Z' in the current directory, Wget
- will assume that it is the first portion of the remote file, and
- will ask the server to continue the retrieval from an offset equal
- to the length of the local file.
-
- Note that you don't need to specify this option if you just want
- the current invocation of Wget to retry downloading a file should
- the connection be lost midway through. This is the default
- behavior. `-c' only affects resumption of downloads started
- _prior_ to this invocation of Wget, and whose local files are
- still sitting around.
-
- Without `-c', the previous example would just download the remote
- file to `ls-lR.Z.1', leaving the truncated `ls-lR.Z' file alone.
-
- Beginning with Wget 1.7, if you use `-c' on a non-empty file, and
- it turns out that the server does not support continued
- downloading, Wget will refuse to start the download from scratch,
- which would effectively ruin existing contents. If you really
- want the download to start from scratch, remove the file.
-
- Also beginning with Wget 1.7, if you use `-c' on a file which is of
- equal size as the one on the server, Wget will refuse to download
- the file and print an explanatory message. The same happens when
- the file is smaller on the server than locally (presumably because
- it was changed on the server since your last download
- attempt)--because "continuing" is not meaningful, no download
- occurs.
-
- On the other side of the coin, while using `-c', any file that's
- bigger on the server than locally will be considered an incomplete
- download and only `(length(remote) - length(local))' bytes will be
- downloaded and tacked onto the end of the local file. This
- behavior can be desirable in certain cases--for instance, you can
- use `wget -c' to download just the new portion that's been
- appended to a data collection or log file.
-
- However, if the file is bigger on the server because it's been
- _changed_, as opposed to just _appended_ to, you'll end up with a
- garbled file. Wget has no way of verifying that the local file is
- really a valid prefix of the remote file. You need to be
- especially careful of this when using `-c' in conjunction with
- `-r', since every file will be considered as an "incomplete
- download" candidate.
-
- Another instance where you'll get a garbled file if you try to use
- `-c' is if you have a lame HTTP proxy that inserts a "transfer
- interrupted" string into the local file. In the future a
- "rollback" option may be added to deal with this case.
-
- Note that `-c' only works with FTP servers and with HTTP servers
- that support the `Range' header.
-
-`--progress=TYPE'
- Select the type of the progress indicator you wish to use. Legal
- indicators are "dot" and "bar".
-
- The "bar" indicator is used by default. It draws an ASCII progress
- bar graphics (a.k.a "thermometer" display) indicating the status of
- retrieval. If the output is not a TTY, the "dot" bar will be used
- by default.
-
- Use `--progress=dot' to switch to the "dot" display. It traces
- the retrieval by printing dots on the screen, each dot
- representing a fixed amount of downloaded data.
-
- When using the dotted retrieval, you may also set the "style" by
- specifying the type as `dot:STYLE'. Different styles assign
- different meaning to one dot. With the `default' style each dot
- represents 1K, there are ten dots in a cluster and 50 dots in a
- line. The `binary' style has a more "computer"-like
- orientation--8K dots, 16-dots clusters and 48 dots per line (which
- makes for 384K lines). The `mega' style is suitable for
- downloading very large files--each dot represents 64K retrieved,
- there are eight dots in a cluster, and 48 dots on each line (so
- each line contains 3M).
-
- Note that you can set the default style using the `progress'
- command in `.wgetrc'. That setting may be overridden from the
- command line. The exception is that, when the output is not a
- TTY, the "dot" progress will be favored over "bar". To force the
- bar output, use `--progress=bar:force'.
-
-`-N'
-`--timestamping'
- Turn on time-stamping. *Note Time-Stamping::, for details.
-
-`-S'
-`--server-response'
- Print the headers sent by HTTP servers and responses sent by FTP
- servers.
-
-`--spider'
- When invoked with this option, Wget will behave as a Web "spider",
- which means that it will not download the pages, just check that
- they are there. For example, you can use Wget to check your
- bookmarks:
-
- wget --spider --force-html -i bookmarks.html
-
- This feature needs much more work for Wget to get close to the
- functionality of real web spiders.
-
-`-T seconds'
-`--timeout=SECONDS'
- Set the network timeout to SECONDS seconds. This is equivalent to
- specifying `--dns-timeout', `--connect-timeout', and
- `--read-timeout', all at the same time.
-
- When interacting with the network, Wget can check for timeout and
- abort the operation if it takes too long. This prevents anomalies
- like hanging reads and infinite connects. The only timeout
- enabled by default is a 900-second read timeout. Setting a
- timeout to 0 disables it altogether. Unless you know what you are
- doing, it is best not to change the default timeout settings.
-
- All timeout-related options accept decimal values, as well as
- subsecond values. For example, `0.1' seconds is a legal (though
- unwise) choice of timeout. Subsecond timeouts are useful for
- checking server response times or for testing network latency.
-
-`--dns-timeout=SECONDS'
- Set the DNS lookup timeout to SECONDS seconds. DNS lookups that
- don't complete within the specified time will fail. By default,
- there is no timeout on DNS lookups, other than that implemented by
- system libraries.
-
-`--connect-timeout=SECONDS'
- Set the connect timeout to SECONDS seconds. TCP connections that
- take longer to establish will be aborted. By default, there is no
- connect timeout, other than that implemented by system libraries.
-
-`--read-timeout=SECONDS'
- Set the read (and write) timeout to SECONDS seconds. The "time"
- of this timeout refers "idle time": if, at any point in the
- download, no data is received for more than the specified number
- of seconds, reading fails and the download is restarted. This
- option does not directly affect the duration of the entire
- download.
-
- Of course, the remote server may choose to terminate the connection
- sooner than this option requires. The default read timeout is 900
- seconds.
-
-`--limit-rate=AMOUNT'
- Limit the download speed to AMOUNT bytes per second. Amount may
- be expressed in bytes, kilobytes with the `k' suffix, or megabytes
- with the `m' suffix. For example, `--limit-rate=20k' will limit
- the retrieval rate to 20KB/s. This is useful when, for whatever
- reason, you don't want Wget to consume the entire available
- bandwidth.
-
- This option allows the use of decimal numbers, usually in
- conjunction with power suffixes; for example, `--limit-rate=2.5k'
- is a legal value.
-
- Note that Wget implements the limiting by sleeping the appropriate
- amount of time after a network read that took less time than
- specified by the rate. Eventually this strategy causes the TCP
- transfer to slow down to approximately the specified rate.
- However, it may take some time for this balance to be achieved, so
- don't be surprised if limiting the rate doesn't work well with
- very small files.
-
-`-w SECONDS'
-`--wait=SECONDS'
- Wait the specified number of seconds between the retrievals. Use
- of this option is recommended, as it lightens the server load by
- making the requests less frequent. Instead of in seconds, the
- time can be specified in minutes using the `m' suffix, in hours
- using `h' suffix, or in days using `d' suffix.
-
- Specifying a large value for this option is useful if the network
- or the destination host is down, so that Wget can wait long enough
- to reasonably expect the network error to be fixed before the
- retry.
-
-`--waitretry=SECONDS'
- If you don't want Wget to wait between _every_ retrieval, but only
- between retries of failed downloads, you can use this option.
- Wget will use "linear backoff", waiting 1 second after the first
- failure on a given file, then waiting 2 seconds after the second
- failure on that file, up to the maximum number of SECONDS you
- specify. Therefore, a value of 10 will actually make Wget wait up
- to (1 + 2 + ... + 10) = 55 seconds per file.
-
- Note that this option is turned on by default in the global
- `wgetrc' file.
-
-`--random-wait'
- Some web sites may perform log analysis to identify retrieval
- programs such as Wget by looking for statistically significant
- similarities in the time between requests. This option causes the
- time between requests to vary between 0 and 2 * WAIT seconds,
- where WAIT was specified using the `--wait' option, in order to
- mask Wget's presence from such analysis.
-
- A recent article in a publication devoted to development on a
- popular consumer platform provided code to perform this analysis
- on the fly. Its author suggested blocking at the class C address
- level to ensure automated retrieval programs were blocked despite
- changing DHCP-supplied addresses.
-
- The `--random-wait' option was inspired by this ill-advised
- recommendation to block many unrelated users from a web site due
- to the actions of one.
-
-`--no-proxy'
- Don't use proxies, even if the appropriate `*_proxy' environment
- variable is defined.
-
- For more information about the use of proxies with Wget, *Note
- Proxies::.
-
-`-Q QUOTA'
-`--quota=QUOTA'
- Specify download quota for automatic retrievals. The value can be
- specified in bytes (default), kilobytes (with `k' suffix), or
- megabytes (with `m' suffix).
-
- Note that quota will never affect downloading a single file. So
- if you specify `wget -Q10k ftp://wuarchive.wustl.edu/ls-lR.gz',
- all of the `ls-lR.gz' will be downloaded. The same goes even when
- several URLs are specified on the command-line. However, quota is
- respected when retrieving either recursively, or from an input
- file. Thus you may safely type `wget -Q2m -i sites'--download
- will be aborted when the quota is exceeded.
-
- Setting quota to 0 or to `inf' unlimits the download quota.
-
-`--no-dns-cache'
- Turn off caching of DNS lookups. Normally, Wget remembers the IP
- addresses it looked up from DNS so it doesn't have to repeatedly
- contact the DNS server for the same (typically small) set of hosts
- it retrieves from. This cache exists in memory only; a new Wget
- run will contact DNS again.
-
- However, it has been reported that in some situations it is not
- desirable to cache host names, even for the duration of a
- short-running application like Wget. With this option Wget issues
- a new DNS lookup (more precisely, a new call to `gethostbyname' or
- `getaddrinfo') each time it makes a new connection. Please note
- that this option will _not_ affect caching that might be performed
- by the resolving library or by an external caching layer, such as
- NSCD.
-
- If you don't understand exactly what this option does, you probably
- won't need it.
-
-`--restrict-file-names=MODE'
- Change which characters found in remote URLs may show up in local
- file names generated from those URLs. Characters that are
- "restricted" by this option are escaped, i.e. replaced with `%HH',
- where `HH' is the hexadecimal number that corresponds to the
- restricted character.
-
- By default, Wget escapes the characters that are not valid as part
- of file names on your operating system, as well as control
- characters that are typically unprintable. This option is useful
- for changing these defaults, either because you are downloading to
- a non-native partition, or because you want to disable escaping of
- the control characters.
-
- When mode is set to "unix", Wget escapes the character `/' and the
- control characters in the ranges 0-31 and 128-159. This is the
- default on Unix-like OS'es.
-
- When mode is set to "windows", Wget escapes the characters `\',
- `|', `/', `:', `?', `"', `*', `<', `>', and the control characters
- in the ranges 0-31 and 128-159. In addition to this, Wget in
- Windows mode uses `+' instead of `:' to separate host and port in
- local file names, and uses `@' instead of `?' to separate the
- query portion of the file name from the rest. Therefore, a URL
- that would be saved as `www.xemacs.org:4300/search.pl?input=blah'
- in Unix mode would be saved as
- `www.xemacs.org+4300/search.pl@input=blah' in Windows mode. This
- mode is the default on Windows.
-
- If you append `,nocontrol' to the mode, as in `unix,nocontrol',
- escaping of the control characters is also switched off. You can
- use `--restrict-file-names=nocontrol' to turn off escaping of
- control characters without affecting the choice of the OS to use
- as file name restriction mode.
-
-`-4'
-`--inet4-only'
-`-6'
-`--inet6-only'
- Force connecting to IPv4 or IPv6 addresses. With `--inet4-only'
- or `-4', Wget will only connect to IPv4 hosts, ignoring AAAA
- records in DNS, and refusing to connect to IPv6 addresses
- specified in URLs. Conversely, with `--inet6-only' or `-6', Wget
- will only connect to IPv6 hosts and ignore A records and IPv4
- addresses.
-
- Neither options should be needed normally. By default, an
- IPv6-aware Wget will use the address family specified by the
- host's DNS record. If the DNS responds with both IPv4 and IPv6
- addresses, Wget will them in sequence until it finds one it can
- connect to. (Also see `--prefer-family' option described below.)
-
- These options can be used to deliberately force the use of IPv4 or
- IPv6 address families on dual family systems, usually to aid
- debugging or to deal with broken network configuration. Only one
- of `--inet6-only' and `--inet4-only' may be specified at the same
- time. Neither option is available in Wget compiled without IPv6
- support.
-
-`--prefer-family=IPv4/IPv6/none'
- When given a choice of several addresses, connect to the addresses
- with specified address family first. IPv4 addresses are preferred
- by default.
-
- This avoids spurious errors and connect attempts when accessing
- hosts that resolve to both IPv6 and IPv4 addresses from IPv4
- networks. For example, `www.kame.net' resolves to
- `2001:200:0:8002:203:47ff:fea5:3085' and to `203.178.141.194'.
- When the preferred family is `IPv4', the IPv4 address is used
- first; when the preferred family is `IPv6', the IPv6 address is
- used first; if the specified value is `none', the address order
- returned by DNS is used without change.
-
- Unlike `-4' and `-6', this option doesn't inhibit access to any
- address family, it only changes the _order_ in which the addresses
- are accessed. Also note that the reordering performed by this
- option is "stable"--it doesn't affect order of addresses of the
- same family. That is, the relative order of all IPv4 addresses
- and of all IPv6 addresses remains intact in all cases.
-
-`--retry-connrefused'
- Consider "connection refused" a transient error and try again.
- Normally Wget gives up on a URL when it is unable to connect to the
- site because failure to connect is taken as a sign that the server
- is not running at all and that retries would not help. This
- option is for mirroring unreliable sites whose servers tend to
- disappear for short periods of time.
-
-`--user=USER'
-`--password=PASSWORD'
- Specify the username USER and password PASSWORD for both FTP and
- HTTP file retrieval. These parameters can be overridden using the
- `--ftp-user' and `--ftp-password' options for FTP connections and
- the `--http-user' and `--http-password' options for HTTP
- connections.
-
-
-File: wget.info, Node: Directory Options, Next: HTTP Options, Prev: Download Options, Up: Invoking
-
-2.6 Directory Options
-=====================
-
-`-nd'
-`--no-directories'
- Do not create a hierarchy of directories when retrieving
- recursively. With this option turned on, all files will get saved
- to the current directory, without clobbering (if a name shows up
- more than once, the filenames will get extensions `.n').
-
-`-x'
-`--force-directories'
- The opposite of `-nd'--create a hierarchy of directories, even if
- one would not have been created otherwise. E.g. `wget -x
- http://fly.srk.fer.hr/robots.txt' will save the downloaded file to
- `fly.srk.fer.hr/robots.txt'.
-
-`-nH'
-`--no-host-directories'
- Disable generation of host-prefixed directories. By default,
- invoking Wget with `-r http://fly.srk.fer.hr/' will create a
- structure of directories beginning with `fly.srk.fer.hr/'. This
- option disables such behavior.
-
-`--protocol-directories'
- Use the protocol name as a directory component of local file
- names. For example, with this option, `wget -r http://HOST' will
- save to `http/HOST/...' rather than just to `HOST/...'.
-
-`--cut-dirs=NUMBER'
- Ignore NUMBER directory components. This is useful for getting a
- fine-grained control over the directory where recursive retrieval
- will be saved.
-
- Take, for example, the directory at
- `ftp://ftp.xemacs.org/pub/xemacs/'. If you retrieve it with `-r',
- it will be saved locally under `ftp.xemacs.org/pub/xemacs/'.
- While the `-nH' option can remove the `ftp.xemacs.org/' part, you
- are still stuck with `pub/xemacs'. This is where `--cut-dirs'
- comes in handy; it makes Wget not "see" NUMBER remote directory
- components. Here are several examples of how `--cut-dirs' option
- works.
-
- No options -> ftp.xemacs.org/pub/xemacs/
- -nH -> pub/xemacs/
- -nH --cut-dirs=1 -> xemacs/
- -nH --cut-dirs=2 -> .
-
- --cut-dirs=1 -> ftp.xemacs.org/xemacs/
- ...
-
- If you just want to get rid of the directory structure, this
- option is similar to a combination of `-nd' and `-P'. However,
- unlike `-nd', `--cut-dirs' does not lose with subdirectories--for
- instance, with `-nH --cut-dirs=1', a `beta/' subdirectory will be
- placed to `xemacs/beta', as one would expect.
-
-`-P PREFIX'
-`--directory-prefix=PREFIX'
- Set directory prefix to PREFIX. The "directory prefix" is the
- directory where all other files and subdirectories will be saved
- to, i.e. the top of the retrieval tree. The default is `.' (the
- current directory).
-
-
-File: wget.info, Node: HTTP Options, Next: HTTPS (SSL/TLS) Options, Prev: Directory Options, Up: Invoking
-
-2.7 HTTP Options
-================
-
-`-E'
-`--html-extension'
- If a file of type `application/xhtml+xml' or `text/html' is
- downloaded and the URL does not end with the regexp
- `\.[Hh][Tt][Mm][Ll]?', this option will cause the suffix `.html'
- to be appended to the local filename. This is useful, for
- instance, when you're mirroring a remote site that uses `.asp'
- pages, but you want the mirrored pages to be viewable on your
- stock Apache server. Another good use for this is when you're
- downloading CGI-generated materials. A URL like
- `http://site.com/article.cgi?25' will be saved as
- `article.cgi?25.html'.
-
- Note that filenames changed in this way will be re-downloaded
- every time you re-mirror a site, because Wget can't tell that the
- local `X.html' file corresponds to remote URL `X' (since it
- doesn't yet know that the URL produces output of type `text/html'
- or `application/xhtml+xml'. To prevent this re-downloading, you
- must use `-k' and `-K' so that the original version of the file
- will be saved as `X.orig' (*note Recursive Retrieval Options::).
-
-`--http-user=USER'
-`--http-password=PASSWORD'
- Specify the username USER and password PASSWORD on an HTTP server.
- According to the type of the challenge, Wget will encode them
- using either the `basic' (insecure) or the `digest' authentication
- scheme.
-
- Another way to specify username and password is in the URL itself
- (*note URL Format::). Either method reveals your password to
- anyone who bothers to run `ps'. To prevent the passwords from
- being seen, store them in `.wgetrc' or `.netrc', and make sure to
- protect those files from other users with `chmod'. If the
- passwords are really important, do not leave them lying in those
- files either--edit the files and delete them after Wget has
- started the download.
-
-`--no-cache'
- Disable server-side cache. In this case, Wget will send the remote
- server an appropriate directive (`Pragma: no-cache') to get the
- file from the remote service, rather than returning the cached
- version. This is especially useful for retrieving and flushing
- out-of-date documents on proxy servers.
-
- Caching is allowed by default.
-
-`--no-cookies'
- Disable the use of cookies. Cookies are a mechanism for
- maintaining server-side state. The server sends the client a
- cookie using the `Set-Cookie' header, and the client responds with
- the same cookie upon further requests. Since cookies allow the
- server owners to keep track of visitors and for sites to exchange
- this information, some consider them a breach of privacy. The
- default is to use cookies; however, _storing_ cookies is not on by
- default.
-
-`--load-cookies FILE'
- Load cookies from FILE before the first HTTP retrieval. FILE is a
- textual file in the format originally used by Netscape's
- `cookies.txt' file.
-
- You will typically use this option when mirroring sites that
- require that you be logged in to access some or all of their
- content. The login process typically works by the web server
- issuing an HTTP cookie upon receiving and verifying your
- credentials. The cookie is then resent by the browser when
- accessing that part of the site, and so proves your identity.
-
- Mirroring such a site requires Wget to send the same cookies your
- browser sends when communicating with the site. This is achieved
- by `--load-cookies'--simply point Wget to the location of the
- `cookies.txt' file, and it will send the same cookies your browser
- would send in the same situation. Different browsers keep textual
- cookie files in different locations:
-
- Netscape 4.x.
- The cookies are in `~/.netscape/cookies.txt'.
-
- Mozilla and Netscape 6.x.
- Mozilla's cookie file is also named `cookies.txt', located
- somewhere under `~/.mozilla', in the directory of your
- profile. The full path usually ends up looking somewhat like
- `~/.mozilla/default/SOME-WEIRD-STRING/cookies.txt'.
-
- Internet Explorer.
- You can produce a cookie file Wget can use by using the File
- menu, Import and Export, Export Cookies. This has been
- tested with Internet Explorer 5; it is not guaranteed to work
- with earlier versions.
-
- Other browsers.
- If you are using a different browser to create your cookies,
- `--load-cookies' will only work if you can locate or produce a
- cookie file in the Netscape format that Wget expects.
-
- If you cannot use `--load-cookies', there might still be an
- alternative. If your browser supports a "cookie manager", you can
- use it to view the cookies used when accessing the site you're
- mirroring. Write down the name and value of the cookie, and
- manually instruct Wget to send those cookies, bypassing the
- "official" cookie support:
-
- wget --no-cookies --header "Cookie: NAME=VALUE"
-
-`--save-cookies FILE'
- Save cookies to FILE before exiting. This will not save cookies
- that have expired or that have no expiry time (so-called "session
- cookies"), but also see `--keep-session-cookies'.
-
-`--keep-session-cookies'
- When specified, causes `--save-cookies' to also save session
- cookies. Session cookies are normally not saved because they are
- meant to be kept in memory and forgotten when you exit the browser.
- Saving them is useful on sites that require you to log in or to
- visit the home page before you can access some pages. With this
- option, multiple Wget runs are considered a single browser session
- as far as the site is concerned.
-
- Since the cookie file format does not normally carry session
- cookies, Wget marks them with an expiry timestamp of 0. Wget's
- `--load-cookies' recognizes those as session cookies, but it might
- confuse other browsers. Also note that cookies so loaded will be
- treated as other session cookies, which means that if you want
- `--save-cookies' to preserve them again, you must use
- `--keep-session-cookies' again.
-
-`--ignore-length'
- Unfortunately, some HTTP servers (CGI programs, to be more
- precise) send out bogus `Content-Length' headers, which makes Wget
- go wild, as it thinks not all the document was retrieved. You can
- spot this syndrome if Wget retries getting the same document again
- and again, each time claiming that the (otherwise normal)
- connection has closed on the very same byte.
-
- With this option, Wget will ignore the `Content-Length' header--as
- if it never existed.
-
-`--header=HEADER-LINE'
- Send HEADER-LINE along with the rest of the headers in each HTTP
- request. The supplied header is sent as-is, which means it must
- contain name and value separated by colon, and must not contain
- newlines.
-
- You may define more than one additional header by specifying
- `--header' more than once.
-
- wget --header='Accept-Charset: iso-8859-2' \
- --header='Accept-Language: hr' \
- http://fly.srk.fer.hr/
-
- Specification of an empty string as the header value will clear all
- previous user-defined headers.
-
- As of Wget 1.10, this option can be used to override headers
- otherwise generated automatically. This example instructs Wget to
- connect to localhost, but to specify `foo.bar' in the `Host'
- header:
-
- wget --header="Host: foo.bar" http://localhost/
-
- In versions of Wget prior to 1.10 such use of `--header' caused
- sending of duplicate headers.
-
-`--proxy-user=USER'
-`--proxy-password=PASSWORD'
- Specify the username USER and password PASSWORD for authentication
- on a proxy server. Wget will encode them using the `basic'
- authentication scheme.
-
- Security considerations similar to those with `--http-password'
- pertain here as well.
-
-`--referer=URL'
- Include `Referer: URL' header in HTTP request. Useful for
- retrieving documents with server-side processing that assume they
- are always being retrieved by interactive web browsers and only
- come out properly when Referer is set to one of the pages that
- point to them.
-
-`--save-headers'
- Save the headers sent by the HTTP server to the file, preceding the
- actual contents, with an empty line as the separator.
-
-`-U AGENT-STRING'
-`--user-agent=AGENT-STRING'
- Identify as AGENT-STRING to the HTTP server.
-
- The HTTP protocol allows the clients to identify themselves using a
- `User-Agent' header field. This enables distinguishing the WWW
- software, usually for statistical purposes or for tracing of
- protocol violations. Wget normally identifies as `Wget/VERSION',
- VERSION being the current version number of Wget.
-
- However, some sites have been known to impose the policy of
- tailoring the output according to the `User-Agent'-supplied
- information. While this is not such a bad idea in theory, it has
- been abused by servers denying information to clients other than
- (historically) Netscape or, more frequently, Microsoft Internet
- Explorer. This option allows you to change the `User-Agent' line
- issued by Wget. Use of this option is discouraged, unless you
- really know what you are doing.
-
- Specifying empty user agent with `--user-agent=""' instructs Wget
- not to send the `User-Agent' header in HTTP requests.
-
-`--post-data=STRING'
-`--post-file=FILE'
- Use POST as the method for all HTTP requests and send the
- specified data in the request body. `--post-data' sends STRING as
- data, whereas `--post-file' sends the contents of FILE. Other than
- that, they work in exactly the same way.
-
- Please be aware that Wget needs to know the size of the POST data
- in advance. Therefore the argument to `--post-file' must be a
- regular file; specifying a FIFO or something like `/dev/stdin'
- won't work. It's not quite clear how to work around this
- limitation inherent in HTTP/1.0. Although HTTP/1.1 introduces
- "chunked" transfer that doesn't require knowing the request length
- in advance, a client can't use chunked unless it knows it's
- talking to an HTTP/1.1 server. And it can't know that until it
- receives a response, which in turn requires the request to have
- been completed - a chicken-and-egg problem.
-
- Note: if Wget is redirected after the POST request is completed, it
- will not send the POST data to the redirected URL. This is because
- URLs that process POST often respond with a redirection to a
- regular page, which does not desire or accept POST. It is not
- completely clear that this behavior is optimal; if it doesn't work
- out, it might be changed in the future.
-
- This example shows how to log to a server using POST and then
- proceed to download the desired pages, presumably only accessible
- to authorized users:
-
- # Log in to the server. This can be done only once.
- wget --save-cookies cookies.txt \
- --post-data 'user=foo&password=bar' \
- http://server.com/auth.php
-
- # Now grab the page or pages we care about.
- wget --load-cookies cookies.txt \
- -p http://server.com/interesting/article.php
-
- If the server is using session cookies to track user
- authentication, the above will not work because `--save-cookies'
- will not save them (and neither will browsers) and the
- `cookies.txt' file will be empty. In that case use
- `--keep-session-cookies' along with `--save-cookies' to force
- saving of session cookies.
-
-
-File: wget.info, Node: HTTPS (SSL/TLS) Options, Next: FTP Options, Prev: HTTP Options, Up: Invoking
-
-2.8 HTTPS (SSL/TLS) Options
-===========================
-
-To support encrypted HTTP (HTTPS) downloads, Wget must be compiled with
-an external SSL library, currently OpenSSL. If Wget is compiled
-without SSL support, none of these options are available.
-
-`--secure-protocol=PROTOCOL'
- Choose the secure protocol to be used. Legal values are `auto',
- `SSLv2', `SSLv3', and `TLSv1'. If `auto' is used, the SSL library
- is given the liberty of choosing the appropriate protocol
- automatically, which is achieved by sending an SSLv2 greeting and
- announcing support for SSLv3 and TLSv1. This is the default.
-
- Specifying `SSLv2', `SSLv3', or `TLSv1' forces the use of the
- corresponding protocol. This is useful when talking to old and
- buggy SSL server implementations that make it hard for OpenSSL to
- choose the correct protocol version. Fortunately, such servers are
- quite rare.
-
-`--no-check-certificate'
- Don't check the server certificate against the available
- certificate authorities. Also don't require the URL host name to
- match the common name presented by the certificate.
-
- As of Wget 1.10, the default is to verify the server's certificate
- against the recognized certificate authorities, breaking the SSL
- handshake and aborting the download if the verification fails.
- Although this provides more secure downloads, it does break
- interoperability with some sites that worked with previous Wget
- versions, particularly those using self-signed, expired, or
- otherwise invalid certificates. This option forces an "insecure"
- mode of operation that turns the certificate verification errors
- into warnings and allows you to proceed.
-
- If you encounter "certificate verification" errors or ones saying
- that "common name doesn't match requested host name", you can use
- this option to bypass the verification and proceed with the
- download. _Only use this option if you are otherwise convinced of
- the site's authenticity, or if you really don't care about the
- validity of its certificate._ It is almost always a bad idea not
- to check the certificates when transmitting confidential or
- important data.
-
-`--certificate=FILE'
- Use the client certificate stored in FILE. This is needed for
- servers that are configured to require certificates from the
- clients that connect to them. Normally a certificate is not
- required and this switch is optional.
-
-`--certificate-type=TYPE'
- Specify the type of the client certificate. Legal values are
- `PEM' (assumed by default) and `DER', also known as `ASN1'.
-
-`--private-key=FILE'
- Read the private key from FILE. This allows you to provide the
- private key in a file separate from the certificate.
-
-`--private-key-type=TYPE'
- Specify the type of the private key. Accepted values are `PEM'
- (the default) and `DER'.
-
-`--ca-certificate=FILE'
- Use FILE as the file with the bundle of certificate authorities
- ("CA") to verify the peers. The certificates must be in PEM
- format.
-
- Without this option Wget looks for CA certificates at the
- system-specified locations, chosen at OpenSSL installation time.
-
-`--ca-directory=DIRECTORY'
- Specifies directory containing CA certificates in PEM format. Each
- file contains one CA certificate, and the file name is based on a
- hash value derived from the certificate. This is achieved by
- processing a certificate directory with the `c_rehash' utility
- supplied with OpenSSL. Using `--ca-directory' is more efficient
- than `--ca-certificate' when many certificates are installed
- because it allows Wget to fetch certificates on demand.
-
- Without this option Wget looks for CA certificates at the
- system-specified locations, chosen at OpenSSL installation time.
-
-`--random-file=FILE'
- Use FILE as the source of random data for seeding the
- pseudo-random number generator on systems without `/dev/random'.
-
- On such systems the SSL library needs an external source of
- randomness to initialize. Randomness may be provided by EGD (see
- `--egd-file' below) or read from an external source specified by
- the user. If this option is not specified, Wget looks for random
- data in `$RANDFILE' or, if that is unset, in `$HOME/.rnd'. If
- none of those are available, it is likely that SSL encryption will
- not be usable.
-
- If you're getting the "Could not seed OpenSSL PRNG; disabling SSL."
- error, you should provide random data using some of the methods
- described above.
-
-`--egd-file=FILE'
- Use FILE as the EGD socket. EGD stands for "Entropy Gathering
- Daemon", a user-space program that collects data from various
- unpredictable system sources and makes it available to other
- programs that might need it. Encryption software, such as the SSL
- library, needs sources of non-repeating randomness to seed the
- random number generator used to produce cryptographically strong
- keys.
-
- OpenSSL allows the user to specify his own source of entropy using
- the `RAND_FILE' environment variable. If this variable is unset,
- or if the specified file does not produce enough randomness,
- OpenSSL will read random data from EGD socket specified using this
- option.
-
- If this option is not specified (and the equivalent startup
- command is not used), EGD is never contacted. EGD is not needed
- on modern Unix systems that support `/dev/random'.
-
-
-File: wget.info, Node: FTP Options, Next: Recursive Retrieval Options, Prev: HTTPS (SSL/TLS) Options, Up: Invoking
-
-2.9 FTP Options
-===============
-
-`--ftp-user=USER'
-`--ftp-password=PASSWORD'
- Specify the username USER and password PASSWORD on an FTP server.
- Without this, or the corresponding startup option, the password
- defaults to `-wget@', normally used for anonymous FTP.
-
- Another way to specify username and password is in the URL itself
- (*note URL Format::). Either method reveals your password to
- anyone who bothers to run `ps'. To prevent the passwords from
- being seen, store them in `.wgetrc' or `.netrc', and make sure to
- protect those files from other users with `chmod'. If the
- passwords are really important, do not leave them lying in those
- files either--edit the files and delete them after Wget has
- started the download.
-
-`--no-remove-listing'
- Don't remove the temporary `.listing' files generated by FTP
- retrievals. Normally, these files contain the raw directory
- listings received from FTP servers. Not removing them can be
- useful for debugging purposes, or when you want to be able to
- easily check on the contents of remote server directories (e.g. to
- verify that a mirror you're running is complete).
-
- Note that even though Wget writes to a known filename for this
- file, this is not a security hole in the scenario of a user making
- `.listing' a symbolic link to `/etc/passwd' or something and
- asking `root' to run Wget in his or her directory. Depending on
- the options used, either Wget will refuse to write to `.listing',
- making the globbing/recursion/time-stamping operation fail, or the
- symbolic link will be deleted and replaced with the actual
- `.listing' file, or the listing will be written to a
- `.listing.NUMBER' file.
-
- Even though this situation isn't a problem, though, `root' should
- never run Wget in a non-trusted user's directory. A user could do
- something as simple as linking `index.html' to `/etc/passwd' and
- asking `root' to run Wget with `-N' or `-r' so the file will be
- overwritten.
-
-`--no-glob'
- Turn off FTP globbing. Globbing refers to the use of shell-like
- special characters ("wildcards"), like `*', `?', `[' and `]' to
- retrieve more than one file from the same directory at once, like:
-
- wget ftp://gnjilux.srk.fer.hr/*.msg
-
- By default, globbing will be turned on if the URL contains a
- globbing character. This option may be used to turn globbing on
- or off permanently.
-
- You may have to quote the URL to protect it from being expanded by
- your shell. Globbing makes Wget look for a directory listing,
- which is system-specific. This is why it currently works only
- with Unix FTP servers (and the ones emulating Unix `ls' output).
-
-`--no-passive-ftp'
- Disable the use of the "passive" FTP transfer mode. Passive FTP
- mandates that the client connect to the server to establish the
- data connection rather than the other way around.
-
- If the machine is connected to the Internet directly, both passive
- and active FTP should work equally well. Behind most firewall and
- NAT configurations passive FTP has a better chance of working.
- However, in some rare firewall configurations, active FTP actually
- works when passive FTP doesn't. If you suspect this to be the
- case, use this option, or set `passive_ftp=off' in your init file.
-
-`--retr-symlinks'
- Usually, when retrieving FTP directories recursively and a symbolic
- link is encountered, the linked-to file is not downloaded.
- Instead, a matching symbolic link is created on the local
- filesystem. The pointed-to file will not be downloaded unless
- this recursive retrieval would have encountered it separately and
- downloaded it anyway.
-
- When `--retr-symlinks' is specified, however, symbolic links are
- traversed and the pointed-to files are retrieved. At this time,
- this option does not cause Wget to traverse symlinks to
- directories and recurse through them, but in the future it should
- be enhanced to do this.
-
- Note that when retrieving a file (not a directory) because it was
- specified on the command-line, rather than because it was recursed
- to, this option has no effect. Symbolic links are always
- traversed in this case.
-
-`--no-http-keep-alive'
- Turn off the "keep-alive" feature for HTTP downloads. Normally,
- Wget asks the server to keep the connection open so that, when you
- download more than one document from the same server, they get
- transferred over the same TCP connection. This saves time and at
- the same time reduces the load on the server.
-
- This option is useful when, for some reason, persistent
- (keep-alive) connections don't work for you, for example due to a
- server bug or due to the inability of server-side scripts to cope
- with the connections.
-
-
-File: wget.info, Node: Recursive Retrieval Options, Next: Recursive Accept/Reject Options, Prev: FTP Options, Up: Invoking
-
-2.10 Recursive Retrieval Options
-================================
-
-`-r'
-`--recursive'
- Turn on recursive retrieving. *Note Recursive Download::, for more
- details.
-
-`-l DEPTH'
-`--level=DEPTH'
- Specify recursion maximum depth level DEPTH (*note Recursive
- Download::). The default maximum depth is 5.
-
-`--delete-after'
- This option tells Wget to delete every single file it downloads,
- _after_ having done so. It is useful for pre-fetching popular
- pages through a proxy, e.g.:
-
- wget -r -nd --delete-after http://whatever.com/~popular/page/
-
- The `-r' option is to retrieve recursively, and `-nd' to not
- create directories.
-
- Note that `--delete-after' deletes files on the local machine. It
- does not issue the `DELE' command to remote FTP sites, for
- instance. Also note that when `--delete-after' is specified,
- `--convert-links' is ignored, so `.orig' files are simply not
- created in the first place.
-
-`-k'
-`--convert-links'
- After the download is complete, convert the links in the document
- to make them suitable for local viewing. This affects not only
- the visible hyperlinks, but any part of the document that links to
- external content, such as embedded images, links to style sheets,
- hyperlinks to non-HTML content, etc.
-
- Each link will be changed in one of the two ways:
-
- * The links to files that have been downloaded by Wget will be
- changed to refer to the file they point to as a relative link.
-
- Example: if the downloaded file `/foo/doc.html' links to
- `/bar/img.gif', also downloaded, then the link in `doc.html'
- will be modified to point to `../bar/img.gif'. This kind of
- transformation works reliably for arbitrary combinations of
- directories.
-
- * The links to files that have not been downloaded by Wget will
- be changed to include host name and absolute path of the
- location they point to.
-
- Example: if the downloaded file `/foo/doc.html' links to
- `/bar/img.gif' (or to `../bar/img.gif'), then the link in
- `doc.html' will be modified to point to
- `http://HOSTNAME/bar/img.gif'.
-
- Because of this, local browsing works reliably: if a linked file
- was downloaded, the link will refer to its local name; if it was
- not downloaded, the link will refer to its full Internet address
- rather than presenting a broken link. The fact that the former
- links are converted to relative links ensures that you can move
- the downloaded hierarchy to another directory.
-
- Note that only at the end of the download can Wget know which
- links have been downloaded. Because of that, the work done by
- `-k' will be performed at the end of all the downloads.
-
-`-K'
-`--backup-converted'
- When converting a file, back up the original version with a `.orig'
- suffix. Affects the behavior of `-N' (*note HTTP Time-Stamping
- Internals::).
-
-`-m'
-`--mirror'
- Turn on options suitable for mirroring. This option turns on
- recursion and time-stamping, sets infinite recursion depth and
- keeps FTP directory listings. It is currently equivalent to `-r
- -N -l inf --no-remove-listing'.
-
-`-p'
-`--page-requisites'
- This option causes Wget to download all the files that are
- necessary to properly display a given HTML page. This includes
- such things as inlined images, sounds, and referenced stylesheets.
-
- Ordinarily, when downloading a single HTML page, any requisite
- documents that may be needed to display it properly are not
- downloaded. Using `-r' together with `-l' can help, but since
- Wget does not ordinarily distinguish between external and inlined
- documents, one is generally left with "leaf documents" that are
- missing their requisites.
-
- For instance, say document `1.html' contains an `<IMG>' tag
- referencing `1.gif' and an `<A>' tag pointing to external document
- `2.html'. Say that `2.html' is similar but that its image is
- `2.gif' and it links to `3.html'. Say this continues up to some
- arbitrarily high number.
-
- If one executes the command:
-
- wget -r -l 2 http://SITE/1.html
-
- then `1.html', `1.gif', `2.html', `2.gif', and `3.html' will be
- downloaded. As you can see, `3.html' is without its requisite
- `3.gif' because Wget is simply counting the number of hops (up to
- 2) away from `1.html' in order to determine where to stop the
- recursion. However, with this command:
-
- wget -r -l 2 -p http://SITE/1.html
-
- all the above files _and_ `3.html''s requisite `3.gif' will be
- downloaded. Similarly,
-
- wget -r -l 1 -p http://SITE/1.html
-
- will cause `1.html', `1.gif', `2.html', and `2.gif' to be
- downloaded. One might think that:
-
- wget -r -l 0 -p http://SITE/1.html
-
- would download just `1.html' and `1.gif', but unfortunately this
- is not the case, because `-l 0' is equivalent to `-l inf'--that
- is, infinite recursion. To download a single HTML page (or a
- handful of them, all specified on the command-line or in a `-i'
- URL input file) and its (or their) requisites, simply leave off
- `-r' and `-l':
-
- wget -p http://SITE/1.html
-
- Note that Wget will behave as if `-r' had been specified, but only
- that single page and its requisites will be downloaded. Links
- from that page to external documents will not be followed.
- Actually, to download a single page and all its requisites (even
- if they exist on separate websites), and make sure the lot
- displays properly locally, this author likes to use a few options
- in addition to `-p':
-
- wget -E -H -k -K -p http://SITE/DOCUMENT
-
- To finish off this topic, it's worth knowing that Wget's idea of an
- external document link is any URL specified in an `<A>' tag, an
- `<AREA>' tag, or a `<LINK>' tag other than `<LINK
- REL="stylesheet">'.
-
-`--strict-comments'
- Turn on strict parsing of HTML comments. The default is to
- terminate comments at the first occurrence of `-->'.
-
- According to specifications, HTML comments are expressed as SGML
- "declarations". Declaration is special markup that begins with
- `<!' and ends with `>', such as `<!DOCTYPE ...>', that may contain
- comments between a pair of `--' delimiters. HTML comments are
- "empty declarations", SGML declarations without any non-comment
- text. Therefore, `<!--foo-->' is a valid comment, and so is
- `<!--one-- --two-->', but `<!--1--2-->' is not.
-
- On the other hand, most HTML writers don't perceive comments as
- anything other than text delimited with `<!--' and `-->', which is
- not quite the same. For example, something like `<!------------>'
- works as a valid comment as long as the number of dashes is a
- multiple of four (!). If not, the comment technically lasts until
- the next `--', which may be at the other end of the document.
- Because of this, many popular browsers completely ignore the
- specification and implement what users have come to expect:
- comments delimited with `<!--' and `-->'.
-
- Until version 1.9, Wget interpreted comments strictly, which
- resulted in missing links in many web pages that displayed fine in
- browsers, but had the misfortune of containing non-compliant
- comments. Beginning with version 1.9, Wget has joined the ranks
- of clients that implements "naive" comments, terminating each
- comment at the first occurrence of `-->'.
-
- If, for whatever reason, you want strict comment parsing, use this
- option to turn it on.
-
-
-File: wget.info, Node: Recursive Accept/Reject Options, Prev: Recursive Retrieval Options, Up: Invoking
-
-2.11 Recursive Accept/Reject Options
-====================================
-
-`-A ACCLIST --accept ACCLIST'
-`-R REJLIST --reject REJLIST'
- Specify comma-separated lists of file name suffixes or patterns to
- accept or reject (*note Types of Files:: for more details).
-
-`-D DOMAIN-LIST'
-`--domains=DOMAIN-LIST'
- Set domains to be followed. DOMAIN-LIST is a comma-separated list
- of domains. Note that it does _not_ turn on `-H'.
-
-`--exclude-domains DOMAIN-LIST'
- Specify the domains that are _not_ to be followed. (*note
- Spanning Hosts::).
-
-`--follow-ftp'
- Follow FTP links from HTML documents. Without this option, Wget
- will ignore all the FTP links.
-
-`--follow-tags=LIST'
- Wget has an internal table of HTML tag / attribute pairs that it
- considers when looking for linked documents during a recursive
- retrieval. If a user wants only a subset of those tags to be
- considered, however, he or she should be specify such tags in a
- comma-separated LIST with this option.
-
-`--ignore-tags=LIST'
- This is the opposite of the `--follow-tags' option. To skip
- certain HTML tags when recursively looking for documents to
- download, specify them in a comma-separated LIST.
-
- In the past, this option was the best bet for downloading a single
- page and its requisites, using a command-line like:
-
- wget --ignore-tags=a,area -H -k -K -r http://SITE/DOCUMENT
-
- However, the author of this option came across a page with tags
- like `<LINK REL="home" HREF="/">' and came to the realization that
- specifying tags to ignore was not enough. One can't just tell
- Wget to ignore `<LINK>', because then stylesheets will not be
- downloaded. Now the best bet for downloading a single page and
- its requisites is the dedicated `--page-requisites' option.
-
-`-H'
-`--span-hosts'
- Enable spanning across hosts when doing recursive retrieving
- (*note Spanning Hosts::).
-
-`-L'
-`--relative'
- Follow relative links only. Useful for retrieving a specific home
- page without any distractions, not even those from the same hosts
- (*note Relative Links::).
-
-`-I LIST'
-`--include-directories=LIST'
- Specify a comma-separated list of directories you wish to follow
- when downloading (*note Directory-Based Limits:: for more
- details.) Elements of LIST may contain wildcards.
-
-`-X LIST'
-`--exclude-directories=LIST'
- Specify a comma-separated list of directories you wish to exclude
- from download (*note Directory-Based Limits:: for more details.)
- Elements of LIST may contain wildcards.
-
-`-np'
-
-`--no-parent'
- Do not ever ascend to the parent directory when retrieving
- recursively. This is a useful option, since it guarantees that
- only the files _below_ a certain hierarchy will be downloaded.
- *Note Directory-Based Limits::, for more details.
-
-
-File: wget.info, Node: Recursive Download, Next: Following Links, Prev: Invoking, Up: Top
-
-3 Recursive Download
-********************
-
-GNU Wget is capable of traversing parts of the Web (or a single HTTP or
-FTP server), following links and directory structure. We refer to this
-as to "recursive retrieval", or "recursion".
-
- With HTTP URLs, Wget retrieves and parses the HTML from the given
-URL, documents, retrieving the files the HTML document was referring
-to, through markup like `href', or `src'. If the freshly downloaded
-file is also of type `text/html' or `application/xhtml+xml', it will be
-parsed and followed further.
-
- Recursive retrieval of HTTP and HTML content is "breadth-first".
-This means that Wget first downloads the requested HTML document, then
-the documents linked from that document, then the documents linked by
-them, and so on. In other words, Wget first downloads the documents at
-depth 1, then those at depth 2, and so on until the specified maximum
-depth.
-
- The maximum "depth" to which the retrieval may descend is specified
-with the `-l' option. The default maximum depth is five layers.
-
- When retrieving an FTP URL recursively, Wget will retrieve all the
-data from the given directory tree (including the subdirectories up to
-the specified depth) on the remote server, creating its mirror image
-locally. FTP retrieval is also limited by the `depth' parameter.
-Unlike HTTP recursion, FTP recursion is performed depth-first.
-
- By default, Wget will create a local directory tree, corresponding to
-the one found on the remote server.
-
- Recursive retrieving can find a number of applications, the most
-important of which is mirroring. It is also useful for WWW
-presentations, and any other opportunities where slow network
-connections should be bypassed by storing the files locally.
-
- You should be warned that recursive downloads can overload the remote
-servers. Because of that, many administrators frown upon them and may
-ban access from your site if they detect very fast downloads of big
-amounts of content. When downloading from Internet servers, consider
-using the `-w' option to introduce a delay between accesses to the
-server. The download will take a while longer, but the server
-administrator will not be alarmed by your rudeness.
-
- Of course, recursive download may cause problems on your machine. If
-left to run unchecked, it can easily fill up the disk. If downloading
-from local network, it can also take bandwidth on the system, as well as
-consume memory and CPU.
-
- Try to specify the criteria that match the kind of download you are
-trying to achieve. If you want to download only one page, use
-`--page-requisites' without any additional recursion. If you want to
-download things under one directory, use `-np' to avoid downloading
-things from other directories. If you want to download all the files
-from one directory, use `-l 1' to make sure the recursion depth never
-exceeds one. *Note Following Links::, for more information about this.
-
- Recursive retrieval should be used with care. Don't say you were not
-warned.
-
-
-File: wget.info, Node: Following Links, Next: Time-Stamping, Prev: Recursive Download, Up: Top
-
-4 Following Links
-*****************
-
-When retrieving recursively, one does not wish to retrieve loads of
-unnecessary data. Most of the time the users bear in mind exactly what
-they want to download, and want Wget to follow only specific links.
-
- For example, if you wish to download the music archive from
-`fly.srk.fer.hr', you will not want to download all the home pages that
-happen to be referenced by an obscure part of the archive.
-
- Wget possesses several mechanisms that allows you to fine-tune which
-links it will follow.
-
-* Menu:
-
-* Spanning Hosts:: (Un)limiting retrieval based on host name.
-* Types of Files:: Getting only certain files.
-* Directory-Based Limits:: Getting only certain directories.
-* Relative Links:: Follow relative links only.
-* FTP Links:: Following FTP links.
-
-
-File: wget.info, Node: Spanning Hosts, Next: Types of Files, Up: Following Links
-
-4.1 Spanning Hosts
-==================
-
-Wget's recursive retrieval normally refuses to visit hosts different
-than the one you specified on the command line. This is a reasonable
-default; without it, every retrieval would have the potential to turn
-your Wget into a small version of google.
-
- However, visiting different hosts, or "host spanning," is sometimes
-a useful option. Maybe the images are served from a different server.
-Maybe you're mirroring a site that consists of pages interlinked between
-three servers. Maybe the server has two equivalent names, and the HTML
-pages refer to both interchangeably.
-
-Span to any host--`-H'
- The `-H' option turns on host spanning, thus allowing Wget's
- recursive run to visit any host referenced by a link. Unless
- sufficient recursion-limiting criteria are applied depth, these
- foreign hosts will typically link to yet more hosts, and so on
- until Wget ends up sucking up much more data than you have
- intended.
-
-Limit spanning to certain domains--`-D'
- The `-D' option allows you to specify the domains that will be
- followed, thus limiting the recursion only to the hosts that
- belong to these domains. Obviously, this makes sense only in
- conjunction with `-H'. A typical example would be downloading the
- contents of `www.server.com', but allowing downloads from
- `images.server.com', etc.:
-
- wget -rH -Dserver.com http://www.server.com/
-
- You can specify more than one address by separating them with a
- comma, e.g. `-Ddomain1.com,domain2.com'.
-
-Keep download off certain domains--`--exclude-domains'
- If there are domains you want to exclude specifically, you can do
- it with `--exclude-domains', which accepts the same type of
- arguments of `-D', but will _exclude_ all the listed domains. For
- example, if you want to download all the hosts from `foo.edu'
- domain, with the exception of `sunsite.foo.edu', you can do it like
- this:
-
- wget -rH -Dfoo.edu --exclude-domains sunsite.foo.edu \
- http://www.foo.edu/
-
-
-
-File: wget.info, Node: Types of Files, Next: Directory-Based Limits, Prev: Spanning Hosts, Up: Following Links
-
-4.2 Types of Files
-==================
-
-When downloading material from the web, you will often want to restrict
-the retrieval to only certain file types. For example, if you are
-interested in downloading GIFs, you will not be overjoyed to get loads
-of PostScript documents, and vice versa.
-
- Wget offers two options to deal with this problem. Each option
-description lists a short name, a long name, and the equivalent command
-in `.wgetrc'.
-
-`-A ACCLIST'
-`--accept ACCLIST'
-`accept = ACCLIST'
- The argument to `--accept' option is a list of file suffixes or
- patterns that Wget will download during recursive retrieval. A
- suffix is the ending part of a file, and consists of "normal"
- letters, e.g. `gif' or `.jpg'. A matching pattern contains
- shell-like wildcards, e.g. `books*' or `zelazny*196[0-9]*'.
-
- So, specifying `wget -A gif,jpg' will make Wget download only the
- files ending with `gif' or `jpg', i.e. GIFs and JPEGs. On the
- other hand, `wget -A "zelazny*196[0-9]*"' will download only files
- beginning with `zelazny' and containing numbers from 1960 to 1969
- anywhere within. Look up the manual of your shell for a
- description of how pattern matching works.
-
- Of course, any number of suffixes and patterns can be combined
- into a comma-separated list, and given as an argument to `-A'.
-
-`-R REJLIST'
-`--reject REJLIST'
-`reject = REJLIST'
- The `--reject' option works the same way as `--accept', only its
- logic is the reverse; Wget will download all files _except_ the
- ones matching the suffixes (or patterns) in the list.
-
- So, if you want to download a whole page except for the cumbersome
- MPEGs and .AU files, you can use `wget -R mpg,mpeg,au'.
- Analogously, to download all files except the ones beginning with
- `bjork', use `wget -R "bjork*"'. The quotes are to prevent
- expansion by the shell.
-
- The `-A' and `-R' options may be combined to achieve even better
-fine-tuning of which files to retrieve. E.g. `wget -A "*zelazny*" -R
-.ps' will download all the files having `zelazny' as a part of their
-name, but _not_ the PostScript files.
-
- Note that these two options do not affect the downloading of HTML
-files; Wget must load all the HTMLs to know where to go at
-all--recursive retrieval would make no sense otherwise.
-
-
-File: wget.info, Node: Directory-Based Limits, Next: Relative Links, Prev: Types of Files, Up: Following Links
-
-4.3 Directory-Based Limits
-==========================
-
-Regardless of other link-following facilities, it is often useful to
-place the restriction of what files to retrieve based on the directories
-those files are placed in. There can be many reasons for this--the
-home pages may be organized in a reasonable directory structure; or some
-directories may contain useless information, e.g. `/cgi-bin' or `/dev'
-directories.
-
- Wget offers three different options to deal with this requirement.
-Each option description lists a short name, a long name, and the
-equivalent command in `.wgetrc'.
-
-`-I LIST'
-`--include LIST'
-`include_directories = LIST'
- `-I' option accepts a comma-separated list of directories included
- in the retrieval. Any other directories will simply be ignored.
- The directories are absolute paths.
-
- So, if you wish to download from `http://host/people/bozo/'
- following only links to bozo's colleagues in the `/people'
- directory and the bogus scripts in `/cgi-bin', you can specify:
-
- wget -I /people,/cgi-bin http://host/people/bozo/
-
-`-X LIST'
-`--exclude LIST'
-`exclude_directories = LIST'
- `-X' option is exactly the reverse of `-I'--this is a list of
- directories _excluded_ from the download. E.g. if you do not want
- Wget to download things from `/cgi-bin' directory, specify `-X
- /cgi-bin' on the command line.
-
- The same as with `-A'/`-R', these two options can be combined to
- get a better fine-tuning of downloading subdirectories. E.g. if
- you want to load all the files from `/pub' hierarchy except for
- `/pub/worthless', specify `-I/pub -X/pub/worthless'.
-
-`-np'
-`--no-parent'
-`no_parent = on'
- The simplest, and often very useful way of limiting directories is
- disallowing retrieval of the links that refer to the hierarchy
- "above" than the beginning directory, i.e. disallowing ascent to
- the parent directory/directories.
-
- The `--no-parent' option (short `-np') is useful in this case.
- Using it guarantees that you will never leave the existing
- hierarchy. Supposing you issue Wget with:
-
- wget -r --no-parent http://somehost/~luzer/my-archive/
-
- You may rest assured that none of the references to
- `/~his-girls-homepage/' or `/~luzer/all-my-mpegs/' will be
- followed. Only the archive you are interested in will be
- downloaded. Essentially, `--no-parent' is similar to
- `-I/~luzer/my-archive', only it handles redirections in a more
- intelligent fashion.
-
-
-File: wget.info, Node: Relative Links, Next: FTP Links, Prev: Directory-Based Limits, Up: Following Links
-
-4.4 Relative Links
-==================
-
-When `-L' is turned on, only the relative links are ever followed.
-Relative links are here defined those that do not refer to the web
-server root. For example, these links are relative:
-
- <a href="foo.gif">
- <a href="foo/bar.gif">
- <a href="../foo/bar.gif">
-
- These links are not relative:
-
- <a href="/foo.gif">
- <a href="/foo/bar.gif">
- <a href="http://www.server.com/foo/bar.gif">
-
- Using this option guarantees that recursive retrieval will not span
-hosts, even without `-H'. In simple cases it also allows downloads to
-"just work" without having to convert links.
-
- This option is probably not very useful and might be removed in a
-future release.
-
-
-File: wget.info, Node: FTP Links, Prev: Relative Links, Up: Following Links
-
-4.5 Following FTP Links
-=======================
-
-The rules for FTP are somewhat specific, as it is necessary for them to
-be. FTP links in HTML documents are often included for purposes of
-reference, and it is often inconvenient to download them by default.
-
- To have FTP links followed from HTML documents, you need to specify
-the `--follow-ftp' option. Having done that, FTP links will span hosts
-regardless of `-H' setting. This is logical, as FTP links rarely point
-to the same host where the HTTP server resides. For similar reasons,
-the `-L' options has no effect on such downloads. On the other hand,
-domain acceptance (`-D') and suffix rules (`-A' and `-R') apply
-normally.
-
- Also note that followed links to FTP directories will not be
-retrieved recursively further.
-
-
-File: wget.info, Node: Time-Stamping, Next: Startup File, Prev: Following Links, Up: Top
-
-5 Time-Stamping
-***************
-
-One of the most important aspects of mirroring information from the
-Internet is updating your archives.
-
- Downloading the whole archive again and again, just to replace a few
-changed files is expensive, both in terms of wasted bandwidth and money,
-and the time to do the update. This is why all the mirroring tools
-offer the option of incremental updating.
-
- Such an updating mechanism means that the remote server is scanned in
-search of "new" files. Only those new files will be downloaded in the
-place of the old ones.
-
- A file is considered new if one of these two conditions are met:
-
- 1. A file of that name does not already exist locally.
-
- 2. A file of that name does exist, but the remote file was modified
- more recently than the local file.
-
- To implement this, the program needs to be aware of the time of last
-modification of both local and remote files. We call this information
-the "time-stamp" of a file.
-
- The time-stamping in GNU Wget is turned on using `--timestamping'
-(`-N') option, or through `timestamping = on' directive in `.wgetrc'.
-With this option, for each file it intends to download, Wget will check
-whether a local file of the same name exists. If it does, and the
-remote file is older, Wget will not download it.
-
- If the local file does not exist, or the sizes of the files do not
-match, Wget will download the remote file no matter what the time-stamps
-say.
-
-* Menu:
-
-* Time-Stamping Usage::
-* HTTP Time-Stamping Internals::
-* FTP Time-Stamping Internals::
-
-
-File: wget.info, Node: Time-Stamping Usage, Next: HTTP Time-Stamping Internals, Up: Time-Stamping
-
-5.1 Time-Stamping Usage
-=======================
-
-The usage of time-stamping is simple. Say you would like to download a
-file so that it keeps its date of modification.
-
- wget -S http://www.gnu.ai.mit.edu/
-
- A simple `ls -l' shows that the time stamp on the local file equals
-the state of the `Last-Modified' header, as returned by the server. As
-you can see, the time-stamping info is preserved locally, even without
-`-N' (at least for HTTP).
-
- Several days later, you would like Wget to check if the remote file
-has changed, and download it if it has.
-
- wget -N http://www.gnu.ai.mit.edu/
-
- Wget will ask the server for the last-modified date. If the local
-file has the same timestamp as the server, or a newer one, the remote
-file will not be re-fetched. However, if the remote file is more
-recent, Wget will proceed to fetch it.
-
- The same goes for FTP. For example:
-
- wget "ftp://ftp.ifi.uio.no/pub/emacs/gnus/*"
-
- (The quotes around that URL are to prevent the shell from trying to
-interpret the `*'.)
-
- After download, a local directory listing will show that the
-timestamps match those on the remote server. Reissuing the command
-with `-N' will make Wget re-fetch _only_ the files that have been
-modified since the last download.
-
- If you wished to mirror the GNU archive every week, you would use a
-command like the following, weekly:
-
- wget --timestamping -r ftp://ftp.gnu.org/pub/gnu/
-
- Note that time-stamping will only work for files for which the server
-gives a timestamp. For HTTP, this depends on getting a `Last-Modified'
-header. For FTP, this depends on getting a directory listing with
-dates in a format that Wget can parse (*note FTP Time-Stamping
-Internals::).
-
-
-File: wget.info, Node: HTTP Time-Stamping Internals, Next: FTP Time-Stamping Internals, Prev: Time-Stamping Usage, Up: Time-Stamping
-
-5.2 HTTP Time-Stamping Internals
-================================
-
-Time-stamping in HTTP is implemented by checking of the `Last-Modified'
-header. If you wish to retrieve the file `foo.html' through HTTP, Wget
-will check whether `foo.html' exists locally. If it doesn't,
-`foo.html' will be retrieved unconditionally.
-
- If the file does exist locally, Wget will first check its local
-time-stamp (similar to the way `ls -l' checks it), and then send a
-`HEAD' request to the remote server, demanding the information on the
-remote file.
-
- The `Last-Modified' header is examined to find which file was
-modified more recently (which makes it "newer"). If the remote file is
-newer, it will be downloaded; if it is older, Wget will give up.(1)
-
- When `--backup-converted' (`-K') is specified in conjunction with
-`-N', server file `X' is compared to local file `X.orig', if extant,
-rather than being compared to local file `X', which will always differ
-if it's been converted by `--convert-links' (`-k').
-
- Arguably, HTTP time-stamping should be implemented using the
-`If-Modified-Since' request.
-
- ---------- Footnotes ----------
-
- (1) As an additional check, Wget will look at the `Content-Length'
-header, and compare the sizes; if they are not the same, the remote
-file will be downloaded no matter what the time-stamp says.
-
-
-File: wget.info, Node: FTP Time-Stamping Internals, Prev: HTTP Time-Stamping Internals, Up: Time-Stamping
-
-5.3 FTP Time-Stamping Internals
-===============================
-
-In theory, FTP time-stamping works much the same as HTTP, only FTP has
-no headers--time-stamps must be ferreted out of directory listings.
-
- If an FTP download is recursive or uses globbing, Wget will use the
-FTP `LIST' command to get a file listing for the directory containing
-the desired file(s). It will try to analyze the listing, treating it
-like Unix `ls -l' output, extracting the time-stamps. The rest is
-exactly the same as for HTTP. Note that when retrieving individual
-files from an FTP server without using globbing or recursion, listing
-files will not be downloaded (and thus files will not be time-stamped)
-unless `-N' is specified.
-
- Assumption that every directory listing is a Unix-style listing may
-sound extremely constraining, but in practice it is not, as many
-non-Unix FTP servers use the Unixoid listing format because most (all?)
-of the clients understand it. Bear in mind that RFC959 defines no
-standard way to get a file list, let alone the time-stamps. We can
-only hope that a future standard will define this.
-
- Another non-standard solution includes the use of `MDTM' command
-that is supported by some FTP servers (including the popular
-`wu-ftpd'), which returns the exact time of the specified file. Wget
-may support this command in the future.
-
-
-File: wget.info, Node: Startup File, Next: Examples, Prev: Time-Stamping, Up: Top
-
-6 Startup File
-**************
-
-Once you know how to change default settings of Wget through command
-line arguments, you may wish to make some of those settings permanent.
-You can do that in a convenient way by creating the Wget startup
-file--`.wgetrc'.
-
- Besides `.wgetrc' is the "main" initialization file, it is
-convenient to have a special facility for storing passwords. Thus Wget
-reads and interprets the contents of `$HOME/.netrc', if it finds it.
-You can find `.netrc' format in your system manuals.
-
- Wget reads `.wgetrc' upon startup, recognizing a limited set of
-commands.
-
-* Menu:
-
-* Wgetrc Location:: Location of various wgetrc files.
-* Wgetrc Syntax:: Syntax of wgetrc.
-* Wgetrc Commands:: List of available commands.
-* Sample Wgetrc:: A wgetrc example.
-
-
-File: wget.info, Node: Wgetrc Location, Next: Wgetrc Syntax, Up: Startup File
-
-6.1 Wgetrc Location
-===================
-
-When initializing, Wget will look for a "global" startup file,
-`/usr/local/etc/wgetrc' by default (or some prefix other than
-`/usr/local', if Wget was not installed there) and read commands from
-there, if it exists.
-
- Then it will look for the user's file. If the environmental variable
-`WGETRC' is set, Wget will try to load that file. Failing that, no
-further attempts will be made.
-
- If `WGETRC' is not set, Wget will try to load `$HOME/.wgetrc'.
-
- The fact that user's settings are loaded after the system-wide ones
-means that in case of collision user's wgetrc _overrides_ the
-system-wide wgetrc (in `/usr/local/etc/wgetrc' by default). Fascist
-admins, away!
-
-
-File: wget.info, Node: Wgetrc Syntax, Next: Wgetrc Commands, Prev: Wgetrc Location, Up: Startup File
-
-6.2 Wgetrc Syntax
-=================
-
-The syntax of a wgetrc command is simple:
-
- variable = value
-
- The "variable" will also be called "command". Valid "values" are
-different for different commands.
-
- The commands are case-insensitive and underscore-insensitive. Thus
-`DIr__PrefiX' is the same as `dirprefix'. Empty lines, lines beginning
-with `#' and lines containing white-space only are discarded.
-
- Commands that expect a comma-separated list will clear the list on an
-empty command. So, if you wish to reset the rejection list specified in
-global `wgetrc', you can do it with:
-
- reject =
-
-
-File: wget.info, Node: Wgetrc Commands, Next: Sample Wgetrc, Prev: Wgetrc Syntax, Up: Startup File
-
-6.3 Wgetrc Commands
-===================
-
-The complete set of commands is listed below. Legal values are listed
-after the `='. Simple Boolean values can be set or unset using `on'
-and `off' or `1' and `0'. A fancier kind of Boolean allowed in some
-cases is the "lockable Boolean", which may be set to `on', `off',
-`always', or `never'. If an option is set to `always' or `never', that
-value will be locked in for the duration of the Wget
-invocation--command-line options will not override.
-
- Some commands take pseudo-arbitrary values. ADDRESS values can be
-hostnames or dotted-quad IP addresses. N can be any positive integer,
-or `inf' for infinity, where appropriate. STRING values can be any
-non-empty string.
-
- Most of these commands have direct command-line equivalents. Also,
-any wgetrc command can be specified on the command line using the
-`--execute' switch (*note Basic Startup Options::.)
-
-accept/reject = STRING
- Same as `-A'/`-R' (*note Types of Files::).
-
-add_hostdir = on/off
- Enable/disable host-prefixed file names. `-nH' disables it.
-
-continue = on/off
- If set to on, force continuation of preexistent partially retrieved
- files. See `-c' before setting it.
-
-background = on/off
- Enable/disable going to background--the same as `-b' (which
- enables it).
-
-backup_converted = on/off
- Enable/disable saving pre-converted files with the suffix
- `.orig'--the same as `-K' (which enables it).
-
-base = STRING
- Consider relative URLs in URL input files forced to be interpreted
- as HTML as being relative to STRING--the same as `--base=STRING'.
-
-bind_address = ADDRESS
- Bind to ADDRESS, like the `--bind-address=ADDRESS'.
-
-ca_certificate = FILE
- Set the certificate authority bundle file to FILE. The same as
- `--ca-certificate=FILE'.
-
-ca_directory = DIRECTORY
- Set the directory used for certificate authorities. The same as
- `--ca-directory=DIRECTORY'.
-
-cache = on/off
- When set to off, disallow server-caching. See the `--no-cache'
- option.
-
-certificate = FILE
- Set the client certificate file name to FILE. The same as
- `--certificate=FILE'.
-
-certificate_type = STRING
- Specify the type of the client certificate, legal values being
- `PEM' (the default) and `DER' (aka ASN1). The same as
- `--certificate-type=STRING'.
-
-check_certificate = on/off
- If this is set to off, the server certificate is not checked
- against the specified client authorities. The default is "on".
- The same as `--check-certificate'.
-
-convert_links = on/off
- Convert non-relative links locally. The same as `-k'.
-
-cookies = on/off
- When set to off, disallow cookies. See the `--cookies' option.
-
-connect_timeout = N
- Set the connect timeout--the same as `--connect-timeout'.
-
-cut_dirs = N
- Ignore N remote directory components. Equivalent to
- `--cut-dirs=N'.
-
-debug = on/off
- Debug mode, same as `-d'.
-
-delete_after = on/off
- Delete after download--the same as `--delete-after'.
-
-dir_prefix = STRING
- Top of directory tree--the same as `-P STRING'.
-
-dirstruct = on/off
- Turning dirstruct on or off--the same as `-x' or `-nd',
- respectively.
-
-dns_cache = on/off
- Turn DNS caching on/off. Since DNS caching is on by default, this
- option is normally used to turn it off and is equivalent to
- `--no-dns-cache'.
-
-dns_timeout = N
- Set the DNS timeout--the same as `--dns-timeout'.
-
-domains = STRING
- Same as `-D' (*note Spanning Hosts::).
-
-dot_bytes = N
- Specify the number of bytes "contained" in a dot, as seen
- throughout the retrieval (1024 by default). You can postfix the
- value with `k' or `m', representing kilobytes and megabytes,
- respectively. With dot settings you can tailor the dot retrieval
- to suit your needs, or you can use the predefined "styles" (*note
- Download Options::).
-
-dots_in_line = N
- Specify the number of dots that will be printed in each line
- throughout the retrieval (50 by default).
-
-dot_spacing = N
- Specify the number of dots in a single cluster (10 by default).
-
-egd_file = FILE
- Use STRING as the EGD socket file name. The same as
- `--egd-file=FILE'.
-
-exclude_directories = STRING
- Specify a comma-separated list of directories you wish to exclude
- from download--the same as `-X STRING' (*note Directory-Based
- Limits::).
-
-exclude_domains = STRING
- Same as `--exclude-domains=STRING' (*note Spanning Hosts::).
-
-follow_ftp = on/off
- Follow FTP links from HTML documents--the same as `--follow-ftp'.
-
-follow_tags = STRING
- Only follow certain HTML tags when doing a recursive retrieval,
- just like `--follow-tags=STRING'.
-
-force_html = on/off
- If set to on, force the input filename to be regarded as an HTML
- document--the same as `-F'.
-
-ftp_password = STRING
- Set your FTP password to STRING. Without this setting, the
- password defaults to `-wget@', which is a useful default for
- anonymous FTP access.
-
- This command used to be named `passwd' prior to Wget 1.10.
-
-ftp_proxy = STRING
- Use STRING as FTP proxy, instead of the one specified in
- environment.
-
-ftp_user = STRING
- Set FTP user to STRING.
-
- This command used to be named `login' prior to Wget 1.10.
-
-glob = on/off
- Turn globbing on/off--the same as `--glob' and `--no-glob'.
-
-header = STRING
- Define a header for HTTP doewnloads, like using `--header=STRING'.
-
-html_extension = on/off
- Add a `.html' extension to `text/html' or `application/xhtml+xml'
- files without it, like `-E'.
-
-http_keep_alive = on/off
- Turn the keep-alive feature on or off (defaults to on). Turning it
- off is equivalent to `--no-http-keep-alive'.
-
-http_password = STRING
- Set HTTP password, equivalent to `--http-password=STRING'.
-
-http_proxy = STRING
- Use STRING as HTTP proxy, instead of the one specified in
- environment.
-
-http_user = STRING
- Set HTTP user to STRING, equivalent to `--http-user=STRING'.
-
-ignore_length = on/off
- When set to on, ignore `Content-Length' header; the same as
- `--ignore-length'.
-
-ignore_tags = STRING
- Ignore certain HTML tags when doing a recursive retrieval, like
- `--ignore-tags=STRING'.
-
-include_directories = STRING
- Specify a comma-separated list of directories you wish to follow
- when downloading--the same as `-I STRING'.
-
-inet4_only = on/off
- Force connecting to IPv4 addresses, off by default. You can put
- this in the global init file to disable Wget's attempts to resolve
- and connect to IPv6 hosts. Available only if Wget was compiled
- with IPv6 support. The same as `--inet4-only' or `-4'.
-
-inet6_only = on/off
- Force connecting to IPv6 addresses, off by default. Available
- only if Wget was compiled with IPv6 support. The same as
- `--inet6-only' or `-6'.
-
-input = FILE
- Read the URLs from STRING, like `-i FILE'.
-
-limit_rate = RATE
- Limit the download speed to no more than RATE bytes per second.
- The same as `--limit-rate=RATE'.
-
-load_cookies = FILE
- Load cookies from FILE. See `--load-cookies FILE'.
-
-logfile = FILE
- Set logfile to FILE, the same as `-o FILE'.
-
-mirror = on/off
- Turn mirroring on/off. The same as `-m'.
-
-netrc = on/off
- Turn reading netrc on or off.
-
-noclobber = on/off
- Same as `-nc'.
-
-no_parent = on/off
- Disallow retrieving outside the directory hierarchy, like
- `--no-parent' (*note Directory-Based Limits::).
-
-no_proxy = STRING
- Use STRING as the comma-separated list of domains to avoid in
- proxy loading, instead of the one specified in environment.
-
-output_document = FILE
- Set the output filename--the same as `-O FILE'.
-
-page_requisites = on/off
- Download all ancillary documents necessary for a single HTML page
- to display properly--the same as `-p'.
-
-passive_ftp = on/off/always/never
- Change setting of passive FTP, equivalent to the `--passive-ftp'
- option. Some scripts and `.pm' (Perl module) files download files
- using `wget --passive-ftp'. If your firewall does not allow this,
- you can set `passive_ftp = never' to override the command-line.
-
-password = STRING
- Specify password STRING for both FTP and HTTP file retrieval.
- This command can be overridden using the `ftp_password' and
- `http_password' command for FTP and HTTP respectively.
-
-post_data = STRING
- Use POST as the method for all HTTP requests and send STRING in
- the request body. The same as `--post-data=STRING'.
-
-post_file = FILE
- Use POST as the method for all HTTP requests and send the contents
- of FILE in the request body. The same as `--post-file=FILE'.
-
-prefer_family = IPv4/IPv6/none
- When given a choice of several addresses, connect to the addresses
- with specified address family first. IPv4 addresses are preferred
- by default. The same as `--prefer-family', which see for a
- detailed discussion of why this is useful.
-
-private_key = FILE
- Set the private key file to FILE. The same as
- `--private-key=FILE'.
-
-private_key_type = STRING
- Specify the type of the private key, legal values being `PEM' (the
- default) and `DER' (aka ASN1). The same as
- `--private-type=STRING'.
-
-progress = STRING
- Set the type of the progress indicator. Legal types are `dot' and
- `bar'. Equivalent to `--progress=STRING'.
-
-protocol_directories = on/off
- When set, use the protocol name as a directory component of local
- file names. The same as `--protocol-directories'.
-
-proxy_user = STRING
- Set proxy authentication user name to STRING, like
- `--proxy-user=STRING'.
-
-proxy_password = STRING
- Set proxy authentication password to STRING, like
- `--proxy-password=STRING'.
-
-quiet = on/off
- Quiet mode--the same as `-q'.
-
-quota = QUOTA
- Specify the download quota, which is useful to put in the global
- `wgetrc'. When download quota is specified, Wget will stop
- retrieving after the download sum has become greater than quota.
- The quota can be specified in bytes (default), kbytes `k'
- appended) or mbytes (`m' appended). Thus `quota = 5m' will set
- the quota to 5 megabytes. Note that the user's startup file
- overrides system settings.
-
-random_file = FILE
- Use FILE as a source of randomness on systems lacking
- `/dev/random'.
-
-read_timeout = N
- Set the read (and write) timeout--the same as `--read-timeout=N'.
-
-reclevel = N
- Recursion level (depth)--the same as `-l N'.
-
-recursive = on/off
- Recursive on/off--the same as `-r'.
-
-referer = STRING
- Set HTTP `Referer:' header just like `--referer=STRING'. (Note it
- was the folks who wrote the HTTP spec who got the spelling of
- "referrer" wrong.)
-
-relative_only = on/off
- Follow only relative links--the same as `-L' (*note Relative
- Links::).
-
-remove_listing = on/off
- If set to on, remove FTP listings downloaded by Wget. Setting it
- to off is the same as `--no-remove-listing'.
-
-restrict_file_names = unix/windows
- Restrict the file names generated by Wget from URLs. See
- `--restrict-file-names' for a more detailed description.
-
-retr_symlinks = on/off
- When set to on, retrieve symbolic links as if they were plain
- files; the same as `--retr-symlinks'.
-
-retry_connrefused = on/off
- When set to on, consider "connection refused" a transient
- error--the same as `--retry-connrefused'.
-
-robots = on/off
- Specify whether the norobots convention is respected by Wget, "on"
- by default. This switch controls both the `/robots.txt' and the
- `nofollow' aspect of the spec. *Note Robot Exclusion::, for more
- details about this. Be sure you know what you are doing before
- turning this off.
-
-save_cookies = FILE
- Save cookies to FILE. The same as `--save-cookies FILE'.
-
-secure_protocol = STRING
- Choose the secure protocol to be used. Legal values are `auto'
- (the default), `SSLv2', `SSLv3', and `TLSv1'. The same as
- `--secure-protocol=STRING'.
-
-server_response = on/off
- Choose whether or not to print the HTTP and FTP server
- responses--the same as `-S'.
-
-span_hosts = on/off
- Same as `-H'.
-
-strict_comments = on/off
- Same as `--strict-comments'.
-
-timeout = N
- Set all applicable timeout values to N, the same as `-T N'.
-
-timestamping = on/off
- Turn timestamping on/off. The same as `-N' (*note
- Time-Stamping::).
-
-tries = N
- Set number of retries per URL--the same as `-t N'.
-
-use_proxy = on/off
- When set to off, don't use proxy even when proxy-related
- environment variables are set. In that case it is the same as
- using `--no-proxy'.
-
-user = STRING
- Specify username STRING for both FTP and HTTP file retrieval.
- This command can be overridden using the `ftp_user' and
- `http_user' command for FTP and HTTP respectively.
-
-verbose = on/off
- Turn verbose on/off--the same as `-v'/`-nv'.
-
-wait = N
- Wait N seconds between retrievals--the same as `-w N'.
-
-waitretry = N
- Wait up to N seconds between retries of failed retrievals
- only--the same as `--waitretry=N'. Note that this is turned on by
- default in the global `wgetrc'.
-
-randomwait = on/off
- Turn random between-request wait times on or off. The same as
- `--random-wait'.
-
-
-File: wget.info, Node: Sample Wgetrc, Prev: Wgetrc Commands, Up: Startup File
-
-6.4 Sample Wgetrc
-=================
-
-This is the sample initialization file, as given in the distribution.
-It is divided in two section--one for global usage (suitable for global
-startup file), and one for local usage (suitable for `$HOME/.wgetrc').
-Be careful about the things you change.
-
- Note that almost all the lines are commented out. For a command to
-have any effect, you must remove the `#' character at the beginning of
-its line.
-
- ###
- ### Sample Wget initialization file .wgetrc
- ###
-
- ## You can use this file to change the default behaviour of wget or to
- ## avoid having to type many many command-line options. This file does
- ## not contain a comprehensive list of commands -- look at the manual
- ## to find out what you can put into this file.
- ##
- ## Wget initialization file can reside in /usr/local/etc/wgetrc
- ## (global, for all users) or $HOME/.wgetrc (for a single user).
- ##
- ## To use the settings in this file, you will have to uncomment them,
- ## as well as change them, in most cases, as the values on the
- ## commented-out lines are the default values (e.g. "off").
-
-
- ##
- ## Global settings (useful for setting up in /usr/local/etc/wgetrc).
- ## Think well before you change them, since they may reduce wget's
- ## functionality, and make it behave contrary to the documentation:
- ##
-
- # You can set retrieve quota for beginners by specifying a value
- # optionally followed by 'K' (kilobytes) or 'M' (megabytes). The
- # default quota is unlimited.
- #quota = inf
-
- # You can lower (or raise) the default number of retries when
- # downloading a file (default is 20).
- #tries = 20
-
- # Lowering the maximum depth of the recursive retrieval is handy to
- # prevent newbies from going too "deep" when they unwittingly start
- # the recursive retrieval. The default is 5.
- #reclevel = 5
-
- # By default Wget uses "passive FTP" transfer where the client
- # initiates the data connection to the server rather than the other
- # way around. That is required on systems behind NAT where the client
- # computer cannot be easily reached from the Internet. However, some
- # firewalls software explicitly supports active FTP and in fact has
- # problems supporting passive transfer. If you are in such
- # environment, use "passive_ftp = off" to revert to active FTP.
- #passive_ftp = off
-
- # The "wait" command below makes Wget wait between every connection.
- # If, instead, you want Wget to wait only between retries of failed
- # downloads, set waitretry to maximum number of seconds to wait (Wget
- # will use "linear backoff", waiting 1 second after the first failure
- # on a file, 2 seconds after the second failure, etc. up to this max).
- waitretry = 10
-
-
- ##
- ## Local settings (for a user to set in his $HOME/.wgetrc). It is
- ## *highly* undesirable to put these settings in the global file, since
- ## they are potentially dangerous to "normal" users.
- ##
- ## Even when setting up your own ~/.wgetrc, you should know what you
- ## are doing before doing so.
- ##
-
- # Set this to on to use timestamping by default:
- #timestamping = off
-
- # It is a good idea to make Wget send your email address in a `From:'
- # header with your request (so that server administrators can contact
- # you in case of errors). Wget does *not* send `From:' by default.
- #header = From: Your Name <username@site.domain>
-
- # You can set up other headers, like Accept-Language. Accept-Language
- # is *not* sent by default.
- #header = Accept-Language: en
-
- # You can set the default proxies for Wget to use for http and ftp.
- # They will override the value in the environment.
- #http_proxy = http://proxy.yoyodyne.com:18023/
- #ftp_proxy = http://proxy.yoyodyne.com:18023/
-
- # If you do not want to use proxy at all, set this to off.
- #use_proxy = on
-
- # You can customize the retrieval outlook. Valid options are default,
- # binary, mega and micro.
- #dot_style = default
-
- # Setting this to off makes Wget not download /robots.txt. Be sure to
- # know *exactly* what /robots.txt is and how it is used before changing
- # the default!
- #robots = on
-
- # It can be useful to make Wget wait between connections. Set this to
- # the number of seconds you want Wget to wait.
- #wait = 0
-
- # You can force creating directory structure, even if a single is being
- # retrieved, by setting this to on.
- #dirstruct = off
-
- # You can turn on recursive retrieving by default (don't do this if
- # you are not sure you know what it means) by setting this to on.
- #recursive = off
-
- # To always back up file X as X.orig before converting its links (due
- # to -k / --convert-links / convert_links = on having been specified),
- # set this variable to on:
- #backup_converted = off
-
- # To have Wget follow FTP links from HTML files by default, set this
- # to on:
- #follow_ftp = off
-
-
-File: wget.info, Node: Examples, Next: Various, Prev: Startup File, Up: Top
-
-7 Examples
-**********
-
-The examples are divided into three sections loosely based on their
-complexity.
-
-* Menu:
-
-* Simple Usage:: Simple, basic usage of the program.
-* Advanced Usage:: Advanced tips.
-* Very Advanced Usage:: The hairy stuff.
-
-
-File: wget.info, Node: Simple Usage, Next: Advanced Usage, Up: Examples
-
-7.1 Simple Usage
-================
-
- * Say you want to download a URL. Just type:
-
- wget http://fly.srk.fer.hr/
-
- * But what will happen if the connection is slow, and the file is
- lengthy? The connection will probably fail before the whole file
- is retrieved, more than once. In this case, Wget will try getting
- the file until it either gets the whole of it, or exceeds the
- default number of retries (this being 20). It is easy to change
- the number of tries to 45, to insure that the whole file will
- arrive safely:
-
- wget --tries=45 http://fly.srk.fer.hr/jpg/flyweb.jpg
-
- * Now let's leave Wget to work in the background, and write its
- progress to log file `log'. It is tiring to type `--tries', so we
- shall use `-t'.
-
- wget -t 45 -o log http://fly.srk.fer.hr/jpg/flyweb.jpg &
-
- The ampersand at the end of the line makes sure that Wget works in
- the background. To unlimit the number of retries, use `-t inf'.
-
- * The usage of FTP is as simple. Wget will take care of login and
- password.
-
- wget ftp://gnjilux.srk.fer.hr/welcome.msg
-
- * If you specify a directory, Wget will retrieve the directory
- listing, parse it and convert it to HTML. Try:
-
- wget ftp://ftp.gnu.org/pub/gnu/
- links index.html
-
-
-File: wget.info, Node: Advanced Usage, Next: Very Advanced Usage, Prev: Simple Usage, Up: Examples
-
-7.2 Advanced Usage
-==================
-
- * You have a file that contains the URLs you want to download? Use
- the `-i' switch:
-
- wget -i FILE
-
- If you specify `-' as file name, the URLs will be read from
- standard input.
-
- * Create a five levels deep mirror image of the GNU web site, with
- the same directory structure the original has, with only one try
- per document, saving the log of the activities to `gnulog':
-
- wget -r http://www.gnu.org/ -o gnulog
-
- * The same as the above, but convert the links in the HTML files to
- point to local files, so you can view the documents off-line:
-
- wget --convert-links -r http://www.gnu.org/ -o gnulog
-
- * Retrieve only one HTML page, but make sure that all the elements
- needed for the page to be displayed, such as inline images and
- external style sheets, are also downloaded. Also make sure the
- downloaded page references the downloaded links.
-
- wget -p --convert-links http://www.server.com/dir/page.html
-
- The HTML page will be saved to `www.server.com/dir/page.html', and
- the images, stylesheets, etc., somewhere under `www.server.com/',
- depending on where they were on the remote server.
-
- * The same as the above, but without the `www.server.com/' directory.
- In fact, I don't want to have all those random server directories
- anyway--just save _all_ those files under a `download/'
- subdirectory of the current directory.
-
- wget -p --convert-links -nH -nd -Pdownload \
- http://www.server.com/dir/page.html
-
- * Retrieve the index.html of `www.lycos.com', showing the original
- server headers:
-
- wget -S http://www.lycos.com/
-
- * Save the server headers with the file, perhaps for post-processing.
-
- wget --save-headers http://www.lycos.com/
- more index.html
-
- * Retrieve the first two levels of `wuarchive.wustl.edu', saving them
- to `/tmp'.
-
- wget -r -l2 -P/tmp ftp://wuarchive.wustl.edu/
-
- * You want to download all the GIFs from a directory on an HTTP
- server. You tried `wget http://www.server.com/dir/*.gif', but that
- didn't work because HTTP retrieval does not support globbing. In
- that case, use:
-
- wget -r -l1 --no-parent -A.gif http://www.server.com/dir/
-
- More verbose, but the effect is the same. `-r -l1' means to
- retrieve recursively (*note Recursive Download::), with maximum
- depth of 1. `--no-parent' means that references to the parent
- directory are ignored (*note Directory-Based Limits::), and
- `-A.gif' means to download only the GIF files. `-A "*.gif"' would
- have worked too.
-
- * Suppose you were in the middle of downloading, when Wget was
- interrupted. Now you do not want to clobber the files already
- present. It would be:
-
- wget -nc -r http://www.gnu.org/
-
- * If you want to encode your own username and password to HTTP or
- FTP, use the appropriate URL syntax (*note URL Format::).
-
- wget ftp://hniksic:mypassword@unix.server.com/.emacs
-
- Note, however, that this usage is not advisable on multi-user
- systems because it reveals your password to anyone who looks at
- the output of `ps'.
-
- * You would like the output documents to go to standard output
- instead of to files?
-
- wget -O - http://jagor.srce.hr/ http://www.srce.hr/
-
- You can also combine the two options and make pipelines to
- retrieve the documents from remote hotlists:
-
- wget -O - http://cool.list.com/ | wget --force-html -i -
-
-
-File: wget.info, Node: Very Advanced Usage, Prev: Advanced Usage, Up: Examples
-
-7.3 Very Advanced Usage
-=======================
-
- * If you wish Wget to keep a mirror of a page (or FTP
- subdirectories), use `--mirror' (`-m'), which is the shorthand for
- `-r -l inf -N'. You can put Wget in the crontab file asking it to
- recheck a site each Sunday:
-
- crontab
- 0 0 * * 0 wget --mirror http://www.gnu.org/ -o /home/me/weeklog
-
- * In addition to the above, you want the links to be converted for
- local viewing. But, after having read this manual, you know that
- link conversion doesn't play well with timestamping, so you also
- want Wget to back up the original HTML files before the
- conversion. Wget invocation would look like this:
-
- wget --mirror --convert-links --backup-converted \
- http://www.gnu.org/ -o /home/me/weeklog
-
- * But you've also noticed that local viewing doesn't work all that
- well when HTML files are saved under extensions other than `.html',
- perhaps because they were served as `index.cgi'. So you'd like
- Wget to rename all the files served with content-type `text/html'
- or `application/xhtml+xml' to `NAME.html'.
-
- wget --mirror --convert-links --backup-converted \
- --html-extension -o /home/me/weeklog \
- http://www.gnu.org/
-
- Or, with less typing:
-
- wget -m -k -K -E http://www.gnu.org/ -o /home/me/weeklog
-
-
-File: wget.info, Node: Various, Next: Appendices, Prev: Examples, Up: Top
-
-8 Various
-*********
-
-This chapter contains all the stuff that could not fit anywhere else.
-
-* Menu:
-
-* Proxies:: Support for proxy servers
-* Distribution:: Getting the latest version.
-* Mailing List:: Wget mailing list for announcements and discussion.
-* Reporting Bugs:: How and where to report bugs.
-* Portability:: The systems Wget works on.
-* Signals:: Signal-handling performed by Wget.
-
-
-File: wget.info, Node: Proxies, Next: Distribution, Up: Various
-
-8.1 Proxies
-===========
-
-"Proxies" are special-purpose HTTP servers designed to transfer data
-from remote servers to local clients. One typical use of proxies is
-lightening network load for users behind a slow connection. This is
-achieved by channeling all HTTP and FTP requests through the proxy
-which caches the transferred data. When a cached resource is requested
-again, proxy will return the data from cache. Another use for proxies
-is for companies that separate (for security reasons) their internal
-networks from the rest of Internet. In order to obtain information
-from the Web, their users connect and retrieve remote data using an
-authorized proxy.
-
- Wget supports proxies for both HTTP and FTP retrievals. The
-standard way to specify proxy location, which Wget recognizes, is using
-the following environment variables:
-
-`http_proxy'
- This variable should contain the URL of the proxy for HTTP
- connections.
-
-`ftp_proxy'
- This variable should contain the URL of the proxy for FTP
- connections. It is quite common that HTTP_PROXY and FTP_PROXY are
- set to the same URL.
-
-`no_proxy'
- This variable should contain a comma-separated list of domain
- extensions proxy should _not_ be used for. For instance, if the
- value of `no_proxy' is `.mit.edu', proxy will not be used to
- retrieve documents from MIT.
-
- In addition to the environment variables, proxy location and settings
-may be specified from within Wget itself.
-
-`--no-proxy'
-`proxy = on/off'
- This option and the corresponding command may be used to suppress
- the use of proxy, even if the appropriate environment variables
- are set.
-
-`http_proxy = URL'
-`ftp_proxy = URL'
-`no_proxy = STRING'
- These startup file variables allow you to override the proxy
- settings specified by the environment.
-
- Some proxy servers require authorization to enable you to use them.
-The authorization consists of "username" and "password", which must be
-sent by Wget. As with HTTP authorization, several authentication
-schemes exist. For proxy authorization only the `Basic' authentication
-scheme is currently implemented.
-
- You may specify your username and password either through the proxy
-URL or through the command-line options. Assuming that the company's
-proxy is located at `proxy.company.com' at port 8001, a proxy URL
-location containing authorization data might look like this:
-
- http://hniksic:mypassword@proxy.company.com:8001/
-
- Alternatively, you may use the `proxy-user' and `proxy-password'
-options, and the equivalent `.wgetrc' settings `proxy_user' and
-`proxy_password' to set the proxy username and password.
-
-
-File: wget.info, Node: Distribution, Next: Mailing List, Prev: Proxies, Up: Various
-
-8.2 Distribution
-================
-
-Like all GNU utilities, the latest version of Wget can be found at the
-master GNU archive site ftp.gnu.org, and its mirrors. For example,
-Wget 1.10.2 can be found at
-`ftp://ftp.gnu.org/pub/gnu/wget/wget-1.10.2.tar.gz'
-
-
-File: wget.info, Node: Mailing List, Next: Reporting Bugs, Prev: Distribution, Up: Various
-
-8.3 Mailing List
-================
-
-There are several Wget-related mailing lists, all hosted by SunSITE.dk.
-The general discussion list is at <wget@sunsite.dk>. It is the
-preferred place for bug reports and suggestions, as well as for
-discussion of development. You are invited to subscribe.
-
- To subscribe, simply send mail to <wget-subscribe@sunsite.dk> and
-follow the instructions. Unsubscribe by mailing to
-<wget-unsubscribe@sunsite.dk>. The mailing list is archived at
-`http://www.mail-archive.com/wget%40sunsite.dk/' and at
-`http://news.gmane.org/gmane.comp.web.wget.general'.
-
- The second mailing list is at <wget-patches@sunsite.dk>, and is used
-to submit patches for review by Wget developers. A "patch" is a
-textual representation of change to source code, readable by both
-humans and programs. The file `PATCHES' that comes with Wget covers
-the creation and submitting of patches in detail. Please don't send
-general suggestions or bug reports to `wget-patches'; use it only for
-patch submissions.
-
- To subscribe, simply send mail to <wget-subscribe@sunsite.dk> and
-follow the instructions. Unsubscribe by mailing to
-<wget-unsubscribe@sunsite.dk>. The mailing list is archived at
-`http://news.gmane.org/gmane.comp.web.wget.patches'.
-
-
-File: wget.info, Node: Reporting Bugs, Next: Portability, Prev: Mailing List, Up: Various
-
-8.4 Reporting Bugs
-==================
-
-You are welcome to send bug reports about GNU Wget to
-<bug-wget@gnu.org>.
-
- Before actually submitting a bug report, please try to follow a few
-simple guidelines.
-
- 1. Please try to ascertain that the behavior you see really is a bug.
- If Wget crashes, it's a bug. If Wget does not behave as
- documented, it's a bug. If things work strange, but you are not
- sure about the way they are supposed to work, it might well be a
- bug.
-
- 2. Try to repeat the bug in as simple circumstances as possible.
- E.g. if Wget crashes while downloading `wget -rl0 -kKE -t5 -Y0
- http://yoyodyne.com -o /tmp/log', you should try to see if the
- crash is repeatable, and if will occur with a simpler set of
- options. You might even try to start the download at the page
- where the crash occurred to see if that page somehow triggered the
- crash.
-
- Also, while I will probably be interested to know the contents of
- your `.wgetrc' file, just dumping it into the debug message is
- probably a bad idea. Instead, you should first try to see if the
- bug repeats with `.wgetrc' moved out of the way. Only if it turns
- out that `.wgetrc' settings affect the bug, mail me the relevant
- parts of the file.
-
- 3. Please start Wget with `-d' option and send us the resulting
- output (or relevant parts thereof). If Wget was compiled without
- debug support, recompile it--it is _much_ easier to trace bugs
- with debug support on.
-
- Note: please make sure to remove any potentially sensitive
- information from the debug log before sending it to the bug
- address. The `-d' won't go out of its way to collect sensitive
- information, but the log _will_ contain a fairly complete
- transcript of Wget's communication with the server, which may
- include passwords and pieces of downloaded data. Since the bug
- address is publically archived, you may assume that all bug
- reports are visible to the public.
-
- 4. If Wget has crashed, try to run it in a debugger, e.g. `gdb `which
- wget` core' and type `where' to get the backtrace. This may not
- work if the system administrator has disabled core files, but it is
- safe to try.
-
-
-File: wget.info, Node: Portability, Next: Signals, Prev: Reporting Bugs, Up: Various
-
-8.5 Portability
-===============
-
-Like all GNU software, Wget works on the GNU system. However, since it
-uses GNU Autoconf for building and configuring, and mostly avoids using
-"special" features of any particular Unix, it should compile (and work)
-on all common Unix flavors.
-
- Various Wget versions have been compiled and tested under many kinds
-of Unix systems, including GNU/Linux, Solaris, SunOS 4.x, OSF (aka
-Digital Unix or Tru64), Ultrix, *BSD, IRIX, AIX, and others. Some of
-those systems are no longer in widespread use and may not be able to
-support recent versions of Wget. If Wget fails to compile on your
-system, we would like to know about it.
-
- Thanks to kind contributors, this version of Wget compiles and works
-on 32-bit Microsoft Windows platforms. It has been compiled
-successfully using MS Visual C++ 6.0, Watcom, Borland C, and GCC
-compilers. Naturally, it is crippled of some features available on
-Unix, but it should work as a substitute for people stuck with Windows.
-Note that Windows-specific portions of Wget are not guaranteed to be
-supported in the future, although this has been the case in practice
-for many years now. All questions and problems in Windows usage should
-be reported to Wget mailing list at <wget@sunsite.dk> where the
-volunteers who maintain the Windows-related features might look at them.
-
-
-File: wget.info, Node: Signals, Prev: Portability, Up: Various
-
-8.6 Signals
-===========
-
-Since the purpose of Wget is background work, it catches the hangup
-signal (`SIGHUP') and ignores it. If the output was on standard
-output, it will be redirected to a file named `wget-log'. Otherwise,
-`SIGHUP' is ignored. This is convenient when you wish to redirect the
-output of Wget after having started it.
-
- $ wget http://www.gnus.org/dist/gnus.tar.gz &
- ...
- $ kill -HUP %%
- SIGHUP received, redirecting output to `wget-log'.
-
- Other than that, Wget will not try to interfere with signals in any
-way. `C-c', `kill -TERM' and `kill -KILL' should kill it alike.
-
-
-File: wget.info, Node: Appendices, Next: Copying, Prev: Various, Up: Top
-
-9 Appendices
-************
-
-This chapter contains some references I consider useful.
-
-* Menu:
-
-* Robot Exclusion:: Wget's support for RES.
-* Security Considerations:: Security with Wget.
-* Contributors:: People who helped.
-
-
-File: wget.info, Node: Robot Exclusion, Next: Security Considerations, Up: Appendices
-
-9.1 Robot Exclusion
-===================
-
-It is extremely easy to make Wget wander aimlessly around a web site,
-sucking all the available data in progress. `wget -r SITE', and you're
-set. Great? Not for the server admin.
-
- As long as Wget is only retrieving static pages, and doing it at a
-reasonable rate (see the `--wait' option), there's not much of a
-problem. The trouble is that Wget can't tell the difference between the
-smallest static page and the most demanding CGI. A site I know has a
-section handled by a CGI Perl script that converts Info files to HTML on
-the fly. The script is slow, but works well enough for human users
-viewing an occasional Info file. However, when someone's recursive Wget
-download stumbles upon the index page that links to all the Info files
-through the script, the system is brought to its knees without providing
-anything useful to the user (This task of converting Info files could be
-done locally and access to Info documentation for all installed GNU
-software on a system is available from the `info' command).
-
- To avoid this kind of accident, as well as to preserve privacy for
-documents that need to be protected from well-behaved robots, the
-concept of "robot exclusion" was invented. The idea is that the server
-administrators and document authors can specify which portions of the
-site they wish to protect from robots and those they will permit access.
-
- The most popular mechanism, and the de facto standard supported by
-all the major robots, is the "Robots Exclusion Standard" (RES) written
-by Martijn Koster et al. in 1994. It specifies the format of a text
-file containing directives that instruct the robots which URL paths to
-avoid. To be found by the robots, the specifications must be placed in
-`/robots.txt' in the server root, which the robots are expected to
-download and parse.
-
- Although Wget is not a web robot in the strictest sense of the word,
-it can downloads large parts of the site without the user's
-intervention to download an individual page. Because of that, Wget
-honors RES when downloading recursively. For instance, when you issue:
-
- wget -r http://www.server.com/
-
- First the index of `www.server.com' will be downloaded. If Wget
-finds that it wants to download more documents from that server, it will
-request `http://www.server.com/robots.txt' and, if found, use it for
-further downloads. `robots.txt' is loaded only once per each server.
-
- Until version 1.8, Wget supported the first version of the standard,
-written by Martijn Koster in 1994 and available at
-`http://www.robotstxt.org/wc/norobots.html'. As of version 1.8, Wget
-has supported the additional directives specified in the internet draft
-`<draft-koster-robots-00.txt>' titled "A Method for Web Robots
-Control". The draft, which has as far as I know never made to an RFC,
-is available at `http://www.robotstxt.org/wc/norobots-rfc.txt'.
-
- This manual no longer includes the text of the Robot Exclusion
-Standard.
-
- The second, less known mechanism, enables the author of an individual
-document to specify whether they want the links from the file to be
-followed by a robot. This is achieved using the `META' tag, like this:
-
- <meta name="robots" content="nofollow">
-
- This is explained in some detail at
-`http://www.robotstxt.org/wc/meta-user.html'. Wget supports this
-method of robot exclusion in addition to the usual `/robots.txt'
-exclusion.
-
- If you know what you are doing and really really wish to turn off the
-robot exclusion, set the `robots' variable to `off' in your `.wgetrc'.
-You can achieve the same effect from the command line using the `-e'
-switch, e.g. `wget -e robots=off URL...'.
-
-
-File: wget.info, Node: Security Considerations, Next: Contributors, Prev: Robot Exclusion, Up: Appendices
-
-9.2 Security Considerations
-===========================
-
-When using Wget, you must be aware that it sends unencrypted passwords
-through the network, which may present a security problem. Here are the
-main issues, and some solutions.
-
- 1. The passwords on the command line are visible using `ps'. The best
- way around it is to use `wget -i -' and feed the URLs to Wget's
- standard input, each on a separate line, terminated by `C-d'.
- Another workaround is to use `.netrc' to store passwords; however,
- storing unencrypted passwords is also considered a security risk.
-
- 2. Using the insecure "basic" authentication scheme, unencrypted
- passwords are transmitted through the network routers and gateways.
-
- 3. The FTP passwords are also in no way encrypted. There is no good
- solution for this at the moment.
-
- 4. Although the "normal" output of Wget tries to hide the passwords,
- debugging logs show them, in all forms. This problem is avoided by
- being careful when you send debug logs (yes, even when you send
- them to me).
-
-
-File: wget.info, Node: Contributors, Prev: Security Considerations, Up: Appendices
-
-9.3 Contributors
-================
-
-GNU Wget was written by Hrvoje Niksic <hniksic@xemacs.org>. However,
-its development could never have gone as far as it has, were it not for
-the help of many people, either with bug reports, feature proposals,
-patches, or letters saying "Thanks!".
-
-Special thanks goes to the following people (no particular order):
-
- * Karsten Thygesen--donated system resources such as the mailing
- list, web space, and FTP space, along with a lot of time to make
- these actually work.
-
- * Shawn McHorse--bug reports and patches.
-
- * Kaveh R. Ghazi--on-the-fly `ansi2knr'-ization. Lots of
- portability fixes.
-
- * Gordon Matzigkeit--`.netrc' support.
-
- * Zlatko Calusic, Tomislav Vujec and Drazen Kacar--feature
- suggestions and "philosophical" discussions.
-
- * Darko Budor--initial port to Windows.
-
- * Antonio Rosella--help and suggestions, plus the Italian
- translation.
-
- * Tomislav Petrovic, Mario Mikocevic--many bug reports and
- suggestions.
-
- * Francois Pinard--many thorough bug reports and discussions.
-
- * Karl Eichwalder--lots of help with internationalization and other
- things.
-
- * Junio Hamano--donated support for Opie and HTTP `Digest'
- authentication.
-
- * The people who provided donations for development, including Brian
- Gough.
-
- The following people have provided patches, bug/build reports, useful
-suggestions, beta testing services, fan mail and all the other things
-that make maintenance so much fun:
-
- Ian Abbott Tim Adam, Adrian Aichner, Martin Baehr, Dieter Baron,
-Roger Beeman, Dan Berger, T. Bharath, Christian Biere, Paul Bludov,
-Daniel Bodea, Mark Boyns, John Burden, Wanderlei Cavassin, Gilles Cedoc,
-Tim Charron, Noel Cragg, Kristijan Conkas, John Daily, Andreas Damm,
-Ahmon Dancy, Andrew Davison, Bertrand Demiddelaer, Andrew Deryabin,
-Ulrich Drepper, Marc Duponcheel, Damir Dzeko, Alan Eldridge,
-Hans-Andreas Engel, Aleksandar Erkalovic, Andy Eskilsson, Christian
-Fraenkel, David Fritz, Charles C. Fu, FUJISHIMA Satsuki, Masashi Fujita,
-Howard Gayle, Marcel Gerrits, Lemble Gregory, Hans Grobler, Mathieu
-Guillaume, Dan Harkless, Aaron Hawley, Herold Heiko, Jochen Hein, Karl
-Heuer, HIROSE Masaaki, Ulf Harnhammar, Gregor Hoffleit, Erik Magnus
-Hulthen, Richard Huveneers, Jonas Jensen, Larry Jones, Simon Josefsson,
-Mario Juric, Hack Kampbjorn, Const Kaplinsky, Goran Kezunovic, Igor
-Khristophorov, Robert Kleine, KOJIMA Haime, Fila Kolodny, Alexander
-Kourakos, Martin Kraemer, Sami Krank, Simos KSenitellis, Christian
-Lackas, Hrvoje Lacko, Daniel S. Lewart, Nicolas Lichtmeier, Dave Love,
-Alexander V. Lukyanov, Thomas Lussnig, Andre Majorel, Aurelien Marchand,
-Matthew J. Mellon, Jordan Mendelson, Lin Zhe Min, Jan Minar, Tim Mooney,
-Keith Moore, Adam D. Moss, Simon Munton, Charlie Negyesi, R. K. Owen,
-Leonid Petrov, Simone Piunno, Andrew Pollock, Steve Pothier, Jan
-Prikryl, Marin Purgar, Csaba Raduly, Keith Refson, Bill Richardson,
-Tyler Riddle, Tobias Ringstrom, Juan Jose Rodriguez, Maciej W. Rozycki,
-Edward J. Sabol, Heinz Salzmann, Robert Schmidt, Nicolas Schodet,
-Andreas Schwab, Chris Seawood, Dennis Smit, Toomas Soome, Tage
-Stabell-Kulo, Philip Stadermann, Daniel Stenberg, Sven Sternberger,
-Markus Strasser, John Summerfield, Szakacsits Szabolcs, Mike Thomas,
-Philipp Thomas, Mauro Tortonesi, Dave Turner, Gisle Vanem, Russell
-Vincent, Zeljko Vrba, Charles G Waldman, Douglas E. Wegscheid, YAMAZAKI
-Makoto, Jasmin Zainul, Bojan Zdrnja, Kristijan Zimmer.
-
- Apologies to all who I accidentally left out, and many thanks to all
-the subscribers of the Wget mailing list.
-
-
-File: wget.info, Node: Copying, Next: Concept Index, Prev: Appendices, Up: Top
-
-10 Copying
-**********
-
-GNU Wget is licensed under the GNU General Public License (GNU GPL),
-which makes it "free software". Please note that "free" in "free
-software" refers to liberty, not price. As some people like to point
-out, it's the "free" of "free speech", not the "free" of "free beer".
-
- The exact and legally binding distribution terms are spelled out
-below. The GPL guarantees that you have the right (freedom) to run and
-change GNU Wget and distribute it to others, and even--if you
-want--charge money for doing any of those things. With these rights
-comes the obligation to distribute the source code along with the
-software and to grant your recipients the same rights and impose the
-same restrictions.
-
- This licensing model is also known as "open source" because it,
-among other things, makes sure that all recipients will receive the
-source code along with the program, and be able to improve it. The GNU
-project prefers the term "free software" for reasons outlined at
-`http://www.gnu.org/philosophy/free-software-for-freedom.html'.
-
- The exact license terms are defined by this paragraph and the GNU
-General Public License it refers to:
-
- GNU Wget is free software; you can redistribute it and/or modify it
- under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- GNU Wget is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- General Public License for more details.
-
- A copy of the GNU General Public License is included as part of
- this manual; if you did not receive it, write to the Free Software
- Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
-
- In addition to this, this manual is free in the same sense:
-
- Permission is granted to copy, distribute and/or modify this
- document under the terms of the GNU Free Documentation License,
- Version 1.2 or any later version published by the Free Software
- Foundation; with the Invariant Sections being "GNU General Public
- License" and "GNU Free Documentation License", with no Front-Cover
- Texts, and with no Back-Cover Texts. A copy of the license is
- included in the section entitled "GNU Free Documentation License".
-
- The full texts of the GNU General Public License and of the GNU Free
-Documentation License are available below.
-
-* Menu:
-
-* GNU General Public License::
-* GNU Free Documentation License::
-
-
-File: wget.info, Node: GNU General Public License, Next: GNU Free Documentation License, Up: Copying
-
-10.1 GNU General Public License
-===============================
-
- Version 2, June 1991
-
- Copyright (C) 1989, 1991 Free Software Foundation, Inc.
- 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
-
- Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
-
-Preamble
-========
-
-The licenses for most software are designed to take away your freedom
-to share and change it. By contrast, the GNU General Public License is
-intended to guarantee your freedom to share and change free
-software--to make sure the software is free for all its users. This
-General Public License applies to most of the Free Software
-Foundation's software and to any other program whose authors commit to
-using it. (Some other Free Software Foundation software is covered by
-the GNU Lesser General Public License instead.) You can apply it to
-your programs, too.
-
- When we speak of free software, we are referring to freedom, not
-price. Our General Public Licenses are designed to make sure that you
-have the freedom to distribute copies of free software (and charge for
-this service if you wish), that you receive source code or can get it
-if you want it, that you can change the software or use pieces of it in
-new free programs; and that you know you can do these things.
-
- To protect your rights, we need to make restrictions that forbid
-anyone to deny you these rights or to ask you to surrender the rights.
-These restrictions translate to certain responsibilities for you if you
-distribute copies of the software, or if you modify it.
-
- For example, if you distribute copies of such a program, whether
-gratis or for a fee, you must give the recipients all the rights that
-you have. You must make sure that they, too, receive or can get the
-source code. And you must show them these terms so they know their
-rights.
-
- We protect your rights with two steps: (1) copyright the software,
-and (2) offer you this license which gives you legal permission to copy,
-distribute and/or modify the software.
-
- Also, for each author's protection and ours, we want to make certain
-that everyone understands that there is no warranty for this free
-software. If the software is modified by someone else and passed on, we
-want its recipients to know that what they have is not the original, so
-that any problems introduced by others will not reflect on the original
-authors' reputations.
-
- Finally, any free program is threatened constantly by software
-patents. We wish to avoid the danger that redistributors of a free
-program will individually obtain patent licenses, in effect making the
-program proprietary. To prevent this, we have made it clear that any
-patent must be licensed for everyone's free use or not licensed at all.
-
- The precise terms and conditions for copying, distribution and
-modification follow.
-
- TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
- 0. This License applies to any program or other work which contains a
- notice placed by the copyright holder saying it may be distributed
- under the terms of this General Public License. The "Program",
- below, refers to any such program or work, and a "work based on
- the Program" means either the Program or any derivative work under
- copyright law: that is to say, a work containing the Program or a
- portion of it, either verbatim or with modifications and/or
- translated into another language. (Hereinafter, translation is
- included without limitation in the term "modification".) Each
- licensee is addressed as "you".
-
- Activities other than copying, distribution and modification are
- not covered by this License; they are outside its scope. The act
- of running the Program is not restricted, and the output from the
- Program is covered only if its contents constitute a work based on
- the Program (independent of having been made by running the
- Program). Whether that is true depends on what the Program does.
-
- 1. You may copy and distribute verbatim copies of the Program's
- source code as you receive it, in any medium, provided that you
- conspicuously and appropriately publish on each copy an appropriate
- copyright notice and disclaimer of warranty; keep intact all the
- notices that refer to this License and to the absence of any
- warranty; and give any other recipients of the Program a copy of
- this License along with the Program.
-
- You may charge a fee for the physical act of transferring a copy,
- and you may at your option offer warranty protection in exchange
- for a fee.
-
- 2. You may modify your copy or copies of the Program or any portion
- of it, thus forming a work based on the Program, and copy and
- distribute such modifications or work under the terms of Section 1
- above, provided that you also meet all of these conditions:
-
- a. You must cause the modified files to carry prominent notices
- stating that you changed the files and the date of any change.
-
- b. You must cause any work that you distribute or publish, that
- in whole or in part contains or is derived from the Program
- or any part thereof, to be licensed as a whole at no charge
- to all third parties under the terms of this License.
-
- c. If the modified program normally reads commands interactively
- when run, you must cause it, when started running for such
- interactive use in the most ordinary way, to print or display
- an announcement including an appropriate copyright notice and
- a notice that there is no warranty (or else, saying that you
- provide a warranty) and that users may redistribute the
- program under these conditions, and telling the user how to
- view a copy of this License. (Exception: if the Program
- itself is interactive but does not normally print such an
- announcement, your work based on the Program is not required
- to print an announcement.)
-
- These requirements apply to the modified work as a whole. If
- identifiable sections of that work are not derived from the
- Program, and can be reasonably considered independent and separate
- works in themselves, then this License, and its terms, do not
- apply to those sections when you distribute them as separate
- works. But when you distribute the same sections as part of a
- whole which is a work based on the Program, the distribution of
- the whole must be on the terms of this License, whose permissions
- for other licensees extend to the entire whole, and thus to each
- and every part regardless of who wrote it.
-
- Thus, it is not the intent of this section to claim rights or
- contest your rights to work written entirely by you; rather, the
- intent is to exercise the right to control the distribution of
- derivative or collective works based on the Program.
-
- In addition, mere aggregation of another work not based on the
- Program with the Program (or with a work based on the Program) on
- a volume of a storage or distribution medium does not bring the
- other work under the scope of this License.
-
- 3. You may copy and distribute the Program (or a work based on it,
- under Section 2) in object code or executable form under the terms
- of Sections 1 and 2 above provided that you also do one of the
- following:
-
- a. Accompany it with the complete corresponding machine-readable
- source code, which must be distributed under the terms of
- Sections 1 and 2 above on a medium customarily used for
- software interchange; or,
-
- b. Accompany it with a written offer, valid for at least three
- years, to give any third party, for a charge no more than your
- cost of physically performing source distribution, a complete
- machine-readable copy of the corresponding source code, to be
- distributed under the terms of Sections 1 and 2 above on a
- medium customarily used for software interchange; or,
-
- c. Accompany it with the information you received as to the offer
- to distribute corresponding source code. (This alternative is
- allowed only for noncommercial distribution and only if you
- received the program in object code or executable form with
- such an offer, in accord with Subsection b above.)
-
- The source code for a work means the preferred form of the work for
- making modifications to it. For an executable work, complete
- source code means all the source code for all modules it contains,
- plus any associated interface definition files, plus the scripts
- used to control compilation and installation of the executable.
- However, as a special exception, the source code distributed need
- not include anything that is normally distributed (in either
- source or binary form) with the major components (compiler,
- kernel, and so on) of the operating system on which the executable
- runs, unless that component itself accompanies the executable.
-
- If distribution of executable or object code is made by offering
- access to copy from a designated place, then offering equivalent
- access to copy the source code from the same place counts as
- distribution of the source code, even though third parties are not
- compelled to copy the source along with the object code.
-
- 4. You may not copy, modify, sublicense, or distribute the Program
- except as expressly provided under this License. Any attempt
- otherwise to copy, modify, sublicense or distribute the Program is
- void, and will automatically terminate your rights under this
- License. However, parties who have received copies, or rights,
- from you under this License will not have their licenses
- terminated so long as such parties remain in full compliance.
-
- 5. You are not required to accept this License, since you have not
- signed it. However, nothing else grants you permission to modify
- or distribute the Program or its derivative works. These actions
- are prohibited by law if you do not accept this License.
- Therefore, by modifying or distributing the Program (or any work
- based on the Program), you indicate your acceptance of this
- License to do so, and all its terms and conditions for copying,
- distributing or modifying the Program or works based on it.
-
- 6. Each time you redistribute the Program (or any work based on the
- Program), the recipient automatically receives a license from the
- original licensor to copy, distribute or modify the Program
- subject to these terms and conditions. You may not impose any
- further restrictions on the recipients' exercise of the rights
- granted herein. You are not responsible for enforcing compliance
- by third parties to this License.
-
- 7. If, as a consequence of a court judgment or allegation of patent
- infringement or for any other reason (not limited to patent
- issues), conditions are imposed on you (whether by court order,
- agreement or otherwise) that contradict the conditions of this
- License, they do not excuse you from the conditions of this
- License. If you cannot distribute so as to satisfy simultaneously
- your obligations under this License and any other pertinent
- obligations, then as a consequence you may not distribute the
- Program at all. For example, if a patent license would not permit
- royalty-free redistribution of the Program by all those who
- receive copies directly or indirectly through you, then the only
- way you could satisfy both it and this License would be to refrain
- entirely from distribution of the Program.
-
- If any portion of this section is held invalid or unenforceable
- under any particular circumstance, the balance of the section is
- intended to apply and the section as a whole is intended to apply
- in other circumstances.
-
- It is not the purpose of this section to induce you to infringe any
- patents or other property right claims or to contest validity of
- any such claims; this section has the sole purpose of protecting
- the integrity of the free software distribution system, which is
- implemented by public license practices. Many people have made
- generous contributions to the wide range of software distributed
- through that system in reliance on consistent application of that
- system; it is up to the author/donor to decide if he or she is
- willing to distribute software through any other system and a
- licensee cannot impose that choice.
-
- This section is intended to make thoroughly clear what is believed
- to be a consequence of the rest of this License.
-
- 8. If the distribution and/or use of the Program is restricted in
- certain countries either by patents or by copyrighted interfaces,
- the original copyright holder who places the Program under this
- License may add an explicit geographical distribution limitation
- excluding those countries, so that distribution is permitted only
- in or among countries not thus excluded. In such case, this
- License incorporates the limitation as if written in the body of
- this License.
-
- 9. The Free Software Foundation may publish revised and/or new
- versions of the General Public License from time to time. Such
- new versions will be similar in spirit to the present version, but
- may differ in detail to address new problems or concerns.
-
- Each version is given a distinguishing version number. If the
- Program specifies a version number of this License which applies
- to it and "any later version", you have the option of following
- the terms and conditions either of that version or of any later
- version published by the Free Software Foundation. If the Program
- does not specify a version number of this License, you may choose
- any version ever published by the Free Software Foundation.
-
- 10. If you wish to incorporate parts of the Program into other free
- programs whose distribution conditions are different, write to the
- author to ask for permission. For software which is copyrighted
- by the Free Software Foundation, write to the Free Software
- Foundation; we sometimes make exceptions for this. Our decision
- will be guided by the two goals of preserving the free status of
- all derivatives of our free software and of promoting the sharing
- and reuse of software generally.
-
- NO WARRANTY
- 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO
- WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE
- LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
- HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT
- WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT
- NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
- FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE
- QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
- PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY
- SERVICING, REPAIR OR CORRECTION.
-
- 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
- WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY
- MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE
- LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL,
- INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR
- INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
- DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU
- OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY
- OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN
- ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
-
- END OF TERMS AND CONDITIONS
-Appendix: How to Apply These Terms to Your New Programs
-=======================================================
-
-If you develop a new program, and you want it to be of the greatest
-possible use to the public, the best way to achieve this is to make it
-free software which everyone can redistribute and change under these
-terms.
-
- To do so, attach the following notices to the program. It is safest
-to attach them to the start of each source file to most effectively
-convey the exclusion of warranty; and each file should have at least
-the "copyright" line and a pointer to where the full notice is found.
-
- ONE LINE TO GIVE THE PROGRAM'S NAME AND A BRIEF IDEA OF WHAT IT DOES.
- Copyright (C) YYYY NAME OF AUTHOR
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program; if not, write to the Free Software
- Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
-
- Also add information on how to contact you by electronic and paper
-mail.
-
- If the program is interactive, make it output a short notice like
-this when it starts in an interactive mode:
-
- Gnomovision version 69, Copyright (C) 19YY NAME OF AUTHOR
- Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
- This is free software, and you are welcome to redistribute it
- under certain conditions; type `show c' for details.
-
- The hypothetical commands `show w' and `show c' should show the
-appropriate parts of the General Public License. Of course, the
-commands you use may be called something other than `show w' and `show
-c'; they could even be mouse-clicks or menu items--whatever suits your
-program.
-
- You should also get your employer (if you work as a programmer) or
-your school, if any, to sign a "copyright disclaimer" for the program,
-if necessary. Here is a sample; alter the names:
-
- Yoyodyne, Inc., hereby disclaims all copyright interest in the program
- `Gnomovision' (which makes passes at compilers) written by James Hacker.
-
- SIGNATURE OF TY COON, 1 April 1989
- Ty Coon, President of Vice
-
- This General Public License does not permit incorporating your
-program into proprietary programs. If your program is a subroutine
-library, you may consider it more useful to permit linking proprietary
-applications with the library. If this is what you want to do, use the
-GNU Lesser General Public License instead of this License.
-
-
-File: wget.info, Node: GNU Free Documentation License, Prev: GNU General Public License, Up: Copying
-
-10.2 GNU Free Documentation License
-===================================
-
- Version 1.2, November 2002
-
- Copyright (C) 2000,2001,2002 Free Software Foundation, Inc.
- 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA
-
- Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
-
- 0. PREAMBLE
-
- The purpose of this License is to make a manual, textbook, or other
- functional and useful document "free" in the sense of freedom: to
- assure everyone the effective freedom to copy and redistribute it,
- with or without modifying it, either commercially or
- noncommercially. Secondarily, this License preserves for the
- author and publisher a way to get credit for their work, while not
- being considered responsible for modifications made by others.
-
- This License is a kind of "copyleft", which means that derivative
- works of the document must themselves be free in the same sense.
- It complements the GNU General Public License, which is a copyleft
- license designed for free software.
-
- We have designed this License in order to use it for manuals for
- free software, because free software needs free documentation: a
- free program should come with manuals providing the same freedoms
- that the software does. But this License is not limited to
- software manuals; it can be used for any textual work, regardless
- of subject matter or whether it is published as a printed book.
- We recommend this License principally for works whose purpose is
- instruction or reference.
-
- 1. APPLICABILITY AND DEFINITIONS
-
- This License applies to any manual or other work, in any medium,
- that contains a notice placed by the copyright holder saying it
- can be distributed under the terms of this License. Such a notice
- grants a world-wide, royalty-free license, unlimited in duration,
- to use that work under the conditions stated herein. The
- "Document", below, refers to any such manual or work. Any member
- of the public is a licensee, and is addressed as "you". You
- accept the license if you copy, modify or distribute the work in a
- way requiring permission under copyright law.
-
- A "Modified Version" of the Document means any work containing the
- Document or a portion of it, either copied verbatim, or with
- modifications and/or translated into another language.
-
- A "Secondary Section" is a named appendix or a front-matter section
- of the Document that deals exclusively with the relationship of the
- publishers or authors of the Document to the Document's overall
- subject (or to related matters) and contains nothing that could
- fall directly within that overall subject. (Thus, if the Document
- is in part a textbook of mathematics, a Secondary Section may not
- explain any mathematics.) The relationship could be a matter of
- historical connection with the subject or with related matters, or
- of legal, commercial, philosophical, ethical or political position
- regarding them.
-
- The "Invariant Sections" are certain Secondary Sections whose
- titles are designated, as being those of Invariant Sections, in
- the notice that says that the Document is released under this
- License. If a section does not fit the above definition of
- Secondary then it is not allowed to be designated as Invariant.
- The Document may contain zero Invariant Sections. If the Document
- does not identify any Invariant Sections then there are none.
-
- The "Cover Texts" are certain short passages of text that are
- listed, as Front-Cover Texts or Back-Cover Texts, in the notice
- that says that the Document is released under this License. A
- Front-Cover Text may be at most 5 words, and a Back-Cover Text may
- be at most 25 words.
-
- A "Transparent" copy of the Document means a machine-readable copy,
- represented in a format whose specification is available to the
- general public, that is suitable for revising the document
- straightforwardly with generic text editors or (for images
- composed of pixels) generic paint programs or (for drawings) some
- widely available drawing editor, and that is suitable for input to
- text formatters or for automatic translation to a variety of
- formats suitable for input to text formatters. A copy made in an
- otherwise Transparent file format whose markup, or absence of
- markup, has been arranged to thwart or discourage subsequent
- modification by readers is not Transparent. An image format is
- not Transparent if used for any substantial amount of text. A
- copy that is not "Transparent" is called "Opaque".
-
- Examples of suitable formats for Transparent copies include plain
- ASCII without markup, Texinfo input format, LaTeX input format,
- SGML or XML using a publicly available DTD, and
- standard-conforming simple HTML, PostScript or PDF designed for
- human modification. Examples of transparent image formats include
- PNG, XCF and JPG. Opaque formats include proprietary formats that
- can be read and edited only by proprietary word processors, SGML or
- XML for which the DTD and/or processing tools are not generally
- available, and the machine-generated HTML, PostScript or PDF
- produced by some word processors for output purposes only.
-
- The "Title Page" means, for a printed book, the title page itself,
- plus such following pages as are needed to hold, legibly, the
- material this License requires to appear in the title page. For
- works in formats which do not have any title page as such, "Title
- Page" means the text near the most prominent appearance of the
- work's title, preceding the beginning of the body of the text.
-
- A section "Entitled XYZ" means a named subunit of the Document
- whose title either is precisely XYZ or contains XYZ in parentheses
- following text that translates XYZ in another language. (Here XYZ
- stands for a specific section name mentioned below, such as
- "Acknowledgements", "Dedications", "Endorsements", or "History".)
- To "Preserve the Title" of such a section when you modify the
- Document means that it remains a section "Entitled XYZ" according
- to this definition.
-
- The Document may include Warranty Disclaimers next to the notice
- which states that this License applies to the Document. These
- Warranty Disclaimers are considered to be included by reference in
- this License, but only as regards disclaiming warranties: any other
- implication that these Warranty Disclaimers may have is void and
- has no effect on the meaning of this License.
-
- 2. VERBATIM COPYING
-
- You may copy and distribute the Document in any medium, either
- commercially or noncommercially, provided that this License, the
- copyright notices, and the license notice saying this License
- applies to the Document are reproduced in all copies, and that you
- add no other conditions whatsoever to those of this License. You
- may not use technical measures to obstruct or control the reading
- or further copying of the copies you make or distribute. However,
- you may accept compensation in exchange for copies. If you
- distribute a large enough number of copies you must also follow
- the conditions in section 3.
-
- You may also lend copies, under the same conditions stated above,
- and you may publicly display copies.
-
- 3. COPYING IN QUANTITY
-
- If you publish printed copies (or copies in media that commonly
- have printed covers) of the Document, numbering more than 100, and
- the Document's license notice requires Cover Texts, you must
- enclose the copies in covers that carry, clearly and legibly, all
- these Cover Texts: Front-Cover Texts on the front cover, and
- Back-Cover Texts on the back cover. Both covers must also clearly
- and legibly identify you as the publisher of these copies. The
- front cover must present the full title with all words of the
- title equally prominent and visible. You may add other material
- on the covers in addition. Copying with changes limited to the
- covers, as long as they preserve the title of the Document and
- satisfy these conditions, can be treated as verbatim copying in
- other respects.
-
- If the required texts for either cover are too voluminous to fit
- legibly, you should put the first ones listed (as many as fit
- reasonably) on the actual cover, and continue the rest onto
- adjacent pages.
-
- If you publish or distribute Opaque copies of the Document
- numbering more than 100, you must either include a
- machine-readable Transparent copy along with each Opaque copy, or
- state in or with each Opaque copy a computer-network location from
- which the general network-using public has access to download
- using public-standard network protocols a complete Transparent
- copy of the Document, free of added material. If you use the
- latter option, you must take reasonably prudent steps, when you
- begin distribution of Opaque copies in quantity, to ensure that
- this Transparent copy will remain thus accessible at the stated
- location until at least one year after the last time you
- distribute an Opaque copy (directly or through your agents or
- retailers) of that edition to the public.
-
- It is requested, but not required, that you contact the authors of
- the Document well before redistributing any large number of
- copies, to give them a chance to provide you with an updated
- version of the Document.
-
- 4. MODIFICATIONS
-
- You may copy and distribute a Modified Version of the Document
- under the conditions of sections 2 and 3 above, provided that you
- release the Modified Version under precisely this License, with
- the Modified Version filling the role of the Document, thus
- licensing distribution and modification of the Modified Version to
- whoever possesses a copy of it. In addition, you must do these
- things in the Modified Version:
-
- A. Use in the Title Page (and on the covers, if any) a title
- distinct from that of the Document, and from those of
- previous versions (which should, if there were any, be listed
- in the History section of the Document). You may use the
- same title as a previous version if the original publisher of
- that version gives permission.
-
- B. List on the Title Page, as authors, one or more persons or
- entities responsible for authorship of the modifications in
- the Modified Version, together with at least five of the
- principal authors of the Document (all of its principal
- authors, if it has fewer than five), unless they release you
- from this requirement.
-
- C. State on the Title page the name of the publisher of the
- Modified Version, as the publisher.
-
- D. Preserve all the copyright notices of the Document.
-
- E. Add an appropriate copyright notice for your modifications
- adjacent to the other copyright notices.
-
- F. Include, immediately after the copyright notices, a license
- notice giving the public permission to use the Modified
- Version under the terms of this License, in the form shown in
- the Addendum below.
-
- G. Preserve in that license notice the full lists of Invariant
- Sections and required Cover Texts given in the Document's
- license notice.
-
- H. Include an unaltered copy of this License.
-
- I. Preserve the section Entitled "History", Preserve its Title,
- and add to it an item stating at least the title, year, new
- authors, and publisher of the Modified Version as given on
- the Title Page. If there is no section Entitled "History" in
- the Document, create one stating the title, year, authors,
- and publisher of the Document as given on its Title Page,
- then add an item describing the Modified Version as stated in
- the previous sentence.
-
- J. Preserve the network location, if any, given in the Document
- for public access to a Transparent copy of the Document, and
- likewise the network locations given in the Document for
- previous versions it was based on. These may be placed in
- the "History" section. You may omit a network location for a
- work that was published at least four years before the
- Document itself, or if the original publisher of the version
- it refers to gives permission.
-
- K. For any section Entitled "Acknowledgements" or "Dedications",
- Preserve the Title of the section, and preserve in the
- section all the substance and tone of each of the contributor
- acknowledgements and/or dedications given therein.
-
- L. Preserve all the Invariant Sections of the Document,
- unaltered in their text and in their titles. Section numbers
- or the equivalent are not considered part of the section
- titles.
-
- M. Delete any section Entitled "Endorsements". Such a section
- may not be included in the Modified Version.
-
- N. Do not retitle any existing section to be Entitled
- "Endorsements" or to conflict in title with any Invariant
- Section.
-
- O. Preserve any Warranty Disclaimers.
-
- If the Modified Version includes new front-matter sections or
- appendices that qualify as Secondary Sections and contain no
- material copied from the Document, you may at your option
- designate some or all of these sections as invariant. To do this,
- add their titles to the list of Invariant Sections in the Modified
- Version's license notice. These titles must be distinct from any
- other section titles.
-
- You may add a section Entitled "Endorsements", provided it contains
- nothing but endorsements of your Modified Version by various
- parties--for example, statements of peer review or that the text
- has been approved by an organization as the authoritative
- definition of a standard.
-
- You may add a passage of up to five words as a Front-Cover Text,
- and a passage of up to 25 words as a Back-Cover Text, to the end
- of the list of Cover Texts in the Modified Version. Only one
- passage of Front-Cover Text and one of Back-Cover Text may be
- added by (or through arrangements made by) any one entity. If the
- Document already includes a cover text for the same cover,
- previously added by you or by arrangement made by the same entity
- you are acting on behalf of, you may not add another; but you may
- replace the old one, on explicit permission from the previous
- publisher that added the old one.
-
- The author(s) and publisher(s) of the Document do not by this
- License give permission to use their names for publicity for or to
- assert or imply endorsement of any Modified Version.
-
- 5. COMBINING DOCUMENTS
-
- You may combine the Document with other documents released under
- this License, under the terms defined in section 4 above for
- modified versions, provided that you include in the combination
- all of the Invariant Sections of all of the original documents,
- unmodified, and list them all as Invariant Sections of your
- combined work in its license notice, and that you preserve all
- their Warranty Disclaimers.
-
- The combined work need only contain one copy of this License, and
- multiple identical Invariant Sections may be replaced with a single
- copy. If there are multiple Invariant Sections with the same name
- but different contents, make the title of each such section unique
- by adding at the end of it, in parentheses, the name of the
- original author or publisher of that section if known, or else a
- unique number. Make the same adjustment to the section titles in
- the list of Invariant Sections in the license notice of the
- combined work.
-
- In the combination, you must combine any sections Entitled
- "History" in the various original documents, forming one section
- Entitled "History"; likewise combine any sections Entitled
- "Acknowledgements", and any sections Entitled "Dedications". You
- must delete all sections Entitled "Endorsements."
-
- 6. COLLECTIONS OF DOCUMENTS
-
- You may make a collection consisting of the Document and other
- documents released under this License, and replace the individual
- copies of this License in the various documents with a single copy
- that is included in the collection, provided that you follow the
- rules of this License for verbatim copying of each of the
- documents in all other respects.
-
- You may extract a single document from such a collection, and
- distribute it individually under this License, provided you insert
- a copy of this License into the extracted document, and follow
- this License in all other respects regarding verbatim copying of
- that document.
-
- 7. AGGREGATION WITH INDEPENDENT WORKS
-
- A compilation of the Document or its derivatives with other
- separate and independent documents or works, in or on a volume of
- a storage or distribution medium, is called an "aggregate" if the
- copyright resulting from the compilation is not used to limit the
- legal rights of the compilation's users beyond what the individual
- works permit. When the Document is included in an aggregate, this
- License does not apply to the other works in the aggregate which
- are not themselves derivative works of the Document.
-
- If the Cover Text requirement of section 3 is applicable to these
- copies of the Document, then if the Document is less than one half
- of the entire aggregate, the Document's Cover Texts may be placed
- on covers that bracket the Document within the aggregate, or the
- electronic equivalent of covers if the Document is in electronic
- form. Otherwise they must appear on printed covers that bracket
- the whole aggregate.
-
- 8. TRANSLATION
-
- Translation is considered a kind of modification, so you may
- distribute translations of the Document under the terms of section
- 4. Replacing Invariant Sections with translations requires special
- permission from their copyright holders, but you may include
- translations of some or all Invariant Sections in addition to the
- original versions of these Invariant Sections. You may include a
- translation of this License, and all the license notices in the
- Document, and any Warranty Disclaimers, provided that you also
- include the original English version of this License and the
- original versions of those notices and disclaimers. In case of a
- disagreement between the translation and the original version of
- this License or a notice or disclaimer, the original version will
- prevail.
-
- If a section in the Document is Entitled "Acknowledgements",
- "Dedications", or "History", the requirement (section 4) to
- Preserve its Title (section 1) will typically require changing the
- actual title.
-
- 9. TERMINATION
-
- You may not copy, modify, sublicense, or distribute the Document
- except as expressly provided for under this License. Any other
- attempt to copy, modify, sublicense or distribute the Document is
- void, and will automatically terminate your rights under this
- License. However, parties who have received copies, or rights,
- from you under this License will not have their licenses
- terminated so long as such parties remain in full compliance.
-
- 10. FUTURE REVISIONS OF THIS LICENSE
-
- The Free Software Foundation may publish new, revised versions of
- the GNU Free Documentation License from time to time. Such new
- versions will be similar in spirit to the present version, but may
- differ in detail to address new problems or concerns. See
- `http://www.gnu.org/copyleft/'.
-
- Each version of the License is given a distinguishing version
- number. If the Document specifies that a particular numbered
- version of this License "or any later version" applies to it, you
- have the option of following the terms and conditions either of
- that specified version or of any later version that has been
- published (not as a draft) by the Free Software Foundation. If
- the Document does not specify a version number of this License,
- you may choose any version ever published (not as a draft) by the
- Free Software Foundation.
-
-10.2.1 ADDENDUM: How to use this License for your documents
------------------------------------------------------------
-
-To use this License in a document you have written, include a copy of
-the License in the document and put the following copyright and license
-notices just after the title page:
-
- Copyright (C) YEAR YOUR NAME.
- Permission is granted to copy, distribute and/or modify this document
- under the terms of the GNU Free Documentation License, Version 1.2
- or any later version published by the Free Software Foundation;
- with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
- Texts. A copy of the license is included in the section entitled ``GNU
- Free Documentation License''.
-
- If you have Invariant Sections, Front-Cover Texts and Back-Cover
-Texts, replace the "with...Texts." line with this:
-
- with the Invariant Sections being LIST THEIR TITLES, with
- the Front-Cover Texts being LIST, and with the Back-Cover Texts
- being LIST.
-
- If you have Invariant Sections without Cover Texts, or some other
-combination of the three, merge those two alternatives to suit the
-situation.
-
- If your document contains nontrivial examples of program code, we
-recommend releasing these examples in parallel under your choice of
-free software license, such as the GNU General Public License, to
-permit their use in free software.
-
-
-File: wget.info, Node: Concept Index, Prev: Copying, Up: Top
-
-Concept Index
-*************
-
-
-* Menu:
-
-* .html extension: HTTP Options. (line 6)
-* .listing files, removing: FTP Options. (line 21)
-* .netrc: Startup File. (line 6)
-* .wgetrc: Startup File. (line 6)
-* accept directories: Directory-Based Limits.
- (line 17)
-* accept suffixes: Types of Files. (line 15)
-* accept wildcards: Types of Files. (line 15)
-* append to log: Logging and Input File Options.
- (line 11)
-* arguments: Invoking. (line 6)
-* authentication <1>: HTTP Options. (line 27)
-* authentication: Download Options. (line 409)
-* backing up converted files: Recursive Retrieval Options.
- (line 71)
-* bandwidth, limit: Download Options. (line 216)
-* base for relative links in input file: Logging and Input File Options.
- (line 68)
-* bind address: Download Options. (line 6)
-* bug reports: Reporting Bugs. (line 6)
-* bugs: Reporting Bugs. (line 6)
-* cache: HTTP Options. (line 43)
-* caching of DNS lookups: Download Options. (line 302)
-* client IP address: Download Options. (line 6)
-* clobbering, file: Download Options. (line 30)
-* command line: Invoking. (line 6)
-* comments, HTML: Recursive Retrieval Options.
- (line 149)
-* connect timeout: Download Options. (line 199)
-* Content-Length, ignore: HTTP Options. (line 132)
-* continue retrieval: Download Options. (line 64)
-* contributors: Contributors. (line 6)
-* conversion of links: Recursive Retrieval Options.
- (line 32)
-* cookies: HTTP Options. (line 52)
-* cookies, loading: HTTP Options. (line 62)
-* cookies, saving: HTTP Options. (line 110)
-* cookies, session: HTTP Options. (line 115)
-* copying: Copying. (line 6)
-* cut directories: Directory Options. (line 32)
-* debug: Logging and Input File Options.
- (line 17)
-* delete after retrieval: Recursive Retrieval Options.
- (line 16)
-* directories: Directory-Based Limits.
- (line 6)
-* directories, exclude: Directory-Based Limits.
- (line 30)
-* directories, include: Directory-Based Limits.
- (line 17)
-* directory limits: Directory-Based Limits.
- (line 6)
-* directory prefix: Directory Options. (line 60)
-* DNS cache: Download Options. (line 302)
-* DNS timeout: Download Options. (line 193)
-* dot style: Download Options. (line 125)
-* downloading multiple times: Download Options. (line 30)
-* EGD: HTTPS (SSL/TLS) Options.
- (line 101)
-* entropy, specifying source of: HTTPS (SSL/TLS) Options.
- (line 85)
-* examples: Examples. (line 6)
-* exclude directories: Directory-Based Limits.
- (line 30)
-* execute wgetrc command: Basic Startup Options.
- (line 19)
-* FDL, GNU Free Documentation License: GNU Free Documentation License.
- (line 6)
-* features: Overview. (line 6)
-* file names, restrict: Download Options. (line 321)
-* filling proxy cache: Recursive Retrieval Options.
- (line 16)
-* follow FTP links: Recursive Accept/Reject Options.
- (line 20)
-* following ftp links: FTP Links. (line 6)
-* following links: Following Links. (line 6)
-* force html: Logging and Input File Options.
- (line 61)
-* free software: Copying. (line 6)
-* ftp authentication: FTP Options. (line 6)
-* ftp password: FTP Options. (line 6)
-* ftp time-stamping: FTP Time-Stamping Internals.
- (line 6)
-* ftp user: FTP Options. (line 6)
-* GFDL: Copying. (line 6)
-* globbing, toggle: FTP Options. (line 45)
-* GPL: Copying. (line 6)
-* hangup: Signals. (line 6)
-* header, add: HTTP Options. (line 143)
-* hosts, spanning: Spanning Hosts. (line 6)
-* HTML comments: Recursive Retrieval Options.
- (line 149)
-* http password: HTTP Options. (line 27)
-* http referer: HTTP Options. (line 178)
-* http time-stamping: HTTP Time-Stamping Internals.
- (line 6)
-* http user: HTTP Options. (line 27)
-* ignore length: HTTP Options. (line 132)
-* include directories: Directory-Based Limits.
- (line 17)
-* incomplete downloads: Download Options. (line 64)
-* incremental updating: Time-Stamping. (line 6)
-* input-file: Logging and Input File Options.
- (line 43)
-* invoking: Invoking. (line 6)
-* IP address, client: Download Options. (line 6)
-* IPv6: Download Options. (line 356)
-* Keep-Alive, turning off: FTP Options. (line 92)
-* latest version: Distribution. (line 6)
-* limit bandwidth: Download Options. (line 216)
-* link conversion: Recursive Retrieval Options.
- (line 32)
-* links: Following Links. (line 6)
-* list: Mailing List. (line 6)
-* loading cookies: HTTP Options. (line 62)
-* location of wgetrc: Wgetrc Location. (line 6)
-* log file: Logging and Input File Options.
- (line 6)
-* mailing list: Mailing List. (line 6)
-* mirroring: Very Advanced Usage. (line 6)
-* no parent: Directory-Based Limits.
- (line 43)
-* no-clobber: Download Options. (line 30)
-* nohup: Invoking. (line 6)
-* number of retries: Download Options. (line 12)
-* operating systems: Portability. (line 6)
-* option syntax: Option Syntax. (line 6)
-* output file: Logging and Input File Options.
- (line 6)
-* overview: Overview. (line 6)
-* page requisites: Recursive Retrieval Options.
- (line 84)
-* passive ftp: FTP Options. (line 61)
-* password: Download Options. (line 409)
-* pause: Download Options. (line 236)
-* Persistent Connections, disabling: FTP Options. (line 92)
-* portability: Portability. (line 6)
-* POST: HTTP Options. (line 211)
-* progress indicator: Download Options. (line 125)
-* proxies: Proxies. (line 6)
-* proxy <1>: HTTP Options. (line 43)
-* proxy: Download Options. (line 279)
-* proxy authentication: HTTP Options. (line 169)
-* proxy filling: Recursive Retrieval Options.
- (line 16)
-* proxy password: HTTP Options. (line 169)
-* proxy user: HTTP Options. (line 169)
-* quiet: Logging and Input File Options.
- (line 28)
-* quota: Download Options. (line 286)
-* random wait: Download Options. (line 261)
-* randomness, specifying source of: HTTPS (SSL/TLS) Options.
- (line 85)
-* rate, limit: Download Options. (line 216)
-* read timeout: Download Options. (line 204)
-* recursion: Recursive Download. (line 6)
-* recursive download: Recursive Download. (line 6)
-* redirecting output: Advanced Usage. (line 88)
-* referer, http: HTTP Options. (line 178)
-* reject directories: Directory-Based Limits.
- (line 30)
-* reject suffixes: Types of Files. (line 34)
-* reject wildcards: Types of Files. (line 34)
-* relative links: Relative Links. (line 6)
-* reporting bugs: Reporting Bugs. (line 6)
-* required images, downloading: Recursive Retrieval Options.
- (line 84)
-* resume download: Download Options. (line 64)
-* retries: Download Options. (line 12)
-* retries, waiting between: Download Options. (line 249)
-* retrieving: Recursive Download. (line 6)
-* robot exclusion: Robot Exclusion. (line 6)
-* robots.txt: Robot Exclusion. (line 6)
-* sample wgetrc: Sample Wgetrc. (line 6)
-* saving cookies: HTTP Options. (line 110)
-* security: Security Considerations.
- (line 6)
-* server maintenance: Robot Exclusion. (line 6)
-* server response, print: Download Options. (line 159)
-* server response, save: HTTP Options. (line 185)
-* session cookies: HTTP Options. (line 115)
-* signal handling: Signals. (line 6)
-* spanning hosts: Spanning Hosts. (line 6)
-* spider: Download Options. (line 164)
-* SSL: HTTPS (SSL/TLS) Options.
- (line 6)
-* SSL certificate: HTTPS (SSL/TLS) Options.
- (line 47)
-* SSL certificate authority: HTTPS (SSL/TLS) Options.
- (line 73)
-* SSL certificate type, specify: HTTPS (SSL/TLS) Options.
- (line 53)
-* SSL certificate, check: HTTPS (SSL/TLS) Options.
- (line 23)
-* SSL protocol, choose: HTTPS (SSL/TLS) Options.
- (line 10)
-* startup: Startup File. (line 6)
-* startup file: Startup File. (line 6)
-* suffixes, accept: Types of Files. (line 15)
-* suffixes, reject: Types of Files. (line 34)
-* symbolic links, retrieving: FTP Options. (line 73)
-* syntax of options: Option Syntax. (line 6)
-* syntax of wgetrc: Wgetrc Syntax. (line 6)
-* tag-based recursive pruning: Recursive Accept/Reject Options.
- (line 24)
-* time-stamping: Time-Stamping. (line 6)
-* time-stamping usage: Time-Stamping Usage. (line 6)
-* timeout: Download Options. (line 175)
-* timeout, connect: Download Options. (line 199)
-* timeout, DNS: Download Options. (line 193)
-* timeout, read: Download Options. (line 204)
-* timestamping: Time-Stamping. (line 6)
-* tries: Download Options. (line 12)
-* types of files: Types of Files. (line 6)
-* updating the archives: Time-Stamping. (line 6)
-* URL: URL Format. (line 6)
-* URL syntax: URL Format. (line 6)
-* usage, time-stamping: Time-Stamping Usage. (line 6)
-* user: Download Options. (line 409)
-* user-agent: HTTP Options. (line 189)
-* various: Various. (line 6)
-* verbose: Logging and Input File Options.
- (line 32)
-* wait: Download Options. (line 236)
-* wait, random: Download Options. (line 261)
-* waiting between retries: Download Options. (line 249)
-* Wget as spider: Download Options. (line 164)
-* wgetrc: Startup File. (line 6)
-* wgetrc commands: Wgetrc Commands. (line 6)
-* wgetrc location: Wgetrc Location. (line 6)
-* wgetrc syntax: Wgetrc Syntax. (line 6)
-* wildcards, accept: Types of Files. (line 15)
-* wildcards, reject: Types of Files. (line 34)
-* Windows file names: Download Options. (line 321)
-
-
-
-Tag Table:
-Node: Top974
-Node: Overview1845
-Node: Invoking5380
-Node: URL Format6217
-Ref: URL Format-Footnote-18790
-Node: Option Syntax8892
-Node: Basic Startup Options11571
-Node: Logging and Input File Options12376
-Node: Download Options15029
-Node: Directory Options34648
-Node: HTTP Options37353
-Node: HTTPS (SSL/TLS) Options49262
-Node: FTP Options54937
-Node: Recursive Retrieval Options59990
-Node: Recursive Accept/Reject Options67858
-Node: Recursive Download70857
-Node: Following Links73968
-Node: Spanning Hosts74905
-Node: Types of Files77078
-Node: Directory-Based Limits79538
-Node: Relative Links82188
-Node: FTP Links83025
-Node: Time-Stamping83892
-Node: Time-Stamping Usage85536
-Node: HTTP Time-Stamping Internals87362
-Ref: HTTP Time-Stamping Internals-Footnote-188638
-Node: FTP Time-Stamping Internals88837
-Node: Startup File90303
-Node: Wgetrc Location91177
-Node: Wgetrc Syntax91976
-Node: Wgetrc Commands92696
-Node: Sample Wgetrc106122
-Node: Examples111324
-Node: Simple Usage111664
-Node: Advanced Usage113068
-Node: Very Advanced Usage116769
-Node: Various118264
-Node: Proxies118789
-Node: Distribution121516
-Node: Mailing List121862
-Node: Reporting Bugs123219
-Node: Portability125576
-Node: Signals127018
-Node: Appendices127701
-Node: Robot Exclusion128023
-Node: Security Considerations131800
-Node: Contributors132984
-Node: Copying136681
-Node: GNU General Public License139393
-Node: GNU Free Documentation License158604
-Node: Concept Index181038
-
-End Tag Table