summaryrefslogtreecommitdiff
path: root/Build/source/utils/xz-4.999.9beta/doc
diff options
context:
space:
mode:
Diffstat (limited to 'Build/source/utils/xz-4.999.9beta/doc')
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/faq.txt122
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/history.txt149
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/lzma-file-format.txt166
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xz-a4.pdfbin0 -> 38867 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzdec-a4.pdfbin0 -> 6914 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzdiff-a4.pdfbin0 -> 4240 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzgrep-a4.pdfbin0 -> 4443 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzless-a4.pdfbin0 -> 4329 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzmore-a4.pdfbin0 -> 4211 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xz-letter.pdfbin0 -> 38846 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzdec-letter.pdfbin0 -> 6909 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzdiff-letter.pdfbin0 -> 4239 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzgrep-letter.pdfbin0 -> 4440 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzless-letter.pdfbin0 -> 4326 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzmore-letter.pdfbin0 -> 4210 bytes
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/txt/xz.txt786
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/txt/xzdec.txt95
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/txt/xzdiff.txt36
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/txt/xzgrep.txt39
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/txt/xzless.txt40
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/man/txt/xzmore.txt34
-rw-r--r--Build/source/utils/xz-4.999.9beta/doc/xz-file-format.txt1150
22 files changed, 2617 insertions, 0 deletions
diff --git a/Build/source/utils/xz-4.999.9beta/doc/faq.txt b/Build/source/utils/xz-4.999.9beta/doc/faq.txt
new file mode 100644
index 00000000000..48c4ec711b8
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/faq.txt
@@ -0,0 +1,122 @@
+
+XZ Utils FAQ
+============
+
+Q: What do the letters XZ mean?
+
+A: Nothing. They are just two letters, which come from the file format
+ suffix .xz. The .xz suffix was selected, because it seemed to be
+ pretty much unused. It is no deeper meaning.
+
+
+Q: What are LZMA and LZMA2?
+
+A: LZMA stands for Lempel-Ziv-Markov chain-Algorithm. It is the name
+ of the compression algorithm designed by Igor Pavlov for 7-Zip.
+ LZMA is based on LZ77 and range encoding.
+
+ LZMA2 is an updated version of the original LZMA to fix a couple of
+ practical issues. In context of XZ Utils, LZMA is called LZMA1 to
+ emphasize that LZMA is not the same thing as LZMA2. LZMA2 is the
+ primary compression algorithm in the .xz file format.
+
+
+Q: There are many LZMA related projects. How does XZ Utils relate to them?
+
+A: 7-Zip and LZMA SDK are the original projects. LZMA SDK is roughly
+ a subset of the 7-Zip source tree.
+
+ p7zip is 7-Zip's command line tools ported to POSIX-like systems.
+
+ LZMA Utils provide a gzip-like lzma tool for POSIX-like systems.
+ LZMA Utils are based on LZMA SDK. XZ Utils are the successor to
+ LZMA Utils.
+
+ There are several other projects using LZMA. Most are more or less
+ based on LZMA SDK.
+
+
+Q: Do XZ Utils support the .7z format?
+
+A: No. Use 7-Zip (Windows) or p7zip (POSIX-like systems) to handle .7z
+ files.
+
+
+Q: I have many .tar.7z files. Can I convert them to .tar.xz without
+ spending hours recompressing the data?
+
+A: In the "extra" directory, there is a script named 7z2lzma.bash which
+ is able to convert some .7z files to the .lzma format (not .xz). It
+ needs the 7za (or 7z) command from p7zip. The script may silently
+ produce corrupt output if certain assumptions are not met, so
+ decompress the resulting .lzma file and compare it against the
+ original before deleting the original file!
+
+
+Q: I have many .lzma files. Can I quickly convert them to the .xz format?
+
+A: For now, no. Since XZ Utils supports the .lzma format, it's usually
+ not too bad to keep the old files in the old format. If you want to
+ do the conversion anyway, you need to decompress the .lzma files and
+ then recompress to the .xz format.
+
+ Technically, there is a way to make the conversion relatively fast
+ (roughly twice the time that normal decompression takes). Writing
+ such a tool would take quite a bit time though, and would probably
+ be useful to only a few people. If you really want such a conversion
+ tool, contact Lasse Collin and offer some money.
+
+
+Q: Can I recover parts of a broken .xz file (e.g. corrupted CD-R)?
+
+A: It may be possible if the file consists of multiple blocks, which
+ typically is not the case if the file was created in single-threaded
+ mode. There is no recovery program yet.
+
+
+Q: Is (some part of) XZ Utils patented?
+
+A: Lasse Collin is not aware of any patents that could affect XZ Utils.
+ However, due to nature of software patents, it's not possible to
+ guarantee that XZ Utils isn't affected by any third party patent(s).
+
+
+Q: Where can I find documentation about the file format and algorithms?
+
+A: The .xz format is documented in xz-file-format.txt. It is a container
+ format only, and doesn't include descriptions of any non-trivial
+ filters.
+
+ Documenting LZMA and LZMA2 is planned, but for now, there is no other
+ documentation that the source code. Before you begin, you should know
+ the basics of LZ77 and range coding algorithms. LZMA is based on LZ77,
+ but LZMA is *a lot* more complex. Range coding is used to compress
+ the final bitstream like Huffman coding is used in Deflate.
+
+
+Q: I cannot find BCJ and BCJ2 filters. Don't they exist in liblzma?
+
+A: BCJ filter is called "x86" in liblzma. BCJ2 is not included,
+ because it requires using more than one encoded output stream.
+
+
+Q: How do I build a program that needs liblzmadec (lzmadec.h)?
+
+A: liblzmadec is part of LZMA Utils. XZ Utils has liblzma, but no
+ liblzmadec. The code using liblzmadec should be ported to use
+ liblzma instead. If you cannot or don't want to do that, download
+ LZMA Utils from <http://tukaani.org/lzma/>.
+
+
+Q: The default build of liblzma is too big. How can I make it smaller?
+
+A: Give --enable-small to the configure script. Use also appropriate
+ --enable or --disable options to include only those filter encoders
+ and decoders and integrity checks that you actually need. Use
+ CFLAGS=-Os (with GCC) or equivalent to tell your compiler to optimize
+ for size. See INSTALL for information about configure options.
+
+ If the result is still too big, take a look at XZ Embedded. It is
+ a separate project, which provides a limited but signinificantly
+ smaller XZ decoder implementation than XZ Utils.
+
diff --git a/Build/source/utils/xz-4.999.9beta/doc/history.txt b/Build/source/utils/xz-4.999.9beta/doc/history.txt
new file mode 100644
index 00000000000..c97492e81a0
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/history.txt
@@ -0,0 +1,149 @@
+
+History of LZMA Utils and XZ Utils
+==================================
+
+Tukaani distribution
+
+ In 2005, there was a small group working on Tukaani distribution, which
+ was a Slackware fork. One of the project goals was to fit the distro on
+ a single 700 MiB ISO-9660 image. Using LZMA instead of gzip helped a
+ lot. Roughly speaking, one could fit data that took 1000 MiB in gzipped
+ form into 700 MiB with LZMA. Naturally compression ratio varied across
+ packages, but this was what we got on average.
+
+ Slackware packages have traditionally had .tgz as the filename suffix,
+ which is an abbreviation of .tar.gz. A logical naming for LZMA
+ compressed packages was .tlz, being an abbreviation of .tar.lzma.
+
+ At the end of the year 2007, there was no distribution under the
+ Tukaani project anymore, but development of LZMA Utils was kept going.
+ Still, there were .tlz packages around, because at least Vector Linux
+ (a Slackware based distribution) used LZMA for its packages.
+
+ First versions of the modified pkgtools used the LZMA_Alone tool from
+ Igor Pavlov's LZMA SDK as is. It was fine, because users wouldn't need
+ to interact with LZMA_Alone directly. But people soon wanted to use
+ LZMA for other files too, and the interface of LZMA_Alone wasn't
+ comfortable for those used to gzip and bzip2.
+
+
+First steps of LZMA Utils
+
+ The first version of LZMA Utils (4.22.0) included a shell script called
+ lzmash. It was wrapper that had gzip-like command line interface. It
+ used the LZMA_Alone tool from LZMA SDK to do all the real work. zgrep,
+ zdiff, and related scripts from gzip were adapted work with LZMA and
+ were part of the first LZMA Utils release too.
+
+ LZMA Utils 4.22.0 included also lzmadec, which was a small (less than
+ 10 KiB) decoder-only command line tool. It was written on top of the
+ decoder-only C code found from the LZMA SDK. lzmadec was convenient in
+ situations where LZMA_Alone (a few hundred KiB) would be too big.
+
+ lzmash and lzmadec were written by Lasse Collin.
+
+
+Second generation
+
+ The lzmash script was an ugly and not very secure hack. The last
+ version of LZMA Utils to use lzmash was 4.27.1.
+
+ LZMA Utils 4.32.0beta1 introduced a new lzma command line tool written
+ by Ville Koskinen. It was written in C++, and used the encoder and
+ decoder from C++ LZMA SDK with little modifications. This tool replaced
+ both the lzmash script and the LZMA_Alone command line tool in LZMA
+ Utils.
+
+ Introducing this new tool caused some temporary incompatibilities,
+ because LZMA_Alone executable was simply named lzma like the new
+ command line tool, but they had completely different command line
+ interface. The file format was still the same.
+
+ Lasse wrote liblzmadec, which was a small decoder-only library based
+ on the C code found from LZMA SDK. liblzmadec had API similar to zlib,
+ although there were some significant differences, which made it
+ non-trivial to use it in some applications designed for zlib and
+ libbzip2.
+
+ The lzmadec command line tool was converted to use liblzmadec.
+
+ Alexandre Sauvé helped converting build system to use GNU Autotools.
+ This made is easier to test for certain less portable features needed
+ by the new command line tool.
+
+ Since the new command line tool never got completely finished (for
+ example, it didn't support LZMA_OPT environment variable), the intent
+ was to not call 4.32.x stable. Similarly, liblzmadec wasn't polished,
+ but appeared to work well enough, so some people started using it too.
+
+ Because the development of the third generation of LZMA Utils was
+ delayed considerably (3-4 years), the 4.32.x branch had to be kept
+ maintained. It got some bug fixes now and then, and finally it was
+ decided to call it stable, although most of the missing features were
+ never added.
+
+
+File format problems
+
+ The file format used by LZMA_Alone was primitive. It was designed for
+ embedded systems in mind, and thus provided only minimal set of
+ features. The two biggest problems for non-embedded use were lack of
+ magic bytes and integrity check.
+
+ Igor and Lasse started developing a new file format with some help
+ from Ville Koskinen. Also Mark Adler, Mikko Pouru, H. Peter Anvin,
+ and Lars Wirzenius helped with some minor things at some point of the
+ development. Designing the new format took quite a long time (actually,
+ too long time would be more appropriate expression). It was mostly
+ because Lasse was quite slow at getting things done due to personal
+ reasons.
+
+ Originally the new format was supposed to use the same .lzma suffix
+ that was already used by the old file format. Switching to the new
+ format wouldn't have caused much trouble when the old format wasn't
+ used by many people. But since the development of the new format took
+ so long time, the old format got quite popular, and it was decided
+ that the new file format must use a different suffix.
+
+ It was decided to use .xz as the suffix of the new file format. The
+ first stable .xz file format specification was finally released in
+ December 2008. In addition to fixing the most obvious problems of
+ the old .lzma format, the .xz format added some new features like
+ support for multiple filters (compression algorithms), filter chaining
+ (like piping on the command line), and limited random-access reading.
+
+ Currently the primary compression algorithm used in .xz is LZMA2.
+ It is an extension on top of the original LZMA to fix some practical
+ problems: LZMA2 adds support for flushing the encoder, uncompressed
+ chunks, eases stateful decoder implementations, and improves support
+ for multithreading. Since LZMA2 is better than the original LZMA, the
+ original LZMA is not supported in .xz.
+
+
+Transition to XZ Utils
+
+ The early versions of XZ Utils were called LZMA Utils. The first
+ releases were 4.42.0alphas. They dropped the rest of the C++ LZMA SDK.
+ The code was still directly based on LZMA SDK but ported to C and
+ converted from callback API to stateful API. Later, Igor Pavlov made
+ C version of the LZMA encoder too; these ports from C++ to C were
+ independent in LZMA SDK and LZMA Utils.
+
+ The core of the new LZMA Utils was liblzma, a compression library with
+ zlib-like API. liblzma supported both the old and new file format. The
+ gzip-like lzma command line tool was rewritten to use liblzma.
+
+ The new LZMA Utils code base was renamed to XZ Utils when the name
+ of the new file format had been decided. The liblzma compression
+ library retained its name though, because changing it would have
+ caused unnecessary breakage in applications already using the early
+ liblzma snapshots.
+
+ The xz command line tool can emulate the gzip-like lzma tool by
+ creating appropriate symlinks (e.g. lzma -> xz). Thus, practically
+ all scripts using the lzma tool from LZMA Utils will work as is with
+ XZ Utils (and will keep using the old .lzma format). Still, the .lzma
+ format is more or less deprecated. XZ Utils will keep supporting it,
+ but new applications should use the .xz format, and migrating old
+ applications to .xz is often a good idea too.
+
diff --git a/Build/source/utils/xz-4.999.9beta/doc/lzma-file-format.txt b/Build/source/utils/xz-4.999.9beta/doc/lzma-file-format.txt
new file mode 100644
index 00000000000..21fcb19fdcd
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/lzma-file-format.txt
@@ -0,0 +1,166 @@
+
+The .lzma File Format
+=====================
+
+ 0. Preface
+ 0.1. Notices and Acknowledgements
+ 0.2. Changes
+ 1. File Format
+ 1.1. Header
+ 1.1.1. Properties
+ 1.1.2. Dictionary Size
+ 1.1.3. Uncompressed Size
+ 1.2. LZMA Compressed Data
+ 2. References
+
+
+0. Preface
+
+ This document describes the .lzma file format, which is
+ sometimes also called LZMA_Alone format. It is a legacy file
+ format, which is being or has been replaced by the .xz format.
+ The MIME type of the .lzma format is `application/x-lzma'.
+
+ The most commonly used software to handle .lzma files are
+ LZMA SDK, LZMA Utils, 7-Zip, and XZ Utils. This document
+ describes some of the differences between these implementations
+ and gives hints what subset of the .lzma format is the most
+ portable.
+
+
+0.1. Notices and Acknowledgements
+
+ This file format was designed by Igor Pavlov for use in
+ LZMA SDK. This document was written by Lasse Collin
+ <lasse.collin@tukaani.org> using the documentation found
+ from the LZMA SDK.
+
+ This document has been put into the public domain.
+
+
+0.2. Changes
+
+ Last modified: 2009-05-01 11:15+0300
+
+
+1. File Format
+
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+==========================+
+ | Header | LZMA Compressed Data |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+==========================+
+
+ The .lzma format file consist of 13-byte Header followed by
+ the LZMA Compressed Data.
+
+ Unlike the .gz, .bz2, and .xz formats, it is not possible to
+ concatenate multiple .lzma files as is and expect the
+ decompression tool to decode the resulting file as if it were
+ a single .lzma file.
+
+ For example, the command line tools from LZMA Utils and
+ LZMA SDK silently ignore all the data after the first .lzma
+ stream. In contrast, the command line tool from XZ Utils
+ considers the .lzma file to be corrupt if there is data after
+ the first .lzma stream.
+
+
+1.1. Header
+
+ +------------+----+----+----+----+--+--+--+--+--+--+--+--+
+ | Properties | Dictionary Size | Uncompressed Size |
+ +------------+----+----+----+----+--+--+--+--+--+--+--+--+
+
+
+1.1.1. Properties
+
+ The Properties field contains three properties. An abbreviation
+ is given in parentheses, followed by the value range of the
+ property. The field consists of
+
+ 1) the number of literal context bits (lc, [0, 8]);
+ 2) the number of literal position bits (lp, [0, 4]); and
+ 3) the number of position bits (pb, [0, 4]).
+
+ The properties are encoded using the following formula:
+
+ Properties = (pb * 5 + lp) * 9 + lc
+
+ The following C code illustrates a straightforward way to
+ decode the Properties field:
+
+ uint8_t lc, lp, pb;
+ uint8_t prop = get_lzma_properties();
+ if (prop > (4 * 5 + 4) * 9 + 8)
+ return LZMA_PROPERTIES_ERROR;
+
+ pb = prop / (9 * 5);
+ prop -= pb * 9 * 5;
+ lp = prop / 9;
+ lc = prop - lp * 9;
+
+ XZ Utils has an additional requirement: lc + lp <= 4. Files
+ which don't follow this requirement cannot be decompressed
+ with XZ Utils. Usually this isn't a problem since the most
+ common lc/lp/pb values are 3/0/2. It is the only lc/lp/pb
+ combination that the files created by LZMA Utils can have,
+ but LZMA Utils can decompress files with any lc/lp/pb.
+
+
+1.1.2. Dictionary Size
+
+ Dictionary Size is stored as an unsigned 32-bit little endian
+ integer. Any 32-bit value is possible, but for maximum
+ portability, only sizes of 2^n and 2^n + 2^(n-1) should be
+ used.
+
+ LZMA Utils creates only files with dictionary size 2^n,
+ 16 <= n <= 25. LZMA Utils can decompress files with any
+ dictionary size.
+
+ XZ Utils creates and decompresses .lzma files only with
+ dictionary sizes 2^n and 2^n + 2^(n-1). If some other
+ dictionary size is specified when compressing, the value
+ stored in the Dictionary Size field is a rounded up, but the
+ specified value is still used in the actual compression code.
+
+
+1.1.3. Uncompressed Size
+
+ Uncompressed Size is stored as unsigned 64-bit little endian
+ integer. A special value of 0xFFFF_FFFF_FFFF_FFFF indicates
+ that Uncompressed Size is unknown. End of Payload Marker (*)
+ is used if and only if Uncompressed Size is unknown.
+
+ XZ Utils rejects files whose Uncompressed Size field specifies
+ a known size that is 256 GiB or more. This is to reject false
+ positives when trying to guess if the input file is in the
+ .lzma format. When Uncompressed Size is unknown, there is no
+ limit for the uncompressed size of the file.
+
+ (*) Some tools use the term End of Stream (EOS) marker
+ instead of End of Payload Marker.
+
+
+1.2. LZMA Compressed Data
+
+ Detailed description of the format of this field is out of
+ scope of this document.
+
+
+2. References
+
+ LZMA SDK - The original LZMA implementation
+ http://7-zip.org/sdk.html
+
+ 7-Zip
+ http://7-zip.org/
+
+ LZMA Utils - LZMA adapted to POSIX-like systems
+ http://tukaani.org/lzma/
+
+ XZ Utils - The next generation of LZMA Utils
+ http://tukaani.org/xz/
+
+ The .xz file format - The successor of the the .lzma format
+ http://tukaani.org/xz/xz-file-format.txt
+
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xz-a4.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xz-a4.pdf
new file mode 100644
index 00000000000..0b6805432e5
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xz-a4.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzdec-a4.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzdec-a4.pdf
new file mode 100644
index 00000000000..92e52eec78e
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzdec-a4.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzdiff-a4.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzdiff-a4.pdf
new file mode 100644
index 00000000000..80aad7a9df2
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzdiff-a4.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzgrep-a4.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzgrep-a4.pdf
new file mode 100644
index 00000000000..2c928946d92
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzgrep-a4.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzless-a4.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzless-a4.pdf
new file mode 100644
index 00000000000..a6374692f4d
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzless-a4.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzmore-a4.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzmore-a4.pdf
new file mode 100644
index 00000000000..b36a5fdfe6d
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-a4/xzmore-a4.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xz-letter.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xz-letter.pdf
new file mode 100644
index 00000000000..6df9ab66aff
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xz-letter.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzdec-letter.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzdec-letter.pdf
new file mode 100644
index 00000000000..f056c2ee348
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzdec-letter.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzdiff-letter.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzdiff-letter.pdf
new file mode 100644
index 00000000000..39bc0f0a287
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzdiff-letter.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzgrep-letter.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzgrep-letter.pdf
new file mode 100644
index 00000000000..677186a1b44
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzgrep-letter.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzless-letter.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzless-letter.pdf
new file mode 100644
index 00000000000..dcbd4871228
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzless-letter.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzmore-letter.pdf b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzmore-letter.pdf
new file mode 100644
index 00000000000..e22857d6894
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/pdf-letter/xzmore-letter.pdf
Binary files differ
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/txt/xz.txt b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xz.txt
new file mode 100644
index 00000000000..e3dacde4a11
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xz.txt
@@ -0,0 +1,786 @@
+XZ(1) XZ Utils XZ(1)
+
+
+
+NAME
+ xz, unxz, xzcat, lzma, unlzma, lzcat - Compress or decompress .xz and
+ .lzma files
+
+SYNOPSIS
+ xz [option]... [file]...
+
+ unxz is equivalent to xz --decompress.
+ xzcat is equivalent to xz --decompress --stdout.
+ lzma is equivalent to xz --format=lzma.
+ unlzma is equivalent to xz --format=lzma --decompress.
+ lzcat is equivalent to xz --format=lzma --decompress --stdout.
+
+ When writing scripts that need to decompress files, it is recommended
+ to always use the name xz with appropriate arguments (xz -d or xz -dc)
+ instead of the names unxz and xzcat.
+
+DESCRIPTION
+ xz is a general-purpose data compression tool with command line syntax
+ similar to gzip(1) and bzip2(1). The native file format is the .xz
+ format, but also the legacy .lzma format and raw compressed streams
+ with no container format headers are supported.
+
+ xz compresses or decompresses each file according to the selected oper-
+ ation mode. If no files are given or file is -, xz reads from standard
+ input and writes the processed data to standard output. xz will refuse
+ (display an error and skip the file) to write compressed data to stan-
+ dard output if it is a terminal. Similarly, xz will refuse to read com-
+ pressed data from standard input if it is a terminal.
+
+ Unless --stdout is specified, files other than - are written to a new
+ file whose name is derived from the source file name:
+
+ o When compressing, the suffix of the target file format (.xz or
+ .lzma) is appended to the source filename to get the target file-
+ name.
+
+ o When decompressing, the .xz or .lzma suffix is removed from the
+ filename to get the target filename. xz also recognizes the suf-
+ fixes .txz and .tlz, and replaces them with the .tar suffix.
+
+ If the target file already exists, an error is displayed and the file
+ is skipped.
+
+ Unless writing to standard output, xz will display a warning and skip
+ the file if any of the following applies:
+
+ o File is not a regular file. Symbolic links are not followed, thus
+ they are never considered to be regular files.
+
+ o File has more than one hardlink.
+
+ o File has setuid, setgid, or sticky bit set.
+
+ o The operation mode is set to compress, and the file already has a
+ suffix of the target file format (.xz or .txz when compressing to
+ the .xz format, and .lzma or .tlz when compressing to the .lzma for-
+ mat).
+
+ o The operation mode is set to decompress, and the file doesn't have a
+ suffix of any of the supported file formats (.xz, .txz, .lzma, or
+ .tlz).
+
+ After successfully compressing or decompressing the file, xz copies the
+ owner, group, permissions, access time, and modification time from the
+ source file to the target file. If copying the group fails, the permis-
+ sions are modified so that the target file doesn't become accessible to
+ users who didn't have permission to access the source file. xz doesn't
+ support copying other metadata like access control lists or extended
+ attributes yet.
+
+ Once the target file has been successfully closed, the source file is
+ removed unless --keep was specified. The source file is never removed
+ if the output is written to standard output.
+
+ Sending SIGINFO or SIGUSR1 to the xz process makes it print progress
+ information to standard error. This has only limited use since when
+ standard error is a terminal, using --verbose will display an automati-
+ cally updating progress indicator.
+
+ Memory usage
+ The memory usage of xz varies from a few hundred kilobytes to several
+ gigabytes depending on the compression settings. The settings used when
+ compressing a file affect also the memory usage of the decompressor.
+ Typically the decompressor needs only 5 % to 20 % of the amount of RAM
+ that the compressor needed when creating the file. Still, the worst-
+ case memory usage of the decompressor is several gigabytes.
+
+ To prevent uncomfortable surprises caused by huge memory usage, xz has
+ a built-in memory usage limiter. The default limit is 40 % of total
+ physical RAM. While operating systems provide ways to limit the memory
+ usage of processes, relying on it wasn't deemed to be flexible enough.
+
+ When compressing, if the selected compression settings exceed the mem-
+ ory usage limit, the settings are automatically adjusted downwards and
+ a notice about this is displayed. As an exception, if the memory usage
+ limit is exceeded when compressing with --format=raw, an error is dis-
+ played and xz will exit with exit status 1.
+
+ If source file cannot be decompressed without exceeding the memory
+ usage limit, an error message is displayed and the file is skipped.
+ Note that compressed files may contain many blocks, which may have been
+ compressed with different settings. Typically all blocks will have
+ roughly the same memory requirements, but it is possible that a block
+ later in the file will exceed the memory usage limit, and an error
+ about too low memory usage limit gets displayed after some data has
+ already been decompressed.
+
+ The absolute value of the active memory usage limit can be seen near
+ the bottom of the output of --long-help. The default limit can be
+ overriden with --memory=limit.
+
+OPTIONS
+ Integer suffixes and special values
+ In most places where an integer argument is expected, an optional suf-
+ fix is supported to easily indicate large integers. There must be no
+ space between the integer and the suffix.
+
+ k or kB
+ The integer is multiplied by 1,000 (10^3). For example, 5k or
+ 5kB equals 5000.
+
+ Ki or KiB
+ The integer is multiplied by 1,024 (2^10).
+
+ M or MB
+ The integer is multiplied by 1,000,000 (10^6).
+
+ Mi or MiB
+ The integer is multiplied by 1,048,576 (2^20).
+
+ G or GB
+ The integer is multiplied by 1,000,000,000 (10^9).
+
+ Gi or GiB
+ The integer is multiplied by 1,073,741,824 (2^30).
+
+ A special value max can be used to indicate the maximum integer value
+ supported by the option.
+
+ Operation mode
+ If multiple operation mode options are given, the last one takes
+ effect.
+
+ -z, --compress
+ Compress. This is the default operation mode when no operation
+ mode option is specified, and no other operation mode is implied
+ from the command name (for example, unxz implies --decompress).
+
+ -d, --decompress, --uncompress
+ Decompress.
+
+ -t, --test
+ Test the integrity of compressed files. No files are created or
+ removed. This option is equivalent to --decompress --stdout
+ except that the decompressed data is discarded instead of being
+ written to standard output.
+
+ -l, --list
+ View information about the compressed files. No uncompressed
+ output is produced, and no files are created or removed. In list
+ mode, the program cannot read the compressed data from standard
+ input or from other unseekable sources.
+
+ This feature has not been implemented yet.
+
+ Operation modifiers
+ -k, --keep
+ Keep (don't delete) the input files.
+
+ -f, --force
+ This option has several effects:
+
+ o If the target file already exists, delete it before compress-
+ ing or decompressing.
+
+ o Compress or decompress even if the input is not a regular
+ file, has more than one hardlink, or has setuid, setgid, or
+ sticky bit set. The setuid, setgid, and sticky bits are not
+ copied to the target file.
+
+ o If combined with --decompress --stdout and xz doesn't recog-
+ nize the type of the source file, xz will copy the source
+ file as is to standard output. This allows using xzcat
+ --force like cat(1) for files that have not been compressed
+ with xz. Note that in future, xz might support new com-
+ pressed file formats, which may make xz decompress more types
+ of files instead of copying them as is to standard output.
+ --format=format can be used to restrict xz to decompress only
+ a single file format.
+
+ o Allow writing compressed data to a terminal, and reading com-
+ pressed data from a terminal.
+
+ -c, --stdout, --to-stdout
+ Write the compressed or decompressed data to standard output
+ instead of a file. This implies --keep.
+
+ -S .suf, --suffix=.suf
+ When compressing, use .suf as the suffix for the target file
+ instead of .xz or .lzma. If not writing to standard output and
+ the source file already has the suffix .suf, a warning is dis-
+ played and the file is skipped.
+
+ When decompressing, recognize also files with the suffix .suf in
+ addition to files with the .xz, .txz, .lzma, or .tlz suffix. If
+ the source file has the suffix .suf, the suffix is removed to
+ get the target filename.
+
+ When compressing or decompressing raw streams (--format=raw),
+ the suffix must always be specified unless writing to standard
+ output, because there is no default suffix for raw streams.
+
+ --files[=file]
+ Read the filenames to process from file; if file is omitted,
+ filenames are read from standard input. Filenames must be termi-
+ nated with the newline character. If filenames are given also as
+ command line arguments, they are processed before the filenames
+ read from file.
+
+ --files0[=file]
+ This is identical to --files[=file] except that the filenames
+ must be terminated with the null character.
+
+ Basic file format and compression options
+ -F format, --format=format
+ Specify the file format to compress or decompress:
+
+ o auto: This is the default. When compressing, auto is equiva-
+ lent to xz. When decompressing, the format of the input file
+ is autodetected. Note that raw streams (created with --for-
+ mat=raw) cannot be autodetected.
+
+ o xz: Compress to the .xz file format, or accept only .xz files
+ when decompressing.
+
+ o lzma or alone: Compress to the legacy .lzma file format, or
+ accept only .lzma files when decompressing. The alternative
+ name alone is provided for backwards compatibility with LZMA
+ Utils.
+
+ o raw: Compress or uncompress a raw stream (no headers). This
+ is meant for advanced users only. To decode raw streams, you
+ need to set not only --format=raw but also specify the filter
+ chain, which would normally be stored in the container format
+ headers.
+
+ -C check, --check=check
+ Specify the type of the integrity check, which is calculated
+ from the uncompressed data. This option has an effect only when
+ compressing into the .xz format; the .lzma format doesn't sup-
+ port integrity checks. The integrity check (if any) is verified
+ when the .xz file is decompressed.
+
+ Supported check types:
+
+ o none: Don't calculate an integrity check at all. This is usu-
+ ally a bad idea. This can be useful when integrity of the
+ data is verified by other means anyway.
+
+ o crc32: Calculate CRC32 using the polynomial from IEEE-802.3
+ (Ethernet).
+
+ o crc64: Calculate CRC64 using the polynomial from ECMA-182.
+ This is the default, since it is slightly better than CRC32
+ at detecting damaged files and the speed difference is negli-
+ gible.
+
+ o sha256: Calculate SHA-256. This is somewhat slower than CRC32
+ and CRC64.
+
+ Integrity of the .xz headers is always verified with CRC32. It
+ is not possible to change or disable it.
+
+ -0 ... -9
+ Select compression preset. If a preset level is specified multi-
+ ple times, the last one takes effect.
+
+ The compression preset levels can be categorised roughly into
+ three categories:
+
+ -0 ... -2
+ Fast presets with relatively low memory usage. -1 and -2
+ should give compression speed and ratios comparable to
+ bzip2 -1 and bzip2 -9, respectively. Currently -0 is not
+ very good (not much faster than -1 but much worse com-
+ pression). In future, -0 may be indicate some fast algo-
+ rithm instead of LZMA2.
+
+ -3 ... -5
+ Good compression ratio with low to medium memory usage.
+ These are significantly slower than levels 0-2.
+
+ -6 ... -9
+ Excellent compression with medium to high memory usage.
+ These are also slower than the lower preset levels. The
+ default is -6. Unless you want to maximize the compres-
+ sion ratio, you probably don't want a higher preset level
+ than -7 due to speed and memory usage.
+
+ The exact compression settings (filter chain) used by each pre-
+ set may vary between xz versions. The settings may also vary
+ between files being compressed, if xz determines that modified
+ settings will probably give better compression ratio without
+ significantly affecting compression time or memory usage.
+
+ Because the settings may vary, the memory usage may vary too.
+ The following table lists the maximum memory usage of each pre-
+ set level, which won't be exceeded even in future versions of
+ xz.
+
+ FIXME: The table below is just a rough idea.
+
+ Preset Compression Decompression
+ -0 6 MiB 1 MiB
+ -1 6 MiB 1 MiB
+ -2 10 MiB 1 MiB
+ -3 20 MiB 2 MiB
+ -4 30 MiB 3 MiB
+ -5 60 MiB 6 MiB
+ -6 100 MiB 10 MiB
+ -7 200 MiB 20 MiB
+ -8 400 MiB 40 MiB
+ -9 800 MiB 80 MiB
+
+ When compressing, xz automatically adjusts the compression set-
+ tings downwards if the memory usage limit would be exceeded, so
+ it is safe to specify a high preset level even on systems that
+ don't have lots of RAM.
+
+ --fast and --best
+ These are somewhat misleading aliases for -0 and -9, respec-
+ tively. These are provided only for backwards compatibility
+ with LZMA Utils. Avoid using these options.
+
+ Especially the name of --best is misleading, because the defini-
+ tion of best depends on the input data, and that usually people
+ don't want the very best compression ratio anyway, because it
+ would be very slow.
+
+ -e, --extreme
+ Modify the compression preset (-0 ... -9) so that a little bit
+ better compression ratio can be achieved without increasing mem-
+ ory usage of the compressor or decompressor (exception: compres-
+ sor memory usage may increase a little with presets -0 ... -2).
+ The downside is that the compression time will increase dramati-
+ cally (it can easily double).
+
+ -M limit, --memory=limit
+ Set the memory usage limit. If this option is specied multiple
+ times, the last one takes effect. The limit can be specified in
+ multiple ways:
+
+ o The limit can be an absolute value in bytes. Using an integer
+ suffix like MiB can be useful. Example: --memory=80MiB
+
+ o The limit can be specified as a percentage of physical RAM.
+ Example: --memory=70%
+
+ o The limit can be reset back to its default value (currently
+ 40 % of physical RAM) by setting it to 0.
+
+ o The memory usage limiting can be effectively disabled by set-
+ ting limit to max. This isn't recommended. It's usually bet-
+ ter to use, for example, --memory=90%.
+
+ The current limit can be seen near the bottom of the output of
+ the --long-help option.
+
+ -T threads, --threads=threads
+ Specify the maximum number of worker threads to use. The default
+ is the number of available CPU cores. You can see the current
+ value of threads near the end of the output of the --long-help
+ option.
+
+ The actual number of worker threads can be less than threads if
+ using more threads would exceed the memory usage limit. In
+ addition to CPU-intensive worker threads, xz may use a few aux-
+ iliary threads, which don't use a lot of CPU time.
+
+ Multithreaded compression and decompression are not implemented
+ yet, so this option has no effect for now.
+
+ Custom compressor filter chains
+ A custom filter chain allows specifying the compression settings in
+ detail instead of relying on the settings associated to the preset lev-
+ els. When a custom filter chain is specified, the compression preset
+ level options (-0 ... -9 and --extreme) are silently ignored.
+
+ A filter chain is comparable to piping on the UN*X command line. When
+ compressing, the uncompressed input goes to the first filter, whose
+ output goes to the next filter (if any). The output of the last filter
+ gets written to the compressed file. The maximum number of filters in
+ the chain is four, but typically a filter chain has only one or two
+ filters.
+
+ Many filters have limitations where they can be in the filter chain:
+ some filters can work only as the last filter in the chain, some only
+ as a non-last filter, and some work in any position in the chain.
+ Depending on the filter, this limitation is either inherent to the fil-
+ ter design or exists to prevent security issues.
+
+ A custom filter chain is specified by using one or more filter options
+ in the order they are wanted in the filter chain. That is, the order of
+ filter options is significant! When decoding raw streams (--for-
+ mat=raw), the filter chain is specified in the same order as it was
+ specified when compressing.
+
+ Filters take filter-specific options as a comma-separated list. Extra
+ commas in options are ignored. Every option has a default value, so you
+ need to specify only those you want to change.
+
+ --lzma1[=options], --lzma2[=options]
+ Add LZMA1 or LZMA2 filter to the filter chain. These filter can
+ be used only as the last filter in the chain.
+
+ LZMA1 is a legacy filter, which is supported almost solely due
+ to the legacy .lzma file format, which supports only LZMA1.
+ LZMA2 is an updated version of LZMA1 to fix some practical
+ issues of LZMA1. The .xz format uses LZMA2, and doesn't support
+ LZMA1 at all. Compression speed and ratios of LZMA1 and LZMA2
+ are practically the same.
+
+ LZMA1 and LZMA2 share the same set of options:
+
+ preset=preset
+ Reset all LZMA1 or LZMA2 options to preset. Preset con-
+ sist of an integer, which may be followed by single-let-
+ ter preset modifiers. The integer can be from 0 to 9,
+ matching the command line options -0 ... -9. The only
+ supported modifier is currently e, which matches
+ --extreme.
+
+ The default preset is 6, from which the default values
+ for the rest of the LZMA1 or LZMA2 options are taken.
+
+ dict=size
+ Dictionary (history buffer) size indicates how many bytes
+ of the recently processed uncompressed data is kept in
+ memory. One method to reduce size of the uncompressed
+ data is to store distance-length pairs, which indicate
+ what data to repeat from the dictionary buffer. The big-
+ ger the dictionary, the better the compression ratio usu-
+ ally is, but dictionaries bigger than the uncompressed
+ data are waste of RAM.
+
+ Typical dictionary size is from 64 KiB to 64 MiB. The
+ minimum is 4 KiB. The maximum for compression is cur-
+ rently 1.5 GiB. The decompressor already supports dictio-
+ naries up to one byte less than 4 GiB, which is the maxi-
+ mum for LZMA1 and LZMA2 stream formats.
+
+ Dictionary size has the biggest effect on compression
+ ratio. Dictionary size and match finder together deter-
+ mine the memory usage of the LZMA1 or LZMA2 encoder. The
+ same dictionary size is required for decompressing that
+ was used when compressing, thus the memory usage of the
+ decoder is determined by the dictionary size used when
+ compressing.
+
+ lc=lc Specify the number of literal context bits. The minimum
+ is 0 and the maximum is 4; the default is 3. In addi-
+ tion, the sum of lc and lp must not exceed 4.
+
+ lp=lp Specify the number of literal position bits. The minimum
+ is 0 and the maximum is 4; the default is 0.
+
+ pb=pb Specify the number of position bits. The minimum is 0 and
+ the maximum is 4; the default is 2.
+
+ mode=mode
+ Compression mode specifies the function used to analyze
+ the data produced by the match finder. Supported modes
+ are fast and normal. The default is fast for presets 0-2
+ and normal for presets 3-9.
+
+ mf=mf Match finder has a major effect on encoder speed, memory
+ usage, and compression ratio. Usually Hash Chain match
+ finders are faster than Binary Tree match finders. Hash
+ Chains are usually used together with mode=fast and
+ Binary Trees with mode=normal. The memory usage formulas
+ are only rough estimates, which are closest to reality
+ when dict is a power of two.
+
+ hc3 Hash Chain with 2- and 3-byte hashing
+ Minimum value for nice: 3
+ Memory usage: dict * 7.5 (if dict <= 16 MiB);
+ dict * 5.5 + 64 MiB (if dict > 16 MiB)
+
+ hc4 Hash Chain with 2-, 3-, and 4-byte hashing
+ Minimum value for nice: 4
+ Memory usage: dict * 7.5
+
+ bt2 Binary Tree with 2-byte hashing
+ Minimum value for nice: 2
+ Memory usage: dict * 9.5
+
+ bt3 Binary Tree with 2- and 3-byte hashing
+ Minimum value for nice: 3
+ Memory usage: dict * 11.5 (if dict <= 16 MiB);
+ dict * 9.5 + 64 MiB (if dict > 16 MiB)
+
+ bt4 Binary Tree with 2-, 3-, and 4-byte hashing
+ Minimum value for nice: 4
+ Memory usage: dict * 11.5
+
+ nice=nice
+ Specify what is considered to be a nice length for a
+ match. Once a match of at least nice bytes is found, the
+ algorithm stops looking for possibly better matches.
+
+ nice can be 2-273 bytes. Higher values tend to give bet-
+ ter compression ratio at expense of speed. The default
+ depends on the preset level.
+
+ depth=depth
+ Specify the maximum search depth in the match finder. The
+ default is the special value 0, which makes the compres-
+ sor determine a reasonable depth from mf and nice.
+
+ Using very high values for depth can make the encoder
+ extremely slow with carefully crafted files. Avoid set-
+ ting the depth over 1000 unless you are prepared to
+ interrupt the compression in case it is taking too long.
+
+ When decoding raw streams (--format=raw), LZMA2 needs only the
+ value of dict. LZMA1 needs also lc, lp, and pb.
+
+ --x86[=options]
+
+ --powerpc[=options]
+
+ --ia64[=options]
+
+ --arm[=options]
+
+ --armthumb[=options]
+
+ --sparc[=options]
+ Add a branch/call/jump (BCJ) filter to the filter chain. These
+ filters can be used only as non-last filter in the filter chain.
+
+ A BCJ filter converts relative addresses in the machine code to
+ their absolute counterparts. This doesn't change the size of the
+ data, but it increases redundancy, which allows e.g. LZMA2 to
+ get better compression ratio.
+
+ The BCJ filters are always reversible, so using a BCJ filter for
+ wrong type of data doesn't cause any data loss. However, apply-
+ ing a BCJ filter for wrong type of data is a bad idea, because
+ it tends to make the compression ratio worse.
+
+ Different instruction sets have have different alignment:
+
+ Filter Alignment Notes
+ x86 1 32-bit and 64-bit x86
+ PowerPC 4 Big endian only
+ ARM 4 Little endian only
+ ARM-Thumb 2 Little endian only
+ IA-64 16 Big or little endian
+ SPARC 4 Big or little endian
+
+ Since the BCJ-filtered data is usually compressed with LZMA2,
+ the compression ratio may be improved slightly if the LZMA2
+ options are set to match the alignment of the selected BCJ fil-
+ ter. For example, with the IA-64 filter, it's good to set pb=4
+ with LZMA2 (2^4=16). The x86 filter is an exception; it's usu-
+ ally good to stick to LZMA2's default four-byte alignment when
+ compressing x86 executables.
+
+ All BCJ filters support the same options:
+
+ start=offset
+ Specify the start offset that is used when converting
+ between relative and absolute addresses. The offset must
+ be a multiple of the alignment of the filter (see the ta-
+ ble above). The default is zero. In practice, the
+ default is good; specifying a custom offset is almost
+ never useful.
+
+ Specifying a non-zero start offset is probably useful
+ only if the executable has multiple sections, and there
+ are many cross-section jumps or calls. Applying a BCJ
+ filter separately for each section with proper start off-
+ set and then compressing the result as a single chunk may
+ give some improvement in compression ratio compared to
+ applying the BCJ filter with the default offset for the
+ whole executable.
+
+ --delta[=options]
+ Add Delta filter to the filter chain. The Delta filter can be
+ used only as non-last filter in the filter chain.
+
+ Currently only simple byte-wise delta calculation is supported.
+ It can be useful when compressing e.g. uncompressed bitmap
+ images or uncompressed PCM audio. However, special purpose algo-
+ rithms may give significantly better results than Delta + LZMA2.
+ This is true especially with audio, which compresses faster and
+ better e.g. with FLAC.
+
+ Supported options:
+
+ dist=distance
+ Specify the distance of the delta calculation as bytes.
+ distance must be 1-256. The default is 1.
+
+ For example, with dist=2 and eight-byte input A1 B1 A2 B3
+ A3 B5 A4 B7, the output will be A1 B1 01 02 01 02 01 02.
+
+ Other options
+ -q, --quiet
+ Suppress warnings and notices. Specify this twice to suppress
+ errors too. This option has no effect on the exit status. That
+ is, even if a warning was suppressed, the exit status to indi-
+ cate a warning is still used.
+
+ -v, --verbose
+ Be verbose. If standard error is connected to a terminal, xz
+ will display a progress indicator. Specifying --verbose twice
+ will give even more verbose output (useful mostly for debug-
+ ging).
+
+ -Q, --no-warn
+ Don't set the exit status to 2 even if a condition worth a warn-
+ ing was detected. This option doesn't affect the verbosity
+ level, thus both --quiet and --no-warn have to be used to not
+ display warnings and to not alter the exit status.
+
+ -h, --help
+ Display a help message describing the most commonly used
+ options, and exit successfully.
+
+ -H, --long-help
+ Display a help message describing all features of xz, and exit
+ successfully
+
+ -V, --version
+ Display the version number of xz and liblzma.
+
+EXIT STATUS
+ 0 All is good.
+
+ 1 An error occurred.
+
+ 2 Something worth a warning occurred, but no actual errors
+ occurred.
+
+ Notices (not warnings or errors) printed on standard error don't affect
+ the exit status.
+
+ENVIRONMENT
+ XZ_OPT A space-separated list of options is parsed from XZ_OPT before
+ parsing the options given on the command line. Note that only
+ options are parsed from XZ_OPT; all non-options are silently
+ ignored. Parsing is done with getopt_long(3) which is used also
+ for the command line arguments.
+
+LZMA UTILS COMPATIBILITY
+ The command line syntax of xz is practically a superset of lzma,
+ unlzma, and lzcat as found from LZMA Utils 4.32.x. In most cases, it is
+ possible to replace LZMA Utils with XZ Utils without breaking existing
+ scripts. There are some incompatibilities though, which may sometimes
+ cause problems.
+
+ Compression preset levels
+ The numbering of the compression level presets is not identical in xz
+ and LZMA Utils. The most important difference is how dictionary sizes
+ are mapped to different presets. Dictionary size is roughly equal to
+ the decompressor memory usage.
+
+ Level xz LZMA Utils
+ -1 64 KiB 64 KiB
+ -2 512 KiB 1 MiB
+ -3 1 MiB 512 KiB
+ -4 2 MiB 1 MiB
+ -5 4 MiB 2 MiB
+ -6 8 MiB 4 MiB
+ -7 16 MiB 8 MiB
+ -8 32 MiB 16 MiB
+ -9 64 MiB 32 MiB
+
+ The dictionary size differences affect the compressor memory usage too,
+ but there are some other differences between LZMA Utils and XZ Utils,
+ which make the difference even bigger:
+
+ Level xz LZMA Utils 4.32.x
+ -1 2 MiB 2 MiB
+ -2 5 MiB 12 MiB
+ -3 13 MiB 12 MiB
+ -4 25 MiB 16 MiB
+ -5 48 MiB 26 MiB
+ -6 94 MiB 45 MiB
+ -7 186 MiB 83 MiB
+ -8 370 MiB 159 MiB
+ -9 674 MiB 311 MiB
+
+ The default preset level in LZMA Utils is -7 while in XZ Utils it is
+ -6, so both use 8 MiB dictionary by default.
+
+ Streamed vs. non-streamed .lzma files
+ Uncompressed size of the file can be stored in the .lzma header. LZMA
+ Utils does that when compressing regular files. The alternative is to
+ mark that uncompressed size is unknown and use end of payload marker to
+ indicate where the decompressor should stop. LZMA Utils uses this
+ method when uncompressed size isn't known, which is the case for exam-
+ ple in pipes.
+
+ xz supports decompressing .lzma files with or without end of payload
+ marker, but all .lzma files created by xz will use end of payload
+ marker and have uncompressed size marked as unknown in the .lzma
+ header. This may be a problem in some (uncommon) situations. For exam-
+ ple, a .lzma decompressor in an embedded device might work only with
+ files that have known uncompressed size. If you hit this problem, you
+ need to use LZMA Utils or LZMA SDK to create .lzma files with known
+ uncompressed size.
+
+ Unsupported .lzma files
+ The .lzma format allows lc values up to 8, and lp values up to 4. LZMA
+ Utils can decompress files with any lc and lp, but always creates files
+ with lc=3 and lp=0. Creating files with other lc and lp is possible
+ with xz and with LZMA SDK.
+
+ The implementation of the LZMA1 filter in liblzma requires that the sum
+ of lc and lp must not exceed 4. Thus, .lzma files which exceed this
+ limitation, cannot be decompressed with xz.
+
+ LZMA Utils creates only .lzma files which have dictionary size of 2^n
+ (a power of 2), but accepts files with any dictionary size. liblzma
+ accepts only .lzma files which have dictionary size of 2^n or 2^n +
+ 2^(n-1). This is to decrease false positives when autodetecting .lzma
+ files.
+
+ These limitations shouldn't be a problem in practice, since practically
+ all .lzma files have been compressed with settings that liblzma will
+ accept.
+
+ Trailing garbage
+ When decompressing, LZMA Utils silently ignore everything after the
+ first .lzma stream. In most situations, this is a bug. This also means
+ that LZMA Utils don't support decompressing concatenated .lzma files.
+
+ If there is data left after the first .lzma stream, xz considers the
+ file to be corrupt. This may break obscure scripts which have assumed
+ that trailing garbage is ignored.
+
+NOTES
+ Compressed output may vary
+ The exact compressed output produced from the same uncompressed input
+ file may vary between XZ Utils versions even if compression options are
+ identical. This is because the encoder can be improved (faster or bet-
+ ter compression) without affecting the file format. The output can vary
+ even between different builds of the same XZ Utils version, if differ-
+ ent build options are used or if the endianness of the hardware is dif-
+ ferent for different builds.
+
+ The above means that implementing --rsyncable to create rsyncable .xz
+ files is not going to happen without freezing a part of the encoder
+ implementation, which can then be used with --rsyncable.
+
+ Embedded .xz decompressors
+ Embedded .xz decompressor implementations like XZ Embedded don't neces-
+ sarily support files created with check types other than none and
+ crc32. Since the default is --check=crc64, you must use --check=none
+ or --check=crc32 when creating files for embedded systems.
+
+ Outside embedded systems, all .xz format decompressors support all the
+ check types, or at least are able to decompress the file without veri-
+ fying the integrity check if the particular check is not supported.
+
+ XZ Embedded supports BCJ filters, but only with the default start off-
+ set.
+
+SEE ALSO
+ xzdec(1), gzip(1), bzip2(1)
+
+ XZ Utils: <http://tukaani.org/xz/>
+ XZ Embedded: <http://tukaani.org/xz/embedded.html>
+ LZMA SDK: <http://7-zip.org/sdk.html>
+
+
+
+Tukaani 2009-08-27 XZ(1)
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzdec.txt b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzdec.txt
new file mode 100644
index 00000000000..ee2b820adad
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzdec.txt
@@ -0,0 +1,95 @@
+XZDEC(1) XZ Utils XZDEC(1)
+
+
+
+NAME
+ xzdec, lzmadec - Small .xz and .lzma decompressors
+
+SYNOPSIS
+ xzdec [option]... [file]...
+ lzmadec [option]... [file]...
+
+DESCRIPTION
+ xzdec is a liblzma-based decompression-only tool for .xz (and only .xz)
+ files. xzdec is intended to work as a drop-in replacement for xz(1) in
+ the most common situations where a script has been written to use xz
+ --decompress --stdout (and possibly a few other commonly used options)
+ to decompress .xz files. lzmadec is identical to xzdec except that
+ lzmadec supports .lzma files instead of .xz files.
+
+ To reduce the size of the executable, xzdec doesn't support multi-
+ threading or localization, and doesn't read options from XZ_OPT envi-
+ ronment variable. xzdec doesn't support displaying intermediate
+ progress information: sending SIGINFO to xzdec does nothing, but send-
+ ing SIGUSR1 terminates the process instead of displaying progress
+ information.
+
+OPTIONS
+ -d, --decompress, --uncompress
+ Ignored for xz(1) compatibility. xzdec supports only decompres-
+ sion.
+
+ -k, --keep
+ Ignored for xz(1) compatibility. xzdec never creates or removes
+ any files.
+
+ -c, --stdout, --to-stdout
+ Ignored for xz(1) compatibility. xzdec always writes the decom-
+ pressed data to standard output.
+
+ -M limit, --memory=limit
+ Set the memory usage limit. If this option is specified multi-
+ ple times, the last one takes effect. The limit can be specified
+ in multiple ways:
+
+ o The limit can be an absolute value in bytes. Using an integer
+ suffix like MiB can be useful. Example: --memory=80MiB
+
+ o The limit can be specified as a percentage of physical RAM.
+ Example: --memory=70%
+
+ o The limit can be reset back to its default value (currently
+ 40 % of physical RAM) by setting it to 0.
+
+ o The memory usage limiting can be effectively disabled by set-
+ ting limit to max. This isn't recommended. It's usually bet-
+ ter to use, for example, --memory=90%.
+
+ The current limit can be seen near the bottom of the output of
+ the --help option.
+
+ -q, --quiet
+ Specifying this once does nothing since xzdec never displays any
+ warnings or notices. Specify this twice to suppress errors.
+
+ -Q, --no-warn
+ Ignored for xz(1) compatibility. xzdec never uses the exit sta-
+ tus 2.
+
+ -h, --help
+ Display a help message and exit successfully.
+
+ -V, --version
+ Display the version number of xzdec and liblzma.
+
+EXIT STATUS
+ 0 All was good.
+
+ 1 An error occurred.
+
+ xzdec doesn't have any warning messages like xz(1) has, thus the exit
+ status 2 is not used by xzdec.
+
+NOTES
+ xzdec and lzmadec are not really that small. The size can be reduced
+ further by dropping features from liblzma at compile time, but that
+ shouldn't usually be done for executables distributed in typical non-
+ embedded operating system distributions. If you need a truly small .xz
+ decompressor, consider using XZ Embedded.
+
+SEE ALSO
+ xz(1)
+
+
+
+Tukaani 2009-06-04 XZDEC(1)
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzdiff.txt b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzdiff.txt
new file mode 100644
index 00000000000..f64568f2ae3
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzdiff.txt
@@ -0,0 +1,36 @@
+XZDIFF(1) XZ Utils XZDIFF(1)
+
+
+
+NAME
+ xzcmp, xzdiff, lzcmp, lzdiff - compare compressed files
+
+SYNOPSIS
+ xzcmp [cmp_options] file1 [file2]
+ xzdiff [diff_options] file1 [file2]
+ lzcmp [cmp_options] file1 [file2]
+ lzdiff [diff_options] file1 [file2]
+
+DESCRIPTION
+ xzcmp and xdiff invoke cmp(1) or diff(1) on files compressed with
+ xz(1), lzma(1), gzip(1), or bzip2(1). All options specified are passed
+ directly to cmp or diff. If only one file is specified, then the files
+ compared are file1 (which must have a suffix of a supported compression
+ format) and file1 from which the compression format suffix has been
+ stripped. If two files are specified, then they are uncompressed if
+ necessary and fed to cmp(1) or diff(1). The exit status from cmp or
+ diff is preserved.
+
+ The names lzcmp and lzdiff are provided for backward compatibility with
+ LZMA Utils.
+
+SEE ALSO
+ cmp(1), diff(1), xz(1), gzip(1), bzip2(1), zdiff(1)
+
+BUGS
+ Messages from the cmp(1) or diff(1) programs refer to temporary file-
+ names instead of those specified.
+
+
+
+Tukaani 2009-07-05 XZDIFF(1)
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzgrep.txt b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzgrep.txt
new file mode 100644
index 00000000000..7f665bce295
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzgrep.txt
@@ -0,0 +1,39 @@
+XZGREP(1) XZ Utils XZGREP(1)
+
+
+
+NAME
+ xzgrep - search compressed files for a regular expression
+
+SYNOPSIS
+ xzgrep [grep_options] [-e] pattern file...
+ xzegrep ...
+ xzfgrep ...
+ lzgrep ...
+ lzegrep ...
+ lzfgrep ...
+
+DESCRIPTION
+ xzgrep invokes grep(1) on files which may be either uncompressed or
+ compressed with xz(1), lzma(1), gzip(1), or bzip2(1). All options
+ specified are passed directly to grep(1).
+
+ If no file is specified, then the standard input is decompressed if
+ necessary and fed to grep(1). When reading from standard input,
+ gzip(1) and bzip2(1) compressed files are not supported.
+
+ If xzgrep is invoked as xzegrep or xzfgrep then egrep(1) or fgrep(1) is
+ used instead of grep(1). The same applies to names lzgrep, lzegrep,
+ and lzfgrep, which are provided for backward compatibility with LZMA
+ Utils.
+
+ENVIRONMENT
+ GREP If the GREP environment variable is set, xzgrep uses it instead
+ of grep(1), egrep(1), or fgrep(1).
+
+SEE ALSO
+ grep(1), xz(1), gzip(1), bzip2(1), zgrep(1)
+
+
+
+Tukaani 2009-07-05 XZGREP(1)
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzless.txt b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzless.txt
new file mode 100644
index 00000000000..2f3dc9fe2c4
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzless.txt
@@ -0,0 +1,40 @@
+XZLESS(1) XZ Utils XZLESS(1)
+
+
+
+NAME
+ xzless, lzless - view xz or lzma compressed (text) files
+
+SYNOPSIS
+ xzless [file...]
+ lzless [file...]
+
+DESCRIPTION
+ xzless is a filter that displays pagefulls of uncompressed text from
+ compressed file(s) to a terminal. It works on files compressed with
+ xz(1) or lzma(1). If no files are given, xzless reads from standard
+ input.
+
+ xzless uses less(1) as its only pager. Unlike xzmore, the choice of
+ pagers is not alterable by an environment variable. Commands are based
+ on both more(1) and vi(1), and allow back and forth movement and
+ searching. See the less(1) manual for more information.
+
+ The command named lzless is provided for backward compatibility with
+ LZMA Utils.
+
+ENVIRONMENT
+ LESSMETACHARS
+ A list of characters special to the shell. Set by xzless unless
+ it is already set in the environment.
+
+ LESSOPEN
+ Set to a command line to invoke the xz(1) decompressor for pre-
+ processing the input files to less(1).
+
+SEE ALSO
+ less(1), xz(1), xzmore(1), zless(1)
+
+
+
+Tukaani 2009-07-05 XZLESS(1)
diff --git a/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzmore.txt b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzmore.txt
new file mode 100644
index 00000000000..6f6cfe93eea
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/man/txt/xzmore.txt
@@ -0,0 +1,34 @@
+XZMORE(1) XZ Utils XZMORE(1)
+
+
+
+NAME
+ xzmore, lzmore - view xz or lzma compressed (text) files
+
+SYNOPSIS
+ xzmore [filename ...]
+ lzmore [filename ...]
+
+DESCRIPTION
+ xzmore is a filter which allows examination of xz(1) or lzma(1) com-
+ pressed text files one screenful at a time on a soft-copy terminal.
+
+ To use a pager other than the default more, set environment variable
+ PAGER to the name of the desired program. The name lzmore is provided
+ for backward compatibility with LZMA Utils.
+
+ e or q When the prompt --More--(Next file: file) is printed, this com-
+ mand causes xzmore to exit.
+
+ s When the prompt --More--(Next file: file) is printed, this com-
+ mand causes xzmore to skip the next file and continue.
+
+ For list of keyboard commands supported while actually viewing the con-
+ tent of a file, refer to manual of the pager you use, usually more(1).
+
+SEE ALSO
+ more(1), xz(1), xzless(1), zmore(1)
+
+
+
+Tukaani 2009-07-05 XZMORE(1)
diff --git a/Build/source/utils/xz-4.999.9beta/doc/xz-file-format.txt b/Build/source/utils/xz-4.999.9beta/doc/xz-file-format.txt
new file mode 100644
index 00000000000..4ed66506018
--- /dev/null
+++ b/Build/source/utils/xz-4.999.9beta/doc/xz-file-format.txt
@@ -0,0 +1,1150 @@
+
+The .xz File Format
+===================
+
+Version 1.0.4 (2009-08-27)
+
+
+ 0. Preface
+ 0.1. Notices and Acknowledgements
+ 0.2. Getting the Latest Version
+ 0.3. Version History
+ 1. Conventions
+ 1.1. Byte and Its Representation
+ 1.2. Multibyte Integers
+ 2. Overall Structure of .xz File
+ 2.1. Stream
+ 2.1.1. Stream Header
+ 2.1.1.1. Header Magic Bytes
+ 2.1.1.2. Stream Flags
+ 2.1.1.3. CRC32
+ 2.1.2. Stream Footer
+ 2.1.2.1. CRC32
+ 2.1.2.2. Backward Size
+ 2.1.2.3. Stream Flags
+ 2.1.2.4. Footer Magic Bytes
+ 2.2. Stream Padding
+ 3. Block
+ 3.1. Block Header
+ 3.1.1. Block Header Size
+ 3.1.2. Block Flags
+ 3.1.3. Compressed Size
+ 3.1.4. Uncompressed Size
+ 3.1.5. List of Filter Flags
+ 3.1.6. Header Padding
+ 3.1.7. CRC32
+ 3.2. Compressed Data
+ 3.3. Block Padding
+ 3.4. Check
+ 4. Index
+ 4.1. Index Indicator
+ 4.2. Number of Records
+ 4.3. List of Records
+ 4.3.1. Unpadded Size
+ 4.3.2. Uncompressed Size
+ 4.4. Index Padding
+ 4.5. CRC32
+ 5. Filter Chains
+ 5.1. Alignment
+ 5.2. Security
+ 5.3. Filters
+ 5.3.1. LZMA2
+ 5.3.2. Branch/Call/Jump Filters for Executables
+ 5.3.3. Delta
+ 5.3.3.1. Format of the Encoded Output
+ 5.4. Custom Filter IDs
+ 5.4.1. Reserved Custom Filter ID Ranges
+ 6. Cyclic Redundancy Checks
+ 7. References
+
+
+0. Preface
+
+ This document describes the .xz file format (filename suffix
+ ".xz", MIME type "application/x-xz"). It is intended that this
+ this format replace the old .lzma format used by LZMA SDK and
+ LZMA Utils.
+
+
+0.1. Notices and Acknowledgements
+
+ This file format was designed by Lasse Collin
+ <lasse.collin@tukaani.org> and Igor Pavlov.
+
+ Special thanks for helping with this document goes to
+ Ville Koskinen. Thanks for helping with this document goes to
+ Mark Adler, H. Peter Anvin, Mikko Pouru, and Lars Wirzenius.
+
+ This document has been put into the public domain.
+
+
+0.2. Getting the Latest Version
+
+ The latest official version of this document can be downloaded
+ from <http://tukaani.org/xz/xz-file-format.txt>.
+
+ Specific versions of this document have a filename
+ xz-file-format-X.Y.Z.txt where X.Y.Z is the version number.
+ For example, the version 1.0.0 of this document is available
+ at <http://tukaani.org/xz/xz-file-format-1.0.0.txt>.
+
+
+0.3. Version History
+
+ Version Date Description
+
+ 1.0.4 2009-08-27 Language improvements in Sections 1.2,
+ 2.1.1.2, 3.1.1, 3.1.2, and 5.3.1
+
+ 1.0.3 2009-06-05 Spelling fixes in Sections 5.1 and 5.4
+
+ 1.0.2 2009-06-04 Typo fixes in Sections 4 and 5.3.1
+
+ 1.0.1 2009-06-01 Typo fix in Section 0.3 and minor
+ clarifications to Sections 2, 2.2,
+ 3.3, 4.4, and 5.3.2
+
+ 1.0.0 2009-01-14 The first official version
+
+
+1. Conventions
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHOULD",
+ "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+ document are to be interpreted as described in [RFC-2119].
+
+ Indicating a warning means displaying a message, returning
+ appropriate exit status, or doing something else to let the
+ user know that something worth warning occurred. The operation
+ SHOULD still finish if a warning is indicated.
+
+ Indicating an error means displaying a message, returning
+ appropriate exit status, or doing something else to let the
+ user know that something prevented successfully finishing the
+ operation. The operation MUST be aborted once an error has
+ been indicated.
+
+
+1.1. Byte and Its Representation
+
+ In this document, byte is always 8 bits.
+
+ A "null byte" has all bits unset. That is, the value of a null
+ byte is 0x00.
+
+ To represent byte blocks, this document uses notation that
+ is similar to the notation used in [RFC-1952]:
+
+ +-------+
+ | Foo | One byte.
+ +-------+
+
+ +---+---+
+ | Foo | Two bytes; that is, some of the vertical bars
+ +---+---+ can be missing.
+
+ +=======+
+ | Foo | Zero or more bytes.
+ +=======+
+
+ In this document, a boxed byte or a byte sequence declared
+ using this notation is called "a field". The example field
+ above would be called "the Foo field" or plain "Foo".
+
+ If there are many fields, they may be split to multiple lines.
+ This is indicated with an arrow ("--->"):
+
+ +=====+
+ | Foo |
+ +=====+
+
+ +=====+
+ ---> | Bar |
+ +=====+
+
+ The above is equivalent to this:
+
+ +=====+=====+
+ | Foo | Bar |
+ +=====+=====+
+
+
+1.2. Multibyte Integers
+
+ Multibyte integers of static length, such as CRC values,
+ are stored in little endian byte order (least significant
+ byte first).
+
+ When smaller values are more likely than bigger values (for
+ example file sizes), multibyte integers are encoded in a
+ variable-length representation:
+ - Numbers in the range [0, 127] are copied as is, and take
+ one byte of space.
+ - Bigger numbers will occupy two or more bytes. All but the
+ last byte of the multibyte representation have the highest
+ (eighth) bit set.
+
+ For now, the value of the variable-length integers is limited
+ to 63 bits, which limits the encoded size of the integer to
+ nine bytes. These limits may be increased in the future if
+ needed.
+
+ The following C code illustrates encoding and decoding of
+ variable-length integers. The functions return the number of
+ bytes occupied by the integer (1-9), or zero on error.
+
+ #include <stddef.h>
+ #include <inttypes.h>
+
+ size_t
+ encode(uint8_t buf[static 9], uint64_t num)
+ {
+ if (num > UINT64_MAX / 2)
+ return 0;
+
+ size_t i = 0;
+
+ while (num >= 0x80) {
+ buf[i++] = (uint8_t)(num) | 0x80;
+ num >>= 7;
+ }
+
+ buf[i++] = (uint8_t)(num);
+
+ return i;
+ }
+
+ size_t
+ decode(const uint8_t buf[], size_t size_max, uint64_t *num)
+ {
+ if (size_max == 0)
+ return 0;
+
+ if (size_max > 9)
+ size_max = 9;
+
+ *num = buf[0] & 0x7F;
+ size_t i = 0;
+
+ while (buf[i++] & 0x80) {
+ if (i >= size_max || buf[i] == 0x00)
+ return 0;
+
+ *num |= (uint64_t)(buf[i] & 0x7F) << (i * 7);
+ }
+
+ return i;
+ }
+
+
+2. Overall Structure of .xz File
+
+ A standalone .xz files consist of one or more Streams which may
+ have Stream Padding between or after them:
+
+ +========+================+========+================+
+ | Stream | Stream Padding | Stream | Stream Padding | ...
+ +========+================+========+================+
+
+ The sizes of Stream and Stream Padding are always multiples
+ of four bytes, thus the size of every valid .xz file MUST be
+ a multiple of four bytes.
+
+ While a typical file contains only one Stream and no Stream
+ Padding, a decoder handling standalone .xz files SHOULD support
+ files that have more than one Stream or Stream Padding.
+
+ In contrast to standalone .xz files, when the .xz file format
+ is used as an internal part of some other file format or
+ communication protocol, it usually is expected that the decoder
+ stops after the first Stream, and doesn't look for Stream
+ Padding or possibly other Streams.
+
+
+2.1. Stream
+
+ +-+-+-+-+-+-+-+-+-+-+-+-+=======+=======+ +=======+
+ | Stream Header | Block | Block | ... | Block |
+ +-+-+-+-+-+-+-+-+-+-+-+-+=======+=======+ +=======+
+
+ +=======+-+-+-+-+-+-+-+-+-+-+-+-+
+ ---> | Index | Stream Footer |
+ +=======+-+-+-+-+-+-+-+-+-+-+-+-+
+
+ All the above fields have a size that is a multiple of four. If
+ Stream is used as an internal part of another file format, it
+ is RECOMMENDED to make the Stream start at an offset that is
+ a multiple of four bytes.
+
+ Stream Header, Index, and Stream Footer are always present in
+ a Stream. The maximum size of the Index field is 16 GiB (2^34).
+
+ There are zero or more Blocks. The maximum number of Blocks is
+ limited only by the maximum size of the Index field.
+
+ Total size of a Stream MUST be less than 8 EiB (2^63 bytes).
+ The same limit applies to the total amount of uncompressed
+ data stored in a Stream.
+
+ If an implementation supports handling .xz files with multiple
+ concatenated Streams, it MAY apply the above limits to the file
+ as a whole instead of limiting per Stream basis.
+
+
+2.1.1. Stream Header
+
+ +---+---+---+---+---+---+-------+------+--+--+--+--+
+ | Header Magic Bytes | Stream Flags | CRC32 |
+ +---+---+---+---+---+---+-------+------+--+--+--+--+
+
+
+2.1.1.1. Header Magic Bytes
+
+ The first six (6) bytes of the Stream are so called Header
+ Magic Bytes. They can be used to identify the file type.
+
+ Using a C array and ASCII:
+ const uint8_t HEADER_MAGIC[6]
+ = { 0xFD, '7', 'z', 'X', 'Z', 0x00 };
+
+ In plain hexadecimal:
+ FD 37 7A 58 5A 00
+
+ Notes:
+ - The first byte (0xFD) was chosen so that the files cannot
+ be erroneously detected as being in .lzma format, in which
+ the first byte is in the range [0x00, 0xE0].
+ - The sixth byte (0x00) was chosen to prevent applications
+ from misdetecting the file as a text file.
+
+ If the Header Magic Bytes don't match, the decoder MUST
+ indicate an error.
+
+
+2.1.1.2. Stream Flags
+
+ The first byte of Stream Flags is always a null byte. In the
+ future, this byte may be used to indicate a new Stream version
+ or other Stream properties.
+
+ The second byte of Stream Flags is a bit field:
+
+ Bit(s) Mask Description
+ 0-3 0x0F Type of Check (see Section 3.4):
+ ID Size Check name
+ 0x00 0 bytes None
+ 0x01 4 bytes CRC32
+ 0x02 4 bytes (Reserved)
+ 0x03 4 bytes (Reserved)
+ 0x04 8 bytes CRC64
+ 0x05 8 bytes (Reserved)
+ 0x06 8 bytes (Reserved)
+ 0x07 16 bytes (Reserved)
+ 0x08 16 bytes (Reserved)
+ 0x09 16 bytes (Reserved)
+ 0x0A 32 bytes SHA-256
+ 0x0B 32 bytes (Reserved)
+ 0x0C 32 bytes (Reserved)
+ 0x0D 64 bytes (Reserved)
+ 0x0E 64 bytes (Reserved)
+ 0x0F 64 bytes (Reserved)
+ 4-7 0xF0 Reserved for future use; MUST be zero for now.
+
+ Implementations SHOULD support at least the Check IDs 0x00
+ (None) and 0x01 (CRC32). Supporting other Check IDs is
+ OPTIONAL. If an unsupported Check is used, the decoder SHOULD
+ indicate a warning or error.
+
+ If any reserved bit is set, the decoder MUST indicate an error.
+ It is possible that there is a new field present which the
+ decoder is not aware of, and can thus parse the Stream Header
+ incorrectly.
+
+
+2.1.1.3. CRC32
+
+ The CRC32 is calculated from the Stream Flags field. It is
+ stored as an unsigned 32-bit little endian integer. If the
+ calculated value does not match the stored one, the decoder
+ MUST indicate an error.
+
+ The idea is that Stream Flags would always be two bytes, even
+ if new features are needed. This way old decoders will be able
+ to verify the CRC32 calculated from Stream Flags, and thus
+ distinguish between corrupt files (CRC32 doesn't match) and
+ files that the decoder doesn't support (CRC32 matches but
+ Stream Flags has reserved bits set).
+
+
+2.1.2. Stream Footer
+
+ +-+-+-+-+---+---+---+---+-------+------+----------+---------+
+ | CRC32 | Backward Size | Stream Flags | Footer Magic Bytes |
+ +-+-+-+-+---+---+---+---+-------+------+----------+---------+
+
+
+2.1.2.1. CRC32
+
+ The CRC32 is calculated from the Backward Size and Stream Flags
+ fields. It is stored as an unsigned 32-bit little endian
+ integer. If the calculated value does not match the stored one,
+ the decoder MUST indicate an error.
+
+ The reason to have the CRC32 field before the Backward Size and
+ Stream Flags fields is to keep the four-byte fields aligned to
+ a multiple of four bytes.
+
+
+2.1.2.2. Backward Size
+
+ Backward Size is stored as a 32-bit little endian integer,
+ which indicates the size of the Index field as multiple of
+ four bytes, minimum value being four bytes:
+
+ real_backward_size = (stored_backward_size + 1) * 4;
+
+ If the stored value does not match the real size of the Index
+ field, the decoder MUST indicate an error.
+
+ Using a fixed-size integer to store Backward Size makes
+ it slightly simpler to parse the Stream Footer when the
+ application needs to parse the Stream backwards.
+
+
+2.1.2.3. Stream Flags
+
+ This is a copy of the Stream Flags field from the Stream
+ Header. The information stored to Stream Flags is needed
+ when parsing the Stream backwards. The decoder MUST compare
+ the Stream Flags fields in both Stream Header and Stream
+ Footer, and indicate an error if they are not identical.
+
+
+2.1.2.4. Footer Magic Bytes
+
+ As the last step of the decoding process, the decoder MUST
+ verify the existence of Footer Magic Bytes. If they don't
+ match, an error MUST be indicated.
+
+ Using a C array and ASCII:
+ const uint8_t FOOTER_MAGIC[2] = { 'Y', 'Z' };
+
+ In hexadecimal:
+ 59 5A
+
+ The primary reason to have Footer Magic Bytes is to make
+ it easier to detect incomplete files quickly, without
+ uncompressing. If the file does not end with Footer Magic Bytes
+ (excluding Stream Padding described in Section 2.2), it cannot
+ be undamaged, unless someone has intentionally appended garbage
+ after the end of the Stream.
+
+
+2.2. Stream Padding
+
+ Only the decoders that support decoding of concatenated Streams
+ MUST support Stream Padding.
+
+ Stream Padding MUST contain only null bytes. To preserve the
+ four-byte alignment of consecutive Streams, the size of Stream
+ Padding MUST be a multiple of four bytes. Empty Stream Padding
+ is allowed. If these requirements are not met, the decoder MUST
+ indicate an error.
+
+ Note that non-empty Stream Padding is allowed at the end of the
+ file; there doesn't need to be a new Stream after non-empty
+ Stream Padding. This can be convenient in certain situations
+ [GNU-tar].
+
+ The possibility of Stream Padding MUST be taken into account
+ when designing an application that parses Streams backwards,
+ and the application supports concatenated Streams.
+
+
+3. Block
+
+ +==============+=================+===============+=======+
+ | Block Header | Compressed Data | Block Padding | Check |
+ +==============+=================+===============+=======+
+
+
+3.1. Block Header
+
+ +-------------------+-------------+=================+
+ | Block Header Size | Block Flags | Compressed Size |
+ +-------------------+-------------+=================+
+
+ +===================+======================+
+ ---> | Uncompressed Size | List of Filter Flags |
+ +===================+======================+
+
+ +================+--+--+--+--+
+ ---> | Header Padding | CRC32 |
+ +================+--+--+--+--+
+
+
+3.1.1. Block Header Size
+
+ This field overlaps with the Index Indicator field (see
+ Section 4.1).
+
+ This field contains the size of the Block Header field,
+ including the Block Header Size field itself. Valid values are
+ in the range [0x01, 0xFF], which indicate the size of the Block
+ Header as multiples of four bytes, minimum size being eight
+ bytes:
+
+ real_header_size = (encoded_header_size + 1) * 4;
+
+ If a Block Header bigger than 1024 bytes is needed in the
+ future, a new field can be added between the Block Header and
+ Compressed Data fields. The presence of this new field would
+ be indicated in the Block Header field.
+
+
+3.1.2. Block Flags
+
+ The Block Flags field is a bit field:
+
+ Bit(s) Mask Description
+ 0-1 0x03 Number of filters (1-4)
+ 2-5 0x3C Reserved for future use; MUST be zero for now.
+ 6 0x40 The Compressed Size field is present.
+ 7 0x80 The Uncompressed Size field is present.
+
+ If any reserved bit is set, the decoder MUST indicate an error.
+ It is possible that there is a new field present which the
+ decoder is not aware of, and can thus parse the Block Header
+ incorrectly.
+
+
+3.1.3. Compressed Size
+
+ This field is present only if the appropriate bit is set in
+ the Block Flags field (see Section 3.1.2).
+
+ The Compressed Size field contains the size of the Compressed
+ Data field, which MUST be non-zero. Compressed Size is stored
+ using the encoding described in Section 1.2. If the Compressed
+ Size doesn't match the size of the Compressed Data field, the
+ decoder MUST indicate an error.
+
+
+3.1.4. Uncompressed Size
+
+ This field is present only if the appropriate bit is set in
+ the Block Flags field (see Section 3.1.2).
+
+ The Uncompressed Size field contains the size of the Block
+ after uncompressing. Uncompressed Size is stored using the
+ encoding described in Section 1.2. If the Uncompressed Size
+ does not match the real uncompressed size, the decoder MUST
+ indicate an error.
+
+ Storing the Compressed Size and Uncompressed Size fields serves
+ several purposes:
+ - The decoder knows how much memory it needs to allocate
+ for a temporary buffer in multithreaded mode.
+ - Simple error detection: wrong size indicates a broken file.
+ - Seeking forwards to a specific location in streamed mode.
+
+ It should be noted that the only reliable way to determine
+ the real uncompressed size is to uncompress the Block,
+ because the Block Header and Index fields may contain
+ (intentionally or unintentionally) invalid information.
+
+
+3.1.5. List of Filter Flags
+
+ +================+================+ +================+
+ | Filter 0 Flags | Filter 1 Flags | ... | Filter n Flags |
+ +================+================+ +================+
+
+ The number of Filter Flags fields is stored in the Block Flags
+ field (see Section 3.1.2).
+
+ The format of each Filter Flags field is as follows:
+
+ +===========+====================+===================+
+ | Filter ID | Size of Properties | Filter Properties |
+ +===========+====================+===================+
+
+ Both Filter ID and Size of Properties are stored using the
+ encoding described in Section 1.2. Size of Properties indicates
+ the size of the Filter Properties field as bytes. The list of
+ officially defined Filter IDs and the formats of their Filter
+ Properties are described in Section 5.3.
+
+ Filter IDs greater than or equal to 0x4000_0000_0000_0000
+ (2^62) are reserved for implementation-specific internal use.
+ These Filter IDs MUST never be used in List of Filter Flags.
+
+
+3.1.6. Header Padding
+
+ This field contains as many null byte as it is needed to make
+ the Block Header have the size specified in Block Header Size.
+ If any of the bytes are not null bytes, the decoder MUST
+ indicate an error. It is possible that there is a new field
+ present which the decoder is not aware of, and can thus parse
+ the Block Header incorrectly.
+
+
+3.1.7. CRC32
+
+ The CRC32 is calculated over everything in the Block Header
+ field except the CRC32 field itself. It is stored as an
+ unsigned 32-bit little endian integer. If the calculated
+ value does not match the stored one, the decoder MUST indicate
+ an error.
+
+ By verifying the CRC32 of the Block Header before parsing the
+ actual contents allows the decoder to distinguish between
+ corrupt and unsupported files.
+
+
+3.2. Compressed Data
+
+ The format of Compressed Data depends on Block Flags and List
+ of Filter Flags. Excluding the descriptions of the simplest
+ filters in Section 5.3, the format of the filter-specific
+ encoded data is out of scope of this document.
+
+
+3.3. Block Padding
+
+ Block Padding MUST contain 0-3 null bytes to make the size of
+ the Block a multiple of four bytes. This can be needed when
+ the size of Compressed Data is not a multiple of four. If any
+ of the bytes in Block Padding are not null bytes, the decoder
+ MUST indicate an error.
+
+
+3.4. Check
+
+ The type and size of the Check field depends on which bits
+ are set in the Stream Flags field (see Section 2.1.1.2).
+
+ The Check, when used, is calculated from the original
+ uncompressed data. If the calculated Check does not match the
+ stored one, the decoder MUST indicate an error. If the selected
+ type of Check is not supported by the decoder, it SHOULD
+ indicate a warning or error.
+
+
+4. Index
+
+ +-----------------+===================+
+ | Index Indicator | Number of Records |
+ +-----------------+===================+
+
+ +=================+===============+-+-+-+-+
+ ---> | List of Records | Index Padding | CRC32 |
+ +=================+===============+-+-+-+-+
+
+ Index serves several purposes. Using it, one can
+ - verify that all Blocks in a Stream have been processed;
+ - find out the uncompressed size of a Stream; and
+ - quickly access the beginning of any Block (random access).
+
+
+4.1. Index Indicator
+
+ This field overlaps with the Block Header Size field (see
+ Section 3.1.1). The value of Index Indicator is always 0x00.
+
+
+4.2. Number of Records
+
+ This field indicates how many Records there are in the List
+ of Records field, and thus how many Blocks there are in the
+ Stream. The value is stored using the encoding described in
+ Section 1.2. If the decoder has decoded all the Blocks of the
+ Stream, and then notices that the Number of Records doesn't
+ match the real number of Blocks, the decoder MUST indicate an
+ error.
+
+
+4.3. List of Records
+
+ List of Records consists of as many Records as indicated by the
+ Number of Records field:
+
+ +========+========+
+ | Record | Record | ...
+ +========+========+
+
+ Each Record contains information about one Block:
+
+ +===============+===================+
+ | Unpadded Size | Uncompressed Size |
+ +===============+===================+
+
+ If the decoder has decoded all the Blocks of the Stream, it
+ MUST verify that the contents of the Records match the real
+ Unpadded Size and Uncompressed Size of the respective Blocks.
+
+ Implementation hint: It is possible to verify the Index with
+ constant memory usage by calculating for example SHA-256 of
+ both the real size values and the List of Records, then
+ comparing the hash values. Implementing this using
+ non-cryptographic hash like CRC32 SHOULD be avoided unless
+ small code size is important.
+
+ If the decoder supports random-access reading, it MUST verify
+ that Unpadded Size and Uncompressed Size of every completely
+ decoded Block match the sizes stored in the Index. If only
+ partial Block is decoded, the decoder MUST verify that the
+ processed sizes don't exceed the sizes stored in the Index.
+
+
+4.3.1. Unpadded Size
+
+ This field indicates the size of the Block excluding the Block
+ Padding field. That is, Unpadded Size is the size of the Block
+ Header, Compressed Data, and Check fields. Unpadded Size is
+ stored using the encoding described in Section 1.2. The value
+ MUST never be zero; with the current structure of Blocks, the
+ actual minimum value for Unpadded Size is five.
+
+ Implementation note: Because the size of the Block Padding
+ field is not included in Unpadded Size, calculating the total
+ size of a Stream or doing random-access reading requires
+ calculating the actual size of the Blocks by rounding Unpadded
+ Sizes up to the next multiple of four.
+
+ The reason to exclude Block Padding from Unpadded Size is to
+ ease making a raw copy of Compressed Data without Block
+ Padding. This can be useful, for example, if someone wants
+ to convert Streams to some other file format quickly.
+
+
+4.3.2. Uncompressed Size
+
+ This field indicates the Uncompressed Size of the respective
+ Block as bytes. The value is stored using the encoding
+ described in Section 1.2.
+
+
+4.4. Index Padding
+
+ This field MUST contain 0-3 null bytes to pad the Index to
+ a multiple of four bytes. If any of the bytes are not null
+ bytes, the decoder MUST indicate an error.
+
+
+4.5. CRC32
+
+ The CRC32 is calculated over everything in the Index field
+ except the CRC32 field itself. The CRC32 is stored as an
+ unsigned 32-bit little endian integer. If the calculated
+ value does not match the stored one, the decoder MUST indicate
+ an error.
+
+
+5. Filter Chains
+
+ The Block Flags field defines how many filters are used. When
+ more than one filter is used, the filters are chained; that is,
+ the output of one filter is the input of another filter. The
+ following figure illustrates the direction of data flow.
+
+ v Uncompressed Data ^
+ | Filter 0 |
+ Encoder | Filter 1 | Decoder
+ | Filter n |
+ v Compressed Data ^
+
+
+5.1. Alignment
+
+ Alignment of uncompressed input data is usually the job of
+ the application producing the data. For example, to get the
+ best results, an archiver tool should make sure that all
+ PowerPC executable files in the archive stream start at
+ offsets that are multiples of four bytes.
+
+ Some filters, for example LZMA2, can be configured to take
+ advantage of specified alignment of input data. Note that
+ taking advantage of aligned input can be beneficial also when
+ a filter is not the first filter in the chain. For example,
+ if you compress PowerPC executables, you may want to use the
+ PowerPC filter and chain that with the LZMA2 filter. Because
+ not only the input but also the output alignment of the PowerPC
+ filter is four bytes, it is now beneficial to set LZMA2
+ settings so that the LZMA2 encoder can take advantage of its
+ four-byte-aligned input data.
+
+ The output of the last filter in the chain is stored to the
+ Compressed Data field, which is is guaranteed to be aligned
+ to a multiple of four bytes relative to the beginning of the
+ Stream. This can increase
+ - speed, if the filtered data is handled multiple bytes at
+ a time by the filter-specific encoder and decoder,
+ because accessing aligned data in computer memory is
+ usually faster; and
+ - compression ratio, if the output data is later compressed
+ with an external compression tool.
+
+
+5.2. Security
+
+ If filters would be allowed to be chained freely, it would be
+ possible to create malicious files, that would be very slow to
+ decode. Such files could be used to create denial of service
+ attacks.
+
+ Slow files could occur when multiple filters are chained:
+
+ v Compressed input data
+ | Filter 1 decoder (last filter)
+ | Filter 0 decoder (non-last filter)
+ v Uncompressed output data
+
+ The decoder of the last filter in the chain produces a lot of
+ output from little input. Another filter in the chain takes the
+ output of the last filter, and produces very little output
+ while consuming a lot of input. As a result, a lot of data is
+ moved inside the filter chain, but the filter chain as a whole
+ gets very little work done.
+
+ To prevent this kind of slow files, there are restrictions on
+ how the filters can be chained. These restrictions MUST be
+ taken into account when designing new filters.
+
+ The maximum number of filters in the chain has been limited to
+ four, thus there can be at maximum of three non-last filters.
+ Of these three non-last filters, only two are allowed to change
+ the size of the data.
+
+ The non-last filters, that change the size of the data, MUST
+ have a limit how much the decoder can compress the data: the
+ decoder SHOULD produce at least n bytes of output when the
+ filter is given 2n bytes of input. This limit is not
+ absolute, but significant deviations MUST be avoided.
+
+ The above limitations guarantee that if the last filter in the
+ chain produces 4n bytes of output, the chain as a whole will
+ produce at least n bytes of output.
+
+
+5.3. Filters
+
+5.3.1. LZMA2
+
+ LZMA (Lempel-Ziv-Markov chain-Algorithm) is a general-purpose
+ compression algorithm with high compression ratio and fast
+ decompression. LZMA is based on LZ77 and range coding
+ algorithms.
+
+ LZMA2 is an extension on top of the original LZMA. LZMA2 uses
+ LZMA internally, but adds support for flushing the encoder,
+ uncompressed chunks, eases stateful decoder implementations,
+ and improves support for multithreading. Thus, the plain LZMA
+ will not be supported in this file format.
+
+ Filter ID: 0x21
+ Size of Filter Properties: 1 byte
+ Changes size of data: Yes
+ Allow as a non-last filter: No
+ Allow as the last filter: Yes
+
+ Preferred alignment:
+ Input data: Adjustable to 1/2/4/8/16 byte(s)
+ Output data: 1 byte
+
+ The format of the one-byte Filter Properties field is as
+ follows:
+
+ Bits Mask Description
+ 0-5 0x3F Dictionary Size
+ 6-7 0xC0 Reserved for future use; MUST be zero for now.
+
+ Dictionary Size is encoded with one-bit mantissa and five-bit
+ exponent. The smallest dictionary size is 4 KiB and the biggest
+ is 4 GiB.
+
+ Raw value Mantissa Exponent Dictionary size
+ 0 2 11 4 KiB
+ 1 3 11 6 KiB
+ 2 2 12 8 KiB
+ 3 3 12 12 KiB
+ 4 2 13 16 KiB
+ 5 3 13 24 KiB
+ 6 2 14 32 KiB
+ ... ... ... ...
+ 35 3 27 768 MiB
+ 36 2 28 1024 MiB
+ 37 3 29 1536 MiB
+ 38 2 30 2048 MiB
+ 39 3 30 3072 MiB
+ 40 2 31 4096 MiB - 1 B
+
+ Instead of having a table in the decoder, the dictionary size
+ can be decoded using the following C code:
+
+ const uint8_t bits = get_dictionary_flags() & 0x3F;
+ if (bits > 40)
+ return DICTIONARY_TOO_BIG; // Bigger than 4 GiB
+
+ uint32_t dictionary_size;
+ if (bits == 40) {
+ dictionary_size = UINT32_MAX;
+ } else {
+ dictionary_size = 2 | (bits & 1);
+ dictionary_size <<= bits / 2 + 11;
+ }
+
+
+5.3.2. Branch/Call/Jump Filters for Executables
+
+ These filters convert relative branch, call, and jump
+ instructions to their absolute counterparts in executable
+ files. This conversion increases redundancy and thus
+ compression ratio.
+
+ Size of Filter Properties: 0 or 4 bytes
+ Changes size of data: No
+ Allow as a non-last filter: Yes
+ Allow as the last filter: No
+
+ Below is the list of filters in this category. The alignment
+ is the same for both input and output data.
+
+ Filter ID Alignment Description
+ 0x04 1 byte x86 filter (BCJ)
+ 0x05 4 bytes PowerPC (big endian) filter
+ 0x06 16 bytes IA64 filter
+ 0x07 4 bytes ARM (little endian) filter
+ 0x08 2 bytes ARM Thumb (little endian) filter
+ 0x09 4 bytes SPARC filter
+
+ If the size of Filter Properties is four bytes, the Filter
+ Properties field contains the start offset used for address
+ conversions. It is stored as an unsigned 32-bit little endian
+ integer. The start offset MUST be a multiple of the alignment
+ of the filter as listed in the table above; if it isn't, the
+ decoder MUST indicate an error. If the size of Filter
+ Properties is zero, the start offset is zero.
+
+ Setting the start offset may be useful if an executable has
+ multiple sections, and there are many cross-section calls.
+ Taking advantage of this feature usually requires usage of
+ the Subblock filter, whose design is not complete yet.
+
+
+5.3.3. Delta
+
+ The Delta filter may increase compression ratio when the value
+ of the next byte correlates with the value of an earlier byte
+ at specified distance.
+
+ Filter ID: 0x03
+ Size of Filter Properties: 1 byte
+ Changes size of data: No
+ Allow as a non-last filter: Yes
+ Allow as the last filter: No
+
+ Preferred alignment:
+ Input data: 1 byte
+ Output data: Same as the original input data
+
+ The Properties byte indicates the delta distance, which can be
+ 1-256 bytes backwards from the current byte: 0x00 indicates
+ distance of 1 byte and 0xFF distance of 256 bytes.
+
+
+5.3.3.1. Format of the Encoded Output
+
+ The code below illustrates both encoding and decoding with
+ the Delta filter.
+
+ // Distance is in the range [1, 256].
+ const unsigned int distance = get_properties_byte() + 1;
+ uint8_t pos = 0;
+ uint8_t delta[256];
+
+ memset(delta, 0, sizeof(delta));
+
+ while (1) {
+ const int byte = read_byte();
+ if (byte == EOF)
+ break;
+
+ uint8_t tmp = delta[(uint8_t)(distance + pos)];
+ if (is_encoder) {
+ tmp = (uint8_t)(byte) - tmp;
+ delta[pos] = (uint8_t)(byte);
+ } else {
+ tmp = (uint8_t)(byte) + tmp;
+ delta[pos] = tmp;
+ }
+
+ write_byte(tmp);
+ --pos;
+ }
+
+
+5.4. Custom Filter IDs
+
+ If a developer wants to use custom Filter IDs, he has two
+ choices. The first choice is to contact Lasse Collin and ask
+ him to allocate a range of IDs for the developer.
+
+ The second choice is to generate a 40-bit random integer,
+ which the developer can use as his personal Developer ID.
+ To minimize the risk of collisions, Developer ID has to be
+ a randomly generated integer, not manually selected "hex word".
+ The following command, which works on many free operating
+ systems, can be used to generate Developer ID:
+
+ dd if=/dev/urandom bs=5 count=1 | hexdump
+
+ The developer can then use his Developer ID to create unique
+ (well, hopefully unique) Filter IDs.
+
+ Bits Mask Description
+ 0-15 0x0000_0000_0000_FFFF Filter ID
+ 16-55 0x00FF_FFFF_FFFF_0000 Developer ID
+ 56-62 0x3F00_0000_0000_0000 Static prefix: 0x3F
+
+ The resulting 63-bit integer will use 9 bytes of space when
+ stored using the encoding described in Section 1.2. To get
+ a shorter ID, see the beginning of this Section how to
+ request a custom ID range.
+
+
+5.4.1. Reserved Custom Filter ID Ranges
+
+ Range Description
+ 0x0000_0300 - 0x0000_04FF Reserved to ease .7z compatibility
+ 0x0002_0000 - 0x0007_FFFF Reserved to ease .7z compatibility
+ 0x0200_0000 - 0x07FF_FFFF Reserved to ease .7z compatibility
+
+
+6. Cyclic Redundancy Checks
+
+ There are several incompatible variations to calculate CRC32
+ and CRC64. For simplicity and clarity, complete examples are
+ provided to calculate the checks as they are used in this file
+ format. Implementations MAY use different code as long as it
+ gives identical results.
+
+ The program below reads data from standard input, calculates
+ the CRC32 and CRC64 values, and prints the calculated values
+ as big endian hexadecimal strings to standard output.
+
+ #include <stddef.h>
+ #include <inttypes.h>
+ #include <stdio.h>
+
+ uint32_t crc32_table[256];
+ uint64_t crc64_table[256];
+
+ void
+ init(void)
+ {
+ static const uint32_t poly32 = UINT32_C(0xEDB88320);
+ static const uint64_t poly64
+ = UINT64_C(0xC96C5795D7870F42);
+
+ for (size_t i = 0; i < 256; ++i) {
+ uint32_t crc32 = i;
+ uint64_t crc64 = i;
+
+ for (size_t j = 0; j < 8; ++j) {
+ if (crc32 & 1)
+ crc32 = (crc32 >> 1) ^ poly32;
+ else
+ crc32 >>= 1;
+
+ if (crc64 & 1)
+ crc64 = (crc64 >> 1) ^ poly64;
+ else
+ crc64 >>= 1;
+ }
+
+ crc32_table[i] = crc32;
+ crc64_table[i] = crc64;
+ }
+ }
+
+ uint32_t
+ crc32(const uint8_t *buf, size_t size, uint32_t crc)
+ {
+ crc = ~crc;
+ for (size_t i = 0; i < size; ++i)
+ crc = crc32_table[buf[i] ^ (crc & 0xFF)]
+ ^ (crc >> 8);
+ return ~crc;
+ }
+
+ uint64_t
+ crc64(const uint8_t *buf, size_t size, uint64_t crc)
+ {
+ crc = ~crc;
+ for (size_t i = 0; i < size; ++i)
+ crc = crc64_table[buf[i] ^ (crc & 0xFF)]
+ ^ (crc >> 8);
+ return ~crc;
+ }
+
+ int
+ main()
+ {
+ init();
+
+ uint32_t value32 = 0;
+ uint64_t value64 = 0;
+ uint64_t total_size = 0;
+ uint8_t buf[8192];
+
+ while (1) {
+ const size_t buf_size
+ = fread(buf, 1, sizeof(buf), stdin);
+ if (buf_size == 0)
+ break;
+
+ total_size += buf_size;
+ value32 = crc32(buf, buf_size, value32);
+ value64 = crc64(buf, buf_size, value64);
+ }
+
+ printf("Bytes: %" PRIu64 "\n", total_size);
+ printf("CRC-32: 0x%08" PRIX32 "\n", value32);
+ printf("CRC-64: 0x%016" PRIX64 "\n", value64);
+
+ return 0;
+ }
+
+
+7. References
+
+ LZMA SDK - The original LZMA implementation
+ http://7-zip.org/sdk.html
+
+ LZMA Utils - LZMA adapted to POSIX-like systems
+ http://tukaani.org/lzma/
+
+ XZ Utils - The next generation of LZMA Utils
+ http://tukaani.org/xz/
+
+ [RFC-1952]
+ GZIP file format specification version 4.3
+ http://www.ietf.org/rfc/rfc1952.txt
+ - Notation of byte boxes in section "2.1. Overall conventions"
+
+ [RFC-2119]
+ Key words for use in RFCs to Indicate Requirement Levels
+ http://www.ietf.org/rfc/rfc2119.txt
+
+ [GNU-tar]
+ GNU tar 1.21 manual
+ http://www.gnu.org/software/tar/manual/html_node/Blocking-Factor.html
+ - Node 9.4.2 "Blocking Factor", paragraph that begins
+ "gzip will complain about trailing garbage"
+ - Note that this URL points to the latest version of the
+ manual, and may some day not contain the note which is in
+ 1.21. For the exact version of the manual, download GNU
+ tar 1.21: ftp://ftp.gnu.org/pub/gnu/tar/tar-1.21.tar.gz
+