diff options
author | Karl Berry <karl@freefriends.org> | 2021-09-16 20:18:12 +0000 |
---|---|---|
committer | Karl Berry <karl@freefriends.org> | 2021-09-16 20:18:12 +0000 |
commit | b33e18a5efa7b022c42aee2adec507dd6917ec28 (patch) | |
tree | bba2f6f8bc7a840ba65e5ec0ec78d3ec5a463deb /Master/texmf-dist/doc/support | |
parent | fd4d90b5caf562a8d951bb4a3b78a1aa74930017 (diff) |
lua-uca (16sep21)
git-svn-id: svn://tug.org/texlive/trunk@60523 c570f23f-e606-0410-a88d-b1316a301751
Diffstat (limited to 'Master/texmf-dist/doc/support')
-rw-r--r-- | Master/texmf-dist/doc/support/lua-uca/CHANGELOG.md | 9 | ||||
-rw-r--r-- | Master/texmf-dist/doc/support/lua-uca/HACKING.md | 98 | ||||
-rw-r--r-- | Master/texmf-dist/doc/support/lua-uca/README.md | 38 | ||||
-rw-r--r-- | Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.pdf | bin | 55816 -> 72038 bytes | |||
-rw-r--r-- | Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.tex | 11 |
5 files changed, 129 insertions, 27 deletions
diff --git a/Master/texmf-dist/doc/support/lua-uca/CHANGELOG.md b/Master/texmf-dist/doc/support/lua-uca/CHANGELOG.md index 5f005ad64dd..2d5816ebfc8 100644 --- a/Master/texmf-dist/doc/support/lua-uca/CHANGELOG.md +++ b/Master/texmf-dist/doc/support/lua-uca/CHANGELOG.md @@ -1,5 +1,14 @@ # Changelog +2021-09-16 + + - added sorting rules for all languages contained in CLDR collation files. + +2020-06-09 + + - moved development information that depends on files not distributed on CTAN to `HACKING.md`. + - extended documentation. + 2020-03-24 - version `0.1` released diff --git a/Master/texmf-dist/doc/support/lua-uca/HACKING.md b/Master/texmf-dist/doc/support/lua-uca/HACKING.md new file mode 100644 index 00000000000..e0b7802ef58 --- /dev/null +++ b/Master/texmf-dist/doc/support/lua-uca/HACKING.md @@ -0,0 +1,98 @@ +# Lua-UCA hacking + +You need the full installation from +[Github](https://github.com/michal-h21/lua-uca) in order to do stuff described +in this section. Package distributed on CTAN doesn't contain all necessary +files. + +## Install + +The package needs to download Unicode collation data and convert it to a Lua +table. It depends on `wget` and `unzip` utilities. All files can be downloaded +using Make: + + make + +To install the package in the local TEXMF tree, run: + + make install + +## New language support + +To add a new language, add new function to `src/lua-uca/lua-uca-languages.lua` +file. The function name should be short language code. Example function for +the Russian language: + + languages.ru = function(collator_obj) + collator_obj:reorder{ "cyrillic" } + return collator_obj + end + +The language function takes the Collator object as a parameter. Methods showed +in the *Change sorting rules* section can be used with this object. + +The `data/common/collation/` directory in the source repository contains files from the `CLDR` project. +They contain rules for many languages. The files needs to be normalized to the +[NFC form](https://en.wikipedia.org/wiki/Unicode_equivalence), for example using: + + cat cs.xml | uconv -x any-nfc -o cs.xml + +The `uconv` utility is a part of the [ICU Project](http://userguide.icu-project.org/). + +Sorting rules for a language are placed in the `<collation>` element. Multiple +`<collation>` elements may be present in the XML file. It is usually best to chose the one with attribute +`type="standard"`. + +The following example contains code from `da.xml`: + + + [caseFirst upper] + &D<<đ<<<Đ<<ð<<<Ð + &th<<<þ + &TH<<<Þ + &Y<<ü<<<Ü<<ű<<<Ű + &[before 1]ǀ<æ<<<Æ<<ä<<<Ä<ø<<<Ø<<ö<<<Ö<<ő<<<Ő<å<<<Å<<<aa<<<Aa<<<AA + &oe<<œ<<<Œ + +This is translated to Lua code in `lua-uca-languages.lua` in the following way: + + + languages.da = function(collator_obj) + -- helper function for more readable tailoring definition + local tailoring = function(s) collator_obj:tailor_string(s) end + collator_obj:uppercase_first() + tailoring("&D<<đ<<<Đ<<ð<<<Ð") + tailoring("&th<<<þ") + tailoring("&TH<<<Þ") + tailoring("&Y<<ü<<<Ü<<ű<<<Ű") + tailoring("&ǀ<æ<<<Æ<<ä<<<Ä<ø<<<Ø<<ö<<<Ö<<ő<<<Ő<å<<<Å<<<aa<<<Aa<<<AA") + tailoring("&oe<<œ<<<Œ") + return collator_obj + end + + + + +Pull requests with new language support are highly appreciated. + +## Support files in the source distribution + +The `xindex` directory contains some examples for configuration of `Xindex`, Lua based indexing system. +Run `make xindex` command to compile them. + +`Xindex` has built-in support for Lua-UCA since version `0.23`, it can be requested using the `-u` option. + +The `tools/indexing-sample.lua` file provides a simple indexing processor, independent of any other tool. + +## Testing + +You can run unit tests using the following command: + + make test + +Testing requires [Busted](https://olivinelabs.com/busted/) testing framework installed on your system. +Tests are placed in the `spec` directory and they provide more examples of the package usage. + + + + diff --git a/Master/texmf-dist/doc/support/lua-uca/README.md b/Master/texmf-dist/doc/support/lua-uca/README.md index b3cef273ea3..37343a83afd 100644 --- a/Master/texmf-dist/doc/support/lua-uca/README.md +++ b/Master/texmf-dist/doc/support/lua-uca/README.md @@ -1,15 +1,9 @@ +\iffalse # The `Lua-UCA` package +\fi This package adds support for the [Unicode collation algorithm](https://unicode.org/reports/tr10/) for Lua 5.3. -## Install - -The package needs to download Unicode collation data and convert it to a Lua table. It depends on `wget` and `unzip` utitilities. - -To install the package in the local TEXMF tree, run: - - make - make install ## Usage @@ -43,18 +37,17 @@ The output: > chochol > jasan -More samples of use can be found in the `spec` directory. -`tools/indexing-sample.lua` is a simple indexing processor. +More samples of the library usage can be found in the source repository of this package on [Github](https://github.com/michal-h21/lua-uca). +% See `HACKING.md` file in the repo for more information. ## Use with Xindex processor [Xindex](https://www.ctan.org/pkg/xindex) is flexible index processor written -in Lua by Herbert Voß. It supports Lua configuration files, which enables use -of Lua-UCA for sorting of the index entries, as shown in [this -example](https://tex.stackexchange.com/a/524014/2891) for Norwegian text. +in Lua by Herbert Voß. It has built-in `Lua-UCA` support starting with version +`0.23`. The support can be requested using the `-u` option: + + xindex -u -l no -c norsk filename.idx -The `xindex` directory in the [source repository](https://github.com/michal-h21/lua-uca/tree/master/xindex) contains more advanced version of such configuration -file together with several examples. Run `make xindex` command to compile them. ## Change sorting rules @@ -90,7 +83,7 @@ It is also possible to expand a letter to multiple letters, like this example fo tailoring "&Ö=Oe" tailoring "&ö=oe" -Some languages, like Norwegian sort uppercase letters before lowercase. This +Some languages, like Norwegian, sort uppercase letters before lowercase. This can be enabled using `collator_obj:uppercase_first()` function: local tailoring = function(s) collator_obj:tailor_string(s) end @@ -102,14 +95,8 @@ can be enabled using `collator_obj:uppercase_first()` function: tailoring("&ǀ<æ<<<Æ<<ä<<<Ä<ø<<<Ø<<ö<<<Ö<<ő<<<Ő<å<<<Å<<<aa<<<Aa<<<AA") tailoring("&oe<<œ<<<Œ") -The `data/common/collation/` directory contains files from the `CLDR` project. -They contain rules for many languages. The files needs to be normalized to the -[NFC form](https://en.wikipedia.org/wiki/Unicode_equivalence), for example -using: - - cat cs.xml | uconv -x any-nfc -o cs.xml - -The `uconv` utility is a part of the [ICU Project](http://userguide.icu-project.org/). +% More information on a new language support is in the `HACKING.md` +% document in the [`Lua-UCA` Github repo](https://github.com/michal-h21/lua-uca/blob/master/HACKING.md). ### Script reordering @@ -117,7 +104,7 @@ Many languages sort different scripts after the script this language uses. As Latin based scripts are sorted first, it is necessary to reorder scripts in such cases. -The `collator_obj:reorder` function takes table with scripts that need to be reorderd. +The `collator_obj:reorder` function takes table with scripts that need to be reordered. For example Cyrillic can be sorted before Latin using: collator_obj:reorder {"cyrillic"} @@ -132,6 +119,5 @@ will be sorted at the very end. # What is missing -- Tailorings for most languages. - Algorithm for setting implicit sort weights of characters that are not explicitly listed in DUCET. - Special handling of CJK scripts. diff --git a/Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.pdf b/Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.pdf Binary files differindex f5d94dfdcf2..049332eda18 100644 --- a/Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.pdf +++ b/Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.pdf diff --git a/Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.tex b/Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.tex index 0ef7914bae9..2d25f519920 100644 --- a/Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.tex +++ b/Master/texmf-dist/doc/support/lua-uca/lua-uca-doc.tex @@ -26,10 +26,12 @@ \begin{document} \maketitle \tableofcontents +\section{Introduction} \markdownInput{README.md} -\section{Available languages} +\section{Available Languages} +\begin{raggedright} The \texttt{lua-uca-languages} library provides the following langauges: \bgroup\ttfamily \begin{luacode*} @@ -44,6 +46,13 @@ table.sort(l) tex.print(table.concat(l, ", ")) \end{luacode*} \egroup +\end{raggedright} + +If you want to requrest language not listed in this listing, or if you had +created support code for one, please contact the package author by mail or using +issue tracker on package's Github profile. + +\markdownInput{HACKING.md} \section{License} \markdownInput{LICENSE} |