datatool (10nov17)

git-svn-id: svn://tug.org/texlive/trunk@45742 c570f23f-e606-0410-a88d-b1316a301751

1 files changed, 179 insertions, 21 deletions

diff --git a/Master/texmf-dist/doc/latex/datatool/datatool-user.tex b/Master/texmf-dist/doc/latex/datatool/datatool-user.tex
index b5e65957904..ca1e0098df3 100644
--- a/Master/texmf-dist/doc/latex/datatool/datatool-user.tex
+++ b/Master/texmf-dist/doc/latex/datatool/datatool-user.tex
@@ -6,6 +6,7 @@
 \usepackage[utf8]{inputenc}
 \usepackage{ifthen}
 \usepackage[verbose=false]{datatool}
+\usepackage{mfirstuc}[2017/11/10]
 \usepackage{datapie,databar,databib,person,datagidx}
 \usepackage{textcomp}
 \usepackage{graphicx}
@@ -102,11 +103,11 @@
 
 \MakeShortVerb{"}
 
- \title{User Manual for datatool bundle version~2.27}
+ \title{User Manual for datatool bundle version~2.28}
  \author{Nicola L.C. Talbot\\
 \url{http://www.dickimaw-books.com/}}
 
- \date{2016-07-28}
+ \date{2017-11-10}
  \maketitle
 
 \pagenumbering{roman}
@@ -121,7 +122,7 @@ The \styfmt{datatool} bundle comes with the following documentation:
   \item[\url{datatool-code.pdf}]
   Advanced users wishing to know more about the inner workings of
   all the packages provided in the \styfmt{datatool} bundle should
-  read \qt{Documented Code for datatool v2.27}
+  read \qt{Documented Code for datatool v2.28}
 
   \item[INSTALL] Installation instructions.
 
@@ -211,6 +212,12 @@ and the second uses the \sty{pgfmath} package:
 
 \end{enumerate}
 
+\begin{important}
+Both \sty{fp} and \sty{pgfmath} have some limitations. These limitations
+will therefore also be present in the various packages provided
+with \sty{datatool}, according to the underlying package used.
+\end{important}
+
 \cleardoublepage
 \pdfbookmark[0]{Contents}{contents}
 \tableofcontents
@@ -244,6 +251,12 @@ ascending or descending).
 \end{itemize}
 Package Options:
   \begin{description}
+   \item[\pkgopt{utf8}] Boolean key. May be used to switch off the UTF-8
+   support described on \pageref{utf8support} in \sectionref{sec:sort}.
+   If you both load \sty{inputenc} with UTF-8
+   support and you use accent commands like \cs{'} or \cs{c} then make
+   sure you have at least version 2.05 of \sty{mfirstuc}
+   if you want to use commands like \ics{makefirstuc}.
    \item[\pkgopt{verbose}] Boolean key. If
     \pkgoptval{true}{verbose}, prints informational messages in
      transcript.
@@ -2063,6 +2076,99 @@ a list of elements separated by commands, for example:
 This isn't the same as comma-separated files, which is dealt
 with in \sectionref{sec:databases}.
 
+\begin{definition}[\DescribeMacro{\DTLformatlist}]
+\cs{DTLformatlist}\marg{list}
+\end{definition}
+Formats the comma-separated list. The unstarred version scopes
+the internal operation. The starred form doesn't.
+The \meta{list} may either be an explicit comma-separated list or 
+a control sequence whose replacement
+text is a comma-separated list. For example:
+\begin{verbatim}
+\newcommand{\mylist}{elephant,ant,zebra,duck}%
+\DTLformatlist{\mylist}
+\end{verbatim}
+which produces:
+\begin{display}
+\newcommand{\mylist}{elephant,ant,zebra,duck}%
+\DTLformatlist{\mylist}
+\end{display}
+The list formatting command inserts
+\begin{definition}[\DescribeMacro\DTLlistformatsep]
+\cs{DTLlistformatsep}
+\end{definition}
+between each item, except for the last pair which uses:
+\begin{definition}[\DescribeMacro\DTLlistformatlastsep]
+\cs{DTLlistformatlastsep}
+\end{definition}
+if there are only two items in the list or
+\begin{definition}[\DescribeMacro\DTLlistformatoxford]
+\cs{DTLlistformatoxford}\cs{DTLlistformatlastsep}
+\end{definition}
+if there are three or more items in the list.
+\cs{DTLlistformatlastsep} uses
+\begin{definition}[\DescribeMacro\DTLandname]
+\cs{DTLandname}
+\end{definition}
+Each item in the list is formatted according to
+\begin{definition}[\DescribeMacro\DTLlistformatitem]
+\cs{DTLlistformatitem}\marg{item}
+\end{definition}
+
+The default definitions are:
+\begin{itemize}
+\item \cs{DTLlistformatitem}\marg{item}: just does \meta{item};
+\item \cs{DTLlistformatsep}: \verb*|, | (comma followed by a space);
+\item \cs{DTLandname}: if \ics{andname} is
+defined this is defined as \cs{andname}
+otherwise it's defined as \verb|\&|;
+\item \cs{DTLlistformatlastsep}: does \verb*| \DTLandname\space|;
+\item \cs{DTLlistformatoxford}: does nothing (so if you want an
+Oxford comma you need to redefine this).
+\end{itemize}
+For example
+\begin{verbatim}
+\renewcommand{\DTLlistformatitem}[1]{\emph{#1}}%
+\renewcommand{\DTLlistformatoxford}{,}%
+\renewcommand{\DTLandname}{and}%
+One: \DTLformatlist{elephant}.
+
+Two: \DTLformatlist{elephant,ant}.
+
+Three: \DTLformatlist{elephant,ant,zebra}.
+
+Four: \DTLformatlist{elephant,ant,zebra,duck}.
+\end{verbatim}
+produces:
+\begin{display}
+\renewcommand{\DTLlistformatitem}[1]{\emph{#1}}%
+\renewcommand{\DTLlistformatoxford}{,}%
+\renewcommand{\DTLandname}{and}%
+One: \DTLformatlist{elephant}.
+
+Two: \DTLformatlist{elephant,ant}.
+
+Three: \DTLformatlist{elephant,ant,zebra}.
+
+Four: \DTLformatlist{elephant,ant,zebra,duck}.
+\end{display}
+
+You can test if an element is contained in a comma-separated list
+using:
+\begin{definition}[\DescribeMacro{\DTLifinlist}]
+\cs{DTLifinlist}\marg{element}\marg{list}\marg{true part}\marg{false part}
+\end{definition}
+If \meta{element} is contained in the comma-separated list given
+by \meta{list}, then this does \meta{true part} otherwise it does false
+part. (Does a one level expansion on \meta{list}, but no
+expansion on \meta{element}.)
+
+\begin{definition}[\DescribeMacro{\DTLnumitemsinlist}]
+\cs{DTLnumitemsinlist}\marg{list}\marg{cmd}
+\end{definition}
+Counts the number of non-empty elements in \meta{list} and stores
+result in \meta{cmd}, which must be a control sequence.
+
 \begin{definition}[\DescribeMacro\dtlsortlist]
 \cs{dtlsortlist}\marg{list cs}\marg{criteria cs}
 \end{definition}
@@ -2075,6 +2181,12 @@ count register should be set to $-1$, if \meta{A} and \meta{B} are
 considered the same then the count register should be set to 0,
 and if \meta{A} is considered greater than \meta{B}, then the count
 register should be set to 1.
+\begin{important}
+It's inefficient to use \TeX\ to sort and comparisons are only
+made according to the character codes and so are inappropriate
+for extended Latin or non-Latin alphabets. It's better to use 
+an external tool where possible.
+\end{important}
 
 The \styfmt{datatool-base} package provides some predefined
 criteria commands:
@@ -2196,6 +2308,18 @@ files saved using \ics{DTLsaverawdb} or \ics{DTLprotectedsaverawdb} in a~graphic
 import data from a~SQL database, a~CSV file or a~\sty{probsoln}
 dataset.
 
+Each database has the data stored internally in a token register and
+the header information is stored in an internal control sequence.
+In general you don't need to worry about the way it's stored, except
+that row indexes start from 1 (the first row of data) and column
+indexes also start from 1. Columns may be referenced by a label
+but rows can only be referenced by the row index. Once the internal
+commands and registers have been set, \styfmt{datatool} doesn't keep
+a record of how or where the information came from (such as a CSV
+file or through a file created by \app{datatooltk}). Changes to the
+data only modify the internal commands and are lost at the end of
+the \LaTeX\ run unless you save the data.
+
 \section{Creating a New Database}
 \label{sec:newdb}
 
@@ -2364,6 +2488,8 @@ can convert it to \sty{datatool}'s internal database format using
 \app{datatooltk} and the input it using \ics{input} or \ics{DTLloaddbtex}. See the 
 \app{datatooltk} documentation for further details.
 \end{important}
+Remember that row numbers in \sty{datatool} refer to the row index
+of the internal data not to line numbers in the CSV file.
 
 Instead of using the commands described in \autoref{sec:newdb}
 to create a new database, you can load a database from an
@@ -2537,7 +2663,7 @@ can use:
 \begin{definition}[\DescribeMacro{\DTLloadrawdb}]%
 \cs{DTLloadrawdb}\oarg{options}\marg{db name}\marg{filename}
 \end{definition}
-This is the same as \ics{DTLloaddb} except that it maps nine of
+This is similar to \ics{DTLloaddb} except that it maps nine of
 the ten special characters onto commands which produce that
 symbol. The only character that retains its active state is the
 backslash character, so you will still need to check the file
@@ -2561,6 +2687,22 @@ As with \ics{DTLloaddb}, you can govern whether or not a new
 database should be created with \ics{DTLnewdbonloadtrue} and
 \ics{DTLnewdbonloadfalse}.
 
+Note that \cs{DTLloadrawdb} is not recommended for CSV files that
+contain commands. Any active characters occurring within the
+file must be mapped. The mapping is done through
+expansion. Version 2.28 now uses \cs{xdef} rather than
+\cs{protected@xdef}, but \cs{DTLloadrawdb} isn't intended for files
+containing \LaTeX\ code, but rather for files generated by a
+non-\TeX\ aware method that doesn't guard against \LaTeX\ special
+characters. 
+
+If you have UTF-8 data, it's best to use an engine that natively
+supports Unicode. Remember that there's no point using
+\cs{DTLloadrawdb} (instead of \cs{DTLloaddb}) if your file doesn't
+contain any special characters that require mapping. It's less
+efficient to load if each row has to be converted, particularly for
+large files.
+
 \begin{table}[htbp]
 \caption[Special character mappings used by 
 \cs{DTLloadrawdb}]{Special character mappings used by 
@@ -4193,7 +4335,9 @@ example:
 \end{verbatim}
 will delete the second row in the database labelled ``scores''.
 There is also a starred version that doesn't check for the 
-existence of the database.
+existence of the database. Remember that the row index refers to the
+internal data not to a reference in the external source if the data
+has been input.
 
 The following commands may be used in the body of the
 \ics{DTLforeach} loop,\footnote{Only the unstarred version of
@@ -4625,6 +4769,12 @@ To determine a person's gender when mail merging, see
 \section{Sorting a Database}
 \label{sec:sort}
 
+The sort methods described here use \TeX\ to sort, which is very
+inefficient. The comparison handlers use a character code comparison
+rather than locale-sensitive alphabetic ordering. It's more
+efficient to use \app{datatooltk} to import and sort at the same
+time.
+
 \begin{definition}[\DescribeMacro{\dtlsort}]%
 \cs{dtlsort}\oarg{replacement key list}\marg{sort criteria}\marg{db
 name}\marg{handler}
@@ -4752,6 +4902,7 @@ code for details on the syntax of the handler. (You may want to
 consider uploading your handler as a separate package to CTAN if you
 think it will be of general use.)
 
+\label{utf8support}%
 As from version 2.24, the predefined handlers now have limited support for UTF-8
 characters. \emph{This is still experimental.} The support will
 automatically be switched on if the \sty{inputenc} package is loaded
@@ -4769,6 +4920,8 @@ or
 \usepackage[utf8]{datatool-base}
 \usepackage[utf8]{inputenc}
 \end{verbatim}
+(As from version 2.28, you can now also pass the \pkgopt{utf8}
+option to \sty{datatool} and \sty{datagidx}.)
 You can also enable this option after \sty{datatool-base} has been loaded
 using
 \begin{definition}[\DescribeMacro\dtlenableUTFviii]
@@ -5494,20 +5647,6 @@ of the rows to be swapped. For example:
 \end{verbatim}
 will swap the third and fifth rows.
 
-\begin{definition}[\DescribeMacro{\DTLifinlist}]
-\cs{DTLifinlist}\marg{element}\marg{list}\marg{true part}\marg{false part}
-\end{definition}
-If \meta{element} is contained in the comma-separated list given
-by \meta{list}, then do \meta{true part} otherwise do false
-part. (Does a one level expansion on \meta{list}, but no
-expansion on \meta{element}.)
-
-\begin{definition}[\DescribeMacro{\DTLnumitemsinlist}]
-\cs{DTLnumitemsinlist}\marg{list}\marg{cmd}
-\end{definition}
-Counts the number of non-empty elements in \meta{list} and stores
-result in \meta{cmd}, which must be a control sequence.
-
 \begin{example}{Two Database Rows Per Tabular Row
 (Column-Wise)}{ex:twoblocks}
 This example adapts \autoref{ex:2rows} so that the list is ordered
@@ -5578,7 +5717,9 @@ This produces \autoref{tab:twoblocks}.
 
 If you want to select from or edit a particular row in a database without having
 to iterate through the database using \cs{DTLforeach}, you can use
-the commands described in this section.
+the commands described in this section. Remember that the row index
+is a reference to the internal data and is unrelated to references
+in the original source (such as line numbers in a CSV file).
 
 \begin{definition}[\DescribeMacro{\DTLgetrowindex}]
 \cs{DTLgetrowindex}\marg{row cs}\marg{db name}\marg{col
@@ -5604,6 +5745,16 @@ cs}\cs{dtlnovalue}. For example:
 \fi
 \end{verbatim}
 
+If you want \meta{value} to be fully expanded before testing
+you can use
+\begin{definition}[\DescribeMacro{\xdtlgetrowindex}]
+\cs{xdtlgetrowindex}\marg{row cs}\marg{db name}\marg{col
+idx}\marg{value}
+\end{definition}
+(The \sty{etoolbox} package provides \cs{expandonce} if you
+only want one level of expansion. See the \sty{etoolbox} manual for
+further details.)
+
 \begin{definition}[\DescribeMacro{\dtlgetrow}]
 \cs{dtlgetrow}\marg{db name}\marg{row idx}
 \end{definition}
@@ -6356,7 +6507,8 @@ be the name of the style. Available styles are listed in
 is only applied if \csopt{printterms}{columns} is greater than~1.
 If \textsf{true}, the columns are balanced. If \textsf{false}, the
 columns aren't balanced. If no value is specified, \textsf{true} is
-assumed.
+assumed. If \verb|balance=false| and \verb|columns=2| 
+\ics{twocolumn} is used instead of \env{multicols*}.
 
 \item[\csopt{newgidx}{heading}] The heading at the start of the
 index/glossary.
@@ -7617,6 +7769,12 @@ the index/glossary database.
 
 The following package options are available for \styfmt{datagidx}:
 \begin{description}
+\item[\pkgopt{utf8}] A boolean option (same as for
+\sty{datatool-base}). If you both load \sty{inputenc} with UTF-8
+support and you use accent commands like \cs{'} or \cs{c} then make
+sure you have at least version 2.05 of \sty{mfirstuc} if you want to
+use commands like \cs{Gls}.
+
 \item[\pkgopt{optimize}] Sets the optimization. (See
 \autoref{sec:optimize}.)