texcount (19sep17)

git-svn-id: svn://tug.org/texlive/trunk@45338 c570f23f-e606-0410-a88d-b1316a301751

16 files changed, 140 insertions, 52 deletions

diff --git a/Master/texmf-dist/doc/support/texcount/QuickReference.pdf b/Master/texmf-dist/doc/support/texcount/QuickReference.pdf
deleted file mode 100644
index fdaa375a5c3..00000000000
--- a/Master/texmf-dist/doc/support/texcount/QuickReference.pdf
+++ /dev/null
diff --git a/Master/texmf-dist/doc/support/texcount/README b/Master/texmf-dist/doc/support/texcount/README
deleted file mode 100644
index 2ca33b6e3b6..00000000000
--- a/Master/texmf-dist/doc/support/texcount/README
+++ /dev/null
@@ -1,9 +0,0 @@
-TeXcount is a Perl script that counts the number of words in the 
-text of LaTeX files. It has rules for handling most of the common 
-macros and provides colour coded output indicating which parts have 
-been counted. Go to
-  http://app.uio.no/ifi/texcount/
-for more information or to access the script online as a web service.
-
-The package, i.e. the script and all accompanying files, is
-distributed under the LaTeX Project Public License.
diff --git a/Master/texmf-dist/doc/support/texcount/README.md b/Master/texmf-dist/doc/support/texcount/README.md
new file mode 100644
index 00000000000..321d09f571a
--- /dev/null
+++ b/Master/texmf-dist/doc/support/texcount/README.md
@@ -0,0 +1,37 @@
+TeXcount
+========
+
+TeXcount is a Perl script that counts the number of words in the 
+text of LaTeX files. It has rules for handling most of the common 
+macros and provides colour coded output indicating which parts have 
+been counted.
+
+
+# Web site and online web service
+
+Go to
+  http://app.uio.no/ifi/texcount/
+for more information or to access the script online as a web service.
+
+
+# Documentation
+
+There is documentation at different levels of detail and technicality:
+
+* **QuickReference** gives a quick overview of options and TeXcount instructions which may be used to control how TeXcount processes TeX documents.
+
+* **TeXcount** is the main documentation giving a detailed explanation of how TeXcount processes TeX documents, and how to use options and TeXcount instructions to customise the word counting and reporting the results.
+
+* **TechDoc** is the technical documentation of the internals of TeXcount.
+
+Additional documentation, including scripts and examples, are available on the TeXcount web site:
+  http://app.uio.no/ifi/texcount/
+
+Questions posted to StackExchange and tagged *texcount* will automatically notify the TeXcount developer:
+  https://tex.stackexchange.com/questions/tagged/texcount
+
+
+# License
+
+The package, i.e. the script and all accompanying files, is
+distributed under the LaTeX Project Public License.
diff --git a/Master/texmf-dist/doc/support/texcount/TeXcount.pdf b/Master/texmf-dist/doc/support/texcount/TeXcount.pdf
deleted file mode 100644
index fe3802e435b..00000000000
--- a/Master/texmf-dist/doc/support/texcount/TeXcount.pdf
+++ /dev/null
diff --git a/Master/texmf-dist/doc/support/texcount/TechDoc.pdf b/Master/texmf-dist/doc/support/texcount/TechDoc.pdf
deleted file mode 100644
index 02aa6690798..00000000000
--- a/Master/texmf-dist/doc/support/texcount/TechDoc.pdf
+++ /dev/null
diff --git a/Master/texmf-dist/doc/support/texcount/doc/QuickReference.pdf b/Master/texmf-dist/doc/support/texcount/doc/QuickReference.pdf
new file mode 100644
index 00000000000..03e45ed9357
--- /dev/null
+++ b/Master/texmf-dist/doc/support/texcount/doc/QuickReference.pdf
diff --git a/Master/texmf-dist/doc/support/texcount/QuickReference.tex b/Master/texmf-dist/doc/support/texcount/doc/QuickReference.tex
index 7115a1bd944..aee811750b6 100644
--- a/Master/texmf-dist/doc/support/texcount/QuickReference.tex
+++ b/Master/texmf-dist/doc/support/texcount/doc/QuickReference.tex
@@ -1,7 +1,6 @@
 \documentclass{article}
 \usepackage[T1]{fontenc}
 \usepackage{a4wide}
-\usepackage{times}
 
 \include{macros}
 
@@ -33,6 +32,8 @@ where the name is use for instructions providing macro handling rules to give th
 %
 \input{sub_tc_other}
 
+\subsection{Adding macro handling rules}
+
 Instructions for adding macro handling rules all take the format
 \codeline{\%TC:\textit{instruction name parameters}}
 where the name indicates the macro (with backslash) or group name for which the rule applies:
diff --git a/Master/texmf-dist/doc/support/texcount/doc/TeXcount.pdf b/Master/texmf-dist/doc/support/texcount/doc/TeXcount.pdf
new file mode 100644
index 00000000000..48f66f008d3
--- /dev/null
+++ b/Master/texmf-dist/doc/support/texcount/doc/TeXcount.pdf
diff --git a/Master/texmf-dist/doc/support/texcount/TeXcount.tex b/Master/texmf-dist/doc/support/texcount/doc/TeXcount.tex
index 16f90ededd4..635df62a714 100644
--- a/Master/texmf-dist/doc/support/texcount/TeXcount.tex
+++ b/Master/texmf-dist/doc/support/texcount/doc/TeXcount.tex
@@ -1,7 +1,6 @@
 \documentclass{article}
 \usepackage[T1]{fontenc}
 \usepackage{a4wide}
-\usepackage{times}
 \usepackage{listings}
 \usepackage{url}
 
@@ -9,6 +8,8 @@
 %TC:envir lstlisting [] xall
 %TC:macro url [ignore]
 
+%TC:subst \begin{abstract} \begin{abstract} ABS
+
 \include{macros}
 
 %\parindent=0pt\parskip=8pt
@@ -84,6 +85,10 @@ Basically, \TeXcount{} has seven different counts plus an additional file count
 \end{description}
 These are stored in an array and sometimes referenced by their index: e.g. in the option \code{-sum=} which takes parameter values corresponding to counts 1 to 7. In other contexts, however, like in the \code{-tempate=} or when incrementing specific counters through the \code{\%TC:macrocount} instruction, the counters may be referred to by keywords rather than the indices 0 to 7.
 
+There is also support for adding additional counters using the \TeXcount{} instruction \code{\%TC:newcounter}. These will then be added to the end of the list of counters and should preferably be referred to by name, not index.
+
+\subsubsection{What \TeXcount{} counts as a word}
+
 The primary role is to count the words. It is not entirely clear what should be considered words, so I have had to make some decisions. A sequence of letters is certainly a word. \TeXcount{} also counts acronyms like \textit{e.g.}, dashed words like \textit{over-all}, and \textit{it's} as one word. It also counts numbers as words unless they are placed in a math group. If \TeXcount{} breaks words that contain special characters, you may try the option \code{-relaxed} which extends the range of characters allowed as part of words.
 
 Alternatively, \TeXcount{} may be asked to count the number of letters/characters (not including spaces). It may also be set to count Chinese or Japanese characters.
@@ -135,6 +140,8 @@ Under Linux/Unix, it should be sufficient to run \code{texcount.pl} provided it
 
 Alternatively, if the above methods do not work, you may have to run \TeXcount{} exclicitly in Perl by executing \code{perl texcount.pl}. You then need to have the \code{perl} executable in the path or give the explicit path.
 
+\subsection{\TeXcount{} command-line options}
+
 For simplicity, I will simply write \code{texcount.pl} in this manual for the code to execute the script. The syntax then becomes
 \codeline{texcount.pl \textit{[options] [files]}}
 where the options may be amongst the following:
@@ -143,6 +150,14 @@ where the options may be amongst the following:
 
 If more than one file is given, \TeXcount{} will perform the count on each of them printing the results for each, then print the total sum at the end. Note that files are parsed one by one in order of appearance and counts made per file; only afterwards are the totals computed. 
 
+\subsubsection{Option handling alternatives and modifications}
+
+In some cases, eg when running from within a script, the option syntax may cause problems. Two minor modifications have been added.
+
+For set options on the form \code{-[options]=[value]}, you may use the alternative syntax \code{-[option]:[value]} to avoid any problems caused by the equal character.
+
+\TeXcount{} will ignore any option starting with \code{@-} which can either be used to temporarily exclude an option in editing an existing call to \TeXcount{}, or to use the \code{@-[option]} to pass options to scripts without having to remove these before passing the arguments on to \TeXcount{}.
+
 
 \subsection{File encoding}
 
@@ -390,7 +405,7 @@ which \LaTeX{} will interpret as a comment but \TeXcount{} will detect.
 
 Adding your own macro handling rules is relatively simple. While it is fairly easy to edit the script to add more rules, this has the disadvantage that the modifications will be lost if updating to a new version of \TeXcount. A better and more flexible solution is to include instructions to \TeXcount{} in the \LaTeX{} documents, alternatively to make a definition file in which new macro handling rules are defined. The \TeXcount{} instructions for doing this take the form
 \codeline{\%TC:\textit{instruction name parameters \alt{option}}}
-where \code{name} is either the macro name (including backslash) or environment name:
+where in some rules \code{name} is the macro name (including backslash), and some rules use the or environment name. Note that rules that take the macro name as an argument can (usually) by applied to \code{\bs begin\{\textit{name}\}} by specifying it as \code{begin\textit{name}} (no backslash) which is how \TeXcount{} represents \code{\bs begin\{\textit{name}\}} internally.
 %
 \input{sub_addrules}
 
@@ -400,42 +415,26 @@ A few additional \TeXcount{} instructions exist to control the overall parsing a
 
 \input{sub_tc_other}
 
+In addition, in part with debugging in mind, the following \TeXcount{} instructions exist:
+
+\begin{description}
+
+\option[log \opt{text-line}]
+Writes a line of text to the verbose output. Counters may be included in the text using the \code{\{name\}} format.
+
+\option[assert \opt{counts} \opt{text-line}]
+This takes a list of counts separated by comma, and writes the text (which may contain counters on the format \code{\{name\}}) if the asserted counts do not match the actual counts.
+
+\end{description}
+
+These may be subject to change at some later point, although the functionality should remain.
+
 
 \subsection{Parameter and content handling rules}\label{subsec:TC_addrule}
 
 There are a set of alternative rules that may be used for parsing macro parameters and environment contents. These rules, or \emph{parser states}, are identified by keywords:
 %
-\begin{description}
-\def\option#1[#2]#3{\item[#1:] (key: \code{#3} formerly \code{#2})}
-\option{Text}[1]{text, word, wd, w}
- Count as text (i.e. count words).
-\option{Header text}[2]{headertext, headerword, hword, hwd, hw}
- Count as header text.
-\option{Other text}[3]{otherword, other, oword, owd, ow}
- Count as float/caption text.
-\option{Displaymath}[7]{displaymath, dsmath, dmath, ds}
- Count as displayed math formulae.
-\option{Inline math}[6]{inlinemath, inline, imath, eq}
- Count as inlined math formulae.
-\option{To header}[4]{header, heading, head}
- Count header, then count text as \code{headertext} (transition state).
-\option{To float}[5]{float, table, figure}
- Count float, then parse contents as \code{isfloat} (transition state).
-\option{Preamble}[-9]{}
- Parse as preamble, i.e. ignore text but look for \code{preambleinclude} macros.
-\option{Ignore}[0]{ignore}
- Ignore text, i.e. do not count, but will still parse the code.
-\option{Float}[-1]{isfloat}
- Float contents, ignore text but look for \code{floatinclude} macros.
-\option{Strong exclude}[-2]{xx}
- Strong ignore which ignore environments, e.g. to use in macro definitions where
- \code{\bs{begin}}--\code{\bs{end}} need not be balanced.
-\option{Stronger exclude}[-3]{xxx}
- Stronger ignore, handles macros as isolated tokens without handling their parameters,
- to use with macro definitions like \code{\bs{newcommand}} and \code{\bs{def}}.
-\option{Exclude all}[-4]{xall}
- Ignore all, including unbalanced braces (e.g. used by \code{\%TC:ignore} and the \code{verbatim} environment). This rule may be used for environment contents, but not for macro or environment parameters or options since the exclusion causes \{ and [ to be ignored.
-\end{description}
+\input{sub_ruletypes}
 
 The keys are used to identify the rule. Environment content rules are simply specified by giving the desired key. Parameter rules are on the form \code{[\textit{rule},\textit{rule},\ldots]} with one rule provided per parameter, where each rule is either one of the above keywords or \code{option:\textit{rule}} to indicate an optional parameter on the form \code{[\ldots]}; alternatively, a single integer can be provided (not enclosed in \code{[]} to indicate that the indicated number of parameters should be ignored.
 
diff --git a/Master/texmf-dist/doc/support/texcount/doc/TechDoc.pdf b/Master/texmf-dist/doc/support/texcount/doc/TechDoc.pdf
new file mode 100644
index 00000000000..e67e81f9541
--- /dev/null
+++ b/Master/texmf-dist/doc/support/texcount/doc/TechDoc.pdf
diff --git a/Master/texmf-dist/doc/support/texcount/TechDoc.tex b/Master/texmf-dist/doc/support/texcount/doc/TechDoc.tex
index af38a49d6f5..2460e68316a 100644
--- a/Master/texmf-dist/doc/support/texcount/TechDoc.tex
+++ b/Master/texmf-dist/doc/support/texcount/doc/TechDoc.tex
@@ -1,7 +1,6 @@
 \documentclass{article}
 \usepackage[T1]{fontenc}
 \usepackage{a4wide}
-\usepackage{times}
 \usepackage{listings}
 \usepackage{url}
 \usepackage{color}
diff --git a/Master/texmf-dist/doc/support/texcount/macros.tex b/Master/texmf-dist/doc/support/texcount/doc/macros.tex
index a79e8f06144..ecc54d32e63 100644
--- a/Master/texmf-dist/doc/support/texcount/macros.tex
+++ b/Master/texmf-dist/doc/support/texcount/doc/macros.tex
@@ -1,4 +1,8 @@
 %% LaTeX macros
+\usepackage{times}
+%\usepackage{garamond}
+%\usepackage{fourier} % math & rm
+\usepackage[scaled=1.05]{helvet} % ss
 \usepackage{relsize}
 \usepackage[dvipsnames]{xcolor}
 
@@ -11,10 +15,10 @@
 \catcode`\_=12
 
 % Set version number
-\newcommand\version{$3.0$}
+\newcommand\version{$3.1$}
 
 \newcommand\copyrightfootnote{
-\footnotetext{Copyright (2008-2013) of Einar Andreas R{\o}dland, distributed
+\footnotetext{Copyright (2008-2017) of Einar Andreas R{\o}dland, distributed
 under the \LaTeX{} Project Public License (LPPL).}
 }
 
@@ -31,7 +35,7 @@ under the \LaTeX{} Project Public License (LPPL).}
 
 % Description items: options, parameters, optional parameters
 % These are also interpreted by dos2html.pl
-\def\option[#1]{\item[\bigcode{#1}]\hskip 0pt plus 10pt}
+\def\option[#1]{\item[\code{#1}]\hskip 0pt plus 10pt}
 \def\parm#1{\textit{\color{OliveGreen}#1}}
 \def\opt#1{\parm{[#1]}}
 \def\alt#1{[#1]}
diff --git a/Master/texmf-dist/doc/support/texcount/sub_addrules.tex b/Master/texmf-dist/doc/support/texcount/doc/sub_addrules.tex
index fb1d3adfec4..ca2fca6811c 100644
--- a/Master/texmf-dist/doc/support/texcount/sub_addrules.tex
+++ b/Master/texmf-dist/doc/support/texcount/doc/sub_addrules.tex
@@ -1,6 +1,10 @@
 % Subsection: TC-commands for adding macro rules
 
 \begin{description}
+%PATTERN: \option[]
+%OUTPUT:  <dt>#1<dt>
+%OUTPUT:  <dd>#x<dd>
+%OUTPUT:
 \sloppy
 
 \option[macro \parm{macroname} \parm{parameter-rules}]
diff --git a/Master/texmf-dist/doc/support/texcount/sub_options.tex b/Master/texmf-dist/doc/support/texcount/doc/sub_options.tex
index 3efa05a0d80..c7078e64c4f 100644
--- a/Master/texmf-dist/doc/support/texcount/sub_options.tex
+++ b/Master/texmf-dist/doc/support/texcount/doc/sub_options.tex
@@ -1,9 +1,14 @@
 % Subsection: command line options
 
 \begin{description}
-\sloppy
+%PATTERN: \option[]
+%OUTPUT:  <dt>#1<dt>
+%OUTPUT:  <dd>#x<dd>
+%OUTPUT:
 \def\option[#1]{\item[\quad\code{#1}]\hskip 0pt plus 10pt}
 \def\alt#1{[#1]}
+\def\alts#1{[[#1]]}
+\sloppy
 
 \option[-v]Verbose (same as \code{-v3}).
 
@@ -17,7 +22,7 @@
 
 \option[-v4]Same as \code{-v3 -showstate}.
 
-\option[-v=\ldots, -v\alt{[0-4]}\ldots]Allows detailed control of which elements are included in the verbose output. The provided values is a list of styles or style categories separated by \code{+} or \code{-} to indicate if they should be added or removed from the list of included styles. Style categories start with capital letter and include \code{Words}, \code{Macros}, \code{Options}; the individual styles are in all lower case and include \code{word}, \code{hword}, \code{option}, \code{ignore}.
+\option[-v=\ldots, -v\alts{0-4}\ldots]Allows detailed control of which elements are included in the verbose output. The provided values is a list of styles or style categories separated by \code{+} or \code{-} to indicate if they should be added or removed from the list of included styles. Style categories start with capital letter and include \code{Words}, \code{Macros}, \code{Options}; the individual styles are in all lower case and include \code{word}, \code{hword}, \code{option}, \code{ignore}.
 
 \option[-showstate]Show internal states (with verbose).
 
@@ -33,7 +38,7 @@
 
 \option[-0]Same as \code{-1}, i.e. \code{-brief} and \code{-total}, but does not put a line shift at the end. This may be useful when the one line output is to be used by another application, e.g. Emacs, for which the line shift would otherwise need to be stripped away.
 
-\option[-template="\ldots"]Specify an output template which is used to generate the summary output for each file and for the total count. Codes \code{\{\textit{label}\}} is used to include values, where \code{\textit{label}} is one of \code{0} to \code{7} (for the counts), \code{SUM}, \code{ERROR} or \code{TITLE} (first character of label is sufficient). Conditional inclusion is done using \code{\{\textit{label}?\textit{text}?\textit{label}\}} or \code{\{\textit{label}?\textit{if non-zero}|\textit{if zero}?\textit{label}\}}. If the count contains at least two subcounts, use \code{\{SUB|\textit{template}|SUB\}} with a separate template for the subcounts, or \code{\{SUB?\textit{prefix}|\textit{template}|\textit{suffix}?SUB\}}.
+\option[-template="\ldots"]Specify an output template which is used to generate the summary output for each file and for the total count. Codes \code{\{\textit{label}\}} is used to include values, where \code{\textit{label}} is one of \code{0} to \code{7} (for the counts), \code{SUM}, \code{ERROR} or \code{TITLE} (first character of label is sufficient). Conditional inclusion is done using \code{\{\textit{label}?\textit{text}?\textit{label}\}} or \code{\{\textit{label}?\textit{if non-zero}|\textit{if zero}?\textit{label}\}}. If the count contains at least two subcounts, use \code{\{SUB?\textit{template}?SUB\}} with a separate template for the subcounts, or \code{\{SUB?\textit{prefix}|\textit{template}|\textit{suffix}?SUB\}}.
 
 \option[-sub\alt{=\ldots}, -subcount\alt{=\ldots}]Generate subcounts. Valid option values are \code{none}, \code{part}, \code{chapter}, \code{section} and \code{subsection} (default), indicating at which level subcounts are generated. (On by default.)
 
@@ -91,10 +96,18 @@
 
 \option[-chinese-only, ..., -korean-words-only]As options \code{-chinese}, ..., \code{-korean-words}, but also excludes other alphabets (e.g. letter-based words) and logographic characters.
 
+\option[-count-all, -all]Allow all alphabets, digits, and logograms as letters, whether counting words or characters. This is the default setting.
+
 \option[-char, -letter]Count letters instead of words. This count does not include spaces.
 
+\option[-char-only, -letter-only]Count letters instead of words, but excludes logograms (like chinese characters). This count does not include spaces.
+
+\option[-all-nonspace-char, -all-nonspace-characters]Counts characters instead of words, including letters, logograms, and punctuation, but not including spaces.
+
 \option[-out=]Send output to file. Takes file name as value.
 
+\option[-out-stderr]Send output to STDERR instead of STDOUT.
+
 \option[-html]Output in HTML format.
 
 \option[-htmlcore]Only HTML body contents.
diff --git a/Master/texmf-dist/doc/support/texcount/doc/sub_ruletypes.tex b/Master/texmf-dist/doc/support/texcount/doc/sub_ruletypes.tex
new file mode 100644
index 00000000000..1e5697d9a9e
--- /dev/null
+++ b/Master/texmf-dist/doc/support/texcount/doc/sub_ruletypes.tex
@@ -0,0 +1,38 @@
+%%
+%
+\begin{description}
+%PATTERN: \option{}[]{}
+%OUTPUT:  <dt>#1 (keys: #3; formerly code #2)<dt>
+%OUTPUT:  <dd>#x<dd>
+%OUTPUT:
+\def\option#1[#2]#3{\item[#1:] (key: \code{#3} formerly \code{#2})}
+
+\option{Text}[1]{text, word, wd, w}
+ Count as text (i.e. count words).
+\option{Header text}[2]{headertext, headerword, hword, hwd, hw}
+ Count as header text.
+\option{Other text}[3]{otherword, other, oword, owd, ow}
+ Count as float/caption text.
+\option{Displaymath}[7]{displaymath, dsmath, dmath, ds}
+ Count as displayed math formulae.
+\option{Inline math}[6]{inlinemath, inline, imath, eq}
+ Count as inlined math formulae.
+\option{To header}[4]{header, heading, head}
+ Count header, then count text as \code{headertext} (transition state).
+\option{To float}[5]{float, table, figure}
+ Count float, then parse contents as \code{isfloat} (transition state).
+\option{Preamble}[-9]{}
+ Parse as preamble, i.e. ignore text but look for \code{preambleinclude} macros.
+\option{Ignore}[0]{ignore}
+ Ignore text, i.e. do not count, but will still parse the code.
+\option{Float}[-1]{isfloat}
+ Float contents, ignore text but look for \code{floatinclude} macros.
+\option{Strong exclude}[-2]{xx}
+ Strong ignore which ignore environments, e.g. to use in macro definitions where
+ \code{\bs{begin}}--\code{\bs{end}} need not be balanced.
+\option{Stronger exclude}[-3]{xxx}
+ Stronger ignore, handles macros as isolated tokens without handling their parameters,
+ to use with macro definitions like \code{\bs{newcommand}} and \code{\bs{def}}.
+\option{Exclude all}[-4]{xall}
+ Ignore all, including unbalanced braces (e.g. used by \code{\%TC:ignore} and the \code{verbatim} environment). This rule may be used for environment contents, but not for macro or environment parameters or options since the exclusion causes \{ and [ to be ignored.
+\end{description}
diff --git a/Master/texmf-dist/doc/support/texcount/sub_tc_other.tex b/Master/texmf-dist/doc/support/texcount/doc/sub_tc_other.tex
index dd6c05fabcc..07d9a6aeb89 100644
--- a/Master/texmf-dist/doc/support/texcount/sub_tc_other.tex
+++ b/Master/texmf-dist/doc/support/texcount/doc/sub_tc_other.tex
@@ -1,6 +1,10 @@
 %% Other TC instructions
 
 \begin{description}
+%PATTERN: \option[]
+%OUTPUT:  <dt>#1<dt>
+%OUTPUT:  <dd>#x<dd>
+%OUTPUT:
 
 \option[break \parm{title}]
 Break point which initiates a new subcount. The title is used to identify the following region in the summary output.
@@ -24,5 +28,3 @@ Define a new counter with the given name and description (optional). A correspon
 Specify a template for the summary output. The first line should just declare a new template using \code{\%TC:newtemplate}, while the subsequent lines use \code{\%TC:template} followed by text specifying the template. The line breaks in the template specification are not of importance: to specify a line break, use \code{\bs{n}}.
 
 \end{description}
-
-