texosquery (29mar17)

git-svn-id: svn://tug.org/texlive/trunk@43641 c570f23f-e606-0410-a88d-b1316a301751

10 files changed, 154 insertions, 21 deletions

diff --git a/Master/texmf-dist/doc/support/texosquery/CHANGES b/Master/texmf-dist/doc/support/texosquery/CHANGES
index ba18c85d3d5..4cc0c383bce 100644
--- a/Master/texmf-dist/doc/support/texosquery/CHANGES
+++ b/Master/texmf-dist/doc/support/texosquery/CHANGES
@@ -1,3 +1,8 @@
+1.2.1 (2017/03/28):
+
+ * Minor bug fix in texosquery.tex (fixed decimal separator
+   for numeric patterns with all # in the fractional part)
+
 1.2 (2017/03/23):
 
  * Now has three .jar files depending on the Java installation:
diff --git a/Master/texmf-dist/doc/support/texosquery/README.md b/Master/texmf-dist/doc/support/texosquery/README.md
index bad3733503a..c9f883ee00b 100644
--- a/Master/texmf-dist/doc/support/texosquery/README.md
+++ b/Master/texmf-dist/doc/support/texosquery/README.md
@@ -91,6 +91,8 @@ There are now three Java applications provided by this package:
  - `texosquery-jre8.jar` (requires at least Java 8)
  - `texosquery-jre5.jar` (requires at least Java 5)
 
+(See the ["Security"](#security) section below.)
+
 There are corresponding bash scripts for Unix-like users (the `.sh`
 extension added by `texosquery.ins` should be removed and the files
 made executable):
@@ -140,7 +142,7 @@ java -version
 ```
 This should display the version information. (For example, `"1.8.0_92"`)
 
-If the version number starts with `1.8` (“Java 8”), then you can use `texosquery-jre8.jar`.
+If the version number starts with `1.8` ("Java 8"), then you can use `texosquery-jre8.jar`.
 This is the full application. You can use `texosquery` or 
 `texosquery-jre5` as well, but there's less locale support with them. 
 The `texosquery-jre8` bash script and `texosquery-jre8.bat` batch script invokes Java with 
@@ -156,19 +158,19 @@ information from the [Unicode Common Locale Data Repository
 pending Java 9 should include this by default, so this will only be
 relevant to Java 8. Earlier versions of Java (7 or below) don't have
 this option, which limits the locale support to that which is
-natively provided by the JRE.
+natively provided by the JRE. Note that Java 7 and earlier versions
+[have reached their end of life](http://www.oracle.com/technetwork/java/eol-135779.html) and are now **deprecated**.
 
-If the version information starts with `1.7` (“Java 7”), then you can use 
+If the version information starts with `1.7` ("Java 7"), then you can use 
 `texosquery.jar`. This is the default application and provides most
 of the functions of the full application, but there's less locale support.
 You can also use the even more limited `texosquery-jre5`, but you can't use
 `texosquery-jre8`, so you can't take advantage of the CLDR.
 
-If the version information starts with `1.5` (“Java 5”) or `1.6`
-(“Java 6”), then you can _only_ use `texosquery-jre5`.  The locale
+If the version information starts with `1.5` ("Java 5") or `1.6`
+("Java 6"), then you can _only_ use `texosquery-jre5`.  The locale
 support is significantly reduced in this case and there's no support
-for language scripts. Note that these versions of Java are
-deprecated.
+for language scripts.
 
 Once you have determined which application you want to use, edit the
 `texosquery.cfg` so that `\TeXOSInvokerName` is defined to the appropriate invocation. For example, with Java 8:
@@ -252,6 +254,91 @@ All jpg and png files in current directory:
 For a full list of available commands, see the documentation
 (`texosquery.pdf`).
 
+## Troubleshooting
+
+If something goes wrong with the call to `texosquery`, the control
+sequence will be set to empty. If this happens, here are the steps
+to diagnose the problem:
+
+  1. Check the log file for any lines starting with `TeXOSQuery:`
+     For example: `TeXOSQuery: texosquery-jre8 --pdfnow`
+     If found, this means that the dry run mode was on and the shell
+     escape wasn't used. Ensure that the shell escape is
+     enabled when you build the document.
+  2. If the log file contains `(|texosquery` _options_`)` then the
+     shell escape was used but `texosquery` returned an empty value.
+     In this case, copy the part between `(|` and `)` and paste it
+     into a command prompt or terminal but insert `--debug` at the
+     start of the options list. For example, if the log file
+     contains `(|texosquery-jre8 --cwd)` then run
+```sh
+texosquery-jre8 --debug --cwd
+```
+
+This should now display an error message explaining the
+problem. (For example, read access forbidden or file not found
+or a security exception.)
+
+## Security
+
+As from version 1.2, all variants of `texosquery` (JRE5, 7 and 8)
+obey TeX's `openin_any` setting. Any of the actions that involve
+reading file information won't work if read access is forbidden by 
+`openin_any` or by the operating system.
+
+In addition to obeying `openin_any`, the file listing actions (such
+as `--list`) for the JRE7 and 8 variants also prohibit listing the
+contents outside of the current working directory's path. This means
+that you can't, for example, list the contents of `..` (the current
+working directory's parent) nor can you try walking the entire file
+system.  The `--walk` action additionally won't descend hidden
+directories. This extra restriction is designed to prevent malicious
+code from trying to use `texosquery` to look around your filing
+system.
+
+The Java 5 version `texosquery-jre5` is the least secure and doesn't
+have this additional restriction on the file listing actions. However,
+it still obeys `openin_any`. The `--walk` action is not available
+with `texosquery-jre5`.
+
+Examples:
+```bash
+texosquery-jre8 --debug --walk ',' '.*' /
+```
+This action is forbidden:
+```
+texosquery-jre8: Can't walk directory: /
+```
+Changing `/` to `.` is allowed, but not for `texosquery-jre5`:
+```bash
+texosquery-jre5 --debug --walk ',' '.*' .
+```
+returns:
+```
+texosquery-jre5: walk requires at least JRE 7 version
+```
+With `openin_any=a`, the `--userhome` action is usually allowed
+(depends on the operating system and Java's security manager).
+```bash
+texosquery-jre8 --debug --userhome
+```
+returns
+```
+\fslh home\fslh nlct
+```
+(which `\TeXOSQuery` converts to `/home/nlct`.)
+
+However, unless this also happens to be the current working
+directory, it's not possible to obtain a file listing:
+```bash
+texosquery-jre8 --debug --list ',' /home/nlct
+```
+returns
+```
+texosquery-jre8: Unable to list contents of: /home/nlct
+texosquery-jre8: Listing outside cwd path not permitted: /home/nlct
+```
+
 ## Source code
 
 The Java source is in the `.java` files and the manifest for each 
diff --git a/Master/texmf-dist/doc/support/texosquery/texosquery.pdf b/Master/texmf-dist/doc/support/texosquery/texosquery.pdf
index a1b3862d24f..f11511e67ee 100644
--- a/Master/texmf-dist/doc/support/texosquery/texosquery.pdf
+++ b/Master/texmf-dist/doc/support/texosquery/texosquery.pdf
diff --git a/Master/texmf-dist/scripts/texosquery/texosquery-jre5.jar b/Master/texmf-dist/scripts/texosquery/texosquery-jre5.jar
index 63d0fc4f9b3..8c2cacc0d66 100755
--- a/Master/texmf-dist/scripts/texosquery/texosquery-jre5.jar
+++ b/Master/texmf-dist/scripts/texosquery/texosquery-jre5.jar
diff --git a/Master/texmf-dist/scripts/texosquery/texosquery-jre8.jar b/Master/texmf-dist/scripts/texosquery/texosquery-jre8.jar
index 06f3b342b51..a8f91be6884 100755
--- a/Master/texmf-dist/scripts/texosquery/texosquery-jre8.jar
+++ b/Master/texmf-dist/scripts/texosquery/texosquery-jre8.jar
diff --git a/Master/texmf-dist/scripts/texosquery/texosquery.jar b/Master/texmf-dist/scripts/texosquery/texosquery.jar
index feb800910ab..44f54fa4337 100755
--- a/Master/texmf-dist/scripts/texosquery/texosquery.jar
+++ b/Master/texmf-dist/scripts/texosquery/texosquery.jar
diff --git a/Master/texmf-dist/source/support/texosquery/java/TeXOSQuery.java b/Master/texmf-dist/source/support/texosquery/java/TeXOSQuery.java
index 882cd3afb44..aad5a50a790 100644
--- a/Master/texmf-dist/source/support/texosquery/java/TeXOSQuery.java
+++ b/Master/texmf-dist/source/support/texosquery/java/TeXOSQuery.java
@@ -53,7 +53,7 @@ import java.io.Serializable;
  * There's a debug mode to print error messages to STDERR to
  * investigate the reason for failure.
  * @author Nicola Talbot
- * @version 1.2
+ * @version 1.2.1
  * @since 1.0
  */
 public class TeXOSQuery implements Serializable
@@ -4299,8 +4299,8 @@ public class TeXOSQuery implements Serializable
     
    public static final int DEFAULT_COMPATIBLE=2;
 
-   private static final String VERSION_NUMBER = "1.2";
-   private static final String VERSION_DATE = "2017-03-23";
+   private static final String VERSION_NUMBER = "1.2.1";
+   private static final String VERSION_DATE = "2017-03-28";
    private static final char BACKSLASH = '\\';
    private static final long ZERO = 0L;
 
diff --git a/Master/texmf-dist/source/support/texosquery/texosquery.dtx b/Master/texmf-dist/source/support/texosquery/texosquery.dtx
index a532e96c577..16dca119d00 100644
--- a/Master/texmf-dist/source/support/texosquery/texosquery.dtx
+++ b/Master/texmf-dist/source/support/texosquery/texosquery.dtx
@@ -53,7 +53,7 @@
 \renewcommand{\meta@font@select}{\rmfamily\itshape}
 \makeatother
 
-\CheckSum{3260}
+\CheckSum{3292}
 
 \RecordChanges
 \PageIndex
@@ -123,7 +123,7 @@ Organization for Standardization}
 %\author{Nicola L. C. Talbot\\
 %\href{http://www.dickimaw-books.com/}{\nolinkurl{dickimaw-books.com}}
 %\and Paulo Cereda}
-%\date{2017-03-23 (v1.2)}
+%\date{2017-03-28 (v1.2.1)}
 %\maketitle
 %
 %\begin{abstract}
@@ -157,7 +157,9 @@ Organization for Standardization}
 %\item \file{texosquery.jar}: requires at least Java~7, has medium locale
 %support, obeys \texttt{openin\_any} but has additional
 %restrictions imposed for security reasons (no listings outside the
-%current working directory path);
+%current working directory path); Note that Java 7 has reached its
+%\href{http://www.oracle.com/technetwork/java/eol-135779.html}{end
+%of life and is now deprecated}.
 %\item \file{texosquery-jre8.jar}: requires at least Java~8, has best locale
 %support, obeys \texttt{openin\_any} but has additional
 %restrictions imposed for security reasons (no listings outside the
@@ -2580,7 +2582,7 @@ Organization for Standardization}
 %    \end{macrocode}
 % Version info.
 %    \begin{macrocode}
-\expandafter\def\csname ver@texosquery.tex\endcsname{2017/03/23 v1.2 (NLCT)}
+\expandafter\def\csname ver@texosquery.tex\endcsname{2017/03/28 v1.2.1 (NLCT)}
 %    \end{macrocode}
 %
 %\begin{macro}{\@texosquery@warn}
@@ -6812,6 +6814,7 @@ Organization for Standardization}
      \let\@texosquery@currentsign\texosquerypatfmtplus
    \fi
    \@texosquery@digitindex=0\relax
+   \let\@texosquery@patfmt@dosep\empty
    \@texosquery@digitfoundfalse
    #1%
   \endgroup
@@ -6833,6 +6836,7 @@ Organization for Standardization}
 %    \end{macrocode}
 %\end{macro}
 %
+%Macros to shift the decimal place.
 %\begin{macro}{\@texosquery@pos@shift}
 %\changes{1.2}{2017-03-23}{new}
 %    \begin{macrocode}
@@ -6904,10 +6908,10 @@ Organization for Standardization}
   \or
     \let\@texosquery@int\@texosquery@frac
     \edef\@texosquery@frac{0}%
-%   \end{macrocode}
+%    \end{macrocode}
 %Anything larger will require scientific notation. Hopefully the
 %pattern supports this.
-%   \begin{macrocode}
+%    \begin{macrocode}
   \fi
 }
 %    \end{macrocode}
@@ -7031,6 +7035,7 @@ Organization for Standardization}
 \def\texosquerypatfmt@plusminus#1#2{%
  \edef\@texosquery@current{\expandafter\@texosquery@gobble\@texosquery@int}%
  \@texosquery@digitindex=0\relax
+ \let\@texosquery@patfmt@dosep\empty
  \@texosquery@digitfoundfalse
  \expandafter\ifx\@texosquery@sgn-%
   #2%
@@ -7079,6 +7084,7 @@ Organization for Standardization}
  \fi
  \edef\@texosquery@current{\expandafter\@texosquery@gobble\@texosquery@mantissa}%
  \@texosquery@digitindex=0\relax
+ \let\@texosquery@patfmt@dosep\empty
  \@texosquery@digitfoundfalse
  #2}}
 %    \end{macrocode}
@@ -7105,6 +7111,7 @@ Organization for Standardization}
 \def\texosquerypatfmt@dec#1#2{%
  \edef\@texosquery@current{\expandafter\@texosquery@gobble\@texosquery@int}%
  \@texosquery@digitindex=0\relax
+ \let\@texosquery@patfmt@dosep\empty
  \@texosquery@digitfoundfalse
  \let\@texosquery@zerodigit\@texosquery@zerodigit@leading
  #1%
@@ -7112,7 +7119,13 @@ Organization for Standardization}
  \else
   \@texosquery@invalidpattern{#1}%
  \fi
- \texosquerypatfmt@decsep
+%    \end{macrocode}
+%\changes{1.2.1}{2017-03-28}{moved decimal sep}
+%We can't display the decimal separator just yet as the pattern may
+%not show any digits for the fractional part. So only do the
+%separator just before the first digit.
+%    \begin{macrocode}
+ \let\@texosquery@patfmt@dosep\texosquerypatfmt@decsep
  \let\@texosquery@current\@texosquery@frac
  \@texosquery@digitindex=0\relax
  \@texosquery@digitfoundfalse
@@ -7256,6 +7269,9 @@ Organization for Standardization}
    \ifx\@texosquery@currentsign\texosquerypatfmtminus
     \texosquerypatfmtminus
     \let\@texosquery@currentsign\empty
+   \else
+    \@texosquery@patfmt@dosep
+    \let\@texosquery@patfmt@dosep\empty
    \fi
  \fi
  \@texosquery@digitfoundtrue
@@ -7324,6 +7340,9 @@ Organization for Standardization}
      \ifx\@texosquery@currentsign\texosquerypatfmtminus
       \texosquerypatfmtminus
       \let\@texosquery@currentsign\empty
+     \else
+       \@texosquery@patfmt@dosep
+       \let\@texosquery@patfmt@dosep\empty
      \fi
    \fi
    \@texosquery@digitfoundtrue
@@ -7369,6 +7388,9 @@ Organization for Standardization}
    \else
      \ifx\texosquerypatminus\texosquerypatfmtminus
       \texosquerypatfmtminus
+     \else
+       \@texosquery@patfmt@dosep
+       \let\@texosquery@patfmt@dosep\empty
      \fi
    \fi
    \@texosquery@digitfoundtrue
@@ -7414,6 +7436,9 @@ Organization for Standardization}
    \else
      \ifx\texosquerypatminus\texosquerypatfmtminus
       \texosquerypatfmtminus
+     \else
+       \@texosquery@patfmt@dosep
+       \let\@texosquery@patfmt@dosep\empty
      \fi
    \fi
    \@texosquery@digitfoundtrue
@@ -7540,7 +7565,7 @@ Organization for Standardization}
 %Identify package:
 %    \begin{macrocode}
 \NeedsTeXFormat{LaTeX2e}
-\ProvidesPackage{texosquery}[2017/03/23 v1.2 (NLCT)]
+\ProvidesPackage{texosquery}[2017/03/28 v1.2.1 (NLCT)]
 %    \end{macrocode}
 %Load \file{texosquery.tex}:
 %    \begin{macrocode}
diff --git a/Master/texmf-dist/tex/latex/texosquery/texosquery.sty b/Master/texmf-dist/tex/latex/texosquery/texosquery.sty
index f88771a6934..95aeff3b3d0 100644
--- a/Master/texmf-dist/tex/latex/texosquery/texosquery.sty
+++ b/Master/texmf-dist/tex/latex/texosquery/texosquery.sty
@@ -43,7 +43,7 @@
 %%   Right brace   \}     Tilde         \~}
 
 \NeedsTeXFormat{LaTeX2e}
-\ProvidesPackage{texosquery}[2017/03/23 v1.2 (NLCT)]
+\ProvidesPackage{texosquery}[2017/03/28 v1.2.1 (NLCT)]
 \input{texosquery}
 \endinput
 %%
diff --git a/Master/texmf-dist/tex/latex/texosquery/texosquery.tex b/Master/texmf-dist/tex/latex/texosquery/texosquery.tex
index 843196b21a7..9bc4aad662a 100644
--- a/Master/texmf-dist/tex/latex/texosquery/texosquery.tex
+++ b/Master/texmf-dist/tex/latex/texosquery/texosquery.tex
@@ -54,7 +54,7 @@
   \@texosquery@restore@at
   \expandafter\endinput
 \fi
-\expandafter\def\csname ver@texosquery.tex\endcsname{2017/03/23 v1.2 (NLCT)}
+\expandafter\def\csname ver@texosquery.tex\endcsname{2017/03/28 v1.2.1 (NLCT)}
 \ifx\@tracklang@pkgwarn\undefined
   \ifx\PackageWarning\undefined
     \def\@texosquery@warn#1{%
@@ -1397,6 +1397,7 @@
      \let\@texosquery@currentsign\texosquerypatfmtplus
    \fi
    \@texosquery@digitindex=0\relax
+   \let\@texosquery@patfmt@dosep\empty
    \@texosquery@digitfoundfalse
    #1%
   \endgroup
@@ -1561,6 +1562,7 @@
 \def\texosquerypatfmt@plusminus#1#2{%
  \edef\@texosquery@current{\expandafter\@texosquery@gobble\@texosquery@int}%
  \@texosquery@digitindex=0\relax
+ \let\@texosquery@patfmt@dosep\empty
  \@texosquery@digitfoundfalse
  \expandafter\ifx\@texosquery@sgn-%
   #2%
@@ -1592,6 +1594,7 @@
  \fi
  \edef\@texosquery@current{\expandafter\@texosquery@gobble\@texosquery@mantissa}%
  \@texosquery@digitindex=0\relax
+ \let\@texosquery@patfmt@dosep\empty
  \@texosquery@digitfoundfalse
  #2}}
 \def\texosquerypatfmtdecsep{.}
@@ -1599,6 +1602,7 @@
 \def\texosquerypatfmt@dec#1#2{%
  \edef\@texosquery@current{\expandafter\@texosquery@gobble\@texosquery@int}%
  \@texosquery@digitindex=0\relax
+ \let\@texosquery@patfmt@dosep\empty
  \@texosquery@digitfoundfalse
  \let\@texosquery@zerodigit\@texosquery@zerodigit@leading
  #1%
@@ -1606,7 +1610,7 @@
  \else
   \@texosquery@invalidpattern{#1}%
  \fi
- \texosquerypatfmt@decsep
+ \let\@texosquery@patfmt@dosep\texosquerypatfmt@decsep
  \let\@texosquery@current\@texosquery@frac
  \@texosquery@digitindex=0\relax
  \@texosquery@digitfoundfalse
@@ -1688,6 +1692,9 @@
    \ifx\@texosquery@currentsign\texosquerypatfmtminus
     \texosquerypatfmtminus
     \let\@texosquery@currentsign\empty
+   \else
+    \@texosquery@patfmt@dosep
+    \let\@texosquery@patfmt@dosep\empty
    \fi
  \fi
  \@texosquery@digitfoundtrue
@@ -1751,6 +1758,9 @@
      \ifx\@texosquery@currentsign\texosquerypatfmtminus
       \texosquerypatfmtminus
       \let\@texosquery@currentsign\empty
+     \else
+       \@texosquery@patfmt@dosep
+       \let\@texosquery@patfmt@dosep\empty
      \fi
    \fi
    \@texosquery@digitfoundtrue
@@ -1790,6 +1800,9 @@
    \else
      \ifx\texosquerypatminus\texosquerypatfmtminus
       \texosquerypatfmtminus
+     \else
+       \@texosquery@patfmt@dosep
+       \let\@texosquery@patfmt@dosep\empty
      \fi
    \fi
    \@texosquery@digitfoundtrue
@@ -1829,6 +1842,9 @@
    \else
      \ifx\texosquerypatminus\texosquerypatfmtminus
       \texosquerypatfmtminus
+     \else
+       \@texosquery@patfmt@dosep
+       \let\@texosquery@patfmt@dosep\empty
      \fi
    \fi
    \@texosquery@digitfoundtrue