summaryrefslogtreecommitdiff
path: root/macros/context/contrib/context-filter
diff options
context:
space:
mode:
authorNorbert Preining <norbert@preining.info>2019-09-02 13:46:59 +0900
committerNorbert Preining <norbert@preining.info>2019-09-02 13:46:59 +0900
commite0c6872cf40896c7be36b11dcc744620f10adf1d (patch)
tree60335e10d2f4354b0674ec22d7b53f0f8abee672 /macros/context/contrib/context-filter
Initial commit
Diffstat (limited to 'macros/context/contrib/context-filter')
-rw-r--r--macros/context/contrib/context-filter/VERSION1
-rw-r--r--macros/context/contrib/context-filter/doc/context/third/filter/filter.txt1005
-rw-r--r--macros/context/contrib/context-filter/tex/context/third/filter/t-filter.mkii508
-rw-r--r--macros/context/contrib/context-filter/tex/context/third/filter/t-filter.mkiv543
-rw-r--r--macros/context/contrib/context-filter/tex/context/third/filter/t-module-catcodes.mkii101
-rw-r--r--macros/context/contrib/context-filter/tex/context/third/filter/t-module-catcodes.mkiv89
6 files changed, 2247 insertions, 0 deletions
diff --git a/macros/context/contrib/context-filter/VERSION b/macros/context/contrib/context-filter/VERSION
new file mode 100644
index 0000000000..95c275e403
--- /dev/null
+++ b/macros/context/contrib/context-filter/VERSION
@@ -0,0 +1 @@
+2018.08.10
diff --git a/macros/context/contrib/context-filter/doc/context/third/filter/filter.txt b/macros/context/contrib/context-filter/doc/context/third/filter/filter.txt
new file mode 100644
index 0000000000..fd7ccee225
--- /dev/null
+++ b/macros/context/contrib/context-filter/doc/context/third/filter/filter.txt
@@ -0,0 +1,1005 @@
+[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](http://www.repostatus.org/badges/latest/active.svg)](http://www.repostatus.org/#active)
+===
+
+The filter module
+=================
+
+History
+-------
+
+This module started with a simple idea. I wanted an environment
+
+ \startmarkdown
+ ...
+ \stopmarkdown
+
+to write content in Markdown. Such an environment requires a parser that
+converts markdown to TeX. A TeX wizard would write such a parser in TeX (which
+after all is Turing complete). A lua wizard would write it in LPEG (and wonder
+why some users still use MkII). I, being no wizard myself, found an existing
+tool that does the job---pandoc. But how could I use pandoc inside ConTeXt?
+
+As for almost everything else in ConTeXt, Hans had already done this; albeit for
+a different tool---the R programming language. The _R
+module_ (`m-r.tex`) allows the user to execute snippets of R code. I wanted the
+same for Markdown.
+
+I copied the R-module, made a couple of changes, and had a Markdown module
+ready. But what if tomorrow I wanted to use ReStructured Text instead of
+Markdown? Or Matlab code instead R? Surely, copying the R-module for each
+program would be a waste of effort. Each new program requires only a few changes
+in the R-module; what I needed was a R-module _template_ so that I could fill in
+the blanks with the appropriate values. And so, the filter module was born.
+
+Compatibility
+------------
+
+This module works with both MkII and MkIV.
+
+Installation
+------------
+
+Writing installation instructions is always boring. If you are using ConTeXt
+standalone, you can install the module using
+
+ first-setup.sh --modules="t-filter"
+
+Depending on your TeX distribution, you may already have the module.
+To verify, check if
+
+ kpsewhich t-filter.mkii
+
+returns a meaningful path. If not, you have to manually install the module.
+
+Download the latest version of the module from
+[http://modules.contextgarden.net/filter](http://modules.contextgarden.net/filter)
+and unzip it either `$TEXMFHOME` or `$TEXMFLOCAL`. Run
+
+ mktexlsr
+
+and
+
+ mtxrun --generate
+
+to refresh the TeX file database (for MkII and MkIV, respectively). If
+everything went well
+
+ kpsewhich t-filter.mkii
+
+will return the path where you stored the file.
+
+Unfortunately, that is not enough. For the module to work, TeX must be able to
+call an external program. This feature is a potential security risk and is
+disabled by default on most TeX distributions. To enable this feature, set
+
+ shell_escape=t
+
+in your `texmf.cnf` file. See this page
+[http://wiki.contextgarden.net/write18](http://wiki.contextgarden.net/write18)
+on the ConTeXt wiki for detailed instructions.
+
+Basic Usage
+-----------
+
+The steps involved in calling a filter on the contents of an environment are:
+
+1. Write the contents to an external file. This file is the input to the filter,
+ and is, therefore, called `\externalfilterinputfile`
+
+2. Run the filter on `\externalfilterinputfile` to generate an output, which is
+ called `\externalfilteroutputfile`.
+
+3. (Optional) Read `\externalfilteroutputfile` in ConTeXt.
+
+Lets start from the simplest case: a filter that inputs a text file and outputs
+a valid ConTeXt file, like `pandoc` to convert from Markdown to ConTeXt. The
+command line syntax of this filter is
+
+ pandoc -t context -o outputfile inputfile
+
+Using this filter from within ConTeXt is pretty simple:
+
+ \usemodule[filter]
+
+ \defineexternalfilter
+ [markdown]
+ [filtercommand={pandoc -t context -o \externalfilteroutputfile\space \externalfilterinputfile}]
+
+Yes, its that easy! The only thing to note is that TeX macros gobble spaces, so
+I inserted a manual space after `\externalfilteroutputfile`.
+
+The above `\defineexternalfilter` macro defines:
+
+1. An environment
+
+ \startmarkdown
+ ...
+ \stopmarkdown
+
+ The contents of the environment are processed by `pandoc` and the output is
+ included back in ConTeXt.
+
+2. A macro
+
+ \inlinemarkdown{...}
+
+ The argument of the macro is processed by `pandoc` and the output is included
+ back in ConTeXt.
+
+3. A macro
+
+ \processmarkdownfile{...}
+
+ The argument to the macro is a filename, which is processed by `pandoc` and
+ the output is included back in ConTeXt.
+
+4. A macro
+
+ \processmarkdownbuffer[...]
+
+ The argument to the macro is the name of a buffer, which is written to an
+ external file, processesd by `pandoc` and the output included back in
+ ConTeXt.
+
+The [wiki](https://github.com/adityam/filter/wiki) page on Github gives the
+setup for common usecases (pandoc, R, etc.)
+
+Inheriting setup from other commands
+-------------------------------------
+
+It is also possible to inherit the settings from another filter. For example,
+
+ \defineexternalfilter
+ [filterstyle]
+ [color=red,
+ style=bold]
+
+ \defineexternalfilter
+ [markdown]
+ [filterstyle]
+ [filter={pandoc -w context -o \externalfilteroutputfile },
+ style=italic]
+
+Notice the three arguments to `\defineexternalfilter`. The first argument
+(`markdown`) is the name of the new filter; the second argument (`filterstyle`)
+is the name of the filter whose settings we want to inherit, and the third
+argument (`filter=...`) are the new settings for `markdown` filter. The above
+definition is same as:
+
+ \defineexternalfilter
+ [filter={pandoc -w context -o \externalfilteroutputfile },
+ style=italic,
+ color=red]
+
+Note that if a setting (like `style` above) is defined both in the new filter
+and the parent filter, then the value of the new filter (`style=italic` above)
+is used.
+
+Dealing with slow filters
+-------------------------
+
+The above definition of a markdown filter creates two additional files: an
+"input" file and an "output" file, *irrespective of the
+number of times the environment is called*. For each markdown environment,
+ConTeXt overwrites the input file and pandoc overwrites the output
+file; as a result, the current directory is not cluttered with temporary files.
+The trade off is that for each document run, the filter is
+invoked as many times as the number of markdown environments. Since getting
+cross-referencing right normally takes two to three runs, effectively the filter
+is run two or three times more than required. A filter like `pandoc` is fairly
+fast, so these extra runs are not noticeable. But some filters, like the
+R-programming language for which simply startup and exit takes about 0.3
+seconds, are slow. In such cases, the additional runs start adding up. A better
+trade off is to store the contents of each environment in a separate file and
+invoke the filter only if a file *changes in between successive runs*.
+
+The second behavior is achieved by adding `cache=yes` option to the
+definition:
+
+ \defineexternalfilter
+ [...]
+ [...
+ cache=yes,
+ ...]
+
+Sometimes you want to force the rerun of a filter, even if the content of the
+environment has not changed. This could be because the filters depend on an
+external script that might have changed. To force a rerun of a filter use
+
+
+ \defineexternalfilter
+ [...]
+ [...
+ cache=force,
+ ...]
+
+Reading the input
+----------------
+
+By default, after the filter is executed, `\externalfilteroutputfile` is read
+using `\ReadFile`. To change this behavior, use the `readcommand` option. For
+example:
+
+ \defineexternalfilter
+ [...]
+ [....
+ readcommand=\typefile,
+ ...]
+
+types the output file verbatim. The value of read command must be a macro that
+takes the name of the output file as a (brace-delimited) argument and does
+something sensible with it.
+
+Sometimes, it is desirable to ignore the output, which is done by
+
+ \defineexternalfilter
+ [...]
+ [....
+ read=no,
+ ...]
+
+Space around the environment
+----------------------------
+
+By default, the `\start<...>` ... `\stop<...>` and the `\type<...>file{...}`
+variant displays the output is _paragraph_ mode (i.e., inserts blanks before and
+after the environment), while the `\inline{...}` variant reads the output in
+_text_ mode (i.e., does not insert blanks before or after the environment).
+
+To change the amount of space inserted before and after the environment, use the
+`spacebefore` and the `spaceafter` keys. For example, if you want big spaces
+around the environment use:
+
+ \defineexternalfilter
+ [...]
+ [....
+ spacebefore=big,
+ spaceafter=big,
+ ...]
+
+The `spacebefore` and `spaceafter` keys accept all values accepted by the
+`\blank[...]` macro.
+
+In the paragraph mode, the next line after `\stop<...>` is indented or not based
+on the value of the `indentnext` key. The default value is `auto` which indents
+the next line if there is an empty line after `\stop<...>`; other options are
+`no`, which never indents the next line and `yes` which always indents the next
+line.
+
+If you want the `\start<...>` ... `\stop<...>` and the `\type<...>file{...}`
+variant to behave in _text_ mode, set:
+
+ \defineexternalfilter
+ [...]
+ [....
+ location=text,
+ ...]
+
+(The default value of `location` is `paragraph`).
+
+**Note** that `location=text` is not equivalent to `\inline{...}`. Inline also
+sets `\endlinechar=\minusone`; therefore no space is inserted when the file is
+read. `location=text` does not change `\endlinechar`. Therefore a space is
+inserted after the file is read.
+
+Stripping leading whitespace (MkIV only)
+----------------------------------------
+
+By default, the leading whitespace is removed before the content of the filter
+environment are saved to an external file. This is useful because one can then
+indent the TeX code without worring how the leading whitespaces will be
+interpretted by the filter. For example,
+
+ \startitemize
+ \item
+ \startmarkdown
+ This is treated as regular text. If the leading spaces were not
+ stripped, this would be treated as a code block in markdown.
+ \stopmarkdown
+ \stopitemize
+
+If you want to keep the leading whitespace, you can set
+
+ \defineexternalfilter
+ [...]
+ [....
+ strip=no,
+ ...]
+
+(The default value is `yes`).
+
+Names of temporary files
+------------------------
+
+By default, `\externalfilterinputfile` is set to `\jobname-temp-<filter>.tmp`, where
+`<filter>` is the first argument of `\defineexternalfilter`. When `cache=yes`
+is set, `\externalfilterinputfile` equals `\jobname-temp-<filter>-<n>.tmp`, where
+`<n>` is the number of filter environments that have appeared so far. In MkII,
+a `\jobname-temp-<filter>-<n>.tmp.md5` file, which stores the `md5` sum of the
+input file is also created.
+
+A macro `\externalfilterbasefile` stores the name of the input file without the
+extension. By default, the value of `\externalfilteroutputfile` is
+`\externalfilterbasefile.tex`. Having a `.tex` extension is not always
+desirable. For example, if the filter generates a PNG file, a `.png` extension
+is more descriptive. The name of the output file is changed using the `output`
+key. For example
+
+ \defineexternalfilter
+ [...]
+ [filtercommand={...},
+ output={\externalfilterbasefile.png}]
+
+changes the output extension to `.png`. To read the generated PNG file, set:
+
+ \defineexternalfilter
+ [...]
+ [....
+ readcommand=\readPNGfile,
+ ...]
+
+where `\readPNGfile` is defined as
+
+ \def\readPNGfile#1{\externalfigure[#1]}
+
+Output Directory
+----------------
+
+This module creates a lot of temporary files that clutter the current directory.
+If you prefer the temporary files to be created in another directory, use
+the `directory` option, e.g.,
+
+ \defineexternalfilter
+ [...]
+ [...
+ directory=output/,
+ ...]
+
+This will create all the temporary files in `output` directory. The name of the
+directory may be specified with or without a trailing slash. Thus,
+`directory=output` and `directory=output/` are both valid.
+
+The directory path **must be relative to the current directory**. Absolute paths
+do not work. If you try to use a absolute path like
+
+ \defineexternalfilter
+ [...]
+ [...
+ directory=/tmp/,
+ ...]
+
+you will get an error message
+
+ t-filter > Fatal Error: Cannot use absolute path /tmp/ as directory
+
+and compilation will stop.
+
+Disabling filters
+----------------
+
+Adding `state=stop` option disables the filters. The
+`\externalfilterinputfile` is still written, but the filter is not run.
+
+When used in conjunction with `cache=yes` and `directory=...`, this
+is useful for sharing your files with others who do not have the
+external program that you are using.
+
+Deleting temporary files
+------------------------
+
+In MkIV, the module automatically deletes the `\externalfilterinputfile` after
+executing the filter unless `\traceexternalfilters` is used. If, for whatever
+reason, you want to keep this file around, use
+
+ \defineexternalfilter
+ [...]
+ [...
+ purge=no,
+ ...]
+
+In MkII, the `\externalfilterinputfile` is not deleted.
+
+All the files generated by the filter module have `-temp-` in their name. As
+such they can be deleted using
+
+ context --purgeall --pattern=filename
+
+Do **not** use the file extension. To delete all temporary files in the
+current directory, use
+
+ context --purgeall
+
+
+Standard options
+---------------
+
+`\defineexternalfilter` accepts the following standard options:
+
+- `spacebefore` and `spaceafter` to specify the blank space to be used
+ before and after the environment.
+- `before` and `after`: to enclose the output in a frame, etc. (only if
+ `location` is `paragraph`)
+- `left` and `right`: same as `before` and `after` but used when `location` is
+ not `paragraph`.
+- `style` and `color`: to set the color and style of the output.
+- `align`: to set the alignment of the output (only if
+ `location` is `paragraph`).
+- `indentnext`: specify if the next line is indented (only if `location` is
+ `paragraph`).
+- `setups`: specify a list of setups (defined using `\startsetups`). These
+ setups may be used to define commands that are needed inside the environment.
+
+The order in which these options are executed are:
+
+1. `\blank[spacebefore]`
+2. `before/left
+3. `align` (if `location=paragraph`)
+4. `style` and `color`
+5. `setups`
+6. `readcommand`
+7. `after/right
+8. `\blank[afterspace]`
+9. check `indentnext`
+
+Options to a specific environment
+---------------------------------
+
+Each `\start<filter>` macro also accepts options. However, unlike other ConTeXt
+environment, these options cannot be on a separate line; they must be on the same
+line as `\start<filter>`. For example, suppose I define an environment to run
+R-code
+
+ \defineexternalfilter
+ [R]
+ [filtercommand={R CMD BATCH -q \externalfilterinputfile\space \externalfilteroutputfile},
+ output=\externalfilterbasefile.out,
+ cache=yes]
+
+I can hide the output of a particular R-environment by
+
+ \startR[read=no]
+ ...
+ \stopR
+
+The macros `\processmarkdownfile` and `\processmarkdownbuffer` also accept user
+options. The usage is
+
+ \processmarkdownfile [.-.=.-.]{filename}
+ \processmarkdownbuffer[...=...][buffer]
+
+
+A setup to control them all
+---------------------------
+
+The macro `\setupexternalfilters` sets the default options for all the filters
+created using `\defineexternalfilter`. This is responsible for the default values
+of all options. The current defaults are
+
+ \setupexternalfilters
+ [before=,
+ after=,
+ setups=,
+ cache=no,
+ read=yes,
+ readcommand=\ReadFile,
+ output=\externalfilterbasefile.tex,
+ ]
+
+
+Passing options to filters
+--------------------------
+
+**NOTE** This option does not work for MkII or for inline snippets
+
+Sometimes it is useful to pass options to a filter. For example, `pandoc`
+converts many different formats to ConTeXt (actually, to many different output
+formats, but that is irrelevant here). Instead of defining a separate
+environment for each input format, can I define a universal pandoc environment
+and specify the input format on a case by case basis. For example,
+
+ \startpandoc
+ ...
+ \stoppandoc
+
+for the default Markdown input,
+
+ \startpandoc[format=rst]
+ ...
+ \stoppandoc
+
+for reStructured Text input, and
+
+ \startpandoc[format=latex]
+ ...
+ \stoppandoc
+
+for LaTeX input. In `pandoc`, the input format is specified as
+
+ pandoc -f format -t context -o outputfile inputfile
+
+So, we need a mechanism to access the value of the `format` option to
+`\startpandoc`. This value is accessed using `\externalfilterparameter{format}`.
+Thus, the pandoc environment may be defined as
+
+ \defineexternalfilter
+ [pandoc]
+ [filtercommand={pandoc -f \externalfilterparameter{format} -t context
+ -o \externalfilteroutputfile\space \externalfilterinputfile},
+ format=markdown]
+
+Macro variant
+-------------
+
+For some cases, a macro `\inline<filter>{...}` is more natural to use rather
+than the environment `\start<filter>` ... `\stop<filter>`. The `\inline...`
+variant is meant for simple cases, so it does not accept any options in square
+brackets. This macro is similar to `\type` macro, and its argument can be
+written in two ways: either as a group `{...}` or delimited by arbitrary tokens.
+Thus, all the following are valid:
+
+ \defineexternalfilter[markdown][...]
+
+ \inlinemarkdown{both braces{}}
+
+ \inlinemarkdown+an opening brace {+
+
+ \inlinemarkdown!a closing brace }!
+
+**Note** Inline mode sets `\endlinechar=\minusone`; therefore no space is
+inserted after a newline. This may lead to unexpected results if the output of
+the filter is wrapped into multiple lines. For example, if the output of the
+filter is
+
+ This is a long line that is wrapped
+ after a fixed number of characters.
+
+Then, when reading the file the space between `wrapped` and `after` will be
+lost! To avoid that pass appropriate options to the filter program so that it
+does not wrap long lines.
+
+Processing existing Files
+-------------------------
+
+A big advantage of a lightweight markup language like markdown is that it is
+easy to convert it into other markups--html, rtf, epub, etc. For that reason, I
+key in markdown in a separate file rather in a start-stop environment of a TeX
+file. To use such markdown files in ConTeXt, I can just use
+
+ \processmarkdownfile{filename.md}
+
+By default, the file is searched the current directory and in the directories
+specified by `\usepath`. In addition, in MkIV, the parent and grand-parent
+directories are also searched. If the file is not in one of these locations,
+specify a full or a relative path to the file.
+
+The general macro is `\process<filter>file{...}`, which takes the name of a file
+**or a url (MkIV only)** as an argument and uses that file as the input file for
+the filter. The rest of the processing is the same as with `\start<filter>` ...
+`\stop<filter>` environment.
+
+The `\process<filter>file` macro also takes an optional argument for setup
+options:
+
+ \process<filter>file[...]{...}
+
+The options in the `[...]` are the same as those for `\defineexternalfilter`.
+
+Processing remote files
+-----------------------
+
+**NOTE** Only works in MkIV
+
+The `\process<filter>file{...}` macro also processes remote files specified
+using URLs. For example, to see a typeset version of this manual, use
+
+ \processmarkdownfile{https://raw.github.com/adityam/filter/master/README.md}
+
+This macro downloads the file in the background, and processes the local file using
+`pandoc`. To prevent frequent downloads, the downloaded file is cached and the
+file is re-downloaded only if the cached file is more than 1 day old. You can
+override the default threshold using `schemes.threshold` directive. For example,
+if you want to re-download the file every 5 minutes (= 300 seconds), add
+
+ \enabledirectives[schemes.threshold=300]
+
+somewhere before `\starttext` or use
+
+ context --directives=schemes.threshold=300 <filename>
+
+to compile the file.
+
+To see where the cached file is stored, add
+
+ \enabletrackers[resolvers.schemes]
+
+or use
+
+ context --trackers=resolvers.schemes <filename>
+
+to compile the file.
+
+Processing existing buffers
+---------------------------
+
+Like all macros built on top of buffers, the `\start<filter>` ...
+`\stop<filter>` environment does not work well inside the argument of another
+command. The `\process<filter>buffer` macro is handy for such macros.
+
+Suppose you want to write some markdown text in a footnote. Using
+
+ \footnote{ ....
+ \startmarkdown
+ ...
+ \stopmarkdown}
+
+gives an error message:
+
+ ! File ended while scanning use of \dododowithbuffer.
+
+ system > tex > error on line 0 in file : File ended while scanning use
+ of \dododowithbuffer ...
+
+ <empty file>
+
+ <inserted text>
+ \par
+
+To avoid this, define a buffer at the outer level
+
+ \startbuffer[footnote-markdown]
+ ...
+ \stopbuffer
+
+and then use
+
+ \footnote{... \processmarkdownbuffer[footnote-markdown]}
+
+The `\process<filter>buffer` macro also takes an optional argument for setup
+options:
+
+ \process<filter>buffer[...][...]
+
+The options in the first `[...]` are the same as those for `\defineexternalfilter`.
+
+
+Prepend and append text
+-----------------------
+
+**NOTE** Only works in MkIV
+
+Some external programs require boilerplate text at the beginning and end of each
+file. Including this boilerplate code in each snippet can get verbose. The
+filter module provides two options `bufferbefore` and `bufferafter` to shorten
+such snippets. Define the boilerplate code in ConTeXt buffers and then use
+
+ \defineexternalfilter
+ [...]
+ [...
+ bufferbefore={...list of buffers...},
+ bufferafter={...list of buffers...},
+ ]
+
+For example, suppose you want to generate images using a LaTeX package that does
+not work well with ConTeXt, say `shak`. One way to use this is as follows: first
+define a file that processes its content using `latex`.
+
+ \defineexternalfilter
+ [chess]
+ [filter=pdflatex,
+ output=\externalfilterbasefile.pdf,
+ readcommand=\readPDFfile,
+ ]
+
+ \def\readPDFfile#1{\externalfigure[#1]}
+
+
+Next create buffers containing boilerplate code needed to run latex:
+
+ \startbuffer[chess::before]
+ \documentclass{minimal}
+ \usepackage{skak}
+ \usepackage[active,tightpage]{preview}
+
+ \begin{document}
+ \begin{preview}
+ \newgame
+ \hidemoves{
+ \stopbuffer
+
+ \startbuffer[chess::after]
+ }
+ \showboard
+ \end{preview}
+ \end{document}
+ \stopbuffer
+
+and tell the filter to prepend and append these buffers
+
+ \setupexternalfilter
+ [chess]
+ [bufferbefore={chess::before},
+ bufferafter={chess::after}]
+
+Then you can use
+
+ \inlinechess{1.e4 e5 2. Nf3 Nc6 3.Bb5}
+
+to get a chess board.
+
+Special use case: `\write18` with caching
+------------------------------------------
+
+Although the raison d'être of the externalfilter module is to process the
+content of an evironment or a macro through an external program, it may also be
+used to simply exectute an external program without processing any content.
+
+For example, suppose we want to include an image with a swirl gradient in our
+document. ImageMagick can generate such an image using:
+
+ convert -size 100x100 gradient: -swirl 180 <output file>
+
+Notice that in this case, the external program does not need any input file. We
+just need to pass the size of the image to the external program.
+
+In such cases, we still want to cache the result, i.e., rerun the external
+program only when the `size` of the image has changed. The `write=no` option
+covers this use case. The basic usage is:
+
+ \defineexternalfilter
+ [...]
+ [
+ ...
+ write=no,
+ cacheoption=....,
+ ...
+ ]
+
+Out of the four macros (see [Basic Usage]) created by `\defineexternalfilter`,
+only `\inline<externalfilter>` makes sense with `write=no`. The usage of this
+macro is
+
+ \inline<externalfilter>[....]
+
+Unlike the default case, this version does not take an argument! (That is
+because, it does not write anything to a file).
+
+When `write=no` is set, `\externalfilterbasefile` is equal to
+`\jobname-temp-<filter>-<cacheoption>` where `<cacheoption>` is the value of the
+`cacheoption` key.
+
+For example, to generate swirl backgrounds described above, define:
+
+ \defineexternalfilter
+ [swirl]
+ [
+ write=no,
+ cacheoptions={\externalfilterparameter{size}},
+ cache=yes,
+ size={1000x1000},
+ output=\externalfilterbasefile.png,
+ filtercommand={convert -size \externalfilterparameter{size} gradient: -swirl 180 \externalfilteroutputfile},
+ readcommand=\ReadFigure,
+ ]
+
+ \def\ReadFigure#1%
+ {\externalfigure[#1]}
+
+
+This creates a macro `\inlineswirl` that uses ImageMagick to generate a file
+`\jobname-temp-swirl-1000x1000.png`.
+
+The result is cached and the external program is rerun only if the value of
+cacheoption changes, that is, only if the value of `size` key changes.
+
+
+Dealing with expansion
+----------------------
+
+All the arguments of `filtercommand` must be fully expandable. Sometimes,
+writing an expandable command is tricky. For example, suppose you want to use
+GNU barcode to draw barcodes. One way to do this is
+
+
+ \defineexternalfilter
+ [barcode]
+ [encoding=code128,
+ output=\externalfilterbasefile.eps,
+ cache=yes,
+ filtercommand=\barcodefiltercommand,
+ readcommand=\barcodereadcommand]
+
+ \def\barcodereadcommand#1%
+ {\externalfigure[#1]}
+
+ \def\barcodefiltercommand
+ {barcode -i \externalfilterinputfile\space -o \externalfilterbasefile.eps\space
+ -E % EPS output
+ -e \externalfilterparameter{encoding}
+
+One of the options that GNU barcode provides is
+
+ -n ``Numeric'' output: don't print the ASCII form of the code, only
+ the bars.
+
+The ideal way to support this option is to provide a `label=(yes|no)` option,
+and in `\barcodefiltercommand` check the value of
+`\externalfilterparameter{label}`. If this value is `no`, add a `-n` flag. That
+is, redefine `\barcodefiltercommand` as follows:
+
+ \def\barcodefiltercommand
+ {barcode -i \externalfilterinputfile\space -o \externalfilterbasefile.eps\space
+ -E % EPS output
+ -e \externalfilterparameter{encoding}
+ \doif{\externalfilterparameter{label}}{no}{-n} }
+
+This approach does not work. The log says:
+
+ t-filter > command : barcode -i barcode-temp-barcode-1.tmp -o barcode-temp-barcode-1.eps -E -e code128 \edef {yes}\edef yes{no}
+
+Instead of `-n`, we get `\edef {yes} \edef yes{no}` in the output. This is
+because `\doif` macro is not fully expandable.
+
+One way to circumvent this limitation is to check for the value of `label`
+outside the `filtercommand`. The filter module provides a `filtersetup` option
+for this. For example, in the above barcode example, use
+
+ \def\barcodelabeloption{}
+
+ \startsetups barcode:options
+ \doifelse{\externalfilterparameter{label}}{no}
+ {\edef\barcodelabeloption{-n}}
+ {\edef\barcodelabeloption{}}
+ \stopsetups
+
+ \defineexternalfilter
+ [barcode]
+ [....
+ filtersetups={barcode:options},
+ filtercommand=\barcodefiltercommand,
+ ...
+ ]
+
+ \def\barcodefiltercommand
+ {barcode -i \externalfilterinputfile\space -o \externalfilterbasefile.eps\space
+ -E % EPS output
+ -e \externalfilterparameter{encoding}
+ \barcodelabeloption % check for label
+ }
+
+Limitations
+------------
+
+- In MkII, the option `cache=yes` does not work correctly with filters that have a
+ pipe `|` in their definition. This is because internally `cache=yes` calls
+
+ mtxrun --ifchanged=filename --direct filtercommand
+
+ and this produces
+
+ MTXrun |
+ MTXrun | executing: filtercommand
+ MTXrun |
+ MTXrun |
+
+ In MkIV, `cache=yes` calls
+
+ \ctxlua{job.files.run("filename", "filtercommand")}
+
+ so filters with a `|` work correctly.
+
+
+
+Messages and Tracing
+-------------------
+
+The filter module outputs some diagnostic information on the console output to
+indicate what is happening. Loading of the module is indicated by:
+
+ loading : ConTeXt User Module / Filter (ver: <date>)
+
+Whenever a filter is defined the expanded name of the command is displayed.
+For example, for the markdown filter we get:
+
+ t-filter > command : pandoc -w context -o markdown-temp-markdown.tex markdown-temp-markdown.tmp
+
+If, for some reason, the output file is not generated, or not found, a message
+similar to
+
+ t-filter > file markdown-temp-markdown.tex cannot be found
+ t-filter > current filter : markdown
+ t-filter > base file : markdown-temp-markdown
+ t-filter > input file : markdown-temp-markdown.tmp
+ t-filter > output file : markdown-temp-markdown.tex
+
+is displayed on the console. At the same time, the string
+
+ [[output file missing]]
+
+is displayed in the PDF output. This data, along with the name of the filter
+command, is useful for debugging what went wrong. To get more debugging
+information add
+
+ \traceexternalfilters
+
+in your tex file. This shows the name of the filters when they are defined.
+In MkIV, `\traceexternalfilters` also enables the trackers for `graphic.run`, so
+when `cache=yes` is used, message like
+
+ graphics > run > processing file, no changes in '<filename>-temp-<filtername>-<n>.tmp', not processed
+
+are shown.
+
+Version History
+--------------
+
+- **2010.09.26**:
+ - First release
+- **2010.10.09**:
+ - Added `\inline<filter>{...}` macro
+ - Changed the syntax of `\process<filter>file`. The filename is now
+ specified in curly brackets rather than square brackets.
+- **2010.10.16**:
+ - Added `\traceexternalfilters` for tracing
+ - Added a message that shows filter command on console
+ - A message is shown on console when output file is not found.
+- **2010.10.30**:
+ - Added `directory=...` option to `\defineexternalfilter` and
+ `\setupexternalfilters`.
+- **2010.12.04**:
+ - Bug fix in `directory` code. The option `directory=../something` was
+ handled incorrectly.
+- **2011.01.28**
+ - Bug fix. The filter counter was not incremented inside a group. Made the
+ increment global.
+- **2011.02.21**
+ - Added `style` and `color` options.
+- **2011.02.27**
+ - The external files are called `\jobname-temp-<filter>*` instead of
+ `\jobname-externalfilter-<filter>*`. As a result, these files are deleted
+ by `context --purgeall`.
+- **2011.03.06**
+ - Complete rewrite of internal macro names. The internal macros are now
+ named `\modulename::command_name`. This is an experiment to see if this
+ style works better than the traditional naming convention in TeX.
+- **2011.06.16**
+ - Added `force` mode to force recompilation of all filters that have
+ `continue=yes`.
+ - Added `reuse` mode to skip running all filters that have
+ `continue=yes`.
+ - Added `state=stop` option to skip running external filter.
+- **2011.08.23**
+ - Added `bufferbefore` and `bufferafter` options
+- **2011.08.28**
+ - Internal change: Defined own macros for setting attributes rather than
+ using built-in ones.
+- **2011.09.03**
+ - Added `filtersetups`
+- **2011.09.14**
+ - `\inline<filter>` now accepts optional arguments.
+ - `before=` and `after=` keys are disabled in `\inline<filter>`
+- **2011.10.22**
+ - Added `\process<filter>buffer`
+- **2011.12.04**
+ - Use `job.files.run` instead of `mtxrun --ifchanged` in MkIV.
+- **2011.12.17**
+ - Split into `.mkii` and `.mkiv` versions
+- **2012.01.26**
+ - Renamed `continue` to `cache`. Using `continue=yes` still works
+ - Removed `force` and `reuse` modes (too easy to clash with user modes).
+ - Functionality of force mode implemented using `cache=force`.
+- **2012.02.05**
+ - Added `purge=yes|no` to control if the input file is deleted or not
+- **2012.03.18**
+ - Process remote files
+- **2012.04.18**
+ - Added `location`, `spacebefore` and `spaceafter` keys.
+- **2012.05.01**
+ - Added `align` key.
+- **2012.06.20**
+ - Support for `\usepath`
+- **2012.01.13**
+ - Support for `write=no` and `cacheoption=...`.
+- **2013.03.31**
+ - Support for `left` and `right`
+- **2018-04-17**
+ - Support for `strip=yes` (which is now default).
diff --git a/macros/context/contrib/context-filter/tex/context/third/filter/t-filter.mkii b/macros/context/contrib/context-filter/tex/context/third/filter/t-filter.mkii
new file mode 100644
index 0000000000..3024d212c8
--- /dev/null
+++ b/macros/context/contrib/context-filter/tex/context/third/filter/t-filter.mkii
@@ -0,0 +1,508 @@
+%D \module
+%D [ file=t-filter,
+%D version=2013.04.15,
+%D title=\CONTEXT\ User Module,
+%D subtitle=Filter,
+%D author=Aditya Mahajan,
+%D date=\currentdate,
+%D copyright=Aditya Mahajan,
+%D email=adityam <at> ieee <dot> org,
+%D license=Simplified BSD License]
+
+\writestatus{loading}{Filter (ver: 2013.04.15)}
+
+\ifx\undefined\normalexpanded \let\normalexpanded\expanded \fi
+
+% Make \doifinset expandable in MkII
+
+\def\p!doifinsetelse#1#2#3#4%
+ {\donefalse
+ \edef\!!stringa{#3}%
+ \ifx\!!stringa\empty
+ \else
+ \processcommacommand[#4]\p!docheckiteminset
+ \fi
+ \ifdone\expandafter#1\else\expandafter#2\fi}
+
+
+\startmodule [filter]
+\usemodule [module-catcodes]
+
+\unprotectmodulecatcodes
+
+%D \subject {Interface}
+%D
+%D Using interface constants allows one to use \type{\c!filter} etc. in
+%D the module definition, and thereby reduces the risk of a typo.
+%D Currently, only English names are provided. If someone wants a
+%D multi-lingual interface, let me know and I will add other language
+%D names as well,
+
+\startinterface all
+ \setinterfaceconstant {filter} {filter}
+ \setinterfaceconstant {filtercommand} {filtercommand}
+ \setinterfaceconstant {output} {output}
+ \setinterfaceconstant {purge} {purge}
+ \setinterfaceconstant {read} {read}
+ \setinterfaceconstant {write} {write}
+ \setinterfaceconstant {readcommand} {readcommand}
+\stopinterface
+
+
+%D \subject {Name space}
+%D
+%D We use logical names to easily catch typos.
+
+\def\externalfilter@id {externalfilter}
+\def\????externalfilter {@@@@\externalfilter@id}
+
+\def\externalfilter@temp_prefix {temp}
+\def\externalfilter@count {\????externalfilter-\currentexternalfilter-counter}
+
+\installparameterhandler \????externalfilter \externalfilter@id
+\installparameterhashhandler \????externalfilter \externalfilter@id
+\installsetuphandler \????externalfilter \externalfilter@id
+\installdefinehandler \????externalfilter \externalfilter@id \????externalfilter
+
+\ifx\undefined\setuvalue
+ \def\setuvalue #1{\normalprotected\expandafter \def\csname#1\endcsname}
+ \def\setuevalue #1{\normalprotected\expandafter\edef\csname#1\endcsname}
+\fi
+
+\appendtoks
+ \externalfilter@show_status{defining filter : \currentexternalfilter}%
+ \doifinset{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\ifcsname\externalfilter@count\endcsname \else
+ \expandafter\newcounter\csname\externalfilter@count\endcsname\fi}%
+ \setuevalue{\e!start\currentexternalfilter}{\externalfilter@start[\currentexternalfilter]}%
+ \setuvalue {\e!stop\currentexternalfilter}{\externalfilter@process_filter}%
+ \setuevalue{process\currentexternalfilter file}{\externalfilter@process_file[\currentexternalfilter]}%
+ \setuevalue{process\currentexternalfilter buffer}{\externalfilter@process_buffer[\currentexternalfilter]}%
+ \setuevalue{inline\currentexternalfilter}{\externalfilter@inline[\currentexternalfilter]}%
+\to \everydefineexternalfilter
+
+
+% For backward compatibility
+\let\setupexternalfilters \setupexternalfilter
+
+%D \subject {Messages}
+
+\def\m!externalfilter{t-filter}
+
+\setinterfacemessage\m!externalfilter{title} {\m!externalfilter}
+\setinterfacemessage\m!externalfilter{notfound} {file -- cannot be found}
+\setinterfacemessage\m!externalfilter{missing} {output file missing}
+\setinterfacemessage\m!externalfilter{missing_cache} {cached output file -- missing. Rerunning filter}
+\setinterfacemessage\m!externalfilter{forbidden} {Fatal Error: Cannot use absolute path -- as directory}
+\setinterfacemessage\m!externalfilter{slash} {Appending / to directory -- }
+\setinterfacemessage\m!externalfilter{reuse} {\c!state=\v!stop : Not running filter on file --}
+\setinterfacemessage\m!externalfilter{writedisabled} {\c!write=\v!no : Not writing output for filter --}
+
+
+%D \subject {Tracing Macros}
+
+\newif\iftraceexternalfilters
+
+\unexpanded\def\traceexternalfilters
+ {\traceexternalfilterstrue}
+
+\starttexdefinition externalfilter@show_filenames
+ \writestatus\m!externalfilter{current filter : \currentexternalfilter}
+ \writestatus\m!externalfilter{base file : \externalfilter@base_file}
+ \writestatus\m!externalfilter{input file : \externalfilter@input_file}
+ \writestatus\m!externalfilter{output file : \externalfilter@output_file}
+\stoptexdefinition
+
+\starttexdefinition externalfilter@show_status #1
+ \iftraceexternalfilters
+ \writestatus\m!externalfilter{#1}
+ \fi
+\stoptexdefinition
+
+\starttexdefinition externalfilter@show_filtercommand
+ \writestatus\m!externalfilter{command : \externalfilterparameter\c!filtercommand}
+ \writestatus\m!externalfilter{state : \externalfilterparameter\c!state}
+\stoptexdefinition
+
+%D \section {The main user macros}
+
+\unexpanded\def\externalfilter@start
+ {\bgroup\obeylines\dodoubleargument\externalfilter@start_indeed}
+
+\starttexdefinition externalfilter@start_indeed [#1][#2]
+ % #1 = filter
+ % #2 = options
+ \egroup %\bgroup in \externalfilter@start
+
+ \begingroup % to keep assignments local
+ \edef\currentexternalfilter{#1}
+
+ \setupexternalfilter[#1][\c!name=,#2]
+
+ \externalfilter@set_filenames
+
+ % Capture the contents of the buffer
+ \dostartbuffer[\externalfilter@buffer_name][\e!start#1][\e!stop#1]
+\stoptexdefinition
+
+\unexpanded\def\externalfilter@process_file
+ {\dodoubleargument\externalfilter@process_file_indeed}
+
+\starttexdefinition externalfilter@process_file_indeed [#1][#2]#3
+ % #1 = filter
+ % #2 = options
+ % #3 = filename
+ \begingroup
+
+ \edef\currentexternalfilter{#1}
+ \setupexternalfilter[#1][\c!name=,#2]
+
+ \processcommacommand[\externalfilterparameter{\c!filter\c!setups}]\directsetup
+ \externalfilter@set_directory
+
+ \locatefilepath{#3}
+ \ifx\locatedfilepath\empty
+ % FIXME: Should we declare an error?
+ \edef\externalfilter@input_file{#3}
+ \else
+ \edef\externalfilter@input_file{\pathplusfile\locatedfilepath{#3}}
+ \fi
+ \splitfilename{#3}
+ %NOTE: \edef doesn not work because \splitoffname is not expandable
+ \def\externalfilter@base_file {\splitoffname}
+
+ % The output is always in the directory specified by
+ % \c!directory; even if the input is from some other directory
+ \def\externalfilter@output_file{\externalfilter@get_directory\externalfilterparameter\c!output}
+
+ \iftraceexternalfilters \externalfilter@show_filenames \fi
+ \externalfilter@execute_filter
+ \externalfilter@read_processed_file
+
+ \endgroup
+\stoptexdefinition
+
+\unexpanded\def\externalfilter@process_buffer
+ {\dotripleargument\externalfilter@process_buffer_indeed}
+
+\starttexdefinition externalfilter@process_buffer_indeed [#1][#2][#3]
+ % #1 = filter
+ % #2 = options
+ % #3 = buffer
+ \begingroup
+
+ \edef\currentexternalfilter{#1}
+ \ifthirdargument
+ \setupexternalfilter[#1][\c!name=,#2]
+ \fi
+
+ \externalfilter@set_directory
+
+ \ifthirdargument
+ \edef\externalfilter@buffer_name{#3}
+ \else
+ \edef\externalfilter@buffer_name{#2}
+ \fi
+
+ \externalfilter@set_filenames_extras
+
+ \iftraceexternalfilters \externalfilter@show_filenames \fi
+
+ \externalfilter@process_filter
+
+\stoptexdefinition
+
+\unexpanded\def\externalfilter@inline
+ {\dodoubleargument\externalfilter@inline_indeed}
+
+\starttexdefinition externalfilter@inline_indeed [#1][#2]
+ \begingroup % to keep assignments local
+ \edef\currentexternalfilter{#1}
+
+ \setupexternalfilter[#1][\c!numbering=,\c!name=,\c!location=\v!text,#2]
+
+ \externalfilter@set_filenames
+
+ \pushcatcodetable
+ \doifelse{\externalfilterparameter\c!write}\v!no
+ \externalfilter@inline_write_disabled
+ \externalfilter@inline_write_enabled
+\stoptexdefinition
+
+\starttexdefinition externalfilter@inline_write_disabled
+ \iftraceexternalfilters \showmessage\m!externalfilter{writedisabled} \currentexternalfilter \fi
+
+ \externalfilter@execute_filter
+ \endlinechar\minusone %to prevent line break after reading file
+ \externalfilter@read_processed_file
+
+ % Finalization
+ \doifinset{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\doglobal\incrementvalue\externalfilter@count}
+ \endgroup
+\stoptexdefinition
+
+\starttexdefinition externalfilter@inline_write_enabled
+ \futurelet\next\externalfilter@inline_grabcontent
+\stoptexdefinition
+
+%D \subsubject {Write argument to file verbatim}
+%D
+%D Surprisingly, there is nothing in the core to define a function that write its
+%D argument to a file verbatim. I basically copied the \type{\type} macro.
+
+\starttexdefinition externalfilter@inline_grabcontent
+ \ifx\next\bgroup
+ \expandafter\externalfilter@inline_group
+ \else
+ \expandafter\externalfilter@inline_other
+ \fi
+\stoptexdefinition
+
+\starttexdefinition externalfilter@inline_group
+ \setcatcodetable \externalfilter@read_catcodes
+ \externalfilter@process_inline
+\stoptexdefinition
+
+\starttexdefinition externalfilter@inline_other #1
+ \setcatcodetable \externalfilter@verb_catcodes
+
+ \def\next##1#1{\externalfilter@process_inline{##1}}
+ \next
+\stoptexdefinition
+
+\newwrite\externalfilter@write
+
+\starttexdefinition externalfilter@process_inline #1
+ \immediate\openout \externalfilter@write\externalfilter@input_file
+ \immediate\write \externalfilter@write{\detokenize{#1}}
+ \immediate\closeout\externalfilter@write
+
+ \popcatcodetable
+
+ \externalfilter@execute_filter
+ \endlinechar\minusone %to prevent line break after reading file
+ \externalfilter@read_processed_file
+
+ % Finalization
+ \doifinset{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\doglobal\incrementvalue\externalfilter@count}
+ \endgroup
+\stoptexdefinition
+
+
+%D \section {Helper Functions}
+%D
+%D \subsubject {First and last character of a string}
+
+\def\externalfilter@get_first_character#1%
+ {\externalfilter@get_first_character_indeed#1\relax}
+
+\def\externalfilter@get_first_character_indeed#1#2\relax{#1}
+
+\def\externalfilter@get_last_character#1%
+ {\@EA\externalfilter@get_last_character_indeed#1\relax}
+
+\def\externalfilter@get_last_character_indeed#1#2%
+ {\ifx#2\relax#1\else\@EA\externalfilter@get_last_character_indeed\@EA#2\fi}
+
+%D \subsubject {Setting font and color attributes}
+
+
+\starttexdefinition externalfilter@attributes_start #1#2#3
+ % id style color
+ \edef\externalfilter@attributes_style{\getvalue{#1parameter}{#2}}
+ \edef\externalfilter@attributes_color{\getvalue{#1parameter}{#3}}
+
+ \expandafter\startcolor\expandafter[\externalfilter@attributes_color]
+ \expandafter\doconvertfont{\externalfilter@attributes_style}
+\stoptexdefinition
+
+\def\externalfilter@attributes_stop{\stopcolor}
+
+%D \subsubject {Set the name of output directory}
+
+\starttexdefinition externalfilter@set_directory
+ \edef\externalfilter@get_directory{\externalfilterparameter\c!directory}
+ \doifsomething{\externalfilter@get_directory}\externalfilter@set_directory_indeed
+\stoptexdefinition
+
+\starttexdefinition externalfilter@set_directory_indeed
+ \doif{\externalfilter@get_first_character\externalfilter@get_directory}{/}
+ {\writeline
+ \showmessage\m!externalfilter{forbidden}\externalfilter@get_directory
+ \batchmode
+ \errmessage{}
+ \normalend}
+
+ \doifnot{\externalfilter@get_last_character\externalfilter@get_directory}{/}
+ {\iftraceexternalfilters \showmessage\m!externalfilter{slash}\externalfilter@get_directory \fi
+ \edef\externalfilter@get_directory{\externalfilter@get_directory/}}
+\stoptexdefinition
+
+
+
+%D \subsubject {Set file names}
+%D
+%D \type{\externalfilter@base_file} is the name of the temporary file without
+%D extension. Its actual value depends on the state of \type{cache} key as
+%D well as the value of \type{name} key.
+
+\starttexdefinition externalfilter@set_filenames
+ \processcommacommand[\externalfilterparameter{\c!filter\c!setups}]\directsetup
+ \externalfilter@set_directory
+
+ % Set the name of temp file for the filter
+ \doifinsetelse{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\edef\externalfilter@buffer_name{\externalfilter@temp_prefix-\currentexternalfilter-\csname\externalfilter@count\endcsname}}
+ {\edef\externalfilter@buffer_name{\externalfilter@temp_prefix-\currentexternalfilter}}
+ \doifsomething{\externalfilterparameter\c!name}
+ {\edef\externalfilter@buffer_name{\externalfilter@temp_prefix-\currentexternalfilter-\externalfilterparameter\c!name}}
+ \doif{\externalfilterparameter\c!write}\v!no
+ {\edef\externalfilter@buffer_name{\externalfilter@temp_prefix-\currentexternalfilter-\externalfilterparameter{\c!cache\c!option}}}
+
+ \externalfilter@set_filenames_extras
+
+ \iftraceexternalfilters \externalfilter@show_filenames \fi
+\stoptexdefinition
+
+\starttexdefinition externalfilter@set_filenames_extras
+ % The following macros are useful for filter= and filtercommand= options
+ % The basename of the external file
+ \edef\externalfilter@base_file {\jobname-\externalfilter@buffer_name}
+
+ % Append directory name to the name of the input file
+ \edef\externalfilter@input_file {\externalfilter@get_directory\externalfilter@base_file.\f!temporaryextension}
+
+ % Append directory name to the name of the output file
+ \edef\externalfilter@output_file{\externalfilter@get_directory\externalfilterparameter\c!output}
+
+ % In MkII, the buffer output is written to \TEXbufferfile{buffername}. So we
+ % redefine \TEXbufferfile to
+
+ \edef\TEXbufferfile##1{\externalfilter@input_file}
+
+\stoptexdefinition
+
+
+
+%D \subsubject {Process Filter}
+%D
+%D Execute filter, read the output and do book-keeping if needed.
+
+\starttexdefinition externalfilter@process_filter
+ \externalfilter@execute_filter
+ \externalfilter@read_processed_file
+
+ % Finalization
+ \doifinset{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\doglobal\incrementvalue\externalfilter@count}
+
+ \doif{\externalfilterparameter\c!location}\v!paragraph
+ {\expanded{\checknextindentation[\externalfilterparameter\c!indentnext]}%
+ \dorechecknextindentation}
+ \endgroup
+\stoptexdefinition
+
+%D \subsubject {Execute Filter}
+
+\starttexdefinition externalfilter@execute_filter
+ \iftraceexternalfilters \externalfilter@show_filtercommand \fi
+
+ \doifelse{\externalfilterparameter\c!cache}\v!yes
+ {\doifelse{\externalfilterparameter\c!state}\v!stop
+ {\showmessage\m!externalfilter{reuse}\externalfilter@input_file}
+ {\doifmode{*first}
+ {\doiffileelse{\externalfilter@output_file}
+ {\doifnot{\externalfilterparameter\c!write}\v!no
+ {\executesystemcommand
+ {mtxrun --ifchanged=\externalfilter@input_file\space
+ --direct \externalfilterparameter\c!filtercommand}}}}
+ {\showmessage\m!externalfilter{missing_cache}\externalfilter@output_file
+ \executesystemcommand
+ {mtxrun --direct \externalfilterparameter\c!filtercommand}}}}
+ {\doifelse{\externalfilterparameter\c!cache}\v!force
+ {\doifmode{*first}
+ {\executesystemcommand
+ {mtxrun --direct \externalfilterparameter\c!filtercommand}}}
+ {\executesystemcommand
+ {\externalfilterparameter\c!filtercommand}}}
+\stoptexdefinition
+
+%D \subsubject {Read output}
+
+\starttexdefinition externalfilter@read_processed_file
+ \externalfilter@show_filenames
+ \doif{\externalfilterparameter\c!read}\v!yes
+ {\doiffileelse{\externalfilter@output_file}
+ {\externalfilter@read_processed_file_indeed}
+ {\showmessage\m!externalfilter{notfound}\externalfilter@output_file
+ \externalfilter@show_filenames
+ \blank
+ {\tttf [[\getmessage\m!externalfilter{missing}]]}
+ \blank}}
+\stoptexdefinition
+
+\starttexdefinition externalfilter@read_processed_file_indeed
+ \doifelse{\externalfilterparameter\c!location}\v!paragraph
+ {\blank[\externalfilterparameter\c!spacebefore]
+ \doifsomething{\externalfilterparameter\c!align}
+ {\setupalign[\externalfilterparameter\c!align]}
+ \externalfilterparameter\c!before}
+ {\externalfilterparameter\c!left}
+
+ \begingroup
+ \externalfilter@attributes_start \externalfilter@id \c!style \c!color
+ \processcommacommand[\externalfilterparameter\c!setups]\directsetup
+ \externalfilterparameter\c!readcommand\externalfilter@output_file
+ \externalfilter@attributes_stop
+ \endgroup
+
+ \doifelse{\externalfilterparameter\c!location}\v!paragraph
+ {\externalfilterparameter\c!after
+ \par\blank[\externalfilterparameter\c!spaceafter]}%
+ {\externalfilterparameter\c!right}
+\stoptexdefinition
+
+%D \section {Default Values}
+
+\setupexternalfilters
+ [
+ \c!location=\v!paragraph,
+ \c!before=,
+ \c!after=,
+ \c!left=\externalfilterparameter\c!before,
+ \c!right=\externalfilterparameter\c!after,
+ \c!spacebefore=,
+ \c!spaceafter=,
+ \c!style=,
+ \c!color=,
+ \c!indentnext=\v!auto,
+ \c!align=,
+ \c!setups=,
+ \c!continue=\v!no,
+ \c!cache=\externalfilterparameter\c!continue, % for backward compatibility
+ \c!cache\c!option=,
+ \c!read=\v!yes,
+ \c!readcommand=\ReadFile,
+ \c!directory=,
+ \c!output=\externalfilterbasefile.tex,
+ \c!filter=,
+ \c!filtercommand={\externalfilterparameter\c!filter\space \externalfilter@input_file},
+ \c!buffer\c!before=,
+ \c!buffer\c!after=,
+ ]
+
+\def\externalfilterbasefile {\externalfilter@base_file}
+\def\externalfilterinputfile {\externalfilter@input_file}
+\def\externalfilteroutputfile{\externalfilter@output_file}
+
+% t-syntax-groups still uses this.
+\def\externalfilter@name {\currentexternalfilter}
+\def\externalfilter@namespace {\????externalfilter}
+% Default value
+\def\externalfilter@input_file {}
+\def\externalfilter@output_file{}
+
+\protectmodulecatcodes
+\stopmodule
diff --git a/macros/context/contrib/context-filter/tex/context/third/filter/t-filter.mkiv b/macros/context/contrib/context-filter/tex/context/third/filter/t-filter.mkiv
new file mode 100644
index 0000000000..408a3d96b3
--- /dev/null
+++ b/macros/context/contrib/context-filter/tex/context/third/filter/t-filter.mkiv
@@ -0,0 +1,543 @@
+%D \module
+%D [ file=t-filter,
+%D version=2018.08.10,
+%D title=\CONTEXT\ User Module,
+%D subtitle=Filter,
+%D author=Aditya Mahajan,
+%D date=\currentdate,
+%D copyright=Aditya Mahajan,
+%D email=adityam <at> ieee <dot> org,
+%D license=Simplified BSD License]
+
+\writestatus{loading}{Filter (ver: 2018.08.10)}
+
+\startmodule [filter]
+\usemodule [module-catcodes]
+\unprotect
+
+% \f!temporaryextension was removed from the core in July 2012.
+\ifx\f!temporaryextension\undefined \def\f!temporaryextension{tmp} \fi
+
+%D \subject {Interface}
+%D
+%D Using interface constants allows one to use \type{\c!filter} etc. in
+%D the module definition, and thereby reduces the risk of a typo.
+%D Currently, only English names are provided. If someone wants a
+%D multi-lingual interface, let me know and I will add other language
+%D names as well,
+
+\startinterface all
+ \setinterfaceconstant {filter} {filter}
+ \setinterfaceconstant {filtercommand} {filtercommand}
+ \setinterfaceconstant {output} {output}
+ \setinterfaceconstant {purge} {purge}
+ \setinterfaceconstant {read} {read}
+ \setinterfaceconstant {write} {write}
+ \setinterfaceconstant {readcommand} {readcommand}
+ \setinterfaceconstant {taglabel} {taglabel}
+\stopinterface
+
+
+%D \subject {Name space}
+%D
+%D We use logical names to easily catch typos.
+
+\def\externalfilter@id {externalfilter}
+
+\def\externalfilter@temp_prefix {temp}
+\def\externalfilter@count {\????externalfilter-\currentexternalfilter-counter}
+
+\definenamespace
+ [externalfilter]
+ [\c!type=module,
+ \c!name=externalfilter,
+ \c!command=\v!yes,
+ setup=\v!list,
+ \c!style=\v!yes,
+ \s!parent=externalfilter]
+
+\appendtoks
+ \externalfilter@show_status{defining filter : \currentexternalfilter}%
+ \doifinset{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\ifcsname\externalfilter@count\endcsname \else
+ \expandafter\newcounter\csname\externalfilter@count\endcsname\fi}%
+ \setuevalue{\e!start\currentexternalfilter}{\externalfilter@start[\currentexternalfilter]}%
+ \setuvalue {\e!stop\currentexternalfilter}{}%
+ \setuevalue{process\currentexternalfilter file}{\externalfilter@process_file[\currentexternalfilter]}%
+ \setuevalue{process\currentexternalfilter buffer}{\externalfilter@process_buffer[\currentexternalfilter]}%
+ \setuevalue{inline\currentexternalfilter}{\externalfilter@inline[\currentexternalfilter]}%
+\to \everydefineexternalfilter
+
+% For backward compatibility
+\let\setupexternalfilters \setupexternalfilter
+
+%D \subject {Messages}
+
+\def\m!externalfilter{t-filter}
+
+\setinterfacemessage\m!externalfilter{title} {\m!externalfilter}
+\setinterfacemessage\m!externalfilter{notfound} {file -- cannot be found}
+\setinterfacemessage\m!externalfilter{missing} {output file missing}
+\setinterfacemessage\m!externalfilter{missing_cache} {cached output file -- missing. Rerunning filter}
+\setinterfacemessage\m!externalfilter{forbidden} {Fatal Error: Cannot use absolute path -- as directory}
+\setinterfacemessage\m!externalfilter{unwriteable} {Fatal Error: Cannot write to file --}
+\setinterfacemessage\m!externalfilter{slash} {Appending / to directory -- }
+\setinterfacemessage\m!externalfilter{reuse} {\c!state=\v!stop : Not running filter on file --}
+\setinterfacemessage\m!externalfilter{writedisabled} {\c!write=\v!no : Not writing output for filter --}
+
+
+%D \subject {Tracing Macros}
+
+\newif\iftraceexternalfilters
+
+\unexpanded\def\traceexternalfilters
+ {\traceexternalfilterstrue
+ \enabletrackers[graphic.runfile]}
+
+\starttexdefinition externalfilter@show_filenames
+ \writestatus\m!externalfilter{current filter : \currentexternalfilter}
+ \writestatus\m!externalfilter{base file : \externalfilter@base_file}
+ \writestatus\m!externalfilter{input file : \externalfilter@input_file}
+ \writestatus\m!externalfilter{output file : \externalfilter@output_file}
+\stoptexdefinition
+
+\starttexdefinition externalfilter@show_status #1
+ \iftraceexternalfilters
+ \writestatus\m!externalfilter{#1}
+ \fi
+\stoptexdefinition
+
+\starttexdefinition externalfilter@show_filtercommand
+ \writestatus\m!externalfilter{command : \externalfilterparameter\c!filtercommand}
+ \writestatus\m!externalfilter{state : \externalfilterparameter\c!state}
+\stoptexdefinition
+
+%D \section {The main user macros}
+
+\unexpanded\def\externalfilter@start
+ {\bgroup\obeylines\dodoubleargument\externalfilter@start_indeed}
+
+\starttexdefinition externalfilter@start_indeed [#1][#2]
+ % #1 = filter
+ % #2 = options
+ \egroup %\bgroup in \externalfilter@start
+
+ \begingroup % to keep assignments local
+ \edef\currentexternalfilter{#1}
+
+ \setupexternalfilter[#1][\c!name=,#2]
+
+ \externalfilter@set_filenames
+
+ % Capture the contents of the buffer
+ \edef\p_strip{\getvalue{\externalfilter@id-\c!strip-\externalfilterparameter\c!strip}}
+ \buff_pickup{\externalfilter@buffer_name}{\e!start#1}{\e!stop#1}{}{\externalfilter@process_filter}{\p_strip}
+\stoptexdefinition
+
+\setvalue{\externalfilter@id-\c!strip-}{\zerocount}
+\setvalue{\externalfilter@id-\c!strip-\v!off}{\zerocount}
+\setvalue{\externalfilter@id-\c!strip-\v!on}{\plusone}
+
+\setvalue{\externalfilter@id-\c!strip-\v!no}{\zerocount}
+\setvalue{\externalfilter@id-\c!strip-\v!yes}{\plusone}
+
+\unexpanded\def\externalfilter@process_file
+ {\dodoubleargument\externalfilter@process_file_indeed}
+
+\starttexdefinition externalfilter@process_file_indeed [#1][#2]#3
+ % #1 = filter
+ % #2 = options
+ % #3 = filename
+ \begingroup
+
+ \edef\currentexternalfilter{#1}
+ \setupexternalfilter[#1][\c!name=,#2]
+
+ \processcommacommand[\externalfilterparameter{\c!filter\c!setups}]\directsetup
+ \externalfilter@set_directory
+
+ \edef\externalfilter@input_file{\externalfilter@any_filename{#3}}
+ \splitfilename{#3}
+ %NOTE: \edef doesn not work because \splitoffname is not expandable
+ \def\externalfilter@base_file {\splitoffname}
+
+ % The output is always in the directory specified by
+ % \c!directory; even if the input is from some other directory
+ \def\externalfilter@output_file{\externalfilter@get_directory\externalfilterparameter\c!output}
+
+ \iftraceexternalfilters \externalfilter@show_filenames \fi
+ \externalfilter@execute_filter
+ \externalfilter@read_processed_file
+
+ \endgroup
+\stoptexdefinition
+
+\unexpanded\def\externalfilter@process_buffer
+ {\dotripleargument\externalfilter@process_buffer_indeed}
+
+\starttexdefinition externalfilter@process_buffer_indeed [#1][#2][#3]
+ % #1 = filter
+ % #2 = options
+ % #3 = buffer
+ \begingroup
+
+ \edef\currentexternalfilter{#1}
+ \ifthirdargument
+ \setupexternalfilter[#1][\c!name=,#2]
+ \fi
+
+ \processcommacommand[\externalfilterparameter{\c!filter\c!setups}]\directsetup
+ \externalfilter@set_directory
+
+ \ifthirdargument
+ \edef\externalfilter@buffer_name{#3}
+ \else
+ \edef\externalfilter@buffer_name{#2}
+ \fi
+
+ \externalfilter@set_filenames_extras
+
+ \iftraceexternalfilters \externalfilter@show_filenames \fi
+
+ \externalfilter@process_filter
+
+\stoptexdefinition
+
+\unexpanded\def\externalfilter@inline
+ {\dodoubleargument\externalfilter@inline_indeed}
+
+\starttexdefinition externalfilter@inline_indeed [#1][#2]
+ \begingroup % to keep assignments local
+ \edef\currentexternalfilter{#1}
+
+ \setupexternalfilter[#1][\c!numbering=,\c!name=,\c!location=\v!text,#2]
+
+ \externalfilter@set_filenames
+ \doifelse{\externalfilterparameter\c!write}\v!no
+ \externalfilter@inline_write_disabled
+ \externalfilter@inline_write_enabled
+\stoptexdefinition
+
+\starttexdefinition externalfilter@inline_write_disabled
+ \iftraceexternalfilters \showmessage\m!externalfilter{writedisabled} \currentexternalfilter \fi
+
+ \externalfilter@execute_filter
+ \endlinechar\minusone %to prevent line break after reading file
+ \externalfilter@read_processed_file
+
+ % Finalization
+ \doifinset{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\doglobal\incrementvalue\externalfilter@count}
+ \endgroup
+\stoptexdefinition
+
+\starttexdefinition externalfilter@inline_write_enabled
+ \pushcatcodetable
+ \futurelet\next\externalfilter@inline_grabcontent
+\stoptexdefinition
+
+%D \subsubject {Write argument to file verbatim}
+%D
+%D Surprisingly, there is nothing in the core to define a function that write its
+%D argument to a file verbatim. I basically copied the \type{\type} macro.
+
+\starttexdefinition externalfilter@inline_grabcontent
+ \ifx\next\bgroup
+ \expandafter\externalfilter@inline_group
+ \else
+ \expandafter\externalfilter@inline_other
+ \fi
+\stoptexdefinition
+
+\starttexdefinition externalfilter@inline_group
+ \setcatcodetable \externalfilter@read_catcodes
+ \externalfilter@process_inline
+\stoptexdefinition
+
+\starttexdefinition externalfilter@inline_other #1
+ \setcatcodetable \externalfilter@verb_catcodes
+
+ \def\next##1#1{\externalfilter@process_inline{##1}}
+ \next
+\stoptexdefinition
+
+\newwrite\externalfilter@write
+
+\starttexdefinition externalfilter@process_inline #1
+ \immediate\openout \externalfilter@write\externalfilter@input_file
+ \immediate\write \externalfilter@write{\detokenize{#1}}
+ \immediate\closeout\externalfilter@write
+
+ \popcatcodetable
+
+ \externalfilter@execute_filter
+ \endlinechar\minusone %to prevent line break after reading file
+ \externalfilter@read_processed_file
+
+ \iftraceexternalfilters \else
+ \doif{\externalfilterparameter\c!purge}\v!yes
+ {\ctxlua{os.remove(\!!bs\externalfilter@input_file\!!es)}}
+ \fi
+ % Finalization
+ \doifinset{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\doglobal\incrementvalue\externalfilter@count}
+ \endgroup
+\stoptexdefinition
+
+
+%D \section {Helper Functions}
+%D
+%D \subsubject {First and last character of a string}
+
+\def\externalfilter@get_first_character#1%
+ {\externalfilter@get_first_character_indeed#1\relax}
+
+\def\externalfilter@get_first_character_indeed#1#2\relax{#1}
+
+\def\externalfilter@get_last_character#1%
+ {\@EA\externalfilter@get_last_character_indeed#1\relax}
+
+\def\externalfilter@get_last_character_indeed#1#2%
+ {\ifx#2\relax#1\else\@EA\externalfilter@get_last_character_indeed\@EA#2\fi}
+
+%D \subsubject {Setting font and color attributes}
+%D I want to use the same interface for MkII and MkIV
+
+\starttexdefinition externalfilter@attributes_start #1#2#3
+ % id style color
+ \getvalue{use#1styleandcolor}{#2}{#3}
+\stoptexdefinition
+
+\def\externalfilter@attributes_stop{}
+
+%D \subsubject {Set the name of output directory}
+
+\starttexdefinition externalfilter@set_directory
+ \edef\externalfilter@get_directory{\externalfilterparameter\c!directory}
+ \doifsomething{\externalfilter@get_directory}\externalfilter@set_directory_indeed
+\stoptexdefinition
+
+\starttexdefinition externalfilter@set_directory_indeed
+ \doif{\externalfilter@get_first_character\externalfilter@get_directory}{/}
+ {\writeline
+ \showmessage\m!externalfilter{forbidden}\externalfilter@get_directory
+ \batchmode
+ \errmessage{}
+ \normalend}
+
+ \doifnot{\externalfilter@get_last_character\externalfilter@get_directory}{/}
+ {\iftraceexternalfilters \showmessage\m!externalfilter{slash}\externalfilter@get_directory \fi
+ \edef\externalfilter@get_directory{\externalfilter@get_directory/}}
+\stoptexdefinition
+
+%D \subsubject {Check if file is writeable}
+
+\starttexdefinition externalfilter@check_writable #1
+
+ \ctxcommand{doifnot(file.is_writable("#1"))}
+ {\showmessage\m!externalfilter{unwriteable}{#1}
+ \batchmode
+ \errmessage{}
+ \normalend}
+\stoptexdefinition
+
+%D \subsubject {Find file name (with search in \usepath)}
+%D
+%D The `\locfilename` macro does not search in the path specified by `\usepath`.
+%D So, we define a macro that is based on `\readfile`.
+
+\def\externalfilter@any_filename#1%
+ {\clf_getreadfilename{any}{.}{#1}}
+
+%D \subsubject {Set file names}
+%D
+%D \type{\externalfilter@base_file} is the name of the temporary file without
+%D extension. Its actual value depends on the state of \type{cache} key as
+%D well as the value of \type{name} key.
+
+\starttexdefinition externalfilter@set_filenames
+ \processcommacommand[\externalfilterparameter{\c!filter\c!setups}]\directsetup
+ \externalfilter@set_directory
+
+ % Set the name of temp file for the filter
+ \doifinsetelse{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\edef\externalfilter@buffer_name{\externalfilter@temp_prefix-\currentexternalfilter-\csname\externalfilter@count\endcsname}}
+ {\edef\externalfilter@buffer_name{\externalfilter@temp_prefix-\currentexternalfilter}}
+ \doifsomething{\externalfilterparameter\c!name}
+ {\edef\externalfilter@buffer_name{\externalfilter@temp_prefix-\currentexternalfilter-\externalfilterparameter\c!name}}
+ \doif{\externalfilterparameter\c!write}\v!no
+ {\edef\externalfilter@buffer_name{\externalfilter@temp_prefix-\currentexternalfilter-\externalfilterparameter{\c!cache\c!option}}}
+
+ \externalfilter@set_filenames_extras
+
+ \iftraceexternalfilters \externalfilter@show_filenames \fi
+\stoptexdefinition
+
+\starttexdefinition externalfilter@set_filenames_extras
+ % The following macros are useful for filter= and filtercommand= options
+ % The basename of the external file
+ \edef\externalfilter@base_file {\jobname-\externalfilter@buffer_name}
+
+ % Append directory name to the name of the input file
+ \edef\externalfilter@input_file {\externalfilter@get_directory\externalfilter@base_file.\f!temporaryextension}
+
+ % Append directory name to the name of the output file
+ \edef\externalfilter@output_file{\externalfilter@get_directory\externalfilterparameter\c!output}
+\stoptexdefinition
+
+
+
+%D \subsubject {Process Filter}
+%D
+%D Execute filter, read the output and do book-keeping if needed.
+
+\starttexdefinition externalfilter@process_filter
+ % By defualt, buffers are in memory in MkIV. So, we save them to disk
+ %
+ % \savebuffer[\externalfilter@buffer_name][\externalfilter@input_file]
+ %
+ % We can also save a list of buffers to a file. The empty
+ % elements of the list are ignored. So, instead we use the following
+ %
+ % \savebuffer[\externalfilterparameter{\c!buffer\c!before},
+ % \externalfilter@buffer_name,
+ % \externalfilterparameter{\c!buffer\c!after}]
+ % [\externalfilter@input_file]}
+ %
+ % but using this method we cannot save the file in another directory.
+ % So, we use the key-value interface for \savebuffer.
+ \externalfilter@check_writable \externalfilter@input_file
+ \savebuffer
+ [
+ \c!list={\externalfilterparameter{\c!buffer\c!before},
+ \externalfilter@buffer_name,
+ \externalfilterparameter{\c!buffer\c!after}},
+ \c!file={\externalfilter@input_file},
+ \c!prefix=\v!no,
+ ]
+ \externalfilter@execute_filter
+ \externalfilter@read_processed_file
+ \iftraceexternalfilters \else
+ \doif{\externalfilterparameter\c!purge}\v!yes
+ {\ctxlua{os.remove(\!!bs\externalfilter@input_file\!!es)}}
+ \fi
+
+ % Finalization
+ \doifinset{\externalfilterparameter\c!cache}{\v!yes,\v!force}
+ {\doglobal\incrementvalue\externalfilter@count}
+
+ \doif{\externalfilterparameter\c!location}\v!paragraph
+ {\useindentnextparameter\externalfilterparameter
+ \aftergroup\dorechecknextindentation}
+ \endgroup
+\stoptexdefinition
+
+%D \subsubject {Execute Filter}
+
+% In MkIV, we use job.files.run to check if a file has changed. This function
+% writes the md5 sum to the tuc file rather than to an external file. So, we
+% must not check for the \type{*first} mode (otherwise the md5 sum is overridden
+% in the next run.
+\starttexdefinition externalfilter@execute_filter
+
+ \externalfilter@check_writable \externalfilter@output_file
+ \iftraceexternalfilters \externalfilter@show_filtercommand \fi
+
+ \doifelsenothing{\externalfilter@input_file}
+ {\showmessage\m!externalfilter{missing}\externalfilter@input_file}
+ {\doifelse{\externalfilterparameter\c!cache}\v!yes
+ {\doifelse{\externalfilterparameter\c!state}\v!stop
+ {\showmessage\m!externalfilter{reuse}\externalfilter@input_file}
+ {\doiffileelse{\externalfilter@output_file}
+ {\doifnot{\externalfilterparameter\c!write}\v!no
+ % When write=no is selected, % base_name=....-cacheoption...
+ % Since there is no input, caching is simply achieved
+ % using the filename.
+ {\ctxlua{job.files.run(\!!bs\externalfilter@input_file\!!es,
+ \!!bs\externalfilterparameter\c!filtercommand\!!es)}}}
+ {\showmessage\m!externalfilter{missing_cache}\externalfilter@output_file
+ \executesystemcommand
+ {\externalfilterparameter\c!filtercommand}}}}
+ {\executesystemcommand
+ {\externalfilterparameter\c!filtercommand}}}
+\stoptexdefinition
+
+%D \subsubject {Read output}
+
+\starttexdefinition externalfilter@read_processed_file
+ \doif{\externalfilterparameter\c!read}\v!yes
+ {\doiffileelse{\externalfilter@output_file}
+ {\externalfilter@read_processed_file_indeed}
+ {\showmessage\m!externalfilter{notfound}\externalfilter@output_file
+ \externalfilter@show_filenames
+ \blank
+ {\tttf [[\getmessage\m!externalfilter{missing}]]}
+ \blank}}
+\stoptexdefinition
+
+\starttexdefinition externalfilter@read_processed_file_indeed
+ \doifelse{\externalfilterparameter\c!location}\v!paragraph
+ {\blank[\externalfilterparameter\c!spacebefore]
+ \usealignparameter\externalfilterparameter
+ \externalfilterparameter\c!before}
+ {\externalfilterparameter\c!left}
+
+ \begingroup
+ \dostarttagged{\externalfilterparameter\c!taglabel}\currentexternalfilter
+ \useexternalfilterstyleandcolor\c!style\c!color
+ \processcommacommand[\externalfilterparameter\c!setups]\directsetup
+ \externalfilterparameter\c!readcommand\externalfilter@output_file
+ \dostoptagged
+ \endgroup
+
+ \doifelse{\externalfilterparameter\c!location}\v!paragraph
+ {\externalfilterparameter\c!after
+ \par\blank[\externalfilterparameter\c!spaceafter]}
+ {\externalfilterparameter\c!right}
+\stoptexdefinition
+
+%D \section {Default Values}
+
+\setupexternalfilters
+ [
+ \c!location=\v!paragraph,
+ \c!before=,
+ \c!after=,
+ \c!left=,
+ \c!right=,
+ \c!spacebefore=,
+ \c!spaceafter=,
+ \c!style=,
+ \c!color=,
+ \c!indentnext=\v!auto,
+ \c!align=,
+ \c!setups=,
+ \c!continue=\v!no,
+ \c!cache=\externalfilterparameter\c!continue, % for backward compatibility
+ \c!cache\c!option=,
+ \c!read=\v!yes,
+ \c!strip=\v!yes,
+ \c!readcommand=\ReadFile,
+ \c!directory=,
+ \c!purge=\v!yes,
+ \c!output=\externalfilterbasefile.tex,
+ \c!filter=,
+ \c!filtercommand={\externalfilterparameter\c!filter\space \externalfilter@input_file},
+ \c!buffer\c!before=,
+ \c!buffer\c!after=,
+ \c!taglabel=\externalfilter@id,
+ ]
+
+\def\externalfilterbasefile {\externalfilter@base_file}
+\def\externalfilterinputfile {\externalfilter@input_file}
+\def\externalfilteroutputfile{\externalfilter@output_file}
+
+% t-syntax-groups still uses this.
+\def\externalfilter@name {\currentexternalfilter}
+
+% Default value
+\def\externalfilter@input_file {}
+\def\externalfilter@output_file{}
+
+\protect
+\stopmodule
diff --git a/macros/context/contrib/context-filter/tex/context/third/filter/t-module-catcodes.mkii b/macros/context/contrib/context-filter/tex/context/third/filter/t-module-catcodes.mkii
new file mode 100644
index 0000000000..052c9fca5b
--- /dev/null
+++ b/macros/context/contrib/context-filter/tex/context/third/filter/t-module-catcodes.mkii
@@ -0,0 +1,101 @@
+%D \module
+%D [ file=t-module-catcodes,
+%D version=2011.12.17,
+%D title=\CONTEXT\ User Module,
+%D subtitle=Module Catcodes,
+%D author=Aditya Mahajan,
+%D date=\currentdate,
+%D copyright=Aditya Mahajan,
+%D email=adityam <at> ieee <dot> org,
+%D license=Simplified BSD License]
+%D
+%D MkII and MkIV use different names and values for commonly used catcode
+%D tables. So, I define a private copy of the catcode tables that are needed
+%D by the filter and vim modules.
+
+
+\writestatus{loading}{Module Catcodes (ver: 2011.12.17)}
+
+\startmodule [module-catcodes]
+
+\newcatcodetable \modulecatcodes
+\startcatcodetable \modulecatcodes % same as \prtcatcodes
+ \catcode\tabasciicode \spacecatcode
+ \catcode\endoflineasciicode \endoflinecatcode
+ \catcode\formfeedasciicode \endoflinecatcode
+ \catcode\spaceasciicode \spacecatcode
+ \catcode\endoffileasciicode \ignorecatcode
+ \catcode\circumflexasciicode\superscriptcatcode
+% \catcode\underscoreasciicode\subscriptcatcode
+ \catcode\underscoreasciicode\lettercatcode
+ \catcode\ampersandasciicode \alignmentcatcode
+ \catcode\backslashasciicode \escapecatcode
+ \catcode\leftbraceasciicode \begingroupcatcode
+ \catcode\rightbraceasciicode\endgroupcatcode
+ \catcode\dollarasciicode \mathshiftcatcode
+ \catcode\hashasciicode \parametercatcode
+ \catcode\commentasciicode \commentcatcode
+ \catcode`\@ \lettercatcode
+ \catcode`\! \lettercatcode
+ \catcode`\? \lettercatcode
+ \catcode\tildeasciicode \activecatcode
+ \catcode\barasciicode \activecatcode
+\stopcatcodetable
+
+\def\unprotectmodulecatcodes
+ {\pushcatcodetable
+ \setcatcodetable\modulecatcodes}
+
+\def\protectmodulecatcodes
+ {\popcatcodetable}
+
+\unprotectmodulecatcodes
+
+%D The following catcode tables are used for reading
+%D and writing to files
+
+\newcatcodetable \externalfilter@read_catcodes
+\newcatcodetable \externalfilter@minimal_catcodes
+\newcatcodetable \externalfilter@verb_catcodes
+
+\startcatcodetable \externalfilter@read_catcodes % same as typcatcodesa
+ \catcode\tabasciicode = \othercatcode
+ \catcode\endoflineasciicode = \othercatcode
+ \catcode\formfeedasciicode = \othercatcode
+ \catcode\spaceasciicode = \othercatcode
+ \catcode\endoffileasciicode = \othercatcode
+ \catcode\leftbraceasciicode = \begingroupcatcode
+ \catcode\rightbraceasciicode = \endgroupcatcode
+\stopcatcodetable
+
+\startcatcodetable \externalfilter@minimal_catcodes % for reading t-vim output
+ \catcode\backslashasciicode = \escapecatcode
+ \catcode\leftbraceasciicode = \begingroupcatcode
+ \catcode\rightbraceasciicode = \endgroupcatcode
+ \catcode\endoflineasciicode = \activecatcode
+ \catcode\formfeedasciicode = \activecatcode
+ \catcode\spaceasciicode = \activecatcode
+\stopcatcodetable
+
+\startcatcodetable \externalfilter@verb_catcodes % same as vrbcatcodes
+ \catcode\tabasciicode = \othercatcode
+ \catcode\endoflineasciicode = \othercatcode
+ \catcode\formfeedasciicode = \othercatcode
+ \catcode\spaceasciicode = \othercatcode
+ \catcode\endoffileasciicode = \othercatcode
+\stopcatcodetable
+
+
+
+\ifdefined \activeendoflinetoken \else
+ % from catc-ini.mkiv
+ \bgroup
+ \catcode \tabasciicode \activecatcode \gdef\activetabtoken {^^I}
+ \gdef\outputnewlinechar {^^J}
+ \catcode \formfeedasciicode \activecatcode \gdef\activeformfeedtoken {^^L}
+ \catcode \endoflineasciicode \activecatcode \gdef\activeendoflinetoken{^^M}
+ \egroup
+\fi
+
+\protectmodulecatcodes
+\stopmodule
diff --git a/macros/context/contrib/context-filter/tex/context/third/filter/t-module-catcodes.mkiv b/macros/context/contrib/context-filter/tex/context/third/filter/t-module-catcodes.mkiv
new file mode 100644
index 0000000000..60e25cc05e
--- /dev/null
+++ b/macros/context/contrib/context-filter/tex/context/third/filter/t-module-catcodes.mkiv
@@ -0,0 +1,89 @@
+%D \module
+%D [ file=t-module-catcodes,
+%D version=2018.04.16,
+%D title=\CONTEXT\ User Module,
+%D subtitle=Module Catcodes,
+%D author=Aditya Mahajan,
+%D date=\currentdate,
+%D copyright=Aditya Mahajan,
+%D email=adityam <at> ieee <dot> org,
+%D license=Simplified BSD License]
+%D
+%D MkII and MkIV use different names and values for commonly used catcode
+%D tables. So, I define a private copy of the catcode tables that are needed
+%D by the filter and vim modules.
+
+
+\writestatus{loading}{Module Catcodes (ver: 2018.04.16)}
+
+\startmodule [module-catcodes]
+
+\newcatcodetable \modulecatcodes
+\startcatcodetable \modulecatcodes % same as \prtcatcodes
+ \catcode\tabasciicode \spacecatcode
+ \catcode\endoflineasciicode \endoflinecatcode
+ \catcode\formfeedasciicode \endoflinecatcode
+ \catcode\spaceasciicode \spacecatcode
+ \catcode\endoffileasciicode \ignorecatcode
+ \catcode\circumflexasciicode\superscriptcatcode
+% \catcode\underscoreasciicode\subscriptcatcode
+ \catcode\underscoreasciicode\lettercatcode
+ \catcode\ampersandasciicode \alignmentcatcode
+ \catcode\backslashasciicode \escapecatcode
+ \catcode\leftbraceasciicode \begingroupcatcode
+ \catcode\rightbraceasciicode\endgroupcatcode
+ \catcode\dollarasciicode \mathshiftcatcode
+ \catcode\hashasciicode \parametercatcode
+ \catcode\commentasciicode \commentcatcode
+ \catcode`\@ \lettercatcode
+ \catcode`\! \lettercatcode
+ \catcode`\? \lettercatcode
+ \catcode\tildeasciicode \activecatcode
+ \catcode\barasciicode \activecatcode
+\stopcatcodetable
+
+\def\unprotectmodulecatcodes
+ {\pushcatcodetable
+ \setcatcodetable\modulecatcodes}
+
+\def\protectmodulecatcodes
+ {\popcatcodetable}
+
+\unprotectmodulecatcodes
+
+%D The following catcode tables are used for reading
+%D and writing to files
+
+\newcatcodetable \externalfilter@read_catcodes
+\newcatcodetable \externalfilter@minimal_catcodes
+\newcatcodetable \externalfilter@verb_catcodes
+
+\startcatcodetable \externalfilter@read_catcodes % same as typcatcodesa
+ \catcode\tabasciicode = \othercatcode
+ \catcode\endoflineasciicode = \othercatcode
+ \catcode\formfeedasciicode = \othercatcode
+ \catcode\spaceasciicode = \othercatcode
+ \catcode\endoffileasciicode = \othercatcode
+ \catcode\leftbraceasciicode = \begingroupcatcode
+ \catcode\rightbraceasciicode = \endgroupcatcode
+\stopcatcodetable
+
+\startcatcodetable \externalfilter@minimal_catcodes % for reading t-vim output
+ \catcode\backslashasciicode = \escapecatcode
+ \catcode\leftbraceasciicode = \begingroupcatcode
+ \catcode\rightbraceasciicode = \endgroupcatcode
+ \catcode\endoflineasciicode = \activecatcode
+ \catcode\formfeedasciicode = \othercatcode
+ \catcode\spaceasciicode = \activecatcode
+\stopcatcodetable
+
+\startcatcodetable \externalfilter@verb_catcodes % same as vrbcatcodes
+ \catcode\tabasciicode = \othercatcode
+ \catcode\endoflineasciicode = \othercatcode
+ \catcode\formfeedasciicode = \othercatcode
+ \catcode\spaceasciicode = \othercatcode
+ \catcode\endoffileasciicode = \othercatcode
+\stopcatcodetable
+
+\protectmodulecatcodes
+\stopmodule