summaryrefslogtreecommitdiff
path: root/macros/generic/expkv-bundle/comparison.tex
diff options
context:
space:
mode:
Diffstat (limited to 'macros/generic/expkv-bundle/comparison.tex')
-rw-r--r--macros/generic/expkv-bundle/comparison.tex246
1 files changed, 246 insertions, 0 deletions
diff --git a/macros/generic/expkv-bundle/comparison.tex b/macros/generic/expkv-bundle/comparison.tex
new file mode 100644
index 0000000000..7736f0a4f1
--- /dev/null
+++ b/macros/generic/expkv-bundle/comparison.tex
@@ -0,0 +1,246 @@
+\section{Comparisons}\label{sec:cmp}
+
+This section makes some basic comparison between \expkv\ and other \kv\
+packages. The comparisons are really concise, regarding speed, feature range
+(without listing the features of each package, comparisons are done against the
+base \expkv\ not counting other packages in \expkvbundle\ that extend it, so
+\enquote{bigger feature set} might not necessarily be true if everything is
+included), and bugs and misfeatures.
+
+Comparisons of speed are done with a very simple test key and the help of the
+\pkg{l3benchmark} package. The key and its usage should be equivalent to
+\begin{enverb}[no-tcb]
+ \protected\ekvdef{test}{height}{\def\myheight{#1}}
+ \ekvsetdef\expkvtest{test}
+ \expkvtest{ height = 6 }
+\end{enverb}
+and only the usage of the key, not its definition, is benchmarked. For the
+impatient, the essence of these comparisons regarding speed and buggy
+behaviour is contained in \autoref{tab:comp}.
+
+As far as I know \expkv\ is the only fully expandable \kv\ parser. I tried to
+compare \expkv\ to every \kv\ package listed on
+\href{https://ctan.org/topic/keyval}{CTAN}, however, one might notice that
+some of those are missing from this list. That's because I didn't get the
+others to work due to bugs, or because they just provide wrappers around other
+packages in this list.
+
+In this subsection is no benchmark of |\ekvparse| and |\keyval_parse:NNn|
+contained, as most other packages don't provide equivalent features to my
+knowledge. |\ekvparse| is slightly faster than |\ekvset|, but keep in mind
+that it does less. The same is true for |\keyval_parse:NNn| compared to
+|\keys_set:nn| of \pkg{expl3} (where the difference is much bigger). Comparing
+just the two, |\ekvparse| is a tad faster than |\keyval_parse:NNn| because of
+two tests (for empty key names and only a single equal sign) which are
+omitted.
+
+\paragraph{\pkg{keyval}} is the fastest \kv\ package there is and has a minimal
+feature set with a slightly different way how it handles keys without values
+compared to \expkv. That might be considered a drawback, as it limits the
+versatility, but also as an advantage, as it might reduce doubled code. Keep in
+mind that as soon as someone loads \pkg{xkeyval} the performance of \pkg{keyval}
+gets replaced by \pkg{xkeyval}'s.
+
+Also \pkg{keyval} has a \pmso{bug}feature, which unfortunately can't really be
+resolved without breaking backwards compatibility for \emph{many} documents,
+namely it strips braces from the argument before stripping spaces if the
+outer most braces aren't surrounded by spaces, also it might strip more than one
+set of braces. Hence all of the following are equivalent in their outcome,
+though the last two lines should result in something different than the first
+two:
+\begin{enverb}[no-tcb]
+ \setkeys{foo}{bar=baz}
+ \setkeys{foo}{bar= {baz}}
+ \setkeys{foo}{bar={ baz}} % should be ` baz'
+ \setkeys{foo}{bar={{baz}}} % should be `{baz}'
+\end{enverb}
+
+\pkg{keyval} doesn't work with non-standard category codes of |=| and |,|. Also
+if a \kv\ pair contains multiple equals signs outside of braces everything post
+the first is silently ignored so the following two inputs yield identical
+outputs:
+\begin{enverb}[no-tcb,lst=belowskip=0pt]
+ \setkeys{foo}{bar=baz}
+ \setkeys{foo}{bar=baz=and more}
+\end{enverb}
+
+\paragraph{\pkg{xkeyval}} is pretty slow (yet not the slowest), but it provides
+more functionality, e.g., it has an interface to disable a list of keys, can
+search multiple sets simultaneously, and has an intriguing mechanism it calls
+\enquote{Pointers} to save the value of particular keys for later reuse. It
+contains the same bug as \pkg{keyval} as it has to be compatible with it by
+design (it replaces \pkg{keyval}'s frontend), but also adds even more cases in
+which braces are stripped that shouldn't be stripped, worsening the situation.
+
+\pkg{xkeyval} does work with non-standard category codes of |=| and |,|, but the
+used mechanism fails if the input contains a mix of different category codes for
+the same character. Just like with \pkg{keyval} equals signs after the first and
+everything after those is ignored.
+
+\paragraph{\pkg{ltxkeys}} is no longer compatible with the \LaTeX\ kernel
+starting with the release 2020-10-01. It is by far the slowest \kv\ package I've
+tested -- which is funny, because it aims to be ``[\ldots] faster [\ldots] than
+these earlier packages [referring to \pkg{keyval} and \pkg{xkeyval}].'' It needs
+more time to parse zero~keys than five of the packages in this comparison need
+to parse 100~keys. Since it aims to have a bigger feature set than
+\pkg{xkeyval}, it most definitely also has a bigger feature set than \expkv.
+Also, it can't parse |\long| input, so as soon as your values contain a |\par|,
+it'll throw errors. Furthermore, \pkg{ltxkeys} doesn't strip outer braces at
+all by design, which, imho, is a weird design choice. Some of the more
+intriguing features (e.g., the |\argpattern| mechanism) didn't work for me. In
+addition \pkg{ltxkeys} loads \pkg{catoptions} which is known to introduce bugs
+(e.g., see \url{https://tex.stackexchange.com/questions/461783}). Because it is
+no longer compatible with the kernel, I stop benchmarking it (so the numbers
+listed here and in \autoref{tab:comp} regarding \pkg{ltxkeys} were last updated
+on 2020-10-05).
+
+\pkg{ltxkeys} works with non-standard category codes, it also silently ignores
+any additional equals signs and the following tokens.
+
+\paragraph{\pkg{l3keys}} is at the slower end of the midfield yet not unusably
+slow, but has an, imho, great interface to define keys. It strips \emph{all}
+outer spaces, even if somehow multiple spaces ended up on either end. It offers
+more features, but has pretty much been bound to \pkg{expl3} code before.
+Nowadays the \LaTeX\ kernel has an interface with the macros |\DeclareKeys|,
+|\SetKeys|, and |\ProcessKeyOptions| that provides access to \pkg{l3keys} from
+the \LaTeXe\ layer as well as parsing package options with it. Because of the
+|\ProcessKeyOptions| macro and its features the only two viable options to
+provide \kv\ options for new projects in my opinion are the kernel's methods and
+\expkvo\ as those are the only two until now up to my knowledge that support
+parsing the raw options, and future options.
+
+\pkg{l3keys} handles active commas and equals signs fine. Multiple equals signs
+lead to an error if additional equals signs aren't nested inside of braces, so
+perfectly predictable behaviour here.
+
+\paragraph{\pkg{pgfkeys}} is among the top~4 of speed if one uses |\pgfqkeys|
+over |\pgfkeys|, else the initialisation parsing the family path takes roughly
+\SI{43}{\ops} and moves it two spots down the list (so in \autoref{tab:comp}
+both $p_0$ and $T_0$ would be about \SI{43}{\ops} bigger if
+|\pgfkeys{|\meta{path}|/.cd,|\meta{keys}|}| was used instead). It has an
+\emph{enormous} feature set. It stores keys in a way that reminds one of folders
+in a Unix system which allows interesting features and has other syntactic
+sugars. It is another package that implements something like the \expnotation\
+with less different options though. To get the best performance |\pgfqkeys| was
+used in the benchmark. It has the same or a very similar bug \pkg{keyval} has.
+The brace bug (and also the category fragility) can be fixed by \pkg{pgfkeyx},
+but this package was last updated in 2012 and it slows down |\pgfkeys| by
+factor~\num{8}. Also \pkg{pgfkeyx} is no longer compatible with versions of
+\pkg{pgfkeys} newer than 2020-05-25.
+
+\pkg{pgfkeys} silently drops anything after the second unbraced equals sign in a
+\kv\ pair.
+
+\paragraph{\pkg{kvsetkeys} with \pkg{kvdefinekeys}} is in the slower midfield,
+but it works even if commas and equals have category codes different from 12
+(just as some other packages in this list). It has quadratic run-time unlike
+most other \kv\ implementations which behave linear. The features of the
+keys are equal to those of \pkg{keyval}, the parser adds handling of unknown
+keys.
+
+\pkg{kvsetkeys} does include any additional equals sign in the value. But any
+active equals sign is turned into one of category code 12 if it's not nested in
+braces. Also spaces around superfluous equals signs are stripped. So the
+following all result in the same:
+\begin{enverb}[no-tcb,lst=belowskip=0pt]
+ \kvsetkeys{foo}{bar=baz=morebaz}
+ \kvsetkeys{foo}{bar=baz =morebaz}
+ \kvsetkeys{foo}{bar=baz= morebaz}
+ \kvsetkeys{foo}{bar=baz = morebaz}
+\end{enverb}
+
+\paragraph{\pkg{options}} is in the midfield of speed. It is faster per
+individual key than \pkg{pgfkeys} but has no shortcut like |\pgfqkeys|.
+It has a much bigger feature set than \expkv. Similar to \pkg{pgfkeys} it uses a
+folder like structure, makes searching multiple paths easy, incorporates package
+options and more. It also features a form of expansion control, predefined
+expansion kinds are limited though one can define additional ones. Unfortunately
+it also suffers from the premature unbracing bug \pkg{keyval} has.
+
+\pkg{options} can't handle non-standard category codes and will silently ignore
+superfluous equals signs and following tokens.
+
+\paragraph{\pkg{simplekv}} is hard to compare because I don't speak French (so
+I don't understand the documentation). There was an update released on
+2020-04-27 which greatly improved the package's performance and added
+functionality so that it can be used more like most of the other \kv\
+packages. Speed wise it is pretty close to \expkv. Regarding unknown keys it got
+a very interesting behaviour. It doesn't throw an error, but stores the \val\ in
+a new entry accessible with \cs[no-index]{useKV}. Also if you omit \val\ it
+stores |true| for that \key.
+
+\pkg{simplekv} can't correctly handle non-standard category codes. It silently
+ignores any unbraced equals sign beyond the first and any following tokens.
+
+\paragraph{\protect\yax} is the second slowest package I've tested. It has a
+pretty strange syntax for the \TeX-world, imho, and again a direct equivalent is
+hard to define (don't understand me wrong, I don't say I don't like the syntax,
+quite the contrary, it's just atypical). It has the premature unbracing bug,
+too. \yax\ features some prefixes one can use to make an assignment use |\edef|,
+|\gdef| or |\xdef| so has something that comes close to expansion control. Also
+somehow loading \yax\ broke \pkg{options} for me. The tested definition was:
+\begin{enverb}[no-tcb]
+ \usepackage{yax}
+ \defactiveparameter yax {\storevalue\myheight yax:height } % setup
+ \setparameterlist{yax}{ height = 6 } % benchmark
+\end{enverb}
+This seems important to state as \yax\ supports two different input syntaxes,
+the tested one was the one closer to traditional \kv\ input.
+
+\yax\ won't handle non-standard category codes correctly. Superfluous equals
+signs end up in the value in an unaltered form (just like with \expkv).
+
+\begin{table}
+ \def\fnsym{\textcolor{red!80!black}{*}}%
+ \sisetup{round-precision=1, round-mode=places}%
+ \begingroup
+ \centering
+ \newcommand*\yes{\textcolor{red!80!black} {yes}}^^A
+ \newcommand*\no {\textcolor{green!80!black}{no}}^^A
+ \caption[Comparison of \kv\ packages]
+ {^^A
+ Comparison of \kv\ packages. The packages are ordered from
+ fastest to slowest for one \kv\ pair. Benchmarking was done using
+ \pkg{l3benchmark} and the scripts in the \file{Benchmarks} folder
+ of \href{https://github.com/Skillmon/tex_expkv}{the original \expkv's git
+ repository}.
+ The columns $p_i$ are the polynomial coefficients of a linear fit to the
+ run-time, $p_0$ can be interpreted as the overhead for initialisation
+ and $p_1$ the cost per key. The $T_0$ column is the actual mean ops
+ needed for an empty list argument, as the linear fit doesn't match that
+ point well in general. The column ``BB'' lists whether the
+ parsing is affected by some sort of brace bug, ``CF'' stands for
+ category code fragile and lists whether the parsing breaks with active
+ commas or equal signs.^^A
+ \label{tab:comp}^^A
+ }
+ \begin{tabular}
+ {>{\collectcell\pkg}l<{\endcollectcell}*3{S[table-format=4.1]}ccc}
+ \toprule
+ \rmfamily Package & {$p_1$} & {$p_0$} & {$T_0$}& BB & CF & Date \\
+ \midrule
+ keyval & 13.557 & 2.202 & 7.185 & \yes & \yes & 2022-05-29 \\
+ \expkv & 16.669 & 3.132 & 5.836 & \no & \no & 2023-01-10 \\
+ simplekv & 19.943 & 2.850 & 15.120 & \no & \yes & 2022-10-01 \\
+ pgfkeys & 24.465 & 2.244 & 10.283 & \yes & \yes & 2021-05-15 \\
+ options & 23.255 & 16.160 & 20.400 & \yes & \yes & 2015-03-01 \\
+ kvsetkeys & {\fnsym} & {\fnsym} & 40.360 & \no & \no & 2022-10-05 \\
+ l3keys & 70.643 & 35.602 & 32.230 & \no & \no & 2022-12-17 \\
+ xkeyval & 255.906 & 221.276 & 173.400 & \yes & \yes & 2022-06-16 \\
+ \yax & 438.242 & 131.846 & 114.800 & \yes & \yes & 2010-01-22 \\
+ ltxkeys & 3400.142 & 4737.958 & 5368.000 & \no & \no & 2012-11-17 \\
+ \bottomrule
+ \end{tabular}
+ \par
+ \endgroup
+ \medskip
+ \fnsym For \pkg{kvsetkeys} the linear model used for the other
+ packages is a poor fit, \pkg{kvsetkeys} seems to have approximately
+ quadratic run-time, the coefficients of the second degree polynomial fit are
+ $p_2=\num{7.617}$, $p_1=\num{47.727}$, and $p_0=\num{57.988}$. Of course the
+ other packages might not really have linear run-time, but at least from 1~to
+ 20~keys the fits don't seem too bad. If one extrapolates the fits for 100
+ \kv\ pairs one finds that most of them match pretty well, the exception
+ being \pkg{ltxkeys}, which behaves quadratic as well with
+ $p_2=\num{23.500}$, $p_1=\num{2906.634}$, and $p_0=\num{6547.489}$.
+\end{table}