summaryrefslogtreecommitdiff
path: root/Master/texmf-dist/doc/support/arara/chapters/concepts.tex
blob: 0a8d168fe7c01008b4726e069f264ff17bc2b8f8 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
% !TeX root = ../arara-manual.tex
\chapter{Important concepts}
\label{chap:importantconcepts}

Time for our first proper contact with \arara! I must stress that is very important to understand a few concepts in which \arara\ relies before we proceed to the usage itself. Do not worry, these concepts are easy to follow, yet they are vital to the comprehension of the application and the logic behind it.

\section{Rules}
\label{sec:rule}

A \emph{rule} is a formal description of how \arara\ handles a certain task. For instance, if we want to use \rbox{pdflatex} with our tool, we should have a rule for that. Directives are mapped to rules, so a call to a non-existent rule \rbox{foo}, for instance, will indeed raise an error:

\begin{codebox}{Terminal}{teal}{\icnote}{white}
  __ _ _ __ __ _ _ __ __ _ 
 / _` | '__/ _` | '__/ _` |
| (_| | | | (_| | | | (_| |
 \__,_|_|  \__,_|_|  \__,_|

Processing 'doc1.tex' (size: 83 bytes, last modified: 05/03/2018
12:10:33), please wait.

I could not find a rule named 'foo' in the provided rule paths.
Perhaps a misspelled word? I was looking for a file named
'foo.yaml' in the following paths in order of priority:
(/opt/paulo/arara/rules)

Total: 0.09 seconds
\end{codebox}

Once a rule is defined, \arara\ automatically provides an access layer to that rule through directives in the source code, a concept to be formally introduced later on, in Section~\ref{sec:directives}. Observe that a directive reflects a particular instance of a rule of the same name (i.e, a \rbox{foo} directive in a certain source code is an instance of the \rbox{foo} rule).

In short, a rule is a plain text file written in the \gls{YAML} format, described in Chapter~\ref{chap:yaml}, on page~\pageref{chap:yaml}. I opted for this format because back then it was cleaner and more intuitive to use than other markup languages such as \gls{XML}, besides of course being a data serialization standard for programming languages.

\begin{messagebox}{Animal jokes}{araracolour}{\icok}{white}
As a bonus, the acronym \emph{YAML} rhymes with the word \emph{camel}, so \arara\ is heavily environmentally friendly. Speaking of camels, there is the programming reference as well, since this amusing animal is usually associated with Perl and friends.
\end{messagebox}

The default rules, i.e, the rules shipped with \arara, are placed inside a special subdirectory named \abox[araracolour]{rules/} inside another special directory named \abox[araracolour]{ARARA\_HOME} (the place where our tool is installed). We will learn later on, in Section~\ref{sec:basicstructure}, on page~\pageref{sec:basicstructure}, that we can add an arbitrary number of paths for storing our own rules, in order of priority, so do not worry too much about the location of the default rules, although it is important to understand and acknowledge their existence.

The following list describes the basic structure of an \arara\ rule by presenting the proper elements (or keys, if we consider the proper \gls{YAML} nomenclature). Observe that elements marked as \rbox[araracolour]{M} are mandatory (i.e, the rule \emph{has} to have them in order to work). Similarly, elements marked as \rbox[araracolour]{O} are optional, so you can safely ignore them when writing a rule for our tool. A key preceded by \rbox{context$\rightarrow$} indicates a context and should be properly defined inside it.

\begin{description}
\item[\describe{M}{!config}] This keyword is mandatory and must be the first line of any \arara\ rule. It denotes the object mapping metadata to be internally used by the tool. Actually, the tool is not too demanding on using it (in fact, you could suppress it entirely and \arara\ will not complain), but it is considered good practice to start all rules with a \abox{!config} keyword regardless.

\item[\describe{M}{identifier}] This key acts as a unique identifier for the rule (as expected). It is highly recommended to use lowercase letters without spaces, accents or punctuation symbols, as good practice (again). As a convention, if you have an identifier named \rbox{pdflatex}, the rule filename must be \rbox{pdflatex.yaml} (like our own instance). Please note that, although \rbox{yml} is known to be a valid \gls{YAML} extension as well, \arara\ only considers files ending with the \rbox{yaml} extension. This is a deliberate decision.

\begin{codebox}{Example}{teal}{\icnote}{white}
identifier: pdflatex
\end{codebox}

\item[\describe{M}{name}] This key holds the name of the \emph{task} (a rule instantiated through a directive) as a plain string. When running \arara, this value will be displayed in the output enclosed in parentheses.

\begin{codebox}{Example}{teal}{\icnote}{white}
name: PDFLaTeX
\end{codebox}

\item[\describe{O}{authors}] We do love blaming people, so \arara\ features a special key to name the rule authors (if any) so you can write stern electronic communications to them! This key holds a list of strings. If the rule has just one author, add it as the first (and only) element of the list.

\begin{codebox}{Example}{teal}{\icnote}{white}
authors:
- Marco Daniel
- Paulo Cereda
\end{codebox}

\item[\describe{M}{commands}] This key is introduced in version 4.0 of \arara\ and denotes a potential list of commands. From the user perspective, each command is called a \emph{subtask} within a task (rule and directive) context. A task may represent only a single command (a single subtask), as well as a sequence of commands (subtasks). For instance, the \rbox{frontespizio} rule requires at least two commands. So, as a means of normalizing the representation, a task composed of a single command (single subtask) is defined as the only element of the list, as opposed to previous versions of \arara, which had a specific key to hold just one command.

\begin{messagebox}{Incompatibility with older versions}{attentioncolour}{\icerror}{black}
Dear reader, note that rules from version 4.0 are incompatible with older versions of \arara. If you are migrating from old versions to version 4.0, we need to replace \abox{command} by \abox{commands} and specify a contextual element, as seen in the following descriptions. Please refer to Section~\ref{sec:migrationguide}, on page~\pageref{sec:migrationguide}, for a comprehensible migration guide.
\end{messagebox}

In order to properly set a subtask, the keys used in this specification are defined inside the \rbox{commands$\rightarrow$} context and presented as follows.

\begin{description}
\item[\describecontext{O}{commands}{name}] This key holds the name of the subtask as a plain string. When running \arara, this value will be displayed in the output. Subtask names are displayed after the main task name. By the way, did you notice that this key is entirely optional? That means that a subtask can simply be unnamed, if you decide so. However, such practice is not recommended, as it's always good to have a visual description of what \arara\ is running at the moment, so name your subtasks properly.

\item[\describecontext{M}{commands}{command}] This key holds the action to be performed, typically a system command. In previous versions, \arara\ would rely solely on a string. For this version on, as a means to enhance the user experience (and also fix serious blockers when handling spaces in file names, as seen in \href{https://github.com/cereda/arara/issues}{previous issues} reported in the repository), the tool offers four types of returned values:

\begin{itemize}[label={--}]
\item A plain string: this is the default (and only) behaviour in older versions of \arara. The plain string is processed as it is by the underlying execution engine. However, automatic argument parsing is problematic, so this approach, although supported, is not recommended any more.

\begin{codebox}{Example}{teal}{\icnote}{white}
command: 'ls'
\end{codebox}

It is important to observe that you can use either a plain string directly or use an \gls{orb-tag} with an explicit \rbox{return} command (as seen in Section~\ref{sec:mvelbasicusage}, on page~\pageref{sec:mvelbasicusage}). Personally, I favour the explicit indication for a quick understanding.

\item A \rbox{Command} object: \arara\ 4.0 features a new approach for handling system commands based on a high level structure with explicit argument parsing named \rbox{Command} (for our curious users, it is a plain Java object). In order to use this approach, we need to rely on \glspl{orb-tag} and use a helper method named \mtbox{getCommand} to obtain the desired result. We will detail this method later on, in Section~\ref{sec:commandsandtriggers}, on page~\pageref{sec:commandsandtriggers}. We highly recommend the adoption of this approach for rule writing instead of using plain strings.

\begin{codebox}{Example}{teal}{\icnote}{white}
command: "@{ return getCommand('ls') }"
\end{codebox}

\item A boolean value: it is also possible to exploit the expressive power of the underlying scripting language available in the rule context (see Chapter~\ref{chap:mvel}, on page~\pageref{chap:mvel}, for more details) for writing complex code. In this particular case, since the computation is being done by \arara\ itself and not the underlying operating system, there will not be a command to be executed, so simply return a boolean value -- either an explicit \rbox{true} or \rbox{false} value or a logical expression -- to indicate whether the computation was successful.

\begin{codebox}{Example}{teal}{\icnote}{white}
command: "@{ return 1 == 1 }"
\end{codebox}

\item A \rbox{Trigger} object: this is surely the least common type of returned value and it is mentioned here just for documentation purposes. In simple terms, a \rbox{Trigger} object constitutes a special command that changes the internal workings of \arara\ at runtime. We have not worked much on this concept, so there is only one trigger available, seen in action in the official \rbox{halt} rule. In order to use this approach, we need to rely on \glspl{orb-tag} and use a helper method named \mtbox{getTrigger} to obtain the desired result.
\end{itemize}

It is also worth mentioning that \arara\ also supports lists of commands represented as plain strings, \rbox{Command} or \rbox{Trigger} objects, boolean values or a mix of them. This is useful if your rule has to decide whether more actions are required in order to accomplish a task. In this case, our tool will take care of the list and execute each element in the specified order.

\begin{codebox}{Example}{teal}{\icnote}{white}
command: "@{ return [ 'ls', 'ls', 'ls' ] }"
\end{codebox}

As an example, please refer to the official \rbox{clean} rule for a real scenario where a list of commands is successfully employed: for each provided extension, the rule creates a new cleaning command and adds it to a list of removals to be processed later.

\begin{messagebox}{Plain string is deprecated}{attentioncolour}{\icattention}{black}
It took me a lot of effort to find out that handling plain strings and employing guesswork to parse arguments are the root of several issues reported by users. Therefore, this approach is being marked as \emph{deprecated} and will be removed in future versions.
\end{messagebox}

There are at least two variables available in the \abox{command} context and are described as follows (note that \gls{MVEL} variables and \glspl{orb-tag} are discussed in Chapter~\ref{chap:mvel}). A variable will be denoted by \varbox{variable} in this list. For each rule argument (defined later on), there will be a corresponding variable in the \abox{command} context, directly accessed through its unique identifier.

\begin{description}
\item[\varbox{file}] This variable holds the file name, without any path reference, as a plain string. It is usually composed from the base name and the extension. This variable is available since the first release of \arara.

\item[\varbox{reference}] This variable is introduced in version 4.0 of \arara\ and holds the canonical, absolute path representation of the \varbox{file} variable as a \rbox{File} object. This is useful if it's necessary to know the hierarchical structure of a project. Since the reference is a Java object, we can use all methods available in the \rbox{File} class.
\end{description}

\begin{messagebox}{Quote handling}{araracolour}{\icinfo}{white}
\setlength{\parskip}{1em}
The \gls{YAML} format disallows key values starting with \rbox{@} without proper quoting. This is the reason we had to use double quotes for the value and internally using single quotes for the command string. Also, we could use the other way around, or even using only one type and then escaping them when needed. This is excessively verbose but needed due to the format requirement. Thankfully, \arara\ offers two solutions for removing the quoting verbosity when writing commands.

The first solution is used in previous versions and it still works like a charm in modern days. We need to precede our command with a special keyword \rbox{<arara>} which will be removed afterwards. This solution works on virtually every key in the rule context, so it is a bonus. The new code will look like this:

\begin{codebox}{Example}{teal}{\icnote}{white}
command: <arara> @{ return getCommand('ls') }
\end{codebox}

The second approach is more of a \gls{YAML} feature rather than a tool exclusive, although we have to do a couple of checks under the hood in order to ensure the correct execution. The idea here is to use the scalar content in folded style, as seen in Section~\ref{sec:yamlscalars}, on page~\pageref{sec:yamlscalars}. The new code will look like this:

\begin{codebox}{Example}{teal}{\icnote}{white}
command: >
  @{
    return getCommand('ls')
  }
\end{codebox}

Mind the indentation, as \gls{YAML} requires it to properly identify blocks. I personally recommend this approach for longer code, as it provides a better visual representation. You will see the second solution all around the default rules, but feel free to use the one you feel more comfortable with.
\end{messagebox}

\item[\describecontext{O}{commands}{exit}] This key holds a special purpose in \arara\ 4.0, as it represents a custom exit status evaluation for the corresponding command. In general, a successful execution has zero as an exit status, but sometimes we end up with tools or situations where we need to override this check for whatever reason. For this purpose, simply write a \gls{MVEL} expression \emph{without \glspl{orb-tag}} as plain string and use the special variable \varbox{value} if you need the actual exit status returned by the command, available at runtime. For example, if the command returns a non-zero value indicating a successful execution, we can write this key as:

\begin{codebox}{Example}{teal}{\icnote}{white}
exit: value > 0
\end{codebox}

If the execution should be marked as successful by \arara\ regardless of the actual exit status, you can simply write \rbox{true} as the key value and this rule will never fail, for obvious reasons.
\end{description}

For instance, consider a full example of the \abox{commands} key, defined with only one command, presented as follows. The hyphen denotes a list element, so mind the indentation for correctly specifying the component keys. Also, note that, in this case, the \abox{exit} key was completely optional, as it does the default checking, and it was included for didactic purposes.

\begin{codebox}{Example}{teal}{\icnote}{white}
commands:
- name: The PDFLaTeX engine
  command: >
    @{
      return getCommand('pdflatex', file)
    }
  exit: value == 0
\end{codebox}

\item[\describe{M}{arguments}] This key holds a list of arguments for the current rule, if any. The arguments specified in this list will be available to the user later on for potential completion through directives. Once instantiated, they will become proper variables in the \abox{command} contexts. This key is mandatory, so even if your rule does not have arguments, you need to specify a list regardless. In this case, use the empty list notation:

\begin{codebox}{Example}{teal}{\icnote}{white}
arguments: []
\end{codebox}

In order to properly set an argument, the keys used in this specification are defined inside the \rbox{arguments$\rightarrow$} context and presented as follows.

\begin{description}
\item[\describecontext{M}{arguments}{identifier}] This key acts as a unique identifier for the argument. It is highly recommended to use lowercase letters without spaces, accents or punctuation symbols, as a good practice. This key will be used later on to set the corresponding value in the directive context.

\begin{codebox}{Example}{teal}{\icnote}{white}
identifier: shell
\end{codebox}

It is important to mention that not all names are valid as argument identifiers. \arara\ has restrictions on three names, described as follows, which cannot be used.

\begin{messagebox}{Reserved names for rule arguments}{attentioncolour}{\icattention}{black}
Our tool has three names reserved for internal use: \abox{file}, \abox{files}, and \abox{reference}. Do not use them as argument identifiers!
\end{messagebox}

\item[\describecontext{O}{arguments}{flag}] This key holds a plain string and is evaluated when the corresponding argument is defined in the directive context.  After being evaluated, the result will be stored in a variable of the same name to be later accessed in the \abox{command} context. In the scenario where the argument is not defined in the directive, the variable will hold an empty string.

\begin{codebox}{Example}{teal}{\icnote}{white}
flag: >
  @{
      isTrue(parameters.shell, '--shell-escape',
             '--no-shell-escape')
  }
\end{codebox}

There are three variables available in the \abox{flag} context, described as follows. Note that are also several helper methods available in the rule context (for instance, \mtbox{isTrue} presented in the previous example) which provide interesting features for rule writing. They are detailed later on, in Chapter~\ref{chap:methods}, on page~\pageref{chap:methods}.

\begin{description}
\item[\varbox{parameters}] This variable holds a map of directive parameters available at runtime. For each argument identifier listed in the \abox{arguments} list in the rule context, there will be an entry in this variable. This is useful to get the actual values provided during execution and take proper actions. If a parameter is not set in the directive context, the reference will still exist in the map, but it will be mapped to an empty string.

\item[\varbox{file}] This variable holds the file name, without any path reference, as a plain string. It is usually composed from the base name and the extension. This variable is available since the first release of \arara.

\item[\varbox{reference}] This variable is introduced in version 4.0 of \arara\ and holds the canonical, absolute path representation of the \varbox{file} variable as a \rbox{File} object. This is useful if it's necessary to know the hierarchical structure of a project. Since the reference is a Java object, we can use all methods available in the \rbox{File} class.
\end{description}

In the previous example, observe that the \gls{MVEL} expression defined in the \abox{flag} key checks if the user provided an affirmative value regarding shell escape, through comparing \varbox{parameters.shell} with a set of predefined affirmative values. In any case, the corresponding command flag is defined as result of such evaluation.

\item[\describecontext{O}{arguments}{default}] As default behaviour, if a parameter is not set in the directive context, the reference will be mapped to an empty string. This key exists for the exact purpose of overriding such behaviour.

\begin{codebox}{Example}{teal}{\icnote}{white}
default: ''
\end{codebox}

There are three variables available in the \abox{default} context, described as follows. Note that are also several helper methods available in the rule context (for instance, \mtbox{isTrue} presented in the previous example) which provide interesting features for rule writing. They are detailed later on, in Chapter~\ref{chap:methods}, on page~\pageref{chap:methods}.

\begin{description}
\item[\varbox{parameters}] This variable holds a map of directive parameters available at runtime. For each argument identifier listed in the \abox{arguments} list in the rule context, there will be an entry in this variable. This is useful to get the actual values provided during execution and take proper actions. If a parameter is not set in the directive context, the reference will still exist in the map, but it will be mapped to an empty string.

\item[\varbox{file}] This variable holds the file name, without any path reference, as a plain string. It is usually composed from the base name and the extension. This variable is available since the first release of \arara.

\item[\varbox{reference}] This variable is introduced in version 4.0 of \arara\ and holds the canonical, absolute path representation of the \varbox{file} variable as a \rbox{File} object. This is useful if it's necessary to know the hierarchical structure of a project. Since the reference is a Java object, we can use all methods available in the \rbox{File} class.
\end{description}

\item[\describecontext{O}{arguments}{required}] There might be certain scenarios in which a rule could make use of required arguments (for instance, a copy operation in which source and target must be provided). The \abox{required} key acts as a boolean switch to indicate whether the corresponding argument should be mandatory. In this case, set the key value to \rbox{true} and the argument becomes required. Later on at runtime, \arara\ will throw an error if a required parameter is missing in the directive.

\begin{codebox}{Example}{teal}{\icnote}{white}
required: false
\end{codebox}

Note that setting the \abox{required} key value to \rbox{false} corresponds to omitting the key completely in the rule context, which resorts to the default behaviour (i.e, all arguments are optional).
\end{description}

\begin{messagebox}{Note on argument keys}{attentioncolour}{\icattention}{black}
As seen previously, both \abox{flag} and \abox{default} are marked as optional, but at least one of them must occur in the argument specification, otherwise \arara\ will throw an error, as it makes no sense to have no argument handling at all. Please make sure to specify at least one of them for a consistent behaviour!
\end{messagebox}

For instance, consider a full example of the \abox{arguments} key, defined with only one argument, presented as follows. The hyphen denotes a list element, so mind the indentation for correctly specifying the component keys. Also, note that, in this case, keys \abox{required} and \abox{default} were completely optional, and they were included for didactic purposes.

\begin{codebox}{Example}{teal}{\icnote}{white}
arguments:
- identifier: shell
  flag: >
    @{
        isTrue(parameters.shell,
               '--shell-escape',
               '--no-shell-escape')
    }
  required: false
  default: ''
\end{codebox}
\end{description}

This is the rule structure in the \gls{YAML} format used by \arara. Keep in mind that all subtasks in a rule are checked against their corresponding exit status. If an abnormal execution is detected, the tool will instantly halt and the rule will fail. Even \arara\ itself will return an exit code different than zero when this situation happens (detailed in Chapter~\ref{chap:commandline}, on page~\pageref{chap:commandline}).

\section{Directives}
\label{sec:directives}

A \emph{directive} is a special comment inserted in the source file in which you indicate how \arara\ should behave. You can insert as many directives as you want and in any position of the file. The tool will read the whole file and extract the directives.

\begin{messagebox}{New features in version 4.0}{araracolour}{\icinfo}{white}
\setlength{\parskip}{1em}
\textbf{Partial directive extraction} -- From version 4.0 of \arara\ on, it is now possible to extract directives only available in the file preamble, i.e, all lines from the beginning that are comments until reaching the first line that is not a comment (excluding blank lines). To this end, a new command line flag is introduced in Section~\ref{sec:basicstructure}, on page~\pageref{sec:basicstructure}.

\textbf{Predefined preambles} -- Common preambles can be predefined and used with files that require the same automation steps, then \arara\ can be invoked based on such specifications. This feature is covered in Section~\ref{sec:basicstructure}, on page~\pageref{sec:basicstructure}.
\end{messagebox}


There are two types of directives in \arara\ which determine the way the corresponding rules will be instantiated. They are listed as follows. Note that directives are always preceded by the \rbox{arara:} pattern.

\begin{description}
\item[empty directive] This type of directive has already been mentioned in Chapter~\ref{chap:introduction}, on page~\pageref{chap:introduction}, it has only the rule name (which refers to the \abox{identifier} key from the rule of the same name). All rule arguments are mapped to empty strings, except the ones with \abox{default} values.

\begin{codebox}{Empty directive}{teal}{\icnote}{white}
% arara: pdflatex
\end{codebox}

\item[parametrized directive] This type of directive also has the rule name (which refers to the \abox{identifier} key from the rule of the same name), and also contains a map of parameters in order to provide additional information to the corresponding rule. This map is defined in the \gls{YAML} format, based on the inline style.

\begin{codebox}{Parametrized directive}{teal}{\icnote}{white}
% arara: pdflatex: { shell: yes }
\end{codebox}

Observe that \arara\ relies on named parameters, so they are mapped by their corresponding argument identifiers and not by their positions. The syntax for a parameter is described as follows. Please refer to the map definition in Section~\ref{sec:yamlcollections}, on page~\pageref{sec:yamlcollections}.

\begin{codebox}{Parameter syntax}{teal}{\icnote}{white}
key : value
\end{codebox}

Note that virtually any type of data can be used as parameter value, so lists, integers, booleans, sets and other maps are available as well. However, there must be the correct handling of such types in the rule context.
\end{description}

When handling parametrized directives, \arara\ always checks if directive parameters and rule arguments match. If we try to inject a non-existent parameter in a parametrized directive, the tool will raise an error about it:

\begin{codebox}{Terminal}{teal}{\icnote}{white}
  __ _ _ __ __ _ _ __ __ _ 
 / _` | '__/ _` | '__/ _` |
| (_| | | | (_| | | | (_| |
 \__,_|_|  \__,_|_|  \__,_|

Processing 'hello.tex' (size: 103 bytes, last modified:
05/03/2018 15:40:16), please wait.

I have spotted an error in rule 'pdflatex' located at
'/opt/paulo/arara/rules'. I found these unknown keys
in the directive: (foo). This should be an easy fix,
just remove them from your map.

Total: 0.21 seconds
\end{codebox}

As the message suggests, we need to remove the unknown parameter key from our directive or rewrite the rule in order to include it as an argument. The first option is, of course, easier.

\begin{messagebox}{New feature in version 4.0}{araracolour}{\icinfo}{white}
\textbf{Helpful messages} -- It is a staple of \arara\ to have friendly and helpful messages. From version 4.0 on, we decided to make messages even friendlier and include suggestions for correcting errors or improving usage. So, whenever possible, make sure to read all messages our tool provides, they will help you!
\end{messagebox}

Sometimes, directives can span several columns of a line, particularly the ones with several parameters. From \arara\ 4.0 on, we can split a directive into multiple lines by using the \rbox{arara: {-}{-}>} mark (also known as \emph{arrow notation} during development) to each line which should compose the directive. We call it a \emph{multiline directive}. Let us see an example:

\begin{codebox}{Multiline directive}{teal}{\icnote}{white}
% arara: pdflatex: {
% arara: --> shell: yes,
% arara: --> synctex: yes
% arara: --> }
\end{codebox}

It is important to observe that there is no need of them to be in contiguous lines, i.e, provided that the syntax for parametrized directives hold for the line composition, lines can be distributed all over the code. In fact, the log file (when enabled) will contain a list of all line numbers that compose a directive. This feature is discussed later on, in Section~\ref{sec:directiveextraction}, on page~\pageref{sec:directiveextraction}.

\begin{messagebox}{Keep lines together}{araracolour}{\icinfo}{white}
Although it is possible to spread lines of a multiline directive all over the code, it is considered good practice to keep them together for easier reading and editing. In any case, you can always see which lines compose a directive by inspecting the log file.
\end{messagebox}

\arara\ 4.0 provides logical expressions, written in the \gls{MVEL} language, and special operators processed at runtime in order to determine whether and how a directive should be processed. This feature is named \emph{directive conditional}, or simply \emph{conditional} as an abbreviation. The following list describes all conditional operators available in the directive context.

\begin{description}
\item[\describeconditional{a priori}{if}] The associated \gls{MVEL} expression is evaluated beforehand, and the directive is interpreted if, and only if, the result of such evaluation is true. This directive, when the conditional holds true, is executed at most once.

\begin{codebox}{Conditional}{teal}{\icnote}{white}
% arara: pdflatex if missing('pdf') || changed('tex')
\end{codebox}

\item[\describeconditional{a posteriori}{until}] The directive is interpreted the first time, then the associated \gls{MVEL} expression evaluation is done. While the result holds false, the directive is interpreted again and again. There are no guarantees of proper halting.

\begin{codebox}{Conditional}{teal}{\icnote}{white}
% arara: pdflatex until !found('log', 'undefined references')
\end{codebox}

\item[\describeconditional{a priori}{unless}] Technically an inverted \cdbox{if} conditional, the associated \gls{MVEL} expression is evaluated beforehand, and the directive is interpreted if, and only if, the result is false. This directive, when the conditional holds false, is executed at most once.

\begin{codebox}{Conditional}{teal}{\icnote}{white}
% arara: pdflatex unless unchanged('tex') && exists('pdf')
\end{codebox}

\item[\describeconditional{a priori}{while}] The associated \gls{MVEL} expression is evaluated beforehand, the directive is interpreted if, and only if, the result is true, and the process is repeated while the result still holds true. There are no guarantees of proper halting.

\begin{codebox}{Conditional}{teal}{\icnote}{white}
% arara: pdflatex while missing('pdf') ||
% arara: --> found('log', 'undefined references')
\end{codebox}
\end{description}

Several methods are available in the directive context in order to ease the writing of conditionals, such as \mtbox{missing}, \mtbox{changed}, \mtbox{found}, \mtbox{unchanged}, and \mtbox{exists} featured in the previous examples. They will be properly detailed later on, in Section~\ref{sec:files}, on page~\pageref{sec:files}.

\begin{messagebox}{No infinite loops}{araracolour}{\icinfo}{white}
Although there are no conceptual guarantees for proper halting of unbounded loops, we have provided a technical solution for potentially infinite iterations: \arara\ has a predefined maximum number of loops. The default value is set to 10, but it can be overridden either in the configuration file or with a command line flag. We discuss this feature later on, in Sections~\ref{sec:options} and~\ref{sec:basicstructure}, on pages~\pageref{sec:options} and~\pageref{sec:basicstructure}, respectively.
\end{messagebox}

All directives, regardless of their type, are internally mapped with both \abox{file} and \abox{reference} parameters, discussed earlier on, in Section~\ref{sec:coreconcepts}, on page~\pageref{sec:coreconcepts}, as special variables in the rule context. When inspecting the log file, you will find all map keys and values for each extracted directive (actually, there is an entire log section devoted to detailing directives found in the code). This feature is covered in Section~\ref{sec:directivenormalization}, on page~\pageref{sec:directivenormalization}. See, for instance, the report of the directive extraction and normalization process performed by \arara\ when inspecting \rbox{doc2.tex}, available in the log file. Note that timestamps were deliberately removed in order to declutter the output, and line breaks were included in order to easily spot the log entries.

\begin{ncodebox}{Source file}{teal}{\icnote}{white}{doc2.tex}
% arara: pdflatex
% arara: pdflatex: { shell: yes }
\documentclass{article}

\begin{document}
Hello world.
\end{document}
\end{ncodebox}

\begin{codebox}{An excerpt of the log file (directive section)}{teal}{\icnote}{white}
Directive: { identifier: pdflatex, parameters:
{reference=/home/paulo/doc2.tex, file=doc2.tex},
conditional: { NONE }, lines: [1] }

Directive: { identifier: pdflatex, parameters:
{shell=yes, file=doc2.tex, reference=/home/paulo/doc2.tex},
conditional: { NONE }, lines: [2] }
\end{codebox}

The directive context also features another special parameter named \abox{files} which expects a non-empty list of file names as plain string values. For each element of this list, \arara\ will replicate the current directive and point the element being iterated as current \abox{file} and \abox{reference} values (being the latter resolved to a proper absolute, canonical path of the former). See, for instance, the report of the directive extraction and normalization process performed by \arara\ when inspecting \rbox{doc3.tex}, available in the log file.

\begin{ncodebox}{Source file}{teal}{\icnote}{white}{doc3.tex}
% arara: pdflatex: { files: [ doc1.tex, doc2.tex ] }
Hello world.
\bye
\end{ncodebox}

\begin{codebox}{An excerpt of the log file (directive section)}{teal}{\icnote}{white}
Directive: { identifier: pdflatex, parameters:
{reference=/home/paulo/doc1.tex, file=doc1.tex},
conditional: { NONE }, lines: [1] }

Directive: { identifier: pdflatex, parameters:
{reference=/home/paulo/doc2.tex, file=doc2.tex},
conditional: { NONE }, lines: [1] }
\end{codebox}

It is important to observe that, in this case, \rbox{doc3.tex} is a plain \TeX\ file, but \rbox{pdflatex} is actually being called on two \LaTeX\ documents, first \rbox{doc1.tex} and then, at last, \rbox{doc2.tex}.

Even when a directive is interpreted with a file other than the one being processed by \arara\ (through the magic of the \abox{files} parameter), it is possible to use helper methods in the rule context to get access to the original file and reference. Such methods are detailed later on, in Section~\ref{sec:files}, on page~\pageref{sec:files}.

\section{Migration guide}
\label{sec:migrationguide}

\begin{messagebox}{A note to users}{araracolour}{\icattention}{white}
If this is your first time using \arara\ or you do not have custom rules in the old format, you can safely ignore this section. All rules shipped with our tool are already written in the new format.
\end{messagebox}

As previously discussed in Section~\ref{sec:rule}, on page~\pageref{sec:rule}, version 4.0 of \arara\ introduces a new rule format. As a result, user-defined rules in the old format are incompatible with the new version of our tool and thus have to be updated. In short, we need to replace \abox{command} by \abox{commands} and specify a contextual element. As an example, consider the following hypothetical rule \rbox{ls} written in the old format:

\begin{codebox}{A rule in the old format}{teal}{\icnote}{white}
!config
identifier: ls
name: LS
command: ls @{details}
arguments:
- identifier: details
  flag: '@{ isTrue(parameters.details, "-l", "") }'
\end{codebox}

This rule does nothing too important, it simply runs the system command \rbox{ls} which lists the contents of the current directory. However, when we try to run \arara\ on a file which contains a directive referencing this rule, we get the following error in the terminal:

\begin{codebox}{Terminal}{teal}{\icnote}{white}
  __ _ _ __ __ _ _ __ __ _ 
 / _` | '__/ _` | '__/ _` |
| (_| | | | (_| | | | (_| |
 \__,_|_|  \__,_|_|  \__,_|

Processing 'doc4.tex' (size: 31 bytes, last modified: 05/27/2018
21:05:57), please wait.

I have spotted an error in rule 'ls' located at '/home/paulo'. I
could not parse the rule, something bad happened. Apparently, the
provided YAML file is invalid. I will do my best to help you in
any way I can. There are more details available on this
exception:

DETAILS ---------------------------------------------------------
Cannot create property=command for
JavaBean=com.github.cereda.arara.model.Rule@29774679
 in
'reader', line 1, column 1:
    !config
    ^
Unable to find
property 'command' on class: com.github.cereda.arara.model.Rule

in 'reader', line 4, column 10:
    command: ls @{details}
      
      ^


Total: 0.03 seconds
\end{codebox}

The above terminal output shows the usual error \arara\ raises when a rule in the old format is used, and thus the corresponding \gls{YAML} file is considered invalid. In order to fix the rule, we need to move the \abox{command} key inside a \rbox{commands$\rightarrow$} context as a list element, as seen as follows.

\begin{codebox}{A rule converted into the new format}{teal}{\icnote}{white}
!config
identifier: ls
name: LS
commands:
- command: ls @{details}
arguments:
- identifier: details
  flag: '@{ isTrue(parameters.details, "-l", "") }'
\end{codebox}

Note that this fix is sufficient to make the rule valid in the new format. Also, it is interesting to observe that the subtask will be unnamed during the execution, as there is no corresponding \abox{name} key  in the list element.

\begin{codebox}{Terminal}{teal}{\icnote}{white}
(LS) Unnamed task ....................................... SUCCESS
\end{codebox}

Now, let us consider an example containing a list of commands, also based on the old format. The \rbox{ls} rule was updated to include two runs of the system command of the same name in the current directory:

\begin{codebox}{A rule in the old format}{teal}{\icnote}{white}
!config
identifier: ls
name: LS
commands:
- ls @{details}
- ls @{details}
arguments:
- identifier: details
  flag: '@{ isTrue(parameters.details, "-l", "") }'
\end{codebox}

Observe that the old format directly represents commands as a list of plain strings. When trying to run \arara\ on a file which contains a directive referencing the updated \rbox{ls} rule, we get the following error in the terminal:

\begin{codebox}{Terminal}{teal}{\icnote}{white}
  __ _ _ __ __ _ _ __ __ _ 
 / _` | '__/ _` | '__/ _` |
| (_| | | | (_| | | | (_| |
 \__,_|_|  \__,_|_|  \__,_|

Processing 'doc4.tex' (size: 31 bytes, last modified: 05/27/2018
21:05:57), please wait.

I have spotted an error in rule 'ls' located at '/home/paulo'. I
could not parse the rule, something bad happened. Apparently, the
provided YAML file is invalid. I will do my best to help you in
any way I can. There are more details available on this
exception:

DETAILS ---------------------------------------------------------
Cannot create property=commands for
JavaBean=com.github.cereda.arara.model.Rule@91161c7
 in 'reader',
line 1, column 1:
    !config
    ^
No single argument
constructor found for class
com.github.cereda.arara.model.RuleCommand
 in 'reader', line 5,
column 1:
    - ls @{details}
    ^


Total: 0.02 seconds
\end{codebox}

The above terminal output shows a slightly different message, but the error is practically the same to the one \arara\ raised before when a rule in the old format was used. The difference relies on the missing rule property being set at the moment, but the idea remains the same. In order to fix the rule, we need to precede every list element in the \rbox{commands$\rightarrow$} context with the \abox{command} key, as seen as follows.

\begin{codebox}{A rule converted into the new format}{teal}{\icnote}{white}
!config
identifier: ls
name: LS
commands:
- command: ls @{details}
- command: ls @{details}
arguments:
- identifier: details
  flag: '@{ isTrue(parameters.details, "-l", "") }'
\end{codebox}

This fix is sufficient to make the rule valid in the new format. Also, as mentioned before, it is interesting to observe that the subtasks will be unnamed during the execution, as there are no corresponding \abox{name} keys in the list elements.

\begin{codebox}{Terminal}{teal}{\icnote}{white}
(LS) Unnamed task ....................................... SUCCESS
(LS) Unnamed task ....................................... SUCCESS
\end{codebox}

There is a helper tool available in the \href{https://github.com/cereda/arara/releases/tag/4.0}{release section} of our project repository that attempts to automatically convert rules in the old format to the new one. If you want to try it, download the \rbox{rc.jar} file from the repository and put it in the same directory where the old rules are located. You can also provide a full path instead. It is important to note that, although the tool might indicate a successful conversion, there are no guarantees that the resulting rule is fully compliant with the new format, due to potential changes in the internal workings of \arara, so your mileage may vary. In general, it should work. The rule converter is written in Java and requires a virtual machine to run. The tool has a straightforward workflow and takes just one parameter referring to the rule to be converted. The entire process should happen without intervention. When invoked without the file name, this is the expected output:

\begin{codebox}{Terminal}{teal}{\icnote}{white}
$ java -jar rc.jar
         _                                _
 ___ _ _| |___    ___ ___ ___ _ _ ___ ___| |_ ___ ___
|  _| | | | -_|  |  _| . |   | | | -_|  _|  _| -_|  _|
|_| |___|_|___|  |___|___|_|_|\_/|___|_| |_| |___|_|

version 1.0 (rules < 4.0)

OH NO! -----------------------------------------------------
This tool expects the YAML rule from previous versions of
arara. Please, provide a proper YAML file containing the old
rule as a parameter and try again. I will do my best to
convert the rule to the new version 4.0 format.
\end{codebox}

Let us invoke the tool with the first version of our hypothetical \rbox{ls} rule, still in the old format. The tool removes all comments from the original file, if any, and constructs a new file with a \rbox{\_v4} suffix attached to the name. The original file is preserved. Just keep in mind that the new rule must be renamed afterwards, as the base name and the corresponding \abox{identifier} key must match. The output is presented as follows.

\begin{codebox}{Terminal}{teal}{\icnote}{white}
$ java -jar rc.jar ls.yaml
         _                                _
 ___ _ _| |___    ___ ___ ___ _ _ ___ ___| |_ ___ ___
|  _| | | | -_|  |  _| . |   | | | -_|  _|  _| -_|  _|
|_| |___|_|___|  |___|___|_|_|\_/|___|_| |_| |___|_|

version 1.0 (rules < 4.0)

The provided YAML rule looks OK. I will try my best to
convert it to the new version 4.0 format adopted by arara.
The new rule name will be written in the same directory of
the original one and will have a '_v4' suffix to it. Keep in
mind that the base name must match the identifier!

YAY! -------------------------------------------------------
Good news, everybody! The provided YAML rule was updated
successfully to the new version 4.0 format of arara! Of
course, there are no guarantees this new rule will work out
of the box, so fingers crossed! Take a closer look at the
manual and update your rule to use the new enhancements of
arara. Have a great time!
\end{codebox}

The resulting rule is identical to the one manually converted in this section. Just note that, when creating the file, the resulting \gls{YAML} file might write the keys in alphabetical order. That means that, although both files semantically represent the same rule, the positions of the keys differ. However, that poses no issue at all, as long as the keys are correctly defined. Also, it is important to note that, due to a conversion policy of the underlying \gls{YAML} library, folded scalars in the old format are transcribed as literal scalars in the new format. We could force a folded style as default, but the resulting rule would be unnecessarily verbose, so we opted for the simpler, cleaner solution. As a direct consequence, we strongly recommend a subsequent verification and potential fix of existing literal scalars into folded ones, if any. Please refer to Section~\ref{sec:yamlscalars}, on page~\pageref{sec:yamlscalars}, for more details on scalars.

\begin{messagebox}{Replace plain strings in commands}{araracolour}{\icattention}{white}
As plain strings are known to be problematic when defining commands, they are marked as deprecated in version 4.0 of \arara\ and will likely be removed in future releases. Since you are migrating from an old format to a new one, please consider replacing plain strings in command by proper \rbox{Command} objects. The helper methods available in the rule context, including the indispensable \mtbox{getCommand} method, for obvious reasons, are detailed in Section~\ref{sec:commandsandtriggers}, on page~\pageref{sec:commandsandtriggers}. It is highly advisable to update your rules on this regard, if applicable.
\end{messagebox}

This section pretty much covered the basics for correctly migrating rules in the old format to the new one. Of course, it is highly advisable to make use of the new features available in \arara\ 4.0 for achieving better results. If you need any help, please do not hesitating in contacting us. See Section~\ref{sec:support}, on page~\pageref{sec:support}, for more details on how to get help.