summaryrefslogtreecommitdiff
path: root/macros/latex/contrib/robust-externalize/robust-externalize.tex
blob: 561cde37a509987aa5980618be3594f839ddff18 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
\documentclass[a4paper,doc2]{ltxdoc} % doc2 is needed to force the old version, or links get colored in a weird red way even with hidelinks. https://github.com/latex3/latex2e/issues/822

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% Packages
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Warning: if you compile and get:
%% ERROR: Argument of \tikz@lib@matrix@with@options has an extra }.
%% make sure to fix catcodes around it as | is given a different meaning in ltxdoc.

\usepackage{amsmath}
\usepackage[margin=3cm]{geometry}
\usepackage{calc}
\usepackage{tikz}
\usetikzlibrary{shadows,fit}
% \usetikzlibrary fails because file is not in current directory, lazy to setup TEXINPUTS
\makeatletter
  \usepackage{robust-externalize}
\makeatother
  
% Loads the great package that produces tikz-like manual (see also tikzcd for examples)
\input{pgfmanual-en-macros.tex} % Is supposed to be included in recent TeX distributions, but I get errors...

%% For verbatim environments:

\NewDocumentEnvironment{codeAndResult}{}{\XSIMfilewritestart{\jobname-codeAndResult-tmp-file-you-can-remove.tmp}}%
{%
  \XSIMfilewritestop%
  \par
  \medskip
  \noindent\colorbox{graphicbackground}{\noindent\begin{minipage}{1.0\linewidth}
  {\input{\jobname-codeAndResult-tmp-file-you-can-remove.tmp}}
  \end{minipage}}
  % I don't know why, but sometimes this prints nothing, while input works:
  %{\codeexample[width=0pt, leave comments, vbox, from file={\jobname-codeAndResult-tmp-file-you-can-remove.tmp}]}%
  {\codeexample[code only, from file={\jobname-codeAndResult-tmp-file-you-can-remove.tmp}]}%
}

\usepackage{makeidx} % Produces an index of commands.
\makeindex % Useful or not index will be created
\usepackage{alertmessage} % For warning, info...
\newcommand{\mylink}[2]{\href{#1}{#2}\footnote{\url{#1}}}
\usepackage{verbatim}
\usepackage{mathtools}
\usepackage{float} %figure inside minipage
\usepackage{pythonhighlight}
\usepackage{tcolorbox}

\usepackage{listings}
\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}

\lstdefinestyle{mystyle}{
    backgroundcolor=\color{backcolour},   
    commentstyle=\color{codegreen},
    keywordstyle=\color{magenta},
    numberstyle=\tiny\color{codegray},
    stringstyle=\color{codepurple},
    basicstyle=\ttfamily\footnotesize,
    breakatwhitespace=false,         
    breaklines=true,                 
    captionpos=b,                    
    keepspaces=true,                 
    numbers=left,                    
    numbersep=5pt,                  
    showspaces=false,                
    showstringspaces=false,
    showtabs=false,                  
    tabsize=2
}

\lstset{style=mystyle}


\usepackage[hidelinks]{hyperref}
\usepackage{cleveref}

\begin{document}
%%% Title: thanks tikzcd for the styling
\begin{center}
  \vspace*{1em} % Thanks tikzcd
  \tikz\node[scale=1.2]{%
    \color{gray}\Huge\ttfamily \char`\{\raisebox{.09em}{\textcolor{red!75!black}{robust\raisebox{-0.1em}{-}externalize}}\char`\}};

  \vspace{0.5em}
  {\Large\bfseries Cache anything (\tikzname, python…),\\in a robust, efficient and pure way.}

  \vspace{1em}
  {Léo Colisson \quad Version 1.0}\\[3mm]
  {\href{https://github.com/leo-colisson/robust-externalize}{\texttt{github.com/leo-colisson/robust-externalize}}}
\end{center}


\tableofcontents

\bigskip

\textbf{WARNING: This library is young and has not been tested extensively (and is an important rewrite of a previous version). Even if we try to stay backward compatible, the only guaranteed way to be immune to changes is to copy/paste the library in your main project folder. Please report any bug to \url{https://github.com/leo-colisson/robust-externalize}, or let us know if it works!}

\section{A taste of this library}

This library allows you to cache any language: not only \LaTeX{} documents and \tikzname{} images, taking into account depth and overlays:
\begin{codeexample}[width=0pt,vbox]
  \robExtConfigure{
    tikz/.append style={
      add to preamble={\usepackage{pifont}}
    }
  }
  The next picture is cached %
  \begin{tikzpictureC}[baseline=(A.base)]
    \node[fill=red, rounded corners](A){My node that respects baseline \ding{164}.};
    \node[fill=red, rounded corners, opacity=.3,overlay] at (A.north east){I am an overlay text};
  \end{tikzpictureC} and you can see that overlay and depth works.
\end{codeexample}


\noindent but also arbitrary code (e.g.\ python). You can also define arbitrary compilation commands, inclusion commands, and presets to fit you need. For instance, you can create a preset to obtain:

\begin{codeAndResult}
\begin{CacheMeCode}{python print code and result, set title={The for loop}}
for name in ["Alice", "Bob"]:
    print(f"Hello {name}")
\end{CacheMeCode}
\end{codeAndResult}

Actually, we also provide this style by default (and explain how to write it yourself), you just make sure to load:
\begin{verbatim}
\usepackage{pythonhighlight}
\usepackage{tcolorbox}
\end{verbatim}

To achieve a pleasant and configurable interface, we introduced placeholders, that can be of independent interest.

\section{Introduction}

\subsection{Why do I need to cache (a.k.a. externalize) parts of my document?}

One often wants to cache (i.e.\ store pre-compiled parts of the document, like figures) operations that are long to do: For instance, TikZ is great, but TikZ figures often take time to compile (it can easily take a few seconds per picture). This can become really annoying with documents containing many pictures, as the compilation can take multiple minutes: for instance my thesis needed roughly 30mn to compile as it contains many tiny figures, and LaTeX needs to compile the document multiple times before converging to the final result. But even on much smaller documents you can easily reach a few minutes of compilation, which is not only high to get a useful feedback in real time, but worse, when using online \LaTeX{} providers (e.g. overleaf), this can be a real pain as you are unable to process your document due to timeouts.

Similarly, you might want to cache the result of some codes, for instance a text or an image generated via python and matplotlib, without manually compiling them externally.

\subsection{Why not using \tikzname{}'s externalize library?}

\tikzname{} has an externalize library to pre-compile these images on the first run. Even if this library is quite simple to use, it has multiple issues:
\begin{itemize}
\item If you add a picture before existing pre-compiled pictures, the pictures that are placed after will be recompiled from scratch. This can be mitigated by manually adding a different prefix to each picture, but it is highly not practical to use.
\item To compile each picture, TikZ's externalize library reads the document's preamble and needs to process (quickly) the whole document. In large documents (or in documents relying on many packages), this can result in a significant loading time, sometimes much bigger than the time to compile the document without the externalize library: for instance, if the document takes 10 seconds to be processed, and if you have 200 pictures that take 1s each to be compiled, the first compilation with the TikZ's externalize library will take roughly half an hour instead of 3mn without the library. And if you add a single picture at the beginning of the document… you need to restart everything from scratch. For these reasons, I was not even able to compile my thesis with TikZ's external library in a reasonable time.
\item  If two pictures share the same code, it will be compiled twice
\item Little purity is enforced: if a macro changes before a pre-compiled picture that uses this macro, the figure will not be updated. This can result in different documents depending on whether the cache is cleared or not.
\item As far as I know, it is made for TikZ picture mostly, and is not really made for inserting other stuff, like matplotlib images generated from python etc...
\item According to some maintainers of TikZ, ``\mylink{https://github.com/pgf-tikz/pgf/issues/758}{the code of the externalization library is mostly unreadable gibberish}'', and therefore most of the above issues are unlikely to be solved in a foreseable future.
\end{itemize}

\subsection{FAQ}

\paragraph{What is supported?}

You can cache most things, including tikz pictures, (including ones with overlays (but not with remember picture), with depth etc.), python code etc. We tried to make the library as customizable as possible to be useful in most scenarios. You can also feed data back to the main document (say that you want to compute a value that takes time to compute, or compute the number of pages of the produced document in order to increase the number of pages accordingly\dots{}).

\paragraph{What is not supported?}

We do not yet support remember picture, and you can't use (yet) cross-references inside your images (at least not without further hacks). Note that this library is quite young, so expect untested things.

\paragraph{What OS are supported?}

I tested the library mostly on Linux systems, but the library should work on all OS. Please let me know if it fails for you.

\paragraph{Do I need to compile using -shell-escape?}

Since we need to compile the images via an external command, the simpler option is to add the argument |-shell-escape| to let the library run the  compilation command automatically (this is also the case of \tikzname's externalize library). However, people worried by security issues of |-shell-escape| (that allows arbitrary code execution if you don't trust the \LaTeX{} code) might be interested by these facts:
\begin{itemize}
\item If images are all already cached, you don't need to enable \texttt{-shell-escape} (this might be interesting e.g. to send the files a pre-cached document to the arxiv or to a publisher: just make sure to include the cache folder).
\item You can choose to display a dummy content until you choose to compile them.
\item You can compile manually the images: all the commands that are left to be executed are listed in \texttt{robExt-compile-missing-figures.sh} and you can just run them, either with \texttt{bash robExt-compile-missing-figures.sh} or by typing them manually (most of the time it's only a matter of running \texttt{pdflatex somefile.tex}).
\end{itemize}


\paragraph{Is it working on overleaf?}

Yes: overleaf automatically compiles documents with |-shell-escape|, so nothing special needs to be done there (of course, if you use this library to run some code, the programming language might not be available, but I heard that python is installed on overleaf servers for instance, even if this needs to be doubled checked). If the first compilation of the document to cache images times out, you can just repeat this operation multiple times until all images are cached.

\paragraph{Do you have some benchmarks?}

On an early draft of a small paper containing 76 small tikz-cd based pictures (from my other zx-calculus library), we measured:
\begin{itemize}
\item 35 seconds for a normal compilation without externalization
\item 75 seconds for the first compilation with this library
\item 2.4 seconds for the next runs
\end{itemize}
So during the first compilation, we lost a x2 factor (roughly an additional time of .5 seconds per picture coming from the time to start \LaTeX{}, it seems like on average a picture takes .5 seconds to be built in my benchmark), but then we have a speedup of x15 (2.43s instead of 34.63s) for all subsequent runs. And I expect this to be even higher with more pictures and more complex documents.

\paragraph{Can I use version-control to keep the cached files in my repository?} Sure, each cached figure is stored in a few files (typically one pdf and one \LaTeX{} file, plus the source) having the same prefix (the hash), avoiding collision between runs. Just commit these files and you are good to go.

\paragraph{Can you deal with baseline position ?} Yes, the depth of the box is automatically computed and used to include the figure by default.

\paragraph{How is purity enforced?} Purity is the property that if you remove the cached files and recompile your document, you should end-up with the same output. To enforce purity, we compute the hash of the final program, including the compilation command and the dependency files used for instance in |\input{include.tex}| (unless you prefer not to, for instance to keep parts of the process impure for efficiency reasons), and put the code in a file named based on this hash. Then we compile it if it has not been used before, and include the output. Changing a single character in the file, the tracked dependencies, or the compilation command will lead to a new hash, and therefore to a new generated picture.

\paragraph{What if I don't want purity for all files?} If you do not want your files to be recompiled if you modify a given file, then just do not add this file to the list of dependencies.

\paragraph{Can I extend it easily?} We tried to take a quite modular approach in order to allow easy extensions. Internally, to support a new cache scheme, we only expect a string containing the program (possibly produced using a template), a list of dependencies, a command to compile this program (e.g. producing a pdf and possibly a tex file with the properties (depth…) of the pdf), and a command to load the result of the compilation into the final document (called after loading the previously mentioned optional tex file). Thanks to pgfkeys, it is then possible to create simple pre-made settings to automatically apply when needed.

\paragraph{How does it compare with \url{https://github.com/sasozivanovic/memoize}?} I recently became aware of the great \url{https://github.com/sasozivanovic/memoize}. While me aim to solve a similar goal, our approaches are quite different. While we focus on purity, and therefore create a different file for each picture, the above project puts all pictures in a single file and compile them all at once to avoid losing time to run the latex command for each picture (this mostly makes a difference for the first compilation). Our understanding of the main differences is the following:

Pros of \url{https://github.com/sasozivanovic/memoize}:
\begin{itemize}
\item The above library is likely to be quicker on the first run since it packs everything in a single file (so you save the time to run latex for each picture). On the other hand, we can load in the preamble exactly what is needed for the picture (we do not load all the preamble of the main file), so our startup time is not huge (it adds .5s per picture in my tests when using the |zx-calculus| library), and you can still commit the cached file to help with recompiling the document elsewhere.
\item It seems to be easier to setup as they do not need to specify the preamble of the file to compile (the preamble of the main file is used), and tikz picture are automatically memoized (we provide |\robExtExternalizeAllTikzpictures| for that, but you still need to specify the preamble once).
\end{itemize}

Cons of \url{https://github.com/sasozivanovic/memoize}:
\begin{itemize}
\item The purity is not enforced so strongly since all images are in the same file. Notably, the hash only depends on the picture, but not on its context. So for instance if you define |\def\mycolor{blue}| before the picture, and use |\mycolor| inside the picture, if you change later the color to, say, |\def\mycolor{red}|, the picture will not be recompiled (so cleaning the cache and recompiling would produce a different result). In our case, the purity is always strictly enforced (unless you choose not to).
\item As a result, the above library has poor support for contexts (in our case, you can easily, for instance, make a picture depend on the current page, and recompile the picture only if the current page changes: you can also do it the other way, and change some counters, say, depending on the cached file).
\item The above library only focuses on \LaTeX{} while our library works for any language
\item The above library can only produce pdf formats, while we can generate any format (text, tex, jpg\dots{} and even videos that I include in my beamer presentation).
\item We have an arguably more complete documentation.
\end{itemize}

Note that |remember picture| is not working in both libraries.

\section{Quickstart}

\subsection{Installation}

To install the library, just copy the |robust-externalize.sty| file into the root of the project. Then, load the library using:\\

|\usepackage{robust-externalize}|


\subsection{Caching a tikz picture}

If you only care about \tikzname's picture, you have 3 options:
\begin{enumerate}
\item Call once |\robExtExternalizeAllTikzpictures| that will redefine |tikzpicture| to use our library (if you use this solution, make sure to read how to disable externalization (\cref{sec:disableExternalization}) as we do not support for instance |remember picture|). Then, configure the default preamble for cached files as explained below.
\item Use |tikzpictureC| instead of |tikzpicture| (this is mostly done to easily convert existing code to this library, but works only for |tikz| pictures).
\item Use the more general |CacheMe| environment, that can cache \tikzname, \LaTeX{}, python, and much more.
\end{enumerate}

These 3 options are illustrated below (note that the newly defined |tikzpicture| and |tikzpictureC| accept a second optional argument that contains the options to pass to |CacheMe| after loading the |tikz| preset):

Option 1:
\begin{codeexample}[width=0pt]
%% We override the default tikzpicture environment
%% to externalize all pictures
%% Warning: it will cause troubles with pictures relying on |remember pictures|
\robExtExternalizeAllTikzpictures

I am a cached picture: \begin{tikzpicture}[baseline=(A.base)]
  \node[draw,rounded corners,fill=pink!60](A){Hello World!};
\end{tikzpicture}.
\end{codeexample}

Option 2:
\begin{codeexample}[width=0pt]
I am a cached picture: \begin{tikzpictureC}[baseline=(A.base)]
  \node[draw,rounded corners,fill=pink!60](A){Hello World!};
\end{tikzpictureC}.
\end{codeexample}


Option 3:
\begin{codeexample}[width=0pt]
I am a cached picture: \begin{CacheMe}{tikz}[baseline=(A.base)]
  \node[draw,rounded corners,fill=pink!60](A){Hello World!};
\end{CacheMe}.
\end{codeexample}

Since |CacheMe| is more general as it applies also to non-tikz pictures (just replace |tikz| with the style of your choice), we will mostly use this syntax from now.

\subsection{Custom preamble}

Note that the pictures are compiled in a separate document, with a different preamble and class (we use the standalone class). This is interesting to reduce the compilation time of each picture (loading a large preamble is really time consuming) and to avoid unnecessary recompilation (do you want to recompile all your pictures when you add a single new macro?) without sacrificing the purity. But of course, you need to provide the preamble of the pictures. The easiest way is probably to modify the |tikz| preset (you can also modify the |latex| preset if you want the change to apply to all \LaTeX{} documents):

\begin{codeexample}[width=0pt,vbox]
  \robExtConfigure{
    tikz/.append style={
      add to preamble={\usetikzlibrary{shadows}},
    },
  }

  See, tikz's style now packs the |shadows| library by default: %
  \begin{CacheMe}{tikz}[even odd rule]
    \filldraw [drop shadow,fill=white] (0,0) circle (.5) (0.5,0) circle (.5);
  \end{CacheMe}
\end{codeexample}

You can also choose to overwrite it for a single picture (or even a block of picture if you run the |\robExtConfigure| and |CacheMe| inside a group |{ ... }|):
\begin{codeexample}[width=0pt,vbox]
  See, you can add something to the preamble of a single picture: %
  \begin{CacheMe}{tikz, add to preamble={\usetikzlibrary{shadows}}}[even odd rule]
    \filldraw [drop shadow,fill=white] (0,0) circle (.5) (0.5,0) circle (.5);
  \end{CacheMe}
\end{codeexample}

Note that if you use the |tikzpictureC| or |tikzpicture| syntax, you want to add the options after the tikz options (leave an empty bracket if there is none):
\begin{codeexample}[width=0pt,vbox]
  See, you can add something to the preamble of a single picture: %
  \begin{tikzpictureC}[even odd rule][add to preamble={\usetikzlibrary{shadows}}]
    \filldraw [drop shadow,fill=white] (0,0) circle (.5) (0.5,0) circle (.5);
  \end{tikzpictureC}
\end{codeexample}

\subsection{Dependencies}\label{sec:dependencies}

It might be handy to have a file that is loaded in both the main document and in the cached pictures. For instance, if you have a file |common_inputs.tex| that you want to input in both the main file and in the cached files, that contains, say:\\
|\def\myValueDefinedInCommonInputs{42}|\\
then you can add it as a dependency this way (here we use the |latex| preset that does not wrap the code inside a |tikzpicture| only to illustrate that we can also cache things that is not generated by tikz):
\begin{codeexample}[width=0pt,vbox]
\begin{CacheMe}{latex,
    add dependencies={common_inputs.tex},
    add to preamble={\input{__ROBEXT_WAY_BACK__/common_inputs.tex}}}
  The answer is \myValueDefinedInCommonInputs.
\end{CacheMe}
\end{codeexample}
Note that the placeholder |__ROBEXT_WAY_BACK__| contains the path from the cache folder (containing the |.tex| that will be cached) to the root folder, and will be replaced when creating the file. This way, you can easily input files contained in the root folder. You can also create your own placeholders, read more below.

You can note that we used |add dependencies={common_inputs.tex}|: this allows us to recompile the files if |common_inputs.tex| changes. If you do not want this behavior (e.g. |common_inputs.tex| changes too often and you do not want to recompile everything at every change), you can remove this line, but beware: if you do a breaking changes in |common_inputs.tex| (e.g.\ redefine |42| to |43|), then the previously cached picture will not be recompiled! (So you will still read 42 instead of 43.)

\subsection{Disabling externalization}\label{sec:disableExternalization}

You can use |disable externalization| to disable externalization (which is particularly practical if you set |\robExtExternalizeAllTikzpictures|). You can configure the exact command run in that case using |command if no externalization/.code={...}|, see \cref{sec:disableExternalization} for details.

\begin{codeexample}[width=0pt,vbox]
  % In theory all pictures should be externalized (so remember picture should fail)
  \tikz[remember picture,baseline=(pointtome1.base)]
    \node[rounded corners, fill=orange](pointtome1){Point to me if you can};\\
  \robExtExternalizeAllTikzpictures
  % But we can disable it temporarily
  \begin{tikzpicture}[remember picture][disable externalization]
    \node[rounded corners, fill=red](A){This figure is not externalized.
      This way, it can use remember picture.};
    \draw[->,overlay] (A) to[bend right] (pointtome1);
  \end{tikzpicture}\\

  % You can also disable it globally/in a group:
  {
    \robExtConfigure{disable externalization}
    
    \begin{tikzpicture}[remember picture]
      \node[rounded corners, fill=red](A){This figure is not externalized.
        This way, it can use remember picture.};
      \draw[->,overlay] (A.west) to[bend left] (pointtome1);
    \end{tikzpicture}\\

    \begin{tikzpicture}[remember picture]
      \node[rounded corners, fill=red](A){This figure is not externalized.
        This way, it can use remember picture.};
      \draw[->,overlay] (A.east) to[bend right] (pointtome1);
    \end{tikzpicture}\\
  }
  
  \begin{tikzpicture}
    \node[rounded corners, fill=green](A){This figure is externalized, but cannot use remember picture.};
  \end{tikzpicture}
\end{codeexample}


\subsection{Feeding data from the main document to the picture}

You can feed data from the main document to the cached file using placeholders, since |set placeholder eval={__foo__}{\bar}| will evaluate |\bar| and put the result in |__foo__|. For instance, if the picture depends on the current page, you can do:

\begin{codeexample}[width=0pt,vbox]
\begin{tikzpictureC}[][set placeholder eval={__thepage__}{\thepage}]
  \node[rounded corners, fill=red]{The current page is __thepage__.};
\end{tikzpictureC}
\end{codeexample}

\subsection{Feeding data back into the main document}

For more advanced usage, you might want to compute a data and cache the result in a macro that you could use later. This is possible if you write into the file |\jobname-out.tex| during the compilation of the cached file (by default, we already open |\writeRobExt| to write to this file). This file will be automatically loaded before loading the pdf (but you can customize all these operations, for instance if you do not want to load the pdf at all; the only requirement is that you should generate a |.pdf| file to specify that the compilation is finished).

For instance:

\begin{codeexample}[width=0pt,vbox]
\begin{CacheMe}{latex, add to preamble={\usepackage{tikz}}, do not include pdf}
We compute this data that is long to compute:
\pgfmathparse{(1 + sqrt(5))/2}% result is stored in \pgfmathresult
% We write the result to the -out file (\string\foo writes \foo to the file without evaluating it,
% so this will write "\gdef\myLongResult{1.61803}"):
% Note that CacheMe is evaluated in a group, so you want to use \gdef to define it
% outside of the group
\immediate\write\writeRobExt{%
  \string\gdef\string\myLongResult{\pgfmathresult}%
}
\end{CacheMe}

We computed the cached value \myLongResult.
\end{codeexample}

\subsection{For non-\LaTeX{} code}

Due to the way \LaTeX{} works, non-\LaTeX{} code can't be reliably read inside macros and some environments that parse their body (e.g. align) as some characters are removed (e.g. percent symbols are comments and are removed). For this reason, we sometimes need to separate the time where we define the code and where we insert it (this is done using placeholders, see |PlaceholderFromCode|), and we need to introduce new environments to populate the template (see \cref{sec:placeholders} for more details, to generate them from filename, to get the path of a file etc).

The environment |CacheMeCode| can be used for this purpose.

\subsubsection{Python code}

\paragraph{Generate an image}

For instance, you can use the default |python| template to generate an image with python. The following code:


\begin{codeexample}[code only]
\begin{CacheMeCode}{python, set includegraphics options={width=.8\linewidth}}
import matplotlib.pyplot as plt
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials')   
plt.savefig("__ROBEXT_OUTPUT_PDF__")  
\end{CacheMeCode}
\end{codeexample}

will produce the image visible in \cref{fig:pythonGeneratedImage}. \textbf{Importantly: you do not want to indent the content of CacheMeCode, or the space will also appear in the final code.}
\begin{figure}
\centering
\begin{CacheMeCode}{python, set includegraphics options={width=.8\linewidth}}
import matplotlib.pyplot as plt
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials')   
plt.savefig("__ROBEXT_OUTPUT_PDF__")
\end{CacheMeCode}
\caption{Image generated with python.}
\label{fig:pythonGeneratedImage}
\end{figure}

\paragraph{Compute a value}

We also provide by default a number of helper functions. For instance, |write_to_out(text)| will write |text| to the |*-out.tex| file that is loaded automatically by \LaTeX{}. This is useful to compute data that is not an image (note that |r"some string"| does not consider backslash as an escape string, which is handy to write \LaTeX{} code in python):

For instance:
\begin{codeAndResult}
\begin{CacheMeCode}{python, do not include pdf}
import math
write_to_out(r"\gdef\cosComputedInPython{" + str(math.cos(1)) + r"}")
\end{CacheMeCode}

$\rightarrow$ The cosinus of 1 is \cosComputedInPython.
\end{codeAndResult}

\paragraph{Improve an existing preset}

If you often use the same code (e.g.\ load matplotlib, save the file etc), you can directly modify the |__ROBEXT_MAIN_CONTENT__| placeholder to add the redundant information (or create a new template from scratch, see below), as this placeholder contains the code typed by the user (this is true for all presets, as |CacheMe*| is in charge of setting this placeholder):

\begin{codeAndResult}
%% Create your style:
\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEMPLATE_BEFORE__}
import matplotlib.pyplot as plt
import sys
\end{PlaceholderFromCode}

\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEMPLATE_AFTER__}
plt.savefig("__ROBEXT_OUTPUT_PDF__")  
\end{PlaceholderFromCode}

\robExtConfigure{
  my matplotlib/.style={
    python,
    add before placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_MATPLOTLIB_TEMPLATE_BEFORE__},
    add to placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_MATPLOTLIB_TEMPLATE_AFTER__},
  },
}

%% Use your style:
%% See, you don't need to load matplotlib or save the file:
\begin{CacheMeCode}{my matplotlib, set includegraphics options={width=.5\linewidth}}
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials')     
\end{CacheMeCode}
\end{codeAndResult}

\paragraph{Custom parameters and placeholders}

Let us say that you would like to define a default font size for your figure, but that you would like to allow the user to change this font size. Then, you should create a new placeholder with your default value, and use |set placeholder| to change this value later (see also the documentation of |CacheMeCode| to see how to create a new command to avoid typing |set placeholder|):

\begin{codeAndResult}
%% Create your style:

\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEMPLATE_BEFORE__}
import matplotlib as mpl
import matplotlib.pyplot as plt
import sys
mpl.rcParams['font.size'] = __MY_MATPLOTLIB_FONT_SIZE__
\end{PlaceholderFromCode}

\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEMPLATE_AFTER__}
plt.savefig("__ROBEXT_OUTPUT_PDF__")  
\end{PlaceholderFromCode}

\robExtConfigure{
  my matplotlib/.style={
    python,
    % We create a new placeholder (it is simple enough that you don't need to use PlaceholderFromCode)
    set placeholder={__MY_MATPLOTLIB_FONT_SIZE__}{12},
    add before placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_MATPLOTLIB_TEMPLATE_BEFORE__},
    add to placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_MATPLOTLIB_TEMPLATE_AFTER__},
  },
}

%% Use your style:
%% See, you don't need to load matplotlib or save the file:
Default font size: \begin{CacheMeCode}{my matplotlib, set includegraphics options={width=.5\linewidth}}
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials')     
\end{CacheMeCode}

With font size 16:
\begin{CacheMeCode}{my matplotlib,
    set includegraphics options={width=.5\linewidth},
    set placeholder={__MY_MATPLOTLIB_FONT_SIZE__}{16}}
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials')     
\end{CacheMeCode}
\end{codeAndResult}

Note that if you manage to move all the code in the template and that the user can configure everything using the options and an empty content, you can use |CacheMeNoContent| that takes no argument and that consider its body as the options.

\paragraph{Custom include command}

There may be some cases where you do not want to include a picture. We already saw the option |do not include pdf| if you do not want to include anything. But you can customize the include function, using notably:\\
|custom include command={your include command}|

For instance, let us say that you would like to display both the source code used to obtain a given code, together with the output of this code. Then, you can write this style:

\begin{codeexample}[code only]
{
%% Create your style:
\begin{PlaceholderFromCode}{__MY_PRINT_BOTH_TEMPLATE_BEFORE__}
# File where print("bla") should be redirected
# get_filename_from_extension("-foo.txt") will give you the path of the file
# in the cache that looks like robExt-somehash-foo.txt
print_file = open(get_filename_from_extension("-print.txt"),  "w")
sys.stdout = print_file
# This code will read the current code, and extract the lines between
# that starts with "### CODESTARTSHERE" and "### CODESTOPSHERE", and will write
# it into the *-code.text (we do not want to print all these functions in
# the final code)
with open(get_filename_from_extension("-code.txt"), "w") as f:
    # The current script has extension .tex
    with open(get_current_script(), "r") as script:
        should_write = False
        for line in script:
            if line.startswith("### CODESTARTSHERE"):
                should_write = True
            elif line.startswith("### CODESTOPSHERE"):
                should_write = False
            elif "HIDEME" in line:
                pass
            else:
                if should_write:
                    f.write(line)
### CODESTARTSHERE
\end{PlaceholderFromCode}


\begin{PlaceholderFromCode}{__MY_PRINT_BOTH_TEMPLATE_AFTER__}
### CODESTOPSHERE
print_file.close()
\end{PlaceholderFromCode}

\robExtConfigure{
  my python print both/.style={
    python,
    add before placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_PRINT_BOTH_TEMPLATE_BEFORE__},
    add to placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_PRINT_BOTH_TEMPLATE_AFTER__},
    set title/.style={
      set placeholder={__MY_TITLE__}{##1},
    },
    set title={Example},
    custom include command={
      % Useful to replace __MY_TITLE__:
      \evalPlaceholder{
        % \verbatiminput{\robExtAddCachePathAndName{\robExtFinalHash-code.txt}}
        \begin{tcolorbox}[colback=red!5!white,colframe=red!75!black,title=__MY_TITLE__]
          \lstinputlisting[frame=single,
            breakindent=.5\textwidth,
            frame=single,
            breaklines=true,
            style=mypython]{\robExtAddCachePathAndName{\robExtFinalHash-code.txt}}
          Output: 
          \verbatiminput{\robExtAddCachePathAndName{\robExtFinalHash-print.txt}}
        \end{tcolorbox}
      }
    },
  },
}
\end{codeexample}

Once the style is defined (actually we already defined in the library under the name |python print code and result|), you can just write:
\begin{codeexample}[code only]
\begin{CacheMeCode}{my python print both, set title={The for loop}}
for name in ["Alice", "Bob"]:
    print(f"Hello {name}")
\end{CacheMeCode}
\end{codeexample}
to get:
{
%% Create your style:
\begin{PlaceholderFromCode}{__MY_PRINT_BOTH_TEMPLATE_BEFORE__}
# File where print("bla") should be redirected
# get_filename_from_extension("-foo.txt") will give you the path of the file
# in the cache that looks like robExt-somehash-foo.txt
print_file = open(get_filename_from_extension("-print.txt"),  "w")
sys.stdout = print_file
# This code will read the current code, and extract the lines between
# that starts with "### CODESTARTSHERE" and "### CODESTOPSHERE", and will write
# it into the *-code.text (we do not want to print all these functions in
# the final code)
with open(get_filename_from_extension("-code.txt"), "w") as f:
    # The current script has extension .tex
    with open(get_current_script(), "r") as script:
        should_write = False
        for line in script:
            if line.startswith("### CODESTARTSHERE"):
                should_write = True
            elif line.startswith("### CODESTOPSHERE"):
                should_write = False
            elif "HIDEME" in line:
                pass
            else:
                if should_write:
                    f.write(line)
### CODESTARTSHERE
\end{PlaceholderFromCode}


\begin{PlaceholderFromCode}{__MY_PRINT_BOTH_TEMPLATE_AFTER__}
### CODESTOPSHERE
print_file.close()
\end{PlaceholderFromCode}

\robExtConfigure{
  my python print both/.style={
    python,
    add before placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_PRINT_BOTH_TEMPLATE_BEFORE__},
    add to placeholder no space={__ROBEXT_MAIN_CONTENT__}{__MY_PRINT_BOTH_TEMPLATE_AFTER__},
    set title/.style={
      set placeholder={__MY_TITLE__}{##1},
    },
    set title={Example},
    custom include command={
      % Useful to replace __MY_TITLE__:
      \evalPlaceholder{
        % \verbatiminput{\robExtAddCachePathAndName{\robExtFinalHash-code.txt}}
        \begin{tcolorbox}[colback=red!5!white,colframe=red!75!black,title=__MY_TITLE__]
          % \noindent Code:
          \lstinputlisting[frame=single,breakindent=.5\textwidth,frame=single,breaklines=true,style=mypython]{\robExtAddCachePathAndName{\robExtFinalHash-code.txt}}
          Output: 
          \verbatiminput{\robExtAddCachePathAndName{\robExtFinalHash-print.txt}}
        \end{tcolorbox}
      }
    },
  },
}

%% Use your style:
\begin{CacheMeCode}{my python print both, set title={The for loop}}
for name in ["Alice", "Bob"]:
    print(f"Hello {name}")
\end{CacheMeCode}
}

\subsubsection{Other languages}

We also provide support for other languages, notably |bash|, but it is relatively easy to add basic support for any new language. You only need to configure |set compilation command| to your command, |set template| to the file to compile (|__ROBEXT_MAIN_CONTENT__| contains the code typed by the user), and possibly a custom include command with |custom include command| if you do not want to do |\includegraphics| on the final pdf. For instance, to define a basic template for bash, you just need to use:

\begin{codeAndResult}
% Create your style
\begin{PlaceholderFromCode}{__MY_BASH_TEMPLATE__}
# Quit if there is an error
set -e  
__ROBEXT_MAIN_CONTENT__
# Create the pdf file to certify that no compilation error occured
touch "__ROBEXT_OUTPUT_PDF__"
\end{PlaceholderFromCode}

\robExtConfigure{
  my bash/.style={
    set compilation command={bash "__ROBEXT_SOURCE_FILE__"},
    set template={__MY_BASH_TEMPLATE__},
    %%% Version 1:
    % verbatim output,
    %%% Version 2:
    custom include command={%
      \evalPlaceholder{%
        \verbatiminput{__ROBEXT_CACHE_FOLDER____ROBEXT_OUTPUT_PREFIX__-out.txt}%
      }%
    },
    % Ensure that the code does not break when externalization is disabled
    print verbatim if no externalization,
  }
}

% Use your style
\begin{CacheMeCode}{my bash}
# Write the system conf to a file *-out.txt
uname -srv > "__ROBEXT_OUTPUT_PREFIX__-out.txt"
\end{CacheMeCode}
\end{codeAndResult}


\paragraph{Code inside a macro}

Due to fundamental \LaTeX{} restrictions, it is impossible to use |CacheMeCode| inside a macro or some environments as \LaTeX{} will strip all lines containing a percent character for instance. The solution here is to define our main content before, and then set it using |set main content| (that simply sets |__ROBEXT_MAIN_CONTENT__|). In this example, we also show how |CacheMeNoContent| can be used when their is no content (the arguments to |CacheMe| are directly given in the body of |CacheMeNoContent|):

\begin{codeAndResult}
\begin{PlaceholderFromCode}{__TMP_MAIN_CONTENT__}
import matplotlib.pyplot as plt
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials')   
plt.savefig("__ROBEXT_OUTPUT_PDF__")  
\end{PlaceholderFromCode}

\fbox{\begin{CacheMeNoContent}
  python,
  set includegraphics options={width=.8\linewidth},
  set main content={__TMP_MAIN_CONTENT__},
\end{CacheMeNoContent}}
\end{codeAndResult}

\section{Documentation}

\subsection{How it works}

This library must be able to generate 3 elements for any cached content:
\begin{itemize}
\item a source file, that will be compiled, and is obtained by expanding the placeholder |__ROBEXT_TEMPLATE__| (see \cref{sec:placeholders}),
\item a compilation command obtained by expanding the placeholder |__ROBEXT_COMPILATION_COMMAND__|,
\item a dependency file, that contains the hash of all the dependencies (see \cref{sec:dependencies} for details) and the compilation command,
\item an inclusion command (this one is not used during the caching process, it is only used when including the compiled document in the main document), that you can set using |custom include command={your command}|.
\end{itemize}

The hash of all these elements is computed in order to obtain a reference hash, denoted |somehash| that looks like a unique random value (note that |__ROBEXT_OUTPUT_PDF__| and alike are expanded after knowing the hash since they depend on the final hash value). This hash |somehash| will change whenever a dependency changes, or if the compilation command changes, ensuring purity. Then, the dependency file and the source file are written in the cache, by default in |robustExternalize/robExt-somehash.tex| and |robustExternalize/robExt-somehash.deps|. Then, the compilation command will be run from the cache folder. At the end, by default, we check if a file |robustExternalize/robExt-somehash.pdf| exists: if not we abort, otherwise we |\input| the file |robustExternalize/robExt-somehash-out.tex| and we run the include command (that includes the pdf by default). As we saw earlier, this command can be customized to use other files. \textbf{Importantly, all the files created during the compilation must be prefixed by} |robExt-somehash|, which can be obtained at runtime using |__ROBEXT_OUTPUT_PREFIX__|. This way, we can easily clean the cache while ensuring maximum purity.

In the following, we will denote by |*-foo.bar| the file in:\\
|robustExternalize/robExt-somehash-foo.bar|.

Note also that we usually define two names for each function, one normal and one prefixed with |robExt| (or |RobExt|) for environments. In this documentation, we only write the first form, but the second form is kept in case a conflicting package redefines some functions.

\subsection{Placeholders}\label{sec:placeholders}

Placeholders are the main concept allowing this library to generate the content of a source file based on a template (a template will itself be a placeholder containing other placeholders). A placeholder is a special strings like |__COLOR_IMAGE__| inserted for instance in a string, that will be given a value later. This value will be used to replace (recursively) the placeholder in the template. For instance, if a placeholder |__LIKES__| contains |I like __FRUIT__ and __VEGETABLE__|, if the placeholder |__FRUIT__| contains |oranges| and if the placeholder |__VEGETABLE__| contains |salad|, then evaluating |__LIKES__| will output |I like oranges and salad.|

Note that the usage of underscore in only a convention, as any name can be used for the placeholder. There is however one rule to follow: the name of a placeholder should be made to avoid ambiguities when replacing the string, notably its name should not contain the name of another placeholder. For instance, if we define a placeholder called |NAME| containing |Foo| and a placeholder named |FULL_NAME| containing |Foo Bar|, then when evaluating the string |My name is FULL_NAME|, we have no way to know if the user wants to get |My name is FULL_Foo| or |My name is Foo Bar|. For this reason, \textbf{we advice users to start and end their placeholder names using two underscores} |__|, while using only upper case letters and single underscore |_| inside the placeholder name (this also improves readability). If you are worried about ambiguities like |__PLACEHOLDER1__PLACEHOLDER2__|, you can also use different separators for the beginning and the end like |__PLACEHOLDER1__|, but beware that |[| requires brackets around when used in pgf styles.

Placeholders are local variables (internally just some \LaTeX{} 3 strings). You can therefore define a placeholder in a local group surrounded by brackets |{ ... }| if you want it to have a reduced scope.

\subsubsection{Reading a placeholder}

\begin{pgfmanualentry}
  \extractcommand\getPlaceholder\opt{\oarg{new placeholder name}}\marg{name placeholder or string}\@@
  \extractcommand\getPlaceholderInResult\opt{\oarg{new placeholder name}}\marg{name placeholder or string}\@@
  \pgfmanualbody

  Get the value of a placeholder after replacing (recursively) all the inner placeholders. |\getPlaceholderInResult| puts the resulting string in a \LaTeX{} 3 string |\l_robExt_result_str| and in |\robExtResult|, while |\getPlaceholder| directly outputs this string. You can also put inside the argument
  any arbitrary string, allowing you, for instance, to concatenate multiple placeholders, copy a placeholder etc. Note that you will get a string, but this string will not be evaluated by \LaTeX{} (see |\evalPlaceholder| for that), for instance math will not be interpreted:
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromContent{__MY_PLACEHOLDER__}{Hello __NAME__, I am a template $\delta_n$.}
  \placeholderFromContent{__NAME__}{Alice __NICKNAME__}
  \placeholderFromContent{__NICKNAME__}{the great}
  The placeholder evaluates to:\\
  \texttt{\getPlaceholder{__MY_PLACEHOLDER__}}\\
  Combining placeholders produces:\\
  \texttt{\getPlaceholder{In ``__MY_PLACEHOLDER__'', the name is __NAME__.}}
\end{codeexample}
You can also specify the optional argument in order to additionally define a new placeholder containing the resulting string (but you might prefer to use its alias |\setPlaceholderRec| described below):
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromContent{__MY_PLACEHOLDER__}{Hello __NAME__, I am a template $\delta_n$.}
  \placeholderFromContent{__NAME__}{Alice __NICKNAME__}
  \placeholderFromContent{__NICKNAME__}{the great}
  \getPlaceholderInResult[__NEW_PLACEHOLDER__]{In ``__MY_PLACEHOLDER__'', the name is __NAME__.}
  \printAllPlaceholdersExceptDefaults
\end{codeexample}
\end{pgfmanualentry}


\begin{pgfmanualentry}
  \extractcommand\evalPlaceholder\marg{name placeholder or string}\@@
  \pgfmanualbody

  Evaluate the value of a placeholder after replacing (recursively) all the inner placeholders. You can also put inside any arbitrary string.
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromContent{__MY_PLACEHOLDER__}{Hello __NAME__, I am a template $\delta_n$.}
  \placeholderFromContent{__NAME__}{Alice __NICKNAME__}
  \placeholderFromContent{__NICKNAME__}{the great}
  % The placeholder evaluates to \texttt{\getPlaceholder{__MY_PLACEHOLDER__}}.
  The placeholder evaluates to:\\
  \evalPlaceholder{__MY_PLACEHOLDER__}\\
  Combining placeholders produces:\\
  \evalPlaceholder{In ``__MY_PLACEHOLDER__'', the name is __NAME__.}
\end{codeexample}
\end{pgfmanualentry}

\subsubsection{List and debug placeholders}

It can sometimes be handy to list all placeholders, print their contents etc. We list here commands that are mostly useful for debugging purposes.

\begin{pgfmanualentry}
  \extractcommand\printAllPlaceholdersExceptDefaults\opt{*}\@@
  \pgfmanualbody

  Prints the verbatim content of all defined placeholders (without performing any replacement of inner placeholders), except for the placeholders that are defined by default in this library (that we identify as they start with |__ROBEXT_|). The stared version does print the name of the placeholder defined in this library, but not their definition. This is mostly for debugging purposes.
\begin{codeexample}[width=0pt,vbox]
\placeholderFromContent{__LIKES__}{Hello __NAME__ I am a really basic template $\delta_n$.}
\placeholderFromContent{__NAME__}{Alice}    
\printAllPlaceholdersExceptDefaults
\end{codeexample}
Compare with:
\begin{codeexample}[width=0pt,vbox]
\placeholderFromContent{__LIKES__}{Hello __NAME__ I am a really basic template $\delta_n$.}
\placeholderFromContent{__NAME__}{Alice}    
\printAllPlaceholdersExceptDefaults*
\end{codeexample}

\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractcommand\printAllPlaceholders\@@
  \pgfmanualbody

  Prints the verbatim content of all defined placeholders (without performing any replacement of inner placeholders), including the placeholders that are defined by default in this library. This is mostly for debugging purposes. Here is the result of |\printAllPlaceholders|:

  \printAllPlaceholders
\end{pgfmanualentry}


\begin{pgfmanualentry}
  \extractcommand\printPlaceholderNoReplacement\marg{name placeholder}\@@
  \pgfmanualbody

  Prints the verbatim content of a given placeholder, without evaluating it and \textbf{without replacing inner placeholders: it is used mostly for debugging purposes} and will be used in this documentation to display the content of the placeholder for educational purposes. The stared version prints it inline.
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromContent{__LIKES__}{Hello NAME I am a really basic template $\delta_n$.}
  \placeholderFromContent{NAME}{Alice}    
  The (unexpanded) template contains \printPlaceholderNoReplacement{__LIKES__}.\\
  The (unexpanded) template contains \printPlaceholderNoReplacement*{__LIKES__}
\end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractcommand\printPlaceholder\marg{name placeholder}\@@
  \pgfmanualbody

  Like |\printPlaceholderNoReplacement| except that it first replaces the inner placeholders. The stared version prints it inline.
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromContent{__LIKES__}{Hello NAME I am a really basic template $\delta_n$.}
  \placeholderFromContent{NAME}{Alice}    
  The (unexpanded) template contains \printPlaceholder{__LIKES__}.\\
  The (unexpanded) template contains \printPlaceholder*{__LIKES__}
\end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractcommand\evalPlaceholderNoReplacement\marg{name placeholder}\@@
  \pgfmanualbody

  Evaluates the content of a given placeholder as a \LaTeX{} code, \textbf{without replacing the placeholders contained inside (mostly used for debugging purposes).}
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromContent{__LIKES__}{Hello NAME I am a really basic template $\delta_n$.}
  \placeholderFromContent{NAME}{Alice}
  The (unexpanded) template evaluates to ``\evalPlaceholderNoReplacement{__LIKES__}''.
\end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractcommand\getPlaceholderNoReplacement\marg{name placeholder}\@@
  \pgfmanualbody

  Like |\evalPlaceholderNoReplacement| except that it only outputs the string without evaluating the macros inside.
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromContent{__LIKES__}{Hello NAME I am a really basic template $\delta_n$.}
  \placeholderFromContent{NAME}{Alice}
  The (unexpanded) template contains \texttt{\getPlaceholderNoReplacement{__LIKES__}}
\end{codeexample}
\end{pgfmanualentry}

\subsubsection{Setting a value to a placeholder}


\begin{pgfmanualentry}
  \extractcommand\placeholderFromContent\marg{name placeholder}\marg{content placeholder}\@@
  \extractcommand\setPlaceholder\marg{name placeholder}\marg{content placeholder}\@@
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set placeholder=\marg{name placeholder}\marg{content placeholder}\@nil
  \extractkey/robExt/set placeholder from content=\marg{name placeholder}\marg{content placeholder}\@nil
  \makeatother%
  \pgfmanualbody

  |\placeholderFromContent| (and its alias |\setPlaceholder| and its equivalent pgf styles |/robExt/set placeholder| and |/robExt/set placeholder from content|) is useful to set a value to a given placeholder.
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromContent{__LIKES__}{Hello I am a basic template with math $\delta_n$ and macros \hello}
  The (unexpanded) template contains \printPlaceholderNoReplacement{__LIKES__} and %
  after evaluation and setting the value of hello,%
  \def\hello{Hello my friend!}%
  you get ``\evalPlaceholder{__LIKES__}''.
\end{codeexample}
  As you can see, \textbf{the precise content is not exactly identical to the original string}: \LaTeX{} comments are removed, spaces are added after macros, some newlines are removed etc. While this is usually not an issue when dealing with \LaTeX{} code, it causes some troubles when dealing with non-\LaTeX{} code. For this reason, we define \textbf{other commands} (see for instance |PlaceholderFromCode| below) that can accept verbatim content; the downside being that \LaTeX{} forbids usage of these verbatim commands inside other macros, so you should always define them at the top level (this seems to be fundamental to how \LaTeX{} works, as any input to a macro gets interpreted first as a \LaTeX{} string, losing all comments for instance). Note that this is not as restrictive as it may sound, as it is always possible to define the needed placeholders before any macro, while using them inside the macro, possibly combining them with other placeholders (defined either before or inside the macro).
\end{pgfmanualentry}

But before seeing how to define placeholder containing arbitrary code, let us first see how we can define a placeholder recursively, by giving it a value based on its previous value (useful for instance in order to add stuff to it).

\begin{pgfmanualentry}
  \extractcommand\setPlaceholderRec\marg{new placeholder}\marg{content with placeholder}\@@
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set placeholder rec=\marg{name placeholder}\marg{content placeholder}\@nil
  \makeatother%
  \pgfmanualbody

  |\setPlaceholderRec{foo}{bar}| is actually an alias for |\getPlaceholderInResult[foo]{bar}|. Note that contrary to |\setPlaceholder|, it recursively replaces all inner placeholders. This is particularly useful to add stuff to an existing (or not) placeholder:
\begin{codeexample}[width=0pt,vbox]
\setPlaceholderRec{__MY_COMMAND__}{pdflatex}
\setPlaceholderRec{__MY_COMMAND__}{__MY_COMMAND__ myfile}
\printAllPlaceholdersExceptDefaults
\end{codeexample}
Note that the if the placeholder content contains at the end the placeholder name, we will automatically remove it to avoid infinite recursion at evaluation time. This has the benefit that you can add something to a placeholder even if this placeholder does not exists yet (in which case it will be understood as the empty string):
\begin{codeexample}[width=0pt,vbox]
\setPlaceholderRec{__COMMAND_ARGS__}{__COMMAND_ARGS__ -l}
\setPlaceholderRec{__COMMAND_ARGS__}{__COMMAND_ARGS__ -s}
\printAllPlaceholdersExceptDefaults
\end{codeexample}
\end{pgfmanualentry}

Note that sometimes, you might not want to use |\setPlaceholderRec| to simply append some data to the placeholder as it will also evaluate the inner placeholders (meaning that you will not be able to redefine them later). For this reason, we also provide functions to add something to the placeholder without evaluating it first:
\begin{pgfmanualentry}
  \extractcommand\addToPlaceholder\opt{*}\marg{placeholder}\marg{content to add}\@@
  \extractcommand\addBeforePlaceholder\opt{*}\marg{placeholder}\marg{content to add}\@@
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/add to placeholder=\marg{name placeholder}\marg{content to add}\@nil
  \extractkey/robExt/add to placeholder no space=\marg{name placeholder}\marg{content to add}\@nil
  \extractkey/robExt/add before placeholder=\marg{name placeholder}\marg{content to add}\@nil
  \extractkey/robExt/add before placeholder no space=\marg{name placeholder}\marg{content to add}\@nil
  \makeatother%
  \pgfmanualbody

  |\addToPlaceholder{foo}{bar}| adds |bar| at the end of the placeholder |foo| (by default it also adds a space, unless you use the star version), creating it if it does not exist (the |before| variants add the content\dots{} before).
\begin{codeexample}[width=0pt,vbox]
\setPlaceholder{__ENGINE__}{pdflatex}
\setPlaceholder{__COMMAND__}{__ENGINE__ --option}
\addToPlaceholder{__COMMAND__}{--other}
\addToPlaceholder*{__COMMAND__}{-option}
\addBeforePlaceholder{__COMMAND__}{time}
\printAllPlaceholdersExceptDefaults
\end{codeexample}
\end{pgfmanualentry}


\begin{pgfmanualentry}
  \extractcommand\evalPlaceholderInplace\marg{name placeholder}\@@
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/eval placeholder inplace=\marg{name placeholder}\@nil
  \makeatother%
  \pgfmanualbody
  This command will update (inplace) the content of a macro by first replacing recursively the placeholders, and finally by expanding the  \LaTeX{} macros.
\begin{codeexample}[width=0pt,vbox]
\def\mymacro{Initial value}    
\placeholderFromContent{__MACRO_NOT_EVALUATED__}{\mymacro}
\placeholderFromContent{__MACRO_EVALUATED__}{\mymacro}
\evalPlaceholderInplace{__MACRO_EVALUATED__}
\printAllPlaceholdersExceptDefaults
\def\mymacro{Final value}    
Compare \evalPlaceholder{__MACRO_EVALUATED__} and \evalPlaceholder{__MACRO_NOT_EVALUATED__}.
\end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set placeholder eval=\marg{name placeholder}\marg{content placeholder}\@nil
  \makeatother%
  \pgfmanualbody
  Alias for |\setPlaceholderRec{#1}{#2}\evalPlaceholderInplace{#1}|: set and evaluate recursively the placeholders and macros. This can be practical to pass the value of a counter/macro to the template (of course, if this value is fixed, you can also directly load it from the preambule):
\begin{codeexample}[width=0pt,vbox]
\begin{CacheMe}{tikz, set placeholder eval={__thepage__}{\thepage}}
  \node[rounded corners, fill=red]{The current page is __thepage__.};
\end{CacheMe}
\end{codeexample}
Note that this works well for commands that expand completely, but some more complex commands might not expand properly (like |cref|). I need to investigate how to solve this issue, meanwhile you can still disable externalization for these pictures.
\end{pgfmanualentry}


\begin{pgfmanualentry}
  \extractenvironement{PlaceholderFromCode}\marg{name placeholder}\@@
  \extractenvironement{setPlaceholderCode}\marg{name placeholder}\@@
  \pgfmanualbody

  These two (aliased) environments are useful to set a verbatim value to a given placeholder: the advantage is that you can put inside any code, including \LaTeX{} comments, the downside is that you cannot use it inside macros and some environments (so you typically define it before the macros and call it inside).
% \begin{codeexample}[width=0pt,vbox]
%   \placeholderFromContent{__PYTHON_CODE__}{}
%   The (unexpanded) template contains \printPlaceholderNoReplacement{__LIKES__} and %
%   after evaluation (no replacement), you get ``\evalPlaceholderNoReplacement{__LIKES__}''.
% \end{codeexample}

\begin{codeexample}[width=0pt,vbox]
\begin{PlaceholderFromCode}{__PYTHON_CODE__}
def my_function(b): # this is a python code
    c = {}
    d[42] = 0
    return b
\end{PlaceholderFromCode}
\printAllPlaceholdersExceptDefaults
\end{codeexample}


Note that |PlaceholderFromCode| should not be used inside other macros or inside some environments (notably the ones that need to evaluate the body of the environment, e.g. using |+b| argument or |environ|) as verbatim content is parsed first by the macro, meaning that some characters might be changed or removed. For instance, any percent character would be considered as a comment, removing the rest of the line. However, this should not be be problem if you use it outside of any macro or environment, or if you load it from a file. For instance this code:
\begin{verbatim}
\begin{PlaceholderFromCode}{__PYTHON_CODE__}
def my_function(b): # this is a python code
    c = {}
    d[42] = 0
    return b % 2
\end{PlaceholderFromCode}
\printAllPlaceholdersExceptDefaults
\end{verbatim}
would produce:

{
\begin{PlaceholderFromCode}{__PYTHON_CODE__}
def my_function(b): # this is a python code
    c = {}
    d[42] = 0
    return b % 2
\end{PlaceholderFromCode}
\begin{codeexample}[width=0pt,vbox]
\printAllPlaceholdersExceptDefaults
\end{codeexample}
}
Note that of course, you can define a placeholder before a macro and call it inside (explaining how we can generate this documentation).
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractcommand\placeholderPathFromFilename\marg{name placeholder}\marg{filename}\@@
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set placeholder path from filename=\marg{name placeholder}\marg{filename}\@nil
  \makeatother%
  \pgfmanualbody

  |\placeholderPathFromFilename{__MYLIB__}{mylib.py}| will copy |mylib.py| in the cache (setting its hash depending on its content), and set the content of the placeholder |__MYLIB__| to the \textbf{path} of the library in the cache. Note that the path is relative to the cache folder (it is easier to use for instance if you want to call this library from a code already in the cache).
\begin{codeexample}[width=0pt,vbox]
  \placeholderPathFromFilename{__MYLIB__}{mylib.py}
  \printAllPlaceholdersExceptDefaults
  You can also get the path relative to the root folder:\\
  \robExtAddCachePath{\getPlaceholderNoReplacement{__MYLIB__}}
\end{codeexample}
\end{pgfmanualentry}


\begin{pgfmanualentry}
  \extractcommand\placeholderFromFileContent\marg{name placeholder}\marg{filename}\@@
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set placeholder from file content=\marg{name placeholder}\marg{filename}\@nil
  \makeatother%
  \pgfmanualbody

  |\placeholderFromFileContent{__MYLIB__}{mylib.py}| will set the content of the placeholder |__MYLIB__| to the content of |mylib.py|.
\begin{codeexample}[width=0pt,vbox]
  \placeholderFromFileContent{__MYLIB__}{mylib.py}
  \printAllPlaceholdersExceptDefaults
\end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractcommand\placeholderPathFromContent\marg{name placeholder}\opt{\oarg{suffix}}\marg{content}\@@
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set placeholder path from content=\marg{name placeholder}\marg{suffix}\marg{content}\@nil
  \makeatother%
  \pgfmanualbody

  |\placeholderPathFromContent{__MYLIB__}{some content}| will copy |some content| in a file in the cache (setting its hash depending on its content, the filename will end with |suffix| that defaults to |.tex|), and set the content of the placeholder |__MYLIB__| to the \textbf{path} of the file in the cache. Note that the path is relative to the cache folder (it is easier to use for instance if you want to call this library from a code already in the cache).
\begin{codeexample}[width=0pt,vbox]
  \placeholderPathFromContent{__MYLIB__}[.py]{some contents b}
  \printAllPlaceholdersExceptDefaults
  You can also get the path relative to the root folder:\\
  \robExtAddCachePath{\getPlaceholderNoReplacement{__MYLIB__}}\\
  As a sanity check, this file contains
  \verbatiminput{\robExtAddCachePath{\getPlaceholderNoReplacement{__MYLIB__}}}
\end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractenvironement{PlaceholderPathFromCode}\opt{\oarg{suffix}}\marg{name placeholder}\@@
  \pgfmanualbody

  This environment is similar to |\placeholderPathFromContent| except that it accepts verbatim code (therefore \LaTeX{} comments, newlines etc. will not be removed). However, due to \LaTeX{} limitations, this environment cannot be used inside macros or some environments, or this property will not be preserved.
  For instance, if you create your placeholder using:
\begin{verbatim}
\begin{PlaceholderPathFromCode}[.py]{__MYLIB__}
def my_function(b): # this is a python code
    c = {}
    d[42] = 0
    return b % 2
\end{PlaceholderPathFromCode}
\end{verbatim}
%% The code cannot be placed inside codeexample as it needs to parse the body:
\begin{PlaceholderPathFromCode}[.py]{__MYLIB__}
def my_function(b): # this is a python code
    c = {}
    d[42] = 0
    return b % 2
\end{PlaceholderPathFromCode}
You can then use it like:
\begin{codeexample}[width=0pt,vbox]
\printAllPlaceholdersExceptDefaults
You can also get the path relative to the root folder:\\
\robExtAddCachePath{\getPlaceholderNoReplacement{__MYLIB__}}\\
As a sanity check, this file contains
\verbatiminput{\robExtAddCachePath{\getPlaceholderNoReplacement{__MYLIB__}}}
\end{codeexample}
\end{pgfmanualentry}


\begin{pgfmanualentry}
  \extractcommand\copyPlaceholder\marg{new placeholder}\marg{old placeholder}\@@
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/copy placeholder=\marg{new placeholder}\marg{old placeholder}\@nil
  \makeatother%
  \pgfmanualbody

  This creates a new placeholder with the content of |old placeholder|. Note that this is different from:\\
  |\setPlaceholder{new placeholder}{old placeholder}|\\
  because if we modify |old placeholder|, this will not affect |new placeholder|.
\begin{codeexample}[width=0pt,vbox]
  \setPlaceholder{__MY_CONTENT__}{Some content}
  \copyPlaceholder{__MY_OLD_CONTENT__}{__MY_CONTENT__}
  \setPlaceholder{__MY_CONTENT__}{The content used to be __MY_OLD_CONTENT__}
  \printAllPlaceholdersExceptDefaults
\end{codeexample}
 It is useful for instance if you want to use a different value for |__ROBEXT_MAIN_CONTENT__|: first copy |__ROBEXT_MAIN_CONTENT__| to another placeholder, say |__NEW_MAIN_CONTENT__|, and then set |__ROBEXT_MAIN_CONTENT__| to point to an arbitrary template that may load |__NEW_MAIN_CONTENT__|.
\end{pgfmanualentry}


\subsection{Caching a content}

\subsubsection{Basics}

\begin{pgfmanualentry}
  \extractcommand\cacheMe\opt{\oarg{preset style}}\marg{content to cache}\@@
  \extractenvironement{CacheMe}\marg{preset style}\@@
  \pgfmanualbody
  This command (and its environment alias) is the main entry point if you want to cache the result of a file. The preset style is a pgfkeys-based style that is used to configure the template that is used, the compilation command, and more. You can either inline the style, or use some presets that configure the style automatically. After evaluating the style, the placeholders |__ROBEXT_TEMPLATE__| (containing the content of the file) and |__ROBEXT_COMPILATION_COMMAND__| (containing the compilation command run in the cache folder, that can use other placeholders internally like |__ROBEXT_SOURCE_FILE__| to get the path to the source file) should be set. Note that we provide some basic styles that allow settings these placeholders easily. See \cref{sec:placeholders} for a list of existing placeholders and presets. The placeholder |__ROBEXT_MAIN_CONTENT__| will automatically be set by this command (or environment) so that it equals the content of the second argument (or the body of the environment). This style can also configure the command to use to include the file and more. By default it will insert the compiled PDF, making sure that the depth is respected (internally, we read the depth from an aux file created by our \LaTeX{} preset), but you can easily change it to anything you like.

  For an educational purpose, we write here an example that does not exploit any preset. In practice, we recommend however to use our presets, or to define new presets based on our presets (see below for examples).
\begin{codeexample}[width=0pt,vbox]
\begin{CacheMe}{set template={
      \documentclass{standalone}
      \begin{document}
      __ROBEXT_MAIN_CONTENT__
      \end{document}
    },
    set compilation command={pdflatex -shell-escape -halt-on-error "__ROBEXT_SOURCE_FILE__"},
    custom include command={%
      \includegraphics[width=4cm,angle=45]{\robExtAddCachePathAndName{\robExtFinalHash.pdf}}%
    },
  }
This content is cached $\delta$.    
\end{CacheMe}
\end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractcommand\robExtConfigure\marg{preset style}\@@
  \pgfmanualbody
  You can then create your own style (or preset) in |\robExtConfigure| (that is basically an alias for |\pgfkeys{/robExt/.cd,#1}|) containing your template, add your own placeholders and commands to configure them etc.

\begin{codeexample}[width=0pt,vbox]
%% Define your presets once:
\robExtConfigure{%
  my latex preset/.style={
    %% Create a default value for my new placeholders:
    set placeholder={__MY_COLOR__}{red},
    set placeholder={__MY_ANGLE__}{45},
    % We can also create custom commands to "hide" the notion of placeholder
    set my angle/.style={
      set placeholder={__MY_ANGLE__}{##1}
    },
    set template={
      \documentclass{standalone}
      \usepackage{xcolor}
      \begin{document}
      \color{__MY_COLOR__}__ROBEXT_MAIN_CONTENT__
      \end{document}
    },
    set compilation command={pdflatex -shell-escape -halt-on-error "__ROBEXT_SOURCE_FILE__"},
    custom include command={%
      % The include command is a regular LaTeX command, but using
      % \evalPlaceholder avoids the need to play with expandafter, getPlaceholder etc...
      \evalPlaceholder{% 
        \includegraphics[width=4cm,angle=__MY_ANGLE__,origin=c]{%
          \robExtAddCachePathAndName{\robExtFinalHash.pdf}%
        }%
      }%
    },
  },
}

% Reuse them later...
\begin{CacheMe}{my latex preset}
This content is cached $\delta$.    
\end{CacheMe}
% And configure them at will
\begin{CacheMe}{my latex preset, set placeholder={__MY_COLOR__}{green}, set my angle=-45}
This content is cached $\delta$.    
\end{CacheMe}
\end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \extractenvironement{CacheMeCode}\marg{preset style}\@@
  \pgfmanualbody
  Like |CacheMe|, except that the code is read verbatim by \LaTeX{}. This way, you can put non-\LaTeX{} code inside safely, but you will not be able to use it inside a macro or some environments that read their body. Here is an example where we define an environment that automatically import matplotlib, save the figure, and insert it into a figure. Note that we define in this example new commands to type |set caption=foo| instead of |set placeholder={__FIG_CAPTION__}{foo}|.
%% codeexample cannot deal with verbatim content

\begin{codeexample}[code only]
%% Define the python code to use as a template
%% (impossible to define it in \robExtConfigure directly since
%% it is a verbatim environment)
\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEMPLATE__}
import matplotlib.pyplot as plt
import sys
__ROBEXT_MAIN_CONTENT__
plt.savefig("__ROBEXT_OUTPUT_PDF__")  
\end{PlaceholderFromCode}

% Create a new preset called matplotlib
\robExtConfigure{
  matplotlib figure/.style={
    set template={__MY_MATPLOTLIB_TEMPLATE__},
    set compilation command={python "__ROBEXT_SOURCE_FILE__"},
    set caption/.style={
      set placeholder={__FIG_CAPTION__}{##1}
    },
    set label/.style={
      set placeholder={__FIG_LABEL__}{##1}
    },
    set includegraphics options/.style={
      set placeholder={__INCLUDEGRAPHICS_OPTIONS__}{##1}
    },
    set caption={},
    set label={},
    set includegraphics options={width=1cm},
    custom include command={%
      \evalPlaceholder{%
        \begin{figure}
          \centering
          \includegraphics[__INCLUDEGRAPHICS_OPTIONS__]{\robExtAddCachePathAndName{\robExtFinalHash.pdf}}%
          \caption{__FIG_CAPTION__a}
          \label{__FIG_LABEL__}
        \end{figure}%
      }%
    },
  },
}

%% Use it
\begin{CacheMeCode}{matplotlib figure, set includegraphics options={width=.8\linewidth}, set caption={Hello}}
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials') 
\end{CacheMeCode}    
\end{codeexample}

%% Define the python code to use as a template
%% (impossible to define it in \robExtConfigure directly since
%% it is a verbatim environment)
\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEMPLATE__}
import matplotlib.pyplot as plt
import sys
__ROBEXT_MAIN_CONTENT__
plt.savefig("__ROBEXT_OUTPUT_PDF__")  
\end{PlaceholderFromCode}

% Create a new preset called matplotlib
\robExtConfigure{
  matplotlib figure/.style={
    set template={__MY_MATPLOTLIB_TEMPLATE__},
    set compilation command={python "__ROBEXT_SOURCE_FILE__"},
    set caption/.style={
      set placeholder={__FIG_CAPTION__}{##1}
    },
    set label/.style={
      set placeholder={__FIG_LABEL__}{##1}
    },
    set includegraphics options/.style={
      set placeholder={__INCLUDEGRAPHICS_OPTIONS__}{##1}
    },
    set caption={},
    set label={},
    set includegraphics options={width=1cm},
    custom include command={%
      \evalPlaceholder{%
        \begin{figure}
          \centering
          \includegraphics[__INCLUDEGRAPHICS_OPTIONS__]{\robExtAddCachePathAndName{\robExtFinalHash.pdf}}%
          \caption{__FIG_CAPTION__}
          \label{__FIG_LABEL__}
        \end{figure}%
      }%
    },
  },
}

%% Use it
\begin{CacheMeCode}{matplotlib figure, set includegraphics options={width=.8\linewidth}, set caption={An example to show how matplotlib pictures can be inserted}}
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials') 
\end{CacheMeCode}

Note that as we explained it before, due to \LaTeX{} limitations, it is impossible to call |CacheMeCode| inside macros and inside some environments that evaluate their body. To avoid that issue, it is always possible to define the macro before and call it inside. We will exemplify this on the previous example, but note that \textbf{this example is only for educational purposes} since the environment |figure| does not evaluate its body, and |CacheMeCode| can therefore safely be used inside without using this trickery:
\begin{codeexample}[code only]
%% Define the python code to use as a template
%% (impossible to define it in \robExtConfigure directly since
%% it is a verbatim environment)
\begin{PlaceholderFromCode}{__MY_MATPLOTLIB_TEMPLATE__}
import matplotlib.pyplot as plt
import sys
__ROBEXT_MAIN_CONTENT__
plt.savefig("__ROBEXT_OUTPUT_PDF__")  
\end{PlaceholderFromCode}

% Create a new preset called matplotlib
\robExtConfigure{
  matplotlib/.style={
    set template={__MY_MATPLOTLIB_TEMPLATE__},
    set compilation command={python "__ROBEXT_SOURCE_FILE__"},
    set includegraphics options/.style={
      set placeholder={__INCLUDEGRAPHICS_OPTIONS__}{##1}
    },
    set includegraphics options={width=1cm},
    custom include command={%
      \evalPlaceholder{%
        \includegraphics[__INCLUDEGRAPHICS_OPTIONS__]{\robExtAddCachePathAndName{\robExtFinalHash.pdf}}%
      }%
    },
  },
}


%% You cannot use CacheMeCode inside some macros or environments due to fundamental LaTeX limitations.
%% But you can always define them before, and call them inside:
\begin{SetPlaceholderCode}{__TMP__}
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials') 
\end{SetPlaceholderCode}    

\begin{figure}
  \centering
  \cacheMe[matplotlib, set includegraphics options={width=.8\linewidth}, set caption={Hello}]{__TMP__}
  \caption{An example to show how code can be inserted into macros or environments that evaluate their contents (this trick is actually not needed for figures)}
\end{figure}
\end{codeexample}
\end{pgfmanualentry}


\subsubsection{Options to configure the template}


\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set template=\marg{content template}\@nil
  \makeatother%
  \pgfmanualbody
  Style that alias to |set placeholder={__ROBEXT_TEMPLATE__}{#1}|, in order to define the placeholder that will hold the template of the final file.
\end{pgfmanualentry}

\subsubsection{Options to configure the compilation command}\label{sec:configureCompilationCommand}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set compilation command=\marg{compilation command}\@nil
  \makeatother%
  \pgfmanualbody
  Style that alias to |set placeholder={__ROBEXT_COMPILATION_COMMAND__}{#1}|, in order to define the placeholder that will hold the compilation command.
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/add argument to compilation command=\marg{argument}\@nil
  \extractkey/robExt/add arguments to compilation command=\marg{argument}\@nil
  \makeatother%
  \pgfmanualbody
  |add argument to compilation command| is a style that alias to:\\
  |set placeholder={__ROBEXT_COMPILATION_COMMAND__}{__ROBEXT_COMPILATION_COMMAND__ "#1"}|
  in order to add an argument to the compilation command. |add arguments to compilation command| (note the |s|) accepts multiple arguments separated by a comma.
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/add key value argument to compilation command=\marg{key=value}\@nil
  \makeatother%
  \pgfmanualbody
  Adds to the command line two arguments |key| and |value|. This is a way to quickly pass arguments to a script: the script just needs to loop over the arguments and consider the odd elements as keys and the next elements as the value. Another option is to insert some placeholders directly in the script.
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/add key and file argument to compilation command=\marg{key=filename}\@nil
  \makeatother%
  \pgfmanualbody
  |filename| is the path to a file in the root folder. This adds, as:\\
  |add key value argument to compilation command|\\
  two arguments, where the first argument is the key, but this time the second argument is the path of |filename| relative to the cache folder (useful since scripts run from this folder). Moreover, it automatically ensures that when |filename| changes, the file gets recompiled. Note that contrary to some other commands, this does not copy the file in the cache, which is practical notably for large files like videos.
\end{pgfmanualentry}

\subsubsection{Options to configure the inclusion command}

The inclusion command is the command that is run to include the cached file back in the pdf (e.g. based on |\includegraphics|). We describe now how to configure this command.

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/custom include command advanced=\marg{include command}\@nil
  \makeatother%
  \pgfmanualbody
  Sets the command to run to include the compiled file. You can use:\\
  |\robExtAddCachePathAndName{\robExtFinalHash.pdf}|\\
  in order to get the path of the compiled pdf file. Note that we recommend rather to use |custom include command| that automatically checks if the file compiled correctly and that load the |*-out.tex| file if it exists (useful to pass information back to the pdf).
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/custom include command=\marg{include command}\@nil
  \makeatother%
  \pgfmanualbody
  Sets the command to run to include the compiled file, after checking if the file has been correctly compiled and loading |*-out.tex| (useful to pass information back to the pdf).
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/do not include pdf\@nil
  \makeatother%
  \pgfmanualbody
  Do not include the pdf. Useful if you only want to compile the file but use it later (note that you should still generate a |.pdf| file, possibly empty, to indicate that the compilation runs smootly). Equivalent to:\\
  |custom include command={}|
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/enable manual mode\@nil
  \extractkey/robExt/disable manual mode\@nil
  \makeatother%
  \pgfmanualbody
  If you do or do not want to ask latex to run the compilation commands itself (for instance for security
  reasons), you can use these commands and run the command manually later:
  \begin{codeexample}[width=0pt,vbox]
    \robExtConfigure{
      enable manual mode
    }

    The next picture must be manually compiled %
    (see JOBNAME-robExt-compile-missing-figures.sh):\\ %
    \begin{tikzpictureC}[baseline=(A.base)][]
      \node[fill=red, rounded corners](A){I must be manually compiled.};
      \node[fill=red, rounded corners, opacity=.3,overlay] at (A.north east){I am an overlay text};
    \end{tikzpictureC}
  \end{codeexample}

  See \cref{sec:operationsCache} for more details.
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/include graphics args\@nil
  \makeatother%
  \pgfmanualbody
  By default, the include commands runs |\includegraphics| on the pdf, and possibly raises it if needed. You can customize the arguments passed to |\includegraphics| here.
\end{pgfmanualentry}

\subsubsection{Configuration of the cache}

If needed, you can configure the cache:

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set filename prefix=\marg{prefix}\@nil
  \makeatother%
  \pgfmanualbody
  By default, the files in the cache starts with |robExt-|. If needed you can change this here, or by manually defining |\def\robExtPrefixFilename{yourPrefix-}|.
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set subfolder and way back=\marg{cache folder}\marg{path to project from cache}\@nil
  \makeatother%
  \pgfmanualbody
  By default, the cache is located in |robustExternalize/|, using:\\
  |set subfolder and way back={robustExternalize/}{../},|\\
  You can customize it the way you want, just be make sure that going to the second arguments after going to the first argument leads you back to the original position.
\end{pgfmanualentry}

\newpage
\subsubsection{Customize or disable externalization}\label{sec:disableExternalization}

You might want (sometimes or always) to disable externalization, for instance to use |remember picture| \tikz[remember picture,baseline=(pointtome.base)] \node[rounded corners, fill=orange](pointtome){Point to me if you can};, even if you used |\robExtExternalizeAllTikzpictures|:

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/disable externalization\@nil
  \extractkey/robExt/enable externalization\@nil
  \makeatother%
  \pgfmanualbody
  Enable or disable externalization.
  \begin{codeexample}[width=0pt,vbox]
    % In theory all pictures should be externalized (so remember picture should fail)
    \robExtExternalizeAllTikzpictures
    % But we can disable it temporarily
    \begin{tikzpicture}[remember picture][disable externalization]
      \node[rounded corners, fill=red](A){This figure is not externalized.
        This way, it can use remember picture.};
      \draw[->,overlay] (A) to[bend right] (pointtome);
    \end{tikzpicture}\\

    % You can also disable it globally/in a group:
    {
      \robExtConfigure{disable externalization}
      
      \begin{tikzpicture}[remember picture]
        \node[rounded corners, fill=red](A){This figure is not externalized.
          This way, it can use remember picture.};
        \draw[->,overlay] (A.west) to[bend left] (pointtome);
      \end{tikzpicture}\\

      \begin{tikzpicture}[remember picture]
        \node[rounded corners, fill=red](A){This figure is not externalized.
          This way, it can use remember picture.};
        \draw[->,overlay] (A.east) to[bend right] (pointtome);
      \end{tikzpicture}
    }
  \end{codeexample}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/command if no externalization\@nil
  \makeatother%
  \pgfmanualbody
  You can easily change the command to run if externalization is disabled using by setting the \textbf{.code} of this key. By default, it is configured as:\\
\begin{verbatim}
command if no externalization/.code={%
  \robExtDisableTikzpictureOverwrite\evalPlaceholder{__ROBEXT_MAIN_CONTENT__}%
}
\end{verbatim}
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/print verbatim if no externalization\@nil
  \makeatother%
  \pgfmanualbody
  Sets |command if no externalization| to print the verbatim content of |__ROBEXT_MAIN_CONTENT__| if externalization is disabled. Internally, it just sets it to:\\
  |\printPlaceholder{__ROBEXT_MAIN_CONTENT__}|\\
  This is mostly useful when typesetting |__ROBEXT_MAIN_CONTENT__| directly does not make sense (e.g. in python code). This style is used for instance in the |python| preset, allowing us to get:
  
\begin{codeAndResult}
\begin{CacheMeCode}{python,
    verbatim output,
    set placeholder eval={__thepage__}{\thepage},
    %% We disable externalization
    disable externalization}
with open("__ROBEXT_OUTPUT_PREFIX__-out.txt", "w") as f:
    for i in range(5):
        f.write(f"Hello {i}, we are on page __thepage__\n")
\end{CacheMeCode}
\end{codeAndResult}

You can also disable the externalization on all elements that use a common preset, for instance you can disable externalization on all |bash| instances (useful if you are on Windows for instance):
\begin{codeAndResult}
\robExtConfigure{
  % bash code will not be compiled (useful on windows for instance)
  bash/.append style={
    disable externalization
  },
}
\begin{CacheMeCode}{bash, verbatim output}
# $outputTxt contains the path of the file that will be printed via \verbatiminput
uname -srv > "${outputTxt}"
\end{CacheMeCode}
\end{codeAndResult}

\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/execute after each externalization\@nil
  \extractkey/robExt/execute before each externalization\@nil
  \makeatother%
  \pgfmanualbody
  By doing |execute after each externalization={some code}|, you will run some code after the externalization. This might be practical for instance to update a counter (e.g. the number of pages\dots) based on the result of the compiled file.
\end{pgfmanualentry}

\subsubsection{Dependencies}

In order to enforce reproducibility, you should tell what are the files that your code depends on, by adding this file as a dependency. This has the advantage that if this file is changed, your code is automatically recompiled. On the other hand, you might not want this behavior (e.g. if this file often changes in a non-important way): in that case, just don't add the file as a dependency (but keep that in mind as you might not be able to recompile your file if you clear the cache if you introduced breaking changes).

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/dependencies=\marg{list,of,dependencies}\@nil
  \extractkey/robExt/add dependencies=\marg{list,of,dependencies}\@nil
  \extractkey/robExt/reset dependencies\@nil
  \makeatother%
  \pgfmanualbody
  Set/add/reset the dependencies (you can put multiple files separated by commas). These files should be relative to the main compiled file. For instance, if you have a file |common_inputs.tex| that you want to input in both the main file and in the cached files, that contains, say:\\
  |\def\myValueDefinedInCommonInputs{42}|\\
  then you can add it as a dependency using:
\begin{codeexample}[width=0pt,vbox]
\begin{CacheMe}{latex,
    add dependencies={common_inputs.tex},
    add to preamble={\input{__ROBEXT_WAY_BACK__/common_inputs.tex}}}
  The answer is \myValueDefinedInCommonInputs.
\end{CacheMe}
\end{codeexample}
Note that the placeholder |__ROBEXT_WAY_BACK__| contains the path from the cache folder (containing the |.tex| that will be cached) to the root folder.\\ This way, you can easily input files contained in the root folder.
\end{pgfmanualentry}


\subsubsection{Pass compiled file to another template}

It can sometimes be handy to use the result of a previous cached file to cache another file, or to do anything else (e.g.\ it can also be practical to debug an issue). |name output| can be used to do that
\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/name output=\marg{macro name}\@nil
  \makeatother%
  \pgfmanualbody
  |name output=foo| will create two global macros |\foo| and |\fooInCache|: |\foo| expands to the prefix of the files created in the template like |robExt-somehash|, and |\fooInCache| also adds the cache folder like |robustExternalize/robExt-somehash|. You can then use |set placeholder eval| to send it to another cached file. It is then your role to add the extension, usually |.tex| to get the source (even if the source is a python file), |.pdf| to get the pdf, |-out.tex| to get the file that is loaded before the import, |-out.txt| if you wanted to make it compatible with |verbatim output| (this list is not exhaustive as each script might decide to create a different output file). Here is a demo:
  
\begin{codeexample}[width=0cm,vbox]
\begin{CacheMe}{tikz, do not add margins, name output=mycode}[baseline=(A.base)]
  \node[draw,rounded corners,fill=pink!60](A){Hello World!};
\end{CacheMe}\\[3mm]

The prefix is \texttt{\mycode} and with the cache folder it is in:\\
\texttt{\mycodeInCache}.\\
It this can be helpful for instance to debug, as you can inspect the source:
\verbatiminput{\mycodeInCache.tex}
but it is also practical to define a template based on the previously cached files:\\

\begin{CacheMe}{tikz, set placeholder eval={__previous__}{\mycode.pdf}}
  \node[rounded corners, fill=green!50]{A cached file can use result from another cached file:
    \includegraphics[width=2cm]{__previous__}\includegraphics[width=2cm]{__previous__}};
\end{CacheMe}  
\end{codeexample}

Note that if you do not want to display the first cached file, you can use |do not include pdf| to hide it.

\end{pgfmanualentry}


\subsection{Default presets}\label{sec:presets}

We provide by default some presets for famous languages (for now \LaTeX{} and python).

\subsubsection{All languages}

First, here are a few options that are available irrespective of the used language.

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/set includegraphics options=\marg{options}\@nil
  \extractkey/robExt/add to includegraphics options=\marg{options}\@nil
  \makeatother%
  \pgfmanualbody
  Set/add options to the |\includegraphics| run when inserting the pdf (by the default include command). By default it is empty, but the |latex| preset sets it to:\\
  |trim=__ROBEXT_LATEX_TRIM_LENGTH__ __ROBEXT_LATEX_TRIM_LENGTH__|\\
  |__ROBEXT_LATEX_TRIM_LENGTH__ __ROBEXT_LATEX_TRIM_LENGTH__|\\
  in order to remove the margin added in the standalone package options, which is needed to display overlay texts.
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/verbatim output\@nil
  \makeatother%
  \pgfmanualbody
  Shortcut for:
\begin{verbatim}
custom include command={%
  \evalPlaceholder{%
    __ROBEXT_VERBATIM_COMMAND__{%
      __ROBEXT_CACHE_FOLDER____ROBEXT_OUTPUT_PREFIX__-out.txt}%
  }%
},
\end{verbatim}
  i.e.\ instead of printing the pdf we print the content of the file |__ROBEXT_OUTPUT_PREFIX__-out.txt| using the command in |__ROBEXT_VERBATIM_COMMAND__|, that defaults to |\verbatiminput|:
\begin{codeAndResult}
\begin{CacheMeCode}{python, verbatim output}
with open("__ROBEXT_OUTPUT_PREFIX__-out.txt", "w") as f:
    for i in range(5):
        f.write(f"Hello {i}\n")
\end{CacheMeCode}
\end{codeAndResult}
\end{pgfmanualentry}


\subsubsection{\LaTeX{}}

The |latex| preset is used to cache any \LaTeX{} content, like tikz pictures. Note that as of today, it supports overlay content out of the box (if the overlay is more than 30cm long, you might want to customize a placeholder), but not images that need to use |remember picture|.

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/latex\@nil
  \makeatother%
  \pgfmanualbody
  This style sets the template |__ROBEXT_LATEX__| and the compilation command:\\ |__ROBEXT_COMPILATION_COMMAND_LATEX__|\\
  (cf \cref{sec:placeholders} for details), and adds a number of styles described below, to easily configure the most common options. You can use it as follows:
  \begin{codeexample}[width=0pt,vbox]
    The next picture is cached %
    \begin{CacheMe}{latex, add to preamble={\usepackage{tikz}}}
      \begin{tikzpicture}[baseline=(A.base)]
        \node[fill=red, rounded corners](A){My node that respects baseline.};
        \node[fill=red, rounded corners, opacity=.3,overlay] at (A.north east){I am an overlay text};
      \end{tikzpicture}
    \end{CacheMe} and you can see that overlay and depth works.
  \end{codeexample}
\end{pgfmanualentry}

To see how to create your own preset or automatically load a library, see \cref{sec:customize}.

The next options can be used after calling the |latex| style:

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/latex/use latexmk\@nil
  \extractkey/robExt/latex/use lualatex\@nil
  \extractkey/robExt/latex/use xelatex\@nil
  \makeatother%
  \pgfmanualbody
  Use latexmk/lualatex/xelatex to compile. It is a shortcut for:\\
  |set placeholder={__ROBEXT_LATEX_ENGINE__}{yourfavoriteengine}|
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/latex/set latex options=\marg{latex options}\@nil
  \extractkey/robExt/latex/add to latex options=\marg{latex options}\@nil
  \makeatother%
  \pgfmanualbody
  Set/add elements to the set of latex options of the |\documentclass| (it will automatically add a comma before if you add an element). Internally it sets |__ROBEXT_LATEX_OPTIONS__|. By default, it sets:\\
  |margin=__ROBEXT_LATEX_TRIM_LENGTH__|
  (where |__ROBEXT_LATEX_TRIM_LENGTH__| is defined as 30cm by default) in order to add a margin that will be trimmed  later in the |\includegraphics|. This is useful not to cut stuff displayed outside of the bounding box (overlays). 
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/latex/set documentclass=\marg{documentclass}\@nil
  \makeatother%
  \pgfmanualbody
  Set the documentclass of the document (defaults to |standalone|). Internally, it sets the placeholder |__ROBEXT_DOCUMENT_CLASS__|.
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/latex/set preamble=\marg{code of preamble}\@nil
  \extractkey/robExt/latex/add to preamble=\marg{code of preamble}\@nil
  \makeatother%
  \pgfmanualbody
  Set/add element to the preamble (defaults to |standalone|). Internally, it sets the placeholder |__ROBEXT_DOCUMENT_CLASS__|.
\end{pgfmanualentry}

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/latex/do not wrap code\@nil
  \makeatother%
  \pgfmanualbody
  By default, the main content is wrapped into a box in order to measure its depth to properly set the baseline. If you do not want to do this wrapping, you can set this option. Internally, it is a shortcut for:\\
  |set placeholder={__ROBEXT_MAIN_CONTENT_WRAPPED__}{__ROBEXT_MAIN_CONTENT__}|
\end{pgfmanualentry}

\subsubsection{Python}

We provide support for python:

\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/python\@nil
  \makeatother%
  \pgfmanualbody
  Load the |python| preset (inspect |__ROBEXT_PYTHON__|) for details on the exact template, but note that this template might be subject to changes. We also provide a few helper functions:
  \begin{itemize}
  \item |write_to_out(text)| writes |text| to the |*-out.tex| file that will be loaded automatically before running the include function
  \item |parse_args()| is a function that returns a dictionary mapping some keys to values depending on the called arguments: for instance, if you call the python file with |python script key1 value1 key2 value2|, then the dictionary will map |key1| to |value1| and |key2| to |value2|. You might like this in conjunction with commands presented in \cref{sec:configureCompilationCommand}. Note that if you place placeholders in your code, you might not need this, but this is used if you plan to use your script outside of this library.
  \item |get_cache_folder()| outputs the cache folder.
  \item |get_file_base()| outputs the prefix of all files that should be created by this script, that looks like |robExt-somehash|.
  \item |get_current_script()| returns the current script.
  \item |get_filename_from_extension(extension)| outputs the prefix |robExt-somehash| concatenated with the extension. You often need this function to get the path of a file that your script is creating, for instance, |get_filename_from_extension("-out.txt")| is the path |*-out.txt| of the file that is read by |verbatim output|.
  \item |get_verbatim_output()| returns |get_filename_from_extension("-out.txt")|
  \item |finished_with_no_error()| creates the pdf file if it does not exists (to certify that the compilation ran without issues). The template automatically runs this function at the end. 
  \end{itemize}
  We demonstrate its usage on a few examples:
\begin{codeAndResult}
\begin{CacheMeCode}{python, verbatim output}
with open(get_verbatim_output(), "w") as f:
    for i in range(5):
        f.write(f"Hello {i}\n")
\end{CacheMeCode}
\end{codeAndResult}

\textbf{Importantly: you do not want to indent the whole content of CacheMeCode, or the spaces will also appear in the final code.}

You can also generate some images. This code will produce the image in \cref{fig:pythonGeneratedImage2}:
\begin{codeexample}[code only]
\begin{CacheMeCode}{python, set includegraphics options={width=.8\linewidth}}
import matplotlib.pyplot as plt
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials')   
plt.savefig("__ROBEXT_OUTPUT_PDF__")  
\end{CacheMeCode}
\end{codeexample}
{\begin{figure}
\centering
\begin{CacheMeCode}{python, set includegraphics options={width=.8\linewidth}}
import matplotlib.pyplot as plt
year = [2014, 2015, 2016, 2017, 2018, 2019]  
tutorial_count = [39, 117, 111, 110, 67, 29]
plt.plot(year, tutorial_count, color="#6c3376", linewidth=3)  
plt.xlabel('Year')  
plt.ylabel('Number of futurestud.io Tutorials')   
plt.savefig("__ROBEXT_OUTPUT_PDF__")
\end{CacheMeCode}
\caption{Image generated with python.}
\label{fig:pythonGeneratedImage2}
\end{figure}
}

Note that by default, the executable called |python| is run. It seems like on windows |python3| is not created and only |python| exists, while on linux the user can choose whether |python| should point to |python3| or |python2| (on NixOs, I directly have |python| pointing to |python3|, and in ubuntu, you might need to install |python-is-python3| or create a symlink, as explained \href{https://askubuntu.com/questions/1296790/python-is-python3-package-in-ubuntu-20-04-what-is-it-and-what-does-it-actually}{here}). In any case, you can customize the name of the executable by setting something like:
\begin{verbatim}
\setPlaceholder{__ROBEXT_PYTHON_EXEC__}{python3}
\end{verbatim}
or using the style |force python3| that forces |python3|.
\end{pgfmanualentry}


\begin{pgfmanualentry}
  \makeatletter%
  \def\extrakeytext{style, }
  \extractkey/robExt/python print code and result\@nil
  \makeatother%
  \pgfmanualbody
  This is a demo style that can print a python code and its result.
\begin{codeAndResult}
\begin{CacheMeCode}{python print code and result, set title={The for loop}}
for name in ["Alice", "Bob"]:
    print(f"Hello {name}")
\end{CacheMeCode}
\end{codeAndResult}
You can set |__ROBEXT_PYTHON_TCOLORBOX_PROPS__| the options of the tcolorbox,\\ |__ROBEXT_PYTHON_CODE_MESSAGE__| and |__ROBEXT_PYTHON_RESULT_MESSAGE__| which are displayed before the corresponding block, |__ROBEXT_PYTHON_LSTINPUT_STYLE__| which contains the default lstinput style and |__MY_TITLE__| (cf |set title|) that contains the title of the box. Make sure to have the following packages to use the default styling:
\begin{verbatim}
\usepackage{pythonhighlight}
\usepackage{tcolorbox}
\end{verbatim}
\end{pgfmanualentry}





\subsubsection{Bash}

We provide a basic bash template, that sets:
\begin{verbatim}
set -e
outputTxt="__ROBEXT_OUTPUT_PREFIX__-out.txt"
outputTex="__ROBEXT_OUTPUT_PREFIX__-out.tex"
outputPdf="__ROBEXT_OUTPUT_PDF__"
\end{verbatim}
in order to quit when an error occurs, and to define two variables containing the path to the |pdf| file and to the file that is read by the |verbatim output| setting (that just apply a |\verbatiminput| on that file). Finally, it also creates the file |outputPdf| with |touch| in order to notify that the compilation succeeded.

In practice:
\begin{codeAndResult}
\begin{CacheMeCode}{bash, verbatim output}
# $outputTxt contains the path of the file that will be printed via \verbatiminput
uname -srv > "${outputTxt}"
\end{CacheMeCode}
\end{codeAndResult}

\subsubsection{Verbatim text}

Sometimes, it might be handy to write the text to a file and use it somehow. This is possible using |verbatim text|, that defaults to calling |\verbatiminput| on that file:

\begin{codeAndResult}
\begin{CacheMeCode}{verbatim text}
def some_verbatim_fct(a):
    # See this is a verbatim code where I can use the % symbol
    return a % b
\end{CacheMeCode}  
\end{codeAndResult}

You can also call |verbatim text no include|: it will not include the text, but it sets a macro |\robExtPathToInput| containing the path to the input file. Use it the way you like! For instance, we define here a macro |codeAndResult| that prints the code and runs it (we use a pretty printer from pgf, so you need to load |\usepackage{tikz}\input{pgfmanual-en-macros.tex}| to use it). It is what we use right now in this documentation for verbatim blocks like here. You can obtain a simpler version using:

\begin{codeAndResult}
\begin{CacheMeCode}{verbatim text no include}
\NewDocumentCommand{\testVerbatim}{+v}{
\begin{flushleft}\ttfamily%
#1
\end{flushleft}}
\testVerbatim{Demo % with percent}
\end{CacheMeCode}
We will input the file \robExtPathToInput{}:
\input{\robExtPathToInput}
This file contains:
\verbatiminput{\robExtPathToInput}
\end{codeAndResult}

\subsection{List of special placeholders and presets}\label{sec:placeholders}

This library defines a number of pre-existing placeholders, or placeholders playing a special role. We list some of them in this section. All placeholders created by this library start with |__ROBEXT_|. Note that you can list all predefined placeholders (at least those globally defined) using |\printAllPlaceholdersExceptDefaults| (note that some other placeholders might be created directly in the style set right before the command, and may not appear in this list if you call it before setting the style).

\subsubsection{Generic placeholders}

We define two special placeholders that should be defined by the user (possibly indirectly, using presets offered by this library):
\begin{itemize}
\item |__ROBEXT_TEMPLATE__| is a placeholder that should contain the code of the file to compile.
\item |__ROBEXT_MAIN_CONTENT__|: is a placeholder that might be used inside |__ROBEXT_TEMPLATE__| and that contains the content that the user is expected to type inside the document. For instance, this might be a tikz picture, a python function without the import etc. This will be automatically set by |CacheMe|, |CacheMeCode| etc, and some styles might add stuff to it (for instance the |tikz| preset adds the |\begin{tikzpicture}| around the user code automatically: this way we do not need to edit the command to disable externalization).
\item |__ROBEXT_COMPILATION_COMMAND__| contains the compilation command to run to compile the file (assuming we are in the cache folder).
\end{itemize}

We also provide a number of predefined placeholders in order to get the name of the source file etc... Note that most of these placeholders are defined (and/or expanded inplace) late during the compilation stage as one needs first to obtain the hash of the file, and therefore all dependencies, the content of the template etc.
\begin{itemize}
\item |__ROBEXT_SOURCE_FILE__| contains the path of the file to compile (containing the content of |__ROBEXT_TEMPLATE__|) like |robExt-somehash.tex|, relative to the cache folder (since we always go to this folder before doing any action, you most likely want to use this directly in the compilation command).
\item |__ROBEXT_OUTPUT_PDF__| contains the path of the pdf file produced after the compilation command relative to the cache folder (like |robExt-somehash.pdf|). Even if you do not plan to output a pdf file, you should still create that file at the end of the compilation so that this library can know whether the compilation succeeded. 
\item |__ROBEXT_OUTPUT_PREFIX__| contains the prefix that all newly created file should follow, like |robExt-somehash|. If you want to create additional files (e.g. a picture, a video, a console output etc...) make sure to make it start with this string. It will not only help to ensure purity, but it also allows us to garbage collect useless files easily.
\item |__ROBEXT_WAY_BACK__| contains the path to go back to the main project from the cache folder, like |../| (internally it is equals to the expanded value of |\robExtPrefixPathWayBack|).
\item |__ROBEXT_CACHE_FOLDER__| contains the path to the cache folder. Since most commands are run from the cache folder, this should not be really useful to the user.
\end{itemize}

You can also use these placeholders to customize the default include function:
\begin{itemize}
\item |__ROBEXT_INCLUDEGRAPHICS_OPTIONS__| contains the options given to |\includegraphics| when loading the pdf
\item |__ROBEXT_INCLUDEGRAPHICS_FILE__| contains the file loaded by |\includegraphics|, defaults to |\robExtAddCachePathAndName{\robExtFinalHash.pdf}|, that is itself equivalent to |__ROBEXT_CACHE_FOLDER____ROBEXT_OUTPUT_PDF__| or\\|__ROBEXT_CACHE_FOLDER____ROBEXT_OUTPUT_PREFIX__.pdf|.
\end{itemize}

\subsubsection{Placeholders related to \LaTeX{}}
Some placeholders are reserved only when dealing with \LaTeX{} code:
\begin{itemize}
\item |__ROBEXT_LATEX__| is the main entrypoint, containing all the latex template. It internally calls other placeholders listed below.
\item |__ROBEXT_LATEX_OPTIONS__|: contains the options to compile the document, like |a4paper|. Empty by default.
\item |__ROBEXT_DOCUMENT_CLASS__|: contains the class of the document. Defaults to |standalone|.
\item |__ROBEXT_PREAMBLE__|: contains the preamble. Is empty by default.
\item |__ROBEXT_MAIN_CONTENT_WRAPPED__|: content inside the |document| environment. It will wrap the actual content typed by the user |__ROBEXT_MAIN_CONTENT__| around a box to compute its depth. If you do not want this behavior, you can set |__ROBEXT_MAIN_CONTENT_WRAPPED__| to be equal to |__ROBEXT_MAIN_CONTENT__|. It calls internally |__ROBEXT_CREATE_OUT_FILE__| and |__ROBEXT_WRITE_DEPTH_TO_OUT_FILE__| to do this computation.
\item |__ROBEXT_CREATE_OUT_FILE__| creates a new file called |\jobname-out.tex| and open it in the handle called |\writeRobExt|
\item |__ROBEXT_WRITE_DEPTH_TO_OUT_FILE__| writes the height, depth and width of the box |\boxRobExt| into the filed opened in |\writeRobExt|.
\item |__ROBEXT_COMPILATION_COMMAND_LATEX__| is the command used to compile a \LaTeX{} document. It uses internally other placeholders:
\item |__ROBEXT_LATEX_ENGINE__| is the engine used to compile the document (defaults to |pdflatex|)
\item |__ROBEXT_COMPILATION_COMMAND_OPTIONS__| contains the options used to compile the document (defaults to |-shell-escape -halt-on-error|)
\end{itemize}

\subsubsection{Placeholders related to python}

\begin{itemize}
\item |__ROBEXT_PYTHON_EXEC__| contains the python executable (defaults to |python|) used to compile
\item |__ROBEXT_PYTHON__| contains the python template
\item |__ROBEXT_PYTHON_IMPORT__| can contain import statements
\item |__ROBEXT_PYTHON_MAIN_CONTENT_WRAPPED__| is used to add all the above functions. You can set it to |__ROBEXT_MAIN_CONTENT__| if you do not want them
\item |__ROBEXT_PYTHON_FINISHED_WITH_NO_ERROR__| is called at the end to create the pdf file even if it is not created, you can set it to the empty string if you do not want to do that.
\end{itemize}

\subsubsection{Placeholders related to bash}

\begin{itemize}
\item |__ROBEXT_BASH_TEMPLATE__| contains the bash template. By default, it sets |set -e|, creates |outputTxt|, |outputTex| and |outputPdf| pointing to the corresponding files, and it created the pdf file at the end.
\item |__ROBEXT_SHELL__| contains the shell (defaults to |bash|).
\end{itemize}


\subsection{Customize presets and create your own style}\label{sec:customize}

Note that you can define your own presets simply by creating a new pgf style (please refer to tikz-pgf's documentation for more details). For instance, we defined the |tikz| style using:
\begin{codeexample}[code only]
\robExtConfigure{
  tikz/.style={
    latex,
    add to preamble={\usepackage{tikz}},
    add before placeholder no space={__ROBEXT_MAIN_CONTENT__}{\begin{tikzpicture}},
    add to placeholder no space={__ROBEXT_MAIN_CONTENT__}{\end{tikzpicture}},
  },
}
\end{codeexample}
in order to automatically load |tikz| and add the surrounding |tikzpicture| (note that the style is always loaded \textbf{after} the definition of |__ROBEXT_MAIN_CONTENT__|, you can therefore do any postprocessing you like on this placeholder). You can also customize an existing style by adding stuff to it using |.append style|. For instance, here, we add the |shadows| library to the |tikz| preset by default:
\begin{codeexample}[width=0pt,vbox]
  \robExtConfigure{
    tikz/.append style={
      add to preamble={\usetikzlibrary{shadows}},
    },
  }
  See, tikz's style now packs the |shadows| library by default: %
  \begin{CacheMe}{tikz}[even odd rule]
    \filldraw [drop shadow,fill=white] (0,0) circle (.5) (0.5,0) circle (.5);
  \end{CacheMe}
\end{codeexample}


\subsection{Operations on the cache}\label{sec:operationsCache}

Every time we compile a document, we create automatically a bunch of files:
\begin{itemize}
\item the cache is located by default in the |robustExternalize| folder. Feel free to remove this folder if you want to completely clear the cache (but then you need to recompile everything). See below if you want to clean it in a better way.
\item |\jobname-robExt-all-figures.txt| contains the list of all figures contained in the document. Mostly useful to help the script that remove other figures.
\item |robExt-remove-old-figures.py| is a python script that will remove all cached files that are not used anymore. Just run |python robExt-remove-old-figures.py| to clean it. You will then see the list of files that the script wants to remove: make sure it does not remove any important data, and press ``y''. Note that it will search for all files that look like |*robExt-all-figures.txt| to see the list of pictures that are still in use, and by default it will only remove the images in the |robustExternalize| folder that start with |robExt-|. If you change the path of the cache or the prefix, edit the script (should not be hard to do).
\item |\jobname-robExt-compile-missing-figures.sh| contains a list of commands that you need to run to compile the images not yet compiled in the cache (this list will only be created if you enable the manual compilation mode).
\item |\jobname-robExt-tmp-file-you-can-remove.tmp| is a temporary file. Feel free to remove it.
\end{itemize}

We go over some of these scripts.

\subsubsection{Cleaning the cache}

You might want to clean the cache. Of course you can remove all generated files, but if you want to keep the picture in use in the latest version of the document, we provide a python script (automatically generated in the root folder) to do this. Just install python3 and run:\\

|python3 robExt-remove-old-figures.py|\\

(on windows, the executable might be called |python|) You will then be prompted for a confirmation after providing the list of files that will be removed.

\subsubsection{Listing all figures in use}

After the compilation of the document, a file |robExt-all-figures.txt| is created with the list of the |.tex| file of all figures used in the current document.

\subsubsection{Manually compiling the figures}

When enabling the manual mode (useful if we don't want to enable |-shell-escape|):

\begin{verbatim}
\robExtConfigure{
  enable manual mode
}
\end{verbatim}

the library creates a file |JOBNAME-robExt-compile-missing-figures.sh| that contains the instructions to build the figures that are not yet in the cache (each line contains the compilation command to run). On Linux (or on Windows with bash/cygwin/… installed, it possibly even work out of the box without) you can easily execute them using:

\begin{verbatim}
bash JOBNAME-robExt-compile-missing-figures.sh
\end{verbatim}


\subsection{How to debug}

If for some reasons you are unable to understand why a build fails, first check if you compiled your document with |-shell-escape| (not that this must appear \textbf{before} the filename). Then, you can look at the log file to get more advices: when a cached document is compiled, we always write the full compilation command before compiling the file in the log file. This way, you can easily check the content of the file and see why it fails to compile. The compilation errors are also displayed directly in the output.

You might often get an error |! Missing $ inserted.|: this is typically when a placeholder was left unreplaced (e.g.\ you forgot to define it, or you forgot to wrap a command in |\evalPlaceholder{}|): since \LaTeX{} is asked to typeset |__something__|, it thinks that you are trying to write a subscript, and asks you to start first the math mode. 

\section{TODO and known bugs:}

\begin{itemize}
\item See how to deal with label and references inside pictures (without disabling externalization).
\item We should create more pre-made settings, e.g. for tikz-cd, zx-calculus etc.
\item Deal with remember picture
\end{itemize}

\section{Acknowledgments}

I am deeply indebted to many users on \url{tex.stackexchange.com} that made the writing of this library possible. I can't list you all, but thank you so much!

\end{document}
% Local Variables:
% TeX-command-extra-options: "-shell-escape -halt-on-error"
% End: