diff options
author | Norbert Preining <norbert@preining.info> | 2022-09-20 03:01:38 +0000 |
---|---|---|
committer | Norbert Preining <norbert@preining.info> | 2022-09-20 03:01:38 +0000 |
commit | 7907f67d07d36a7e20421a1ec3a0eceecac4aacd (patch) | |
tree | 7aae41d00037cc944480cecfa75b88048a672970 /support | |
parent | fe872ca5d5d1bb989da6b31802b8e0bfb0342e45 (diff) |
CTAN sync 202209200301
Diffstat (limited to 'support')
-rw-r--r-- | support/texaccents/README.md | 3 | ||||
-rw-r--r-- | support/texaccents/texaccents.1 | 15 | ||||
-rw-r--r-- | support/texaccents/texaccents.md | 139 | ||||
-rw-r--r-- | support/texaccents/texaccents.pdf | bin | 47667 -> 50028 bytes | |||
-rwxr-xr-x[-rw-r--r--] | support/texaccents/texaccents.sno | 25 | ||||
-rw-r--r-- | support/texaccents/texaccents.spt | 278 | ||||
-rw-r--r-- | support/texaccents/texaccents.tex | 258 |
7 files changed, 712 insertions, 6 deletions
diff --git a/support/texaccents/README.md b/support/texaccents/README.md index ef977ce16b..4137f26b00 100644 --- a/support/texaccents/README.md +++ b/support/texaccents/README.md @@ -2,7 +2,8 @@ A standalone utility designed to convert legacy (La)TeX (text mode, no math) and BibTeX codes for "accented" characters to Unicode equivalents. For example, `\={a}` ('a' with macron) will be converted to `ā`. -Current version: 2022/08/27 version 1.0.0 +Current version: 2022/09/18 version 1.0.1 +Windows version: 2022/09/18 version 1.0.1 This project is licensed under the MIT License: see https://opensource.org/licenses/MIT diff --git a/support/texaccents/texaccents.1 b/support/texaccents/texaccents.1 new file mode 100644 index 0000000000..9ffd33f074 --- /dev/null +++ b/support/texaccents/texaccents.1 @@ -0,0 +1,15 @@ +.\" Manpage for texaccents +.\" Contact guido.milanese@unicatt.it to correct errors or typos. +.TH man 8 "17th September 2022" "1.0.1" "texaccents man page" +.SH NAME +texaccents - Converts legacy (La)TeX accents and ligatures to Unicode +.SH DESCRIPTION +A Snobol script. Example: \={a} ('a' with macron) will be converted to ā. +.SH OPTIONS +No options besides --help and --version. +.SH AUTHOR +Guido Milanese (guido.milanese@unicatt.it) +.SH BUGS +Please send bug reports or other correspondence to guido.milanese@unicatt.it +.SH SEE ALSO +CTAN page of the package: https://www.ctan.org/pkg/texaccents diff --git a/support/texaccents/texaccents.md b/support/texaccents/texaccents.md new file mode 100644 index 0000000000..feaec47dec --- /dev/null +++ b/support/texaccents/texaccents.md @@ -0,0 +1,139 @@ +--- +abstract: | + TeXaccents is a standalone utility designed to convert legacy (La)TeX ligatures and codes for "accented" characters to Unicode equivalents (text mode, no math) . For example, `\={a}` ('a' with macron) will be converted to `ā`. +author: +- "Guido Milanese[^1]" +date: 17^th^ September 2022 +lang: en +title: | + TeXaccents\ + version 1.0.1 +fontfamily: libertine +fontsize: 12pt +--- + +# General information + +Even if modern compilers handle Unicode encoding, (La) and files +featuring "legacy" encoding for non-Ascii characters are still very +common, and users may need to incorporate old code into new texts that +make use of modern text encoding. + +Several utilities are available online that claim to be able to convert +legacy (La) encoding to standard Unicode. See: + +- *Simple LaTeX to Text Converter*. A complex programme, able to deal + with maths. Insofar as non-Ascii chars are concerned, it fails + sometimes, at least according to my tests. See + <https://pylatexenc.readthedocs.io/en/latest/latexwalker/>. Written + in Python. + +- *LaTeX handler*. Converts non-Ascii (La) encoding to Unicode. + However, it does not seem to be able to deal with the legacy + encoding, e.g. `{\a}` instead of `\{a}` or `\a`. It does not convert + simple ligatures as `\ae{}` `\oe{}`. I used the tables provided by + this programme as a starting point. Written in Python. See + <https://github.com/hayk314/LaTex-handler>. + +- *Pandoc* is the standard programme for any text format conversion + (<https://pandoc.org/>). It converts almost all the accents (thorn + and eth missing?), but (if I have checked this correctly) normalises + files stripping non-standard fields. This can be a problem for + scholars who frequently use non-standard fields, such as e.g. + "shorttitle", required by not a few bibliographic styles. + +*TeXaccents* should be able to transform (La) normal text or "accents" +(not "math" accents) to their Unicode equivalent. The programme deals +with the following codes (*not all the fonts are able to output all the +required Unicode glyphs of this table!*): + + | NAME | \tex | Unicode | + |--------------- |------- |---------| + | Umlaut | \"{a} | ä | + | acute | \'{a} | á | + | double acute | \H{a} | a̋ | + | grave | \`{a} | à | + | circumflex | \^{a} | â | + | caron hraceck | \v{a} | ǎ | + | breve | \u{a} | ă | + | cedilla | \c{c} | ç | + | dot | \.{a} | ȧ | + | dot under | \d{a} | ạ | + | ogonek | \k{a} | ą | + | tilde | \~{a} | ã | + | macron | \={a} | ā | + | bar under | \b{a} | a̱ | + | ring over | \r{a} | å | + +The programme should recognize the following varieties: + +::: {.center} +`\'a` -- `\'{a}` -- `{\'a}` -- `{{\'a}}` +::: + +It transforms also the encoding for : `æ œ Æ Œ ð Ð þ Þ ø Ø ł Ł`. +Checking the page +<https://www.utf8-chartable.de/unicode-utf8-table.pl?number=512> I could +not find a legacy text mode encoding for: +**ƀ Ƀ đ Đ ǥ Ǥ ħ Ħ ɨ Ɨ ŧ Ŧ ƶ Ƶ** (some chars are accessible in math +mode). + +# Setup + +## From source +The programme is written in Snobol +(<https://en.wikipedia.org/wiki/SNOBOL> or +<https://it.wikipedia.org/wiki/SNOBOL>) and should run on any platform. +Steps: + +1. Install Snobol4 (version 2.3, March 2022) from + <http://www.regressive.org/snobol4/csnobol4/curr/>. Make sure to + install the compiler in a folder listed in your `PATH` or add the + folder to your path. On Linux the folder `snobol4` is installed + under `/usr/local/bin/`, which is normally listed in the PATH of a + standard Linux system. + +2. Test the compiler running `snobol4` from the command line. Leave the + compiler with `Ctr-C` or writing `end`. + +3. Copy `texaccents.sno` and all the provided `*.inc` files + +> `compiler.inc` `delete.inc` `grepl.inc` `newline.inc` `systype.inc` + +to a folder of your choice (e.g. `/home/<user>/bin`). + +4. In this folder, run + `snobol4 texaccents.sno testaccents-in testaccents-out` to test the + programme. The test file contains all the accents listed above. See + the result typing `cat testaccents-out` (Unixes / Powershell) or + `type testaccents-out` (Windows/Dos prompt), or open the file with + your text editor. The output file name is just a suggestion, of + course. + +## Windows standalone version +If preferred, a Windows EXE standalone file is provided. It was compiled using +Spitbol (see <https://github.com/spitbol/windows-nt>); the source code has been slightly adapted to Spitbol (basically only input/output syntax). From any directory, run `texaccents.exe INPUT OUTPUT`. To test the programme, run `texaccents.exe testaccents-in testaccents-out`. As above, the output file name is just a suggestion. + +# History + +- 25^th^ July 2022. First version (after trying unsuccesfully to + convert an old file with existing utilities) +- 17^th^ August 2022. First complete version (0.9). +- 27^rd^ August 2022. This version (1.0) with documentation and + comments. +- 17^th^ September 2022. Windows standalone executable. Manual page written. + Version message added; help message improved. In the source, a regular + shebang according to the recommendation of CTAN + (<https://tug.org/texlive/pkgcontrib.html>) was added. Documentation updated + accordingly. + +# Contacts / todo + +Bugs / suggestions / improvements: please write to +[guido.milanese\@unicatt.it](guido.milanese@unicatt.it) using +*TeXaccents* as subject of the mail. + +Genoa, Italy, 17^th^ September 2022 + +[^1]: Università Cattolica d.S.C., Dipartimento di scienze storiche e + filologiche, via Trieste 17, I-25121 Brescia diff --git a/support/texaccents/texaccents.pdf b/support/texaccents/texaccents.pdf Binary files differindex 6922882441..e41ab0abe0 100644 --- a/support/texaccents/texaccents.pdf +++ b/support/texaccents/texaccents.pdf diff --git a/support/texaccents/texaccents.sno b/support/texaccents/texaccents.sno index 5c20e75e47..41f655bad2 100644..100755 --- a/support/texaccents/texaccents.sno +++ b/support/texaccents/texaccents.sno @@ -1,6 +1,7 @@ +#!/usr/bin/env snobol4 # texaccents.sno -# version 1.0 -# 2022.08.23 +# version 1.0.1 +# 2022.08.23 -- 2022.09.17 # guido.milanese@unicatt.it # Transforms LaTeX accents to their UTF8 equivalents # accepts both LaTeX and Bibtex codes: @@ -32,15 +33,29 @@ initialise_bg -include "grepl.inc" ;* essential! calls repl internally MAX = 8000000 ;* max size of input/output files = 8 megs NL = newline() - HELP_MSG = "texaccents.sno 1.0" NL -+ " (c) 2022 Guido Milanese guido.milanese@unicatt.it" NL -+ " Usage: texaccents.sno INFILE OUTFILE" NL + HELP_MSG = "texaccents 1.0.1" NL ++ "Converts legacy (La)TeX accents and ligatures to Unicode" NL ++ "Usage: texaccents.sno INFILE OUTFILE" NL ++ "--help print this help, then exit" NL ++ "--version print version number, then exit" NL ++ "Report bugs to <guido.milanese@unicatt.it>" NL ++ "CTAN page of the package: <https://www.ctan.org/pkg/texaccents>" + + VERS_MSG = "texaccents 1.0.1" NL NL ++ "Copyright (c) 2022 Guido Milanese" NL NL ++ "This is free software; see the source for copying conditions. There is NO" NL ++ "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." NL nl ++ "Written by Guido Milanese <guido.milanese@unicatt.it>" + + ####################################################### # opens input / output files ####################################################### * Defines and checks Input & Output * help request -- exits. Both "--help" and "-help" are valid ((host(2,2) ? "-help") (Terminal = HELP_MSG)) :s(end) +* version request -- exits. Both "--version" and "-version" are valid + ((host(2,2) ? "-version") (Terminal = VERS_MSG)) :s(end) * Normal input-output with error messages. * Error messages are used by MAIN if necessary (~(infile = host(2,2)) ((Terminal = HELP_MSG) (ERROR_MSG = "Input error"))) :s(freturn) diff --git a/support/texaccents/texaccents.spt b/support/texaccents/texaccents.spt new file mode 100644 index 0000000000..33d435df83 --- /dev/null +++ b/support/texaccents/texaccents.spt @@ -0,0 +1,278 @@ +* WARNING - SPITBOL REQUIRES CR+LF LINE ENDINGS
+* THE CTAN TEAM WAS INFORMED ABOUT THIS SPECIAL NEED
+*
+* texaccents.spt
+* version 1.0.1
+* 2022.09.17
+* Windows version of texaccents.sno
+* guido.milanese@unicatt.it
+* Transforms LaTeX accents to their UTF8 equivalents
+* accepts both LaTeX and Bibtex codes:
+* examples: \'a \'{a} {\"a} and even {\'{a}}
+* tables of accents imported from:
+* https://github.com/hayk314/LaTex-handler -- Author: Hayk Aleksanyan
+* with my integrations, including ligatures as \ae{}
+* requires spitbol: https://github.com/spitbol/windows-nt
+
+*****************************************************************
+* MIT License - Copyright (c) 2022 Guido Milanese
+* See file LICENSE in this package
+*****************************************************************
+
+******************************************************
+* FUNCTIONS
+******************************************************
+
+*-- Function INITIALISE
+*-- opens files, imports functions
+ define("initialise()")
+ :(initialise_end)
+initialise
+initialise_bg
+* INCLUDE FILES
+
+-include "args.inc"
+-include "delete.inc"
+-include "host.inc"
+-include "grepl.inc" ;* essential! calls repl internally
+ MAX = 4000000 ;* max size of input/output files = 4 megs
+ NL = char(13) char(10)
+
+ HELP_MSG = "texaccents 1.0.1" NL
++ "Converts legacy (La)TeX accents and ligatures to Unicode" NL
++ "Usage: texaccents.sno INFILE OUTFILE" NL
++ "--help print this help, then exit" NL
++ "--version print version number, then exit" NL
++ "Report bugs to <guido.milanese@unicatt.it>" NL
++ "CTAN page of the package: <https://www.ctan.org/pkg/texaccents>"
+
+ VERS_MSG = "texaccents 1.0.1" NL NL
++ "Copyright (c) 2022 Guido Milanese" NL NL
++ "This is free software; see the source for copying conditions. There is NO" NL
++ "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." NL nl
++ "Written by Guido Milanese <guido.milanese@unicatt.it>"
+
+*******************************************************
+* opens input / output files
+*******************************************************
+* Defines and checks Input & Output
+* help request -- exits. Both "--help" and "-help" are valid
+ ((argv[1] ? "-help") (Terminal = HELP_MSG)) :s(end)
+* version request -- exits. Both "--version" and "-version" are valid
+ ((argv[1] ? "-version") (Terminal = VERS_MSG)) :s(end)
+* Normal input-output with error messages.
+* Error messages are used by MAIN if necessary
+ (~(infile = argv[1]) ((Terminal = HELP_MSG) (ERROR_MSG = "Input error"))) :s(freturn)
+ (~(outfile = argv[2]) ((Terminal = HELP_MSG) (ERROR_MSG = "Output error"))) :s(freturn)
+ INPUT('r_f',1, infile '[-R' MAX ']') :f(freturn)
+ Terminal = "Reading text from " Infile
+ OUTPUT('w_f',2,Outfile '[-r' MAX ']') :f(freturn)
+initialise_rt initialise = r_f :(return)
+initialise_end
+
+*************************************************************************************
+*-- Function NORMACC
+*-- Changes TeX accents without curly brackets to the standard form
+*-- \'a --> \'{a}
+*-- and replaces ligatures such as \ae --> æ | \oe --> œ
+*-- requires GREPL
+*************************************************************************************
+ define("normacc(Pass)")
+ P_shortacc = notany('{')
++ ((('\' any("'`" '"' "^=~.")) . Pre_shortacc)
++ (any(&ucase &lcase) . Letter_shortacc)) . V_shortacc
+ :(normacc_end)
+normacc
+normacc_bg Pass ? P_shortacc :f(normacc_lg)
+ Pass ? V_shortacc = Pre_shortacc '{' Letter_shortacc '}' :(normacc_bg)
+normacc_lg Pass2 = grepl(Pass,
++ "\ae{} \oe{} \AE{} \OE{} \dh{} \DH{} \th{} \TH{} \o{} \O{} \l{} \L{} ",
++ "æ œ Æ Œ ð Ð þ Þ ø Ø ł Ł ")
+ Terminal = "Ligatures tranformed"
+normacc_rt normacc = Pass2 :(return)
+normacc_end
+*
+* The pattern to normalise accents -- as \'a -- is defined as follows:
+* 1. there must be no '{' before backslash: otherwise the pattern would trap also
+* BibTex codes such as {\'a}
+* 2. find a backslash followed by any char of the list. Saved as "Pre_sortacc"
+* 3. find the accented letter. Saved as Letter_shortacc
+* 4. The whole structure is saved as V_shortacc
+* The programme will save the whole structure and substitute it in the text with:
+* Pre_shortacc (as \') + '{' + Letter_shortacc + '}'
+* in this way the short form -- \'a -- is transformed to the canonical \'{a}
+*
+* Ligatures and special chars are simply replaced with GREPL.
+
+*************************************************************************************
+*-- Function CLEANACC
+*-- Substitutes plain letters to Unicode accents, using the sets provides by TRANSACC
+*-- \'{a} will output á
+*************************************************************************************
+ define("cleanacc(Text,Acc1,Acc2)") :(cleanacc_end)
+* Acc1 is the original LaTeX code e.g. \"{a}
+* Acc2, at the time of calling the function, is e.g. \"{ä}
+cleanacc
+cleanacc_bg
+* removes paretheses: \"{ä} --> \"ä
+ Acc2 = delete(Acc2,"{}")
+* moves 2 chars and saves what remains, i.e. the letter
+* \"ä --> ä
+ Acc2 ? len(2) (rem . Acc2)
+* in text replaces e.g. \"{a} with ä
+* the substitution is done ONCE for the whole text
+ Text = repl(Text,Acc1,Acc2)
+ Terminal = "Working on " Acc1
+cleanacc_rt cleanacc = Text :(return)
+cleanacc_end
+
+*******************************************************
+*-- Function TRANSACC
+*-- transforms LaTeX/Bibtex accents to Unicode
+*******************************************************
+ define("transacc(Pass)")
+* Pass is the input text
+*
+* Accents used by Latex, e.g. \'{a} = acute accent over 'a'
+ Lat_acc =
++ any('"' "'" "`" 'H' '^' 'v' 'u' 'c' '.' 'd' 'k' '~' '=' 'b' 'r' )
++ . V_lat_acc
+*
+* The pattern says:
+* '\' + Lat_acc OR ('{\' + Lat_acc + '{' OR nothing)
+* any letter, saved as C_noacc + '}' + '}' OR nothing
+* Lat_acc is already defined above and saved ad V_lat_acc
+* Examples \'{a} -- {\'a} -- {\'{a}} (\'a was already "normalised")
+ P_lat_code =
++ (
++ (('\' Lat_acc '{') | ('{\' Lat_acc ('{' | '') ))
++ (any(&lcase &ucase) . C_noacc) '}' ('}' | '') ) . V_lat_code
+ :(transacc_end)
+transacc
+transacc_bg
+latcode_bg V_lat_code = ;* cleans previous value
+ V_lat_code_2 = ;* cleans previous value
+* Scans input text to locate the pattern
+* if no other LaTeX accents, fails and returns to main
+* if yes, goes to the section that transforms this accent.
+* e.g. if the accent is umlaut, will jump to TR"
+* After each transformation we have:
+* 1. the original code, e.g. \"{a} or \"a or {\"a}
+* 2. the transformed code, e.g. \"{ä} or \"ä or {\"ä}
+* Each section calls CLEANCLODE
+* CLEANCODE will remove all the {}\ and the LaTeX accent
+* and substitute in the input text e.g. \"{ä} with ä
+ Pass ? P_lat_code :f(latcode_nd)
+ :($('TR' V_lat_acc))
+
+TR" ;* umlaut
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c E e H h I i K k M m N n O o P p Q q S s T t U u V v W w X x Y y Z z ",
++ "Ä ä B̈ b̈ C̈ c̈ Ë ë Ḧ ḧ Ï ï K̈ k̈ M̈ m̈ N̈ n̈ Ö ö P̈ p̈ Q̈ q̈ S̈ s̈ T̈ ẗ Ü ü V̈ v̈ Ẅ ẅ Ẍ ẍ Ÿ ÿ Z̈ z̈ ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TR' ;* acute
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e F f G g H h I i J j K k L l M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ",
++ "Á á B́ b́ Ć ć D́ d́ É é F́ f́ Ǵ ǵ H́ h́ Í í J́ ȷ́ Ḱ ḱ Ĺ ĺ Ḿ ḿ Ń ń Ó ó Ṕ ṕ Q́ q́ Ŕ ŕ Ś ś T́ t́ Ú ú V́ v́ Ẃ ẃ X́ x́ Ý ý Ź ź ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TRH ;* double acute - \H{o} Hungarian
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a E e I i M m O o U u ",
++ "A̋ a̋ E̋ e̋ I̋ i̋ M̋ m̋ Ő ő Ű ű ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TR` ;* grave
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a Æ æ E e H h I i K k M m N n O o R r S s T t U u V v W w X x Y y Z z ",
++ "À à Æ̀ æ̀ È è H̀ h̀ Ì ì K̀ k̀ M̀ m̀ Ǹ ǹ Ò ò R̀ r̀ S̀ s̀ T̀ t̀ Ù ù V̀ v̀ Ẁ ẁ X̀ x̀ Ỳ ỳ Z̀ z̀ ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TR^ ;* circumflex \^{o}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e G g H h I i J j K k L l M m N n O o R r S s T t U u V v W w X x Y y Z z ",
++ "Â â B̂ b̂ Ĉ ĉ D̂ d̂ Ê ê Ĝ ĝ Ĥ ĥ Î î Ĵ ĵ K̂ k̂ L̂ l̂ M̂ m̂ N̂ n̂ Ô ô R̂ r̂ Ŝ ŝ T̂ t̂ Û û V̂ v̂ Ŵ ŵ X̂ x̂ Ŷ ŷ Ẑ ẑ ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TRv ;* caron hraceck \v{s}
+* For accents that are also normal letters
+* we need to restore the accent to its original value
+* This applies to: v u c d k b r
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e F f G g H h I i J j K k L l M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ",
++ "Ǎ ǎ B̌ b̌ Č č Ď ď Ě ě F̌ f̌ Ǧ ǧ Ȟ ȟ Ǐ ǐ J̌ ǰ Ǩ ǩ Ľ ľ M̌ m̌ Ň ň Ǒ ǒ P̌ p̌ Q̌ q̌ Ř ř Š š Ť ť Ǔ ǔ V̌ v̌ W̌ w̌ X̌ x̌ Y̌ y̌ Ž ž ")
+ V_lat_code_2 = repl(V_lat_code_2,"\v̌","\v")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TRu ;* breve \u{o}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a C c E e G g I i K k M m N n O o P p R r T t U u V v X x Y y ",
++ "Ă ă C̆ c̆ Ĕ ĕ Ğ ğ Ĭ ĭ K̆ k̆ M̆ m̆ N̆ n̆ Ŏ ŏ P̆ p̆ R̆ r̆ T̆ t̆ Ŭ ŭ V̆ v̆ X̆ x̆ Y̆ y̆ ")
+ V_lat_code_2 = repl(V_lat_code_2,"\ŭ","\u")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TRc ;* cedilla \c{c}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e G g H h I i K k L l M m N n O o Q q R r S s T t U u X x Z z ",
++ "A̧ a̧ B̧ b̧ Ç ç Ḑ ḑ Ȩ ȩ Ģ ģ Ḩ ḩ I̧ i̧ Ķ ķ Ļ ļ M̧ m̧ Ņ ņ O̧ o̧ Q̧ q̧ Ŗ ŗ Ş ş Ţ ţ U̧ u̧ X̧ x̧ Z̧ z̧ ")
+ V_lat_code_2 = repl(V_lat_code_2,"\ç","\c")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TR. ;* dot \.{o}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e F f G g H h I i K k L l M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ",
++ "Ȧ ȧ Ḃ ḃ Ċ ċ Ḋ ḋ Ė ė Ḟ ḟ Ġ ġ Ḣ ḣ İ i̇̀ K̇ k̇ L̇ l̇ Ṁ ṁ Ṅ ṅ Ȯ ȯ Ṗ ṗ Q̇ q̇ Ṙ ṙ Ṡ ṡ Ṫ ṫ U̇ u̇ V̇ v̇ Ẇ ẇ Ẋ ẋ Ẏ ẏ Ż ż ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TRd ;* dot under the letter \d{u}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e F f G g H h I i J j K k L l M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ",
++ "Ạ ạ Ḅ ḅ C̣ c̣ Ḍ ḍ Ẹ ẹ F̣ f̣ G̣ g̣ Ḥ ḥ Ị ị J̣ j̣ Ḳ ḳ Ḷ ḷ Ṃ ṃ Ṇ ṇ Ọ ọ P̣ p̣ Q̣ q̣ Ṛ ṛ Ṣ ṣ Ṭ ṭ Ụ ụ Ṿ ṿ Ẉ ẉ X̣ x̣ Ỵ ỵ Ẓ ẓ ")
+ V_lat_code_2 = repl(V_lat_code_2,"\ḍ","\d")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TRk ;* ogonek \k{a}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a E e I i O o U u Y y ",
++ "Ą ą Ę ę Į į Ǫ ǫ Ų ų Y̨ y̨ ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TR~ ;* tilde \~o
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a E e I i N n O o U u V v Y y ",
++ "Ã ã Ẽ ẽ Ĩ ĩ Ñ ñ Õ õ Ũ ũ Ṽ ṽ Ỹ ỹ ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TR= ;* macron \=a \={a}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e G g I i J j K k M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ",
++ "Ā ā B̄ b̄ C̄ c̄ D̄ d̄ Ē ē Ḡ ḡ Ī ī J̄ j̄ K̄ k̄ M̄ m̄ N̄ n̄ Ō ō P̄ p̄ Q̄ q̄ R̄ r̄ S̄ s̄ T̄ t̄ Ū ū V̄ v̄ W̄ w̄ X̄ x̄ Ȳ ȳ Z̄ z̄ ")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TRb ;* bar under the letter \b{a}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e G g H h I i J j K k L l M m N n O o P p R r S s T t U u X x Y y Z z ",
++ "A̱ a̱ Ḇ ḇ C̱ c̱ Ḏ ḏ E̱ e̱ G̱ g̱ H̱ ẖ I̱ i̱ J̱ j̱ Ḵ ḵ Ḻ ḻ M̱ m̱ Ṉ ṉ O̱ o̱ P̱ p̱ Ṟ ṟ S̱ s̱ Ṯ ṯ U̱ u̱ X̱ x̱ Y̱ y̱ Ẕ ẕ ")
+ V_lat_code_2 = repl(V_lat_code_2,"\ḇ","\b")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+TRr ;* ring over the letter \r{a}
+ V_lat_code_2 = grepl(V_lat_code,
++ "A a B b C c D d E e F f G g H h I i J j K k L l M m N n O o P p Q q r R S s U u V v W w X x Y y Z z ",
++ "Å å B̊ b̊ C̊ c̊ D̊ d̊ E̊ e̊ F̊ f̊ G̊ g̊ H̊ h̊ I̊ i̊ J̊ j̊ K̊ k̊ L̊ l̊ M̊ m̊ N̊ n̊ O̊ o̊ P̊ p̊ Q̊ q̊ r̊ R̊ S̊ s̊ Ů ů V̊ v̊ W̊ ẘ X̊ x̊ Y̊ ẙ Z̊ z̊ ")
+ V_lat_code_2 = repl(V_lat_code_2,"\r̊","\r")
+ Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg)
+
+latcode_nd
+ transacc = Pass :(return)
+transacc_end
+
+*******************************************************
+* MAIN
+*******************************************************
+ (~(texfile = initialise()) (terminal = "Could not start programme - " ERROR_MSG)) :s(end)
+ Texfile = normacc(Texfile)
+ (((w_f = transacc(texfile)) (Terminal = "Done. File " Outfile " written")),
++ (Terminal = "Conversion failed"))
+END
diff --git a/support/texaccents/texaccents.tex b/support/texaccents/texaccents.tex new file mode 100644 index 0000000000..ec63276852 --- /dev/null +++ b/support/texaccents/texaccents.tex @@ -0,0 +1,258 @@ +% TEXACCENTS.TEX +% Documentation for texaccents.sno / texaccents.spt +% version 1.0.1 -- 17th september 2022 +% author: guido.milanese@unicatt.it +% tex file produced from markdown source +% using `pandoc texaccents.md -s -o texaccents.tex` +% and compiled with lualatex +% Line 35 was added: \microtypesetup{nopatch=item} +% See: https://github.com/schlcht/microtype/issues/4 +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% +% Options for packages loaded elsewhere +\PassOptionsToPackage{unicode}{hyperref} +\PassOptionsToPackage{hyphens}{url} +% +\documentclass[ + 12pt, + english, +]{article} +\usepackage{amsmath,amssymb} +\usepackage[]{libertine} +\usepackage{iftex} +\ifPDFTeX + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} + \usepackage{textcomp} % provide euro and other symbols +\else % if luatex or xetex + \usepackage{unicode-math} + \defaultfontfeatures{Scale=MatchLowercase} + \defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1} +\fi +% Use upquote if available, for straight quotes in verbatim environments +\IfFileExists{upquote.sty}{\usepackage{upquote}}{} +\IfFileExists{microtype.sty}{% use microtype if available + \usepackage[]{microtype} + \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts + \microtypesetup{nopatch=item} +}{} +\makeatletter +\@ifundefined{KOMAClassName}{% if non-KOMA class + \IfFileExists{parskip.sty}{% + \usepackage{parskip} + }{% else + \setlength{\parindent}{0pt} + \setlength{\parskip}{6pt plus 2pt minus 1pt}} +}{% if KOMA class + \KOMAoptions{parskip=half}} +\makeatother +\usepackage{xcolor} +\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available +\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}} +\hypersetup{ + pdftitle={TeXaccents version 1.0.1}, + pdfauthor={Guido Milanese}, + pdflang={en}, + hidelinks, + pdfcreator={LaTeX via pandoc}} +\urlstyle{same} % disable monospaced font for URLs +\setlength{\emergencystretch}{3em} % prevent overfull lines +\providecommand{\tightlist}{% + \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}} +\setcounter{secnumdepth}{-\maxdimen} % remove section numbering +\ifXeTeX + % Load polyglossia as late as possible: uses bidi with RTL langages (e.g. Hebrew, Arabic) + \usepackage{polyglossia} + \setmainlanguage[]{english} +\else + \usepackage[main=english]{babel} +% get rid of language-specific shorthands (see #6817): +\let\LanguageShortHands\languageshorthands +\def\languageshorthands#1{} +\fi +\ifLuaTeX + \usepackage{selnolig} % disable illegal ligatures +\fi + +\title{TeXaccents\\ +version 1.0.1} +\author{Guido Milanese\footnote{Università Cattolica d.S.C., + Dipartimento di scienze storiche e filologiche, via Trieste 17, + I-25121 Brescia}} +\date{17\textsuperscript{th} September 2022} + +\begin{document} +\maketitle +\begin{abstract} +TeXaccents is a standalone utility designed to convert legacy (La)TeX +ligatures and codes for ``accented'' characters to Unicode equivalents +(text mode, no math) . For example, \texttt{\textbackslash{}=\{a\}} (`a' +with macron) will be converted to \texttt{ā}. +\end{abstract} + +\hypertarget{general-information}{% +\section{General information}\label{general-information}} + +Even if modern compilers handle Unicode encoding, (La) and files +featuring ``legacy'' encoding for non-Ascii characters are still very +common, and users may need to incorporate old code into new texts that +make use of modern text encoding. + +Several utilities are available online that claim to be able to convert +legacy (La) encoding to standard Unicode. See: + +\begin{itemize} +\item + \emph{Simple LaTeX to Text Converter}. A complex programme, able to + deal with maths. Insofar as non-Ascii chars are concerned, it fails + sometimes, at least according to my tests. See + \url{https://pylatexenc.readthedocs.io/en/latest/latexwalker/}. + Written in Python. +\item + \emph{LaTeX handler}. Converts non-Ascii (La) encoding to Unicode. + However, it does not seem to be able to deal with the legacy encoding, + e.g.~\texttt{\{\textbackslash{}a\}} instead of + \texttt{\textbackslash{}\{a\}} or \texttt{\textbackslash{}a}. It does + not convert simple ligatures as \texttt{\textbackslash{}ae\{\}} + \texttt{\textbackslash{}oe\{\}}. I used the tables provided by this + programme as a starting point. Written in Python. See + \url{https://github.com/hayk314/LaTex-handler}. +\item + \emph{Pandoc} is the standard programme for any text format conversion + (\url{https://pandoc.org/}). It converts almost all the accents (thorn + and eth missing?), but (if I have checked this correctly) normalises + files stripping non-standard fields. This can be a problem for + scholars who frequently use non-standard fields, such as e.g. + ``shorttitle'', required by not a few bibliographic styles. +\end{itemize} + +\emph{TeXaccents} should be able to transform (La) normal text or +``accents'' (not ``math'' accents) to their Unicode equivalent. The +programme deals with the following codes (\emph{not all the fonts are +able to output all the required Unicode glyphs of this table!}): + +\begin{verbatim} +| NAME | \tex | Unicode | +|--------------- |------- |---------| +| Umlaut | \"{a} | ä | +| acute | \'{a} | á | +| double acute | \H{a} | a̋ | +| grave | \`{a} | à | +| circumflex | \^{a} | â | +| caron hraceck | \v{a} | ǎ | +| breve | \u{a} | ă | +| cedilla | \c{c} | ç | +| dot | \.{a} | ȧ | +| dot under | \d{a} | ạ | +| ogonek | \k{a} | ą | +| tilde | \~{a} | ã | +| macron | \={a} | ā | +| bar under | \b{a} | a̱ | +| ring over | \r{a} | å | +\end{verbatim} + +The programme should recognize the following varieties: + +\texttt{\textbackslash{}\textquotesingle{}a} -- +\texttt{\textbackslash{}\textquotesingle{}\{a\}} -- +\texttt{\{\textbackslash{}\textquotesingle{}a\}} -- +\texttt{\{\{\textbackslash{}\textquotesingle{}a\}\}} + +It transforms also the encoding for : \texttt{æ~œ~Æ~Œ~ð~Ð~þ~Þ~ø~Ø~ł~Ł}. +Checking the page +\url{https://www.utf8-chartable.de/unicode-utf8-table.pl?number=512} I +could not find a legacy text mode encoding for: +\textbf{ƀ~Ƀ~đ~Đ~ǥ~Ǥ~ħ~Ħ~ɨ~Ɨ~ŧ~Ŧ~ƶ~Ƶ} (some chars are accessible in math +mode). + +\hypertarget{setup}{% +\section{Setup}\label{setup}} + +\hypertarget{from-source}{% +\subsection{From source}\label{from-source}} + +The programme is written in Snobol +(\url{https://en.wikipedia.org/wiki/SNOBOL} or +\url{https://it.wikipedia.org/wiki/SNOBOL}) and should run on any +platform. Steps: + +\begin{enumerate} +\def\labelenumi{\arabic{enumi}.} +\item + Install Snobol4 (version 2.3, March 2022) from + \url{http://www.regressive.org/snobol4/csnobol4/curr/}. Make sure to + install the compiler in a folder listed in your \texttt{PATH} or add + the folder to your path. On Linux the folder \texttt{snobol4} is + installed under \texttt{/usr/local/bin/}, which is normally listed in + the PATH of a standard Linux system. +\item + Test the compiler running \texttt{snobol4} from the command line. + Leave the compiler with \texttt{Ctr-C} or writing \texttt{end}. +\item + Copy \texttt{texaccents.sno} and all the provided \texttt{*.inc} files +\end{enumerate} + +\begin{quote} +\texttt{compiler.inc}~\texttt{delete.inc}~\texttt{grepl.inc}~\texttt{newline.inc}~\texttt{systype.inc} +\end{quote} + +to a folder of your choice +(e.g.~\texttt{/home/\textless{}user\textgreater{}/bin}). + +\begin{enumerate} +\def\labelenumi{\arabic{enumi}.} +\setcounter{enumi}{3} +\tightlist +\item + In this folder, run + \texttt{snobol4\ texaccents.sno\ testaccents-in\ testaccents-out} to + test the programme. The test file contains all the accents listed + above. See the result typing \texttt{cat~testaccents-out} (Unixes / + Powershell) or \texttt{type~testaccents-out} (Windows/Dos prompt), or + open the file with your text editor. The output file name is just a + suggestion, of course. +\end{enumerate} + +\hypertarget{windows-standalone-version}{% +\subsection{Windows standalone +version}\label{windows-standalone-version}} + +If preferred, a Windows EXE standalone file is provided. It was compiled +using Spitbol (see \url{https://github.com/spitbol/windows-nt}); the +source code has been slightly adapted to Spitbol (basically only +input/output syntax). From any directory, run +\texttt{texaccents.exe\ INPUT\ OUTPUT}. To test the programme, run +\texttt{texaccents.exe\ testaccents-in\ testaccents-out}. As above, the +output file name is just a suggestion. + +\hypertarget{history}{% +\section{History}\label{history}} + +\begin{itemize} +\tightlist +\item + 25\textsuperscript{th} July 2022. First version (after trying + unsuccesfully to convert an old file with existing utilities) +\item + 17\textsuperscript{th} August 2022. First complete version (0.9). +\item + 27\textsuperscript{rd} August 2022. This version (1.0) with + documentation and comments. +\item + 17\textsuperscript{th} September 2022. Windows standalone executable. + Manual page written. Version message added; help message improved. In + the source, a regular shebang according to the recommendation of CTAN + (\url{https://tug.org/texlive/pkgcontrib.html}) was added. + Documentation updated accordingly. +\end{itemize} + +\hypertarget{contacts-todo}{% +\section{Contacts / todo}\label{contacts-todo}} + +Bugs / suggestions / improvements: please write to +\url{guido.milanese@unicatt.it} using \emph{TeXaccents} as subject of +the mail. + +Genoa, Italy, 17\textsuperscript{th} September 2022 + +\end{document} |