From 7907f67d07d36a7e20421a1ec3a0eceecac4aacd Mon Sep 17 00:00:00 2001 From: Norbert Preining Date: Tue, 20 Sep 2022 03:01:38 +0000 Subject: CTAN sync 202209200301 --- support/texaccents/README.md | 3 +- support/texaccents/texaccents.1 | 15 ++ support/texaccents/texaccents.md | 139 +++++++++++++++++++ support/texaccents/texaccents.pdf | Bin 47667 -> 50028 bytes support/texaccents/texaccents.sno | 25 +++- support/texaccents/texaccents.spt | 278 ++++++++++++++++++++++++++++++++++++++ support/texaccents/texaccents.tex | 258 +++++++++++++++++++++++++++++++++++ 7 files changed, 712 insertions(+), 6 deletions(-) create mode 100644 support/texaccents/texaccents.1 create mode 100644 support/texaccents/texaccents.md mode change 100644 => 100755 support/texaccents/texaccents.sno create mode 100644 support/texaccents/texaccents.spt create mode 100644 support/texaccents/texaccents.tex (limited to 'support') diff --git a/support/texaccents/README.md b/support/texaccents/README.md index ef977ce16b..4137f26b00 100644 --- a/support/texaccents/README.md +++ b/support/texaccents/README.md @@ -2,7 +2,8 @@ A standalone utility designed to convert legacy (La)TeX (text mode, no math) and BibTeX codes for "accented" characters to Unicode equivalents. For example, `\={a}` ('a' with macron) will be converted to `ā`. -Current version: 2022/08/27 version 1.0.0 +Current version: 2022/09/18 version 1.0.1 +Windows version: 2022/09/18 version 1.0.1 This project is licensed under the MIT License: see https://opensource.org/licenses/MIT diff --git a/support/texaccents/texaccents.1 b/support/texaccents/texaccents.1 new file mode 100644 index 0000000000..9ffd33f074 --- /dev/null +++ b/support/texaccents/texaccents.1 @@ -0,0 +1,15 @@ +.\" Manpage for texaccents +.\" Contact guido.milanese@unicatt.it to correct errors or typos. +.TH man 8 "17th September 2022" "1.0.1" "texaccents man page" +.SH NAME +texaccents - Converts legacy (La)TeX accents and ligatures to Unicode +.SH DESCRIPTION +A Snobol script. Example: \={a} ('a' with macron) will be converted to ā. +.SH OPTIONS +No options besides --help and --version. +.SH AUTHOR +Guido Milanese (guido.milanese@unicatt.it) +.SH BUGS +Please send bug reports or other correspondence to guido.milanese@unicatt.it +.SH SEE ALSO +CTAN page of the package: https://www.ctan.org/pkg/texaccents diff --git a/support/texaccents/texaccents.md b/support/texaccents/texaccents.md new file mode 100644 index 0000000000..feaec47dec --- /dev/null +++ b/support/texaccents/texaccents.md @@ -0,0 +1,139 @@ +--- +abstract: | + TeXaccents is a standalone utility designed to convert legacy (La)TeX ligatures and codes for "accented" characters to Unicode equivalents (text mode, no math) . For example, `\={a}` ('a' with macron) will be converted to `ā`. +author: +- "Guido Milanese[^1]" +date: 17^th^ September 2022 +lang: en +title: | + TeXaccents\ + version 1.0.1 +fontfamily: libertine +fontsize: 12pt +--- + +# General information + +Even if modern compilers handle Unicode encoding, (La) and files +featuring "legacy" encoding for non-Ascii characters are still very +common, and users may need to incorporate old code into new texts that +make use of modern text encoding. + +Several utilities are available online that claim to be able to convert +legacy (La) encoding to standard Unicode. See: + +- *Simple LaTeX to Text Converter*. A complex programme, able to deal + with maths. Insofar as non-Ascii chars are concerned, it fails + sometimes, at least according to my tests. See + . Written + in Python. + +- *LaTeX handler*. Converts non-Ascii (La) encoding to Unicode. + However, it does not seem to be able to deal with the legacy + encoding, e.g. `{\a}` instead of `\{a}` or `\a`. It does not convert + simple ligatures as `\ae{}` `\oe{}`. I used the tables provided by + this programme as a starting point. Written in Python. See + . + +- *Pandoc* is the standard programme for any text format conversion + (). It converts almost all the accents (thorn + and eth missing?), but (if I have checked this correctly) normalises + files stripping non-standard fields. This can be a problem for + scholars who frequently use non-standard fields, such as e.g. + "shorttitle", required by not a few bibliographic styles. + +*TeXaccents* should be able to transform (La) normal text or "accents" +(not "math" accents) to their Unicode equivalent. The programme deals +with the following codes (*not all the fonts are able to output all the +required Unicode glyphs of this table!*): + + | NAME | \tex | Unicode | + |--------------- |------- |---------| + | Umlaut | \"{a} | ä | + | acute | \'{a} | á | + | double acute | \H{a} | a̋ | + | grave | \`{a} | à | + | circumflex | \^{a} | â | + | caron hraceck | \v{a} | ǎ | + | breve | \u{a} | ă | + | cedilla | \c{c} | ç | + | dot | \.{a} | ȧ | + | dot under | \d{a} | ạ | + | ogonek | \k{a} | ą | + | tilde | \~{a} | ã | + | macron | \={a} | ā | + | bar under | \b{a} | a̱ | + | ring over | \r{a} | å | + +The programme should recognize the following varieties: + +::: {.center} +`\'a` -- `\'{a}` -- `{\'a}` -- `{{\'a}}` +::: + +It transforms also the encoding for : `æ œ Æ Œ ð Ð þ Þ ø Ø ł Ł`. +Checking the page + I could +not find a legacy text mode encoding for: +**ƀ Ƀ đ Đ ǥ Ǥ ħ Ħ ɨ Ɨ ŧ Ŧ ƶ Ƶ** (some chars are accessible in math +mode). + +# Setup + +## From source +The programme is written in Snobol +( or +) and should run on any platform. +Steps: + +1. Install Snobol4 (version 2.3, March 2022) from + . Make sure to + install the compiler in a folder listed in your `PATH` or add the + folder to your path. On Linux the folder `snobol4` is installed + under `/usr/local/bin/`, which is normally listed in the PATH of a + standard Linux system. + +2. Test the compiler running `snobol4` from the command line. Leave the + compiler with `Ctr-C` or writing `end`. + +3. Copy `texaccents.sno` and all the provided `*.inc` files + +> `compiler.inc` `delete.inc` `grepl.inc` `newline.inc` `systype.inc` + +to a folder of your choice (e.g. `/home//bin`). + +4. In this folder, run + `snobol4 texaccents.sno testaccents-in testaccents-out` to test the + programme. The test file contains all the accents listed above. See + the result typing `cat testaccents-out` (Unixes / Powershell) or + `type testaccents-out` (Windows/Dos prompt), or open the file with + your text editor. The output file name is just a suggestion, of + course. + +## Windows standalone version +If preferred, a Windows EXE standalone file is provided. It was compiled using +Spitbol (see ); the source code has been slightly adapted to Spitbol (basically only input/output syntax). From any directory, run `texaccents.exe INPUT OUTPUT`. To test the programme, run `texaccents.exe testaccents-in testaccents-out`. As above, the output file name is just a suggestion. + +# History + +- 25^th^ July 2022. First version (after trying unsuccesfully to + convert an old file with existing utilities) +- 17^th^ August 2022. First complete version (0.9). +- 27^rd^ August 2022. This version (1.0) with documentation and + comments. +- 17^th^ September 2022. Windows standalone executable. Manual page written. + Version message added; help message improved. In the source, a regular + shebang according to the recommendation of CTAN + () was added. Documentation updated + accordingly. + +# Contacts / todo + +Bugs / suggestions / improvements: please write to +[guido.milanese\@unicatt.it](guido.milanese@unicatt.it) using +*TeXaccents* as subject of the mail. + +Genoa, Italy, 17^th^ September 2022 + +[^1]: Università Cattolica d.S.C., Dipartimento di scienze storiche e + filologiche, via Trieste 17, I-25121 Brescia diff --git a/support/texaccents/texaccents.pdf b/support/texaccents/texaccents.pdf index 6922882441..e41ab0abe0 100644 Binary files a/support/texaccents/texaccents.pdf and b/support/texaccents/texaccents.pdf differ diff --git a/support/texaccents/texaccents.sno b/support/texaccents/texaccents.sno old mode 100644 new mode 100755 index 5c20e75e47..41f655bad2 --- a/support/texaccents/texaccents.sno +++ b/support/texaccents/texaccents.sno @@ -1,6 +1,7 @@ +#!/usr/bin/env snobol4 # texaccents.sno -# version 1.0 -# 2022.08.23 +# version 1.0.1 +# 2022.08.23 -- 2022.09.17 # guido.milanese@unicatt.it # Transforms LaTeX accents to their UTF8 equivalents # accepts both LaTeX and Bibtex codes: @@ -32,15 +33,29 @@ initialise_bg -include "grepl.inc" ;* essential! calls repl internally MAX = 8000000 ;* max size of input/output files = 8 megs NL = newline() - HELP_MSG = "texaccents.sno 1.0" NL -+ " (c) 2022 Guido Milanese guido.milanese@unicatt.it" NL -+ " Usage: texaccents.sno INFILE OUTFILE" NL + HELP_MSG = "texaccents 1.0.1" NL ++ "Converts legacy (La)TeX accents and ligatures to Unicode" NL ++ "Usage: texaccents.sno INFILE OUTFILE" NL ++ "--help print this help, then exit" NL ++ "--version print version number, then exit" NL ++ "Report bugs to " NL ++ "CTAN page of the package: " + + VERS_MSG = "texaccents 1.0.1" NL NL ++ "Copyright (c) 2022 Guido Milanese" NL NL ++ "This is free software; see the source for copying conditions. There is NO" NL ++ "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." NL nl ++ "Written by Guido Milanese " + + ####################################################### # opens input / output files ####################################################### * Defines and checks Input & Output * help request -- exits. Both "--help" and "-help" are valid ((host(2,2) ? "-help") (Terminal = HELP_MSG)) :s(end) +* version request -- exits. Both "--version" and "-version" are valid + ((host(2,2) ? "-version") (Terminal = VERS_MSG)) :s(end) * Normal input-output with error messages. * Error messages are used by MAIN if necessary (~(infile = host(2,2)) ((Terminal = HELP_MSG) (ERROR_MSG = "Input error"))) :s(freturn) diff --git a/support/texaccents/texaccents.spt b/support/texaccents/texaccents.spt new file mode 100644 index 0000000000..33d435df83 --- /dev/null +++ b/support/texaccents/texaccents.spt @@ -0,0 +1,278 @@ +* WARNING - SPITBOL REQUIRES CR+LF LINE ENDINGS +* THE CTAN TEAM WAS INFORMED ABOUT THIS SPECIAL NEED +* +* texaccents.spt +* version 1.0.1 +* 2022.09.17 +* Windows version of texaccents.sno +* guido.milanese@unicatt.it +* Transforms LaTeX accents to their UTF8 equivalents +* accepts both LaTeX and Bibtex codes: +* examples: \'a \'{a} {\"a} and even {\'{a}} +* tables of accents imported from: +* https://github.com/hayk314/LaTex-handler -- Author: Hayk Aleksanyan +* with my integrations, including ligatures as \ae{} +* requires spitbol: https://github.com/spitbol/windows-nt + +***************************************************************** +* MIT License - Copyright (c) 2022 Guido Milanese +* See file LICENSE in this package +***************************************************************** + +****************************************************** +* FUNCTIONS +****************************************************** + +*-- Function INITIALISE +*-- opens files, imports functions + define("initialise()") + :(initialise_end) +initialise +initialise_bg +* INCLUDE FILES + +-include "args.inc" +-include "delete.inc" +-include "host.inc" +-include "grepl.inc" ;* essential! calls repl internally + MAX = 4000000 ;* max size of input/output files = 4 megs + NL = char(13) char(10) + + HELP_MSG = "texaccents 1.0.1" NL ++ "Converts legacy (La)TeX accents and ligatures to Unicode" NL ++ "Usage: texaccents.sno INFILE OUTFILE" NL ++ "--help print this help, then exit" NL ++ "--version print version number, then exit" NL ++ "Report bugs to " NL ++ "CTAN page of the package: " + + VERS_MSG = "texaccents 1.0.1" NL NL ++ "Copyright (c) 2022 Guido Milanese" NL NL ++ "This is free software; see the source for copying conditions. There is NO" NL ++ "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." NL nl ++ "Written by Guido Milanese " + +******************************************************* +* opens input / output files +******************************************************* +* Defines and checks Input & Output +* help request -- exits. Both "--help" and "-help" are valid + ((argv[1] ? "-help") (Terminal = HELP_MSG)) :s(end) +* version request -- exits. Both "--version" and "-version" are valid + ((argv[1] ? "-version") (Terminal = VERS_MSG)) :s(end) +* Normal input-output with error messages. +* Error messages are used by MAIN if necessary + (~(infile = argv[1]) ((Terminal = HELP_MSG) (ERROR_MSG = "Input error"))) :s(freturn) + (~(outfile = argv[2]) ((Terminal = HELP_MSG) (ERROR_MSG = "Output error"))) :s(freturn) + INPUT('r_f',1, infile '[-R' MAX ']') :f(freturn) + Terminal = "Reading text from " Infile + OUTPUT('w_f',2,Outfile '[-r' MAX ']') :f(freturn) +initialise_rt initialise = r_f :(return) +initialise_end + +************************************************************************************* +*-- Function NORMACC +*-- Changes TeX accents without curly brackets to the standard form +*-- \'a --> \'{a} +*-- and replaces ligatures such as \ae --> æ | \oe --> œ +*-- requires GREPL +************************************************************************************* + define("normacc(Pass)") + P_shortacc = notany('{') ++ ((('\' any("'`" '"' "^=~.")) . Pre_shortacc) ++ (any(&ucase &lcase) . Letter_shortacc)) . V_shortacc + :(normacc_end) +normacc +normacc_bg Pass ? P_shortacc :f(normacc_lg) + Pass ? V_shortacc = Pre_shortacc '{' Letter_shortacc '}' :(normacc_bg) +normacc_lg Pass2 = grepl(Pass, ++ "\ae{} \oe{} \AE{} \OE{} \dh{} \DH{} \th{} \TH{} \o{} \O{} \l{} \L{} ", ++ "æ œ Æ Œ ð Ð þ Þ ø Ø ł Ł ") + Terminal = "Ligatures tranformed" +normacc_rt normacc = Pass2 :(return) +normacc_end +* +* The pattern to normalise accents -- as \'a -- is defined as follows: +* 1. there must be no '{' before backslash: otherwise the pattern would trap also +* BibTex codes such as {\'a} +* 2. find a backslash followed by any char of the list. Saved as "Pre_sortacc" +* 3. find the accented letter. Saved as Letter_shortacc +* 4. The whole structure is saved as V_shortacc +* The programme will save the whole structure and substitute it in the text with: +* Pre_shortacc (as \') + '{' + Letter_shortacc + '}' +* in this way the short form -- \'a -- is transformed to the canonical \'{a} +* +* Ligatures and special chars are simply replaced with GREPL. + +************************************************************************************* +*-- Function CLEANACC +*-- Substitutes plain letters to Unicode accents, using the sets provides by TRANSACC +*-- \'{a} will output á +************************************************************************************* + define("cleanacc(Text,Acc1,Acc2)") :(cleanacc_end) +* Acc1 is the original LaTeX code e.g. \"{a} +* Acc2, at the time of calling the function, is e.g. \"{ä} +cleanacc +cleanacc_bg +* removes paretheses: \"{ä} --> \"ä + Acc2 = delete(Acc2,"{}") +* moves 2 chars and saves what remains, i.e. the letter +* \"ä --> ä + Acc2 ? len(2) (rem . Acc2) +* in text replaces e.g. \"{a} with ä +* the substitution is done ONCE for the whole text + Text = repl(Text,Acc1,Acc2) + Terminal = "Working on " Acc1 +cleanacc_rt cleanacc = Text :(return) +cleanacc_end + +******************************************************* +*-- Function TRANSACC +*-- transforms LaTeX/Bibtex accents to Unicode +******************************************************* + define("transacc(Pass)") +* Pass is the input text +* +* Accents used by Latex, e.g. \'{a} = acute accent over 'a' + Lat_acc = ++ any('"' "'" "`" 'H' '^' 'v' 'u' 'c' '.' 'd' 'k' '~' '=' 'b' 'r' ) ++ . V_lat_acc +* +* The pattern says: +* '\' + Lat_acc OR ('{\' + Lat_acc + '{' OR nothing) +* any letter, saved as C_noacc + '}' + '}' OR nothing +* Lat_acc is already defined above and saved ad V_lat_acc +* Examples \'{a} -- {\'a} -- {\'{a}} (\'a was already "normalised") + P_lat_code = ++ ( ++ (('\' Lat_acc '{') | ('{\' Lat_acc ('{' | '') )) ++ (any(&lcase &ucase) . C_noacc) '}' ('}' | '') ) . V_lat_code + :(transacc_end) +transacc +transacc_bg +latcode_bg V_lat_code = ;* cleans previous value + V_lat_code_2 = ;* cleans previous value +* Scans input text to locate the pattern +* if no other LaTeX accents, fails and returns to main +* if yes, goes to the section that transforms this accent. +* e.g. if the accent is umlaut, will jump to TR" +* After each transformation we have: +* 1. the original code, e.g. \"{a} or \"a or {\"a} +* 2. the transformed code, e.g. \"{ä} or \"ä or {\"ä} +* Each section calls CLEANCLODE +* CLEANCODE will remove all the {}\ and the LaTeX accent +* and substitute in the input text e.g. \"{ä} with ä + Pass ? P_lat_code :f(latcode_nd) + :($('TR' V_lat_acc)) + +TR" ;* umlaut + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c E e H h I i K k M m N n O o P p Q q S s T t U u V v W w X x Y y Z z ", ++ "Ä ä B̈ b̈ C̈ c̈ Ë ë Ḧ ḧ Ï ï K̈ k̈ M̈ m̈ N̈ n̈ Ö ö P̈ p̈ Q̈ q̈ S̈ s̈ T̈ ẗ Ü ü V̈ v̈ Ẅ ẅ Ẍ ẍ Ÿ ÿ Z̈ z̈ ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TR' ;* acute + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e F f G g H h I i J j K k L l M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ", ++ "Á á B́ b́ Ć ć D́ d́ É é F́ f́ Ǵ ǵ H́ h́ Í í J́ ȷ́ Ḱ ḱ Ĺ ĺ Ḿ ḿ Ń ń Ó ó Ṕ ṕ Q́ q́ Ŕ ŕ Ś ś T́ t́ Ú ú V́ v́ Ẃ ẃ X́ x́ Ý ý Ź ź ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TRH ;* double acute - \H{o} Hungarian + V_lat_code_2 = grepl(V_lat_code, ++ "A a E e I i M m O o U u ", ++ "A̋ a̋ E̋ e̋ I̋ i̋ M̋ m̋ Ő ő Ű ű ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TR` ;* grave + V_lat_code_2 = grepl(V_lat_code, ++ "A a Æ æ E e H h I i K k M m N n O o R r S s T t U u V v W w X x Y y Z z ", ++ "À à Æ̀ æ̀ È è H̀ h̀ Ì ì K̀ k̀ M̀ m̀ Ǹ ǹ Ò ò R̀ r̀ S̀ s̀ T̀ t̀ Ù ù V̀ v̀ Ẁ ẁ X̀ x̀ Ỳ ỳ Z̀ z̀ ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TR^ ;* circumflex \^{o} + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e G g H h I i J j K k L l M m N n O o R r S s T t U u V v W w X x Y y Z z ", ++ " â B̂ b̂ Ĉ ĉ D̂ d̂ Ê ê Ĝ ĝ Ĥ ĥ Î î Ĵ ĵ K̂ k̂ L̂ l̂ M̂ m̂ N̂ n̂ Ô ô R̂ r̂ Ŝ ŝ T̂ t̂ Û û V̂ v̂ Ŵ ŵ X̂ x̂ Ŷ ŷ Ẑ ẑ ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TRv ;* caron hraceck \v{s} +* For accents that are also normal letters +* we need to restore the accent to its original value +* This applies to: v u c d k b r + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e F f G g H h I i J j K k L l M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ", ++ "Ǎ ǎ B̌ b̌ Č č Ď ď Ě ě F̌ f̌ Ǧ ǧ Ȟ ȟ Ǐ ǐ J̌ ǰ Ǩ ǩ Ľ ľ M̌ m̌ Ň ň Ǒ ǒ P̌ p̌ Q̌ q̌ Ř ř Š š Ť ť Ǔ ǔ V̌ v̌ W̌ w̌ X̌ x̌ Y̌ y̌ Ž ž ") + V_lat_code_2 = repl(V_lat_code_2,"\v̌","\v") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TRu ;* breve \u{o} + V_lat_code_2 = grepl(V_lat_code, ++ "A a C c E e G g I i K k M m N n O o P p R r T t U u V v X x Y y ", ++ "Ă ă C̆ c̆ Ĕ ĕ Ğ ğ Ĭ ĭ K̆ k̆ M̆ m̆ N̆ n̆ Ŏ ŏ P̆ p̆ R̆ r̆ T̆ t̆ Ŭ ŭ V̆ v̆ X̆ x̆ Y̆ y̆ ") + V_lat_code_2 = repl(V_lat_code_2,"\ŭ","\u") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TRc ;* cedilla \c{c} + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e G g H h I i K k L l M m N n O o Q q R r S s T t U u X x Z z ", ++ "A̧ a̧ B̧ b̧ Ç ç Ḑ ḑ Ȩ ȩ Ģ ģ Ḩ ḩ I̧ i̧ Ķ ķ Ļ ļ M̧ m̧ Ņ ņ O̧ o̧ Q̧ q̧ Ŗ ŗ Ş ş Ţ ţ U̧ u̧ X̧ x̧ Z̧ z̧ ") + V_lat_code_2 = repl(V_lat_code_2,"\ç","\c") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TR. ;* dot \.{o} + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e F f G g H h I i K k L l M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ", ++ "Ȧ ȧ Ḃ ḃ Ċ ċ Ḋ ḋ Ė ė Ḟ ḟ Ġ ġ Ḣ ḣ İ i̇̀ K̇ k̇ L̇ l̇ Ṁ ṁ Ṅ ṅ Ȯ ȯ Ṗ ṗ Q̇ q̇ Ṙ ṙ Ṡ ṡ Ṫ ṫ U̇ u̇ V̇ v̇ Ẇ ẇ Ẋ ẋ Ẏ ẏ Ż ż ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TRd ;* dot under the letter \d{u} + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e F f G g H h I i J j K k L l M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ", ++ "Ạ ạ Ḅ ḅ C̣ c̣ Ḍ ḍ Ẹ ẹ F̣ f̣ G̣ g̣ Ḥ ḥ Ị ị J̣ j̣ Ḳ ḳ Ḷ ḷ Ṃ ṃ Ṇ ṇ Ọ ọ P̣ p̣ Q̣ q̣ Ṛ ṛ Ṣ ṣ Ṭ ṭ Ụ ụ Ṿ ṿ Ẉ ẉ X̣ x̣ Ỵ ỵ Ẓ ẓ ") + V_lat_code_2 = repl(V_lat_code_2,"\ḍ","\d") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TRk ;* ogonek \k{a} + V_lat_code_2 = grepl(V_lat_code, ++ "A a E e I i O o U u Y y ", ++ "Ą ą Ę ę Į į Ǫ ǫ Ų ų Y̨ y̨ ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TR~ ;* tilde \~o + V_lat_code_2 = grepl(V_lat_code, ++ "A a E e I i N n O o U u V v Y y ", ++ "à ã Ẽ ẽ Ĩ ĩ Ñ ñ Õ õ Ũ ũ Ṽ ṽ Ỹ ỹ ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TR= ;* macron \=a \={a} + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e G g I i J j K k M m N n O o P p Q q R r S s T t U u V v W w X x Y y Z z ", ++ "Ā ā B̄ b̄ C̄ c̄ D̄ d̄ Ē ē Ḡ ḡ Ī ī J̄ j̄ K̄ k̄ M̄ m̄ N̄ n̄ Ō ō P̄ p̄ Q̄ q̄ R̄ r̄ S̄ s̄ T̄ t̄ Ū ū V̄ v̄ W̄ w̄ X̄ x̄ Ȳ ȳ Z̄ z̄ ") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TRb ;* bar under the letter \b{a} + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e G g H h I i J j K k L l M m N n O o P p R r S s T t U u X x Y y Z z ", ++ "A̱ a̱ Ḇ ḇ C̱ c̱ Ḏ ḏ E̱ e̱ G̱ g̱ H̱ ẖ I̱ i̱ J̱ j̱ Ḵ ḵ Ḻ ḻ M̱ m̱ Ṉ ṉ O̱ o̱ P̱ p̱ Ṟ ṟ S̱ s̱ Ṯ ṯ U̱ u̱ X̱ x̱ Y̱ y̱ Ẕ ẕ ") + V_lat_code_2 = repl(V_lat_code_2,"\ḇ","\b") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +TRr ;* ring over the letter \r{a} + V_lat_code_2 = grepl(V_lat_code, ++ "A a B b C c D d E e F f G g H h I i J j K k L l M m N n O o P p Q q r R S s U u V v W w X x Y y Z z ", ++ "Å å B̊ b̊ C̊ c̊ D̊ d̊ E̊ e̊ F̊ f̊ G̊ g̊ H̊ h̊ I̊ i̊ J̊ j̊ K̊ k̊ L̊ l̊ M̊ m̊ N̊ n̊ O̊ o̊ P̊ p̊ Q̊ q̊ r̊ R̊ S̊ s̊ Ů ů V̊ v̊ W̊ ẘ X̊ x̊ Y̊ ẙ Z̊ z̊ ") + V_lat_code_2 = repl(V_lat_code_2,"\r̊","\r") + Pass = cleanacc(Pass,V_lat_code,V_lat_code_2) :(latcode_bg) + +latcode_nd + transacc = Pass :(return) +transacc_end + +******************************************************* +* MAIN +******************************************************* + (~(texfile = initialise()) (terminal = "Could not start programme - " ERROR_MSG)) :s(end) + Texfile = normacc(Texfile) + (((w_f = transacc(texfile)) (Terminal = "Done. File " Outfile " written")), ++ (Terminal = "Conversion failed")) +END diff --git a/support/texaccents/texaccents.tex b/support/texaccents/texaccents.tex new file mode 100644 index 0000000000..ec63276852 --- /dev/null +++ b/support/texaccents/texaccents.tex @@ -0,0 +1,258 @@ +% TEXACCENTS.TEX +% Documentation for texaccents.sno / texaccents.spt +% version 1.0.1 -- 17th september 2022 +% author: guido.milanese@unicatt.it +% tex file produced from markdown source +% using `pandoc texaccents.md -s -o texaccents.tex` +% and compiled with lualatex +% Line 35 was added: \microtypesetup{nopatch=item} +% See: https://github.com/schlcht/microtype/issues/4 +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% +% Options for packages loaded elsewhere +\PassOptionsToPackage{unicode}{hyperref} +\PassOptionsToPackage{hyphens}{url} +% +\documentclass[ + 12pt, + english, +]{article} +\usepackage{amsmath,amssymb} +\usepackage[]{libertine} +\usepackage{iftex} +\ifPDFTeX + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} + \usepackage{textcomp} % provide euro and other symbols +\else % if luatex or xetex + \usepackage{unicode-math} + \defaultfontfeatures{Scale=MatchLowercase} + \defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1} +\fi +% Use upquote if available, for straight quotes in verbatim environments +\IfFileExists{upquote.sty}{\usepackage{upquote}}{} +\IfFileExists{microtype.sty}{% use microtype if available + \usepackage[]{microtype} + \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts + \microtypesetup{nopatch=item} +}{} +\makeatletter +\@ifundefined{KOMAClassName}{% if non-KOMA class + \IfFileExists{parskip.sty}{% + \usepackage{parskip} + }{% else + \setlength{\parindent}{0pt} + \setlength{\parskip}{6pt plus 2pt minus 1pt}} +}{% if KOMA class + \KOMAoptions{parskip=half}} +\makeatother +\usepackage{xcolor} +\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available +\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}} +\hypersetup{ + pdftitle={TeXaccents version 1.0.1}, + pdfauthor={Guido Milanese}, + pdflang={en}, + hidelinks, + pdfcreator={LaTeX via pandoc}} +\urlstyle{same} % disable monospaced font for URLs +\setlength{\emergencystretch}{3em} % prevent overfull lines +\providecommand{\tightlist}{% + \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}} +\setcounter{secnumdepth}{-\maxdimen} % remove section numbering +\ifXeTeX + % Load polyglossia as late as possible: uses bidi with RTL langages (e.g. Hebrew, Arabic) + \usepackage{polyglossia} + \setmainlanguage[]{english} +\else + \usepackage[main=english]{babel} +% get rid of language-specific shorthands (see #6817): +\let\LanguageShortHands\languageshorthands +\def\languageshorthands#1{} +\fi +\ifLuaTeX + \usepackage{selnolig} % disable illegal ligatures +\fi + +\title{TeXaccents\\ +version 1.0.1} +\author{Guido Milanese\footnote{Università Cattolica d.S.C., + Dipartimento di scienze storiche e filologiche, via Trieste 17, + I-25121 Brescia}} +\date{17\textsuperscript{th} September 2022} + +\begin{document} +\maketitle +\begin{abstract} +TeXaccents is a standalone utility designed to convert legacy (La)TeX +ligatures and codes for ``accented'' characters to Unicode equivalents +(text mode, no math) . For example, \texttt{\textbackslash{}=\{a\}} (`a' +with macron) will be converted to \texttt{ā}. +\end{abstract} + +\hypertarget{general-information}{% +\section{General information}\label{general-information}} + +Even if modern compilers handle Unicode encoding, (La) and files +featuring ``legacy'' encoding for non-Ascii characters are still very +common, and users may need to incorporate old code into new texts that +make use of modern text encoding. + +Several utilities are available online that claim to be able to convert +legacy (La) encoding to standard Unicode. See: + +\begin{itemize} +\item + \emph{Simple LaTeX to Text Converter}. A complex programme, able to + deal with maths. Insofar as non-Ascii chars are concerned, it fails + sometimes, at least according to my tests. See + \url{https://pylatexenc.readthedocs.io/en/latest/latexwalker/}. + Written in Python. +\item + \emph{LaTeX handler}. Converts non-Ascii (La) encoding to Unicode. + However, it does not seem to be able to deal with the legacy encoding, + e.g.~\texttt{\{\textbackslash{}a\}} instead of + \texttt{\textbackslash{}\{a\}} or \texttt{\textbackslash{}a}. It does + not convert simple ligatures as \texttt{\textbackslash{}ae\{\}} + \texttt{\textbackslash{}oe\{\}}. I used the tables provided by this + programme as a starting point. Written in Python. See + \url{https://github.com/hayk314/LaTex-handler}. +\item + \emph{Pandoc} is the standard programme for any text format conversion + (\url{https://pandoc.org/}). It converts almost all the accents (thorn + and eth missing?), but (if I have checked this correctly) normalises + files stripping non-standard fields. This can be a problem for + scholars who frequently use non-standard fields, such as e.g. + ``shorttitle'', required by not a few bibliographic styles. +\end{itemize} + +\emph{TeXaccents} should be able to transform (La) normal text or +``accents'' (not ``math'' accents) to their Unicode equivalent. The +programme deals with the following codes (\emph{not all the fonts are +able to output all the required Unicode glyphs of this table!}): + +\begin{verbatim} +| NAME | \tex | Unicode | +|--------------- |------- |---------| +| Umlaut | \"{a} | ä | +| acute | \'{a} | á | +| double acute | \H{a} | a̋ | +| grave | \`{a} | à | +| circumflex | \^{a} | â | +| caron hraceck | \v{a} | ǎ | +| breve | \u{a} | ă | +| cedilla | \c{c} | ç | +| dot | \.{a} | ȧ | +| dot under | \d{a} | ạ | +| ogonek | \k{a} | ą | +| tilde | \~{a} | ã | +| macron | \={a} | ā | +| bar under | \b{a} | a̱ | +| ring over | \r{a} | å | +\end{verbatim} + +The programme should recognize the following varieties: + +\texttt{\textbackslash{}\textquotesingle{}a} -- +\texttt{\textbackslash{}\textquotesingle{}\{a\}} -- +\texttt{\{\textbackslash{}\textquotesingle{}a\}} -- +\texttt{\{\{\textbackslash{}\textquotesingle{}a\}\}} + +It transforms also the encoding for : \texttt{æ~œ~Æ~Œ~ð~Ð~þ~Þ~ø~Ø~ł~Ł}. +Checking the page +\url{https://www.utf8-chartable.de/unicode-utf8-table.pl?number=512} I +could not find a legacy text mode encoding for: +\textbf{ƀ~Ƀ~đ~Đ~ǥ~Ǥ~ħ~Ħ~ɨ~Ɨ~ŧ~Ŧ~ƶ~Ƶ} (some chars are accessible in math +mode). + +\hypertarget{setup}{% +\section{Setup}\label{setup}} + +\hypertarget{from-source}{% +\subsection{From source}\label{from-source}} + +The programme is written in Snobol +(\url{https://en.wikipedia.org/wiki/SNOBOL} or +\url{https://it.wikipedia.org/wiki/SNOBOL}) and should run on any +platform. Steps: + +\begin{enumerate} +\def\labelenumi{\arabic{enumi}.} +\item + Install Snobol4 (version 2.3, March 2022) from + \url{http://www.regressive.org/snobol4/csnobol4/curr/}. Make sure to + install the compiler in a folder listed in your \texttt{PATH} or add + the folder to your path. On Linux the folder \texttt{snobol4} is + installed under \texttt{/usr/local/bin/}, which is normally listed in + the PATH of a standard Linux system. +\item + Test the compiler running \texttt{snobol4} from the command line. + Leave the compiler with \texttt{Ctr-C} or writing \texttt{end}. +\item + Copy \texttt{texaccents.sno} and all the provided \texttt{*.inc} files +\end{enumerate} + +\begin{quote} +\texttt{compiler.inc}~\texttt{delete.inc}~\texttt{grepl.inc}~\texttt{newline.inc}~\texttt{systype.inc} +\end{quote} + +to a folder of your choice +(e.g.~\texttt{/home/\textless{}user\textgreater{}/bin}). + +\begin{enumerate} +\def\labelenumi{\arabic{enumi}.} +\setcounter{enumi}{3} +\tightlist +\item + In this folder, run + \texttt{snobol4\ texaccents.sno\ testaccents-in\ testaccents-out} to + test the programme. The test file contains all the accents listed + above. See the result typing \texttt{cat~testaccents-out} (Unixes / + Powershell) or \texttt{type~testaccents-out} (Windows/Dos prompt), or + open the file with your text editor. The output file name is just a + suggestion, of course. +\end{enumerate} + +\hypertarget{windows-standalone-version}{% +\subsection{Windows standalone +version}\label{windows-standalone-version}} + +If preferred, a Windows EXE standalone file is provided. It was compiled +using Spitbol (see \url{https://github.com/spitbol/windows-nt}); the +source code has been slightly adapted to Spitbol (basically only +input/output syntax). From any directory, run +\texttt{texaccents.exe\ INPUT\ OUTPUT}. To test the programme, run +\texttt{texaccents.exe\ testaccents-in\ testaccents-out}. As above, the +output file name is just a suggestion. + +\hypertarget{history}{% +\section{History}\label{history}} + +\begin{itemize} +\tightlist +\item + 25\textsuperscript{th} July 2022. First version (after trying + unsuccesfully to convert an old file with existing utilities) +\item + 17\textsuperscript{th} August 2022. First complete version (0.9). +\item + 27\textsuperscript{rd} August 2022. This version (1.0) with + documentation and comments. +\item + 17\textsuperscript{th} September 2022. Windows standalone executable. + Manual page written. Version message added; help message improved. In + the source, a regular shebang according to the recommendation of CTAN + (\url{https://tug.org/texlive/pkgcontrib.html}) was added. + Documentation updated accordingly. +\end{itemize} + +\hypertarget{contacts-todo}{% +\section{Contacts / todo}\label{contacts-todo}} + +Bugs / suggestions / improvements: please write to +\url{guido.milanese@unicatt.it} using \emph{TeXaccents} as subject of +the mail. + +Genoa, Italy, 17\textsuperscript{th} September 2022 + +\end{document} -- cgit v1.2.3