summaryrefslogtreecommitdiff
path: root/Master/texmf-dist/source/generic/hyph-utf8
diff options
context:
space:
mode:
Diffstat (limited to 'Master/texmf-dist/source/generic/hyph-utf8')
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/README75
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/data/encodings/ec.dat257
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/data/encodings/il3.dat6
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/data/encodings/lmc.dat70
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/data/encodings/qx.dat256
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/data/encodings/t2a.dat256
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/data/encodings/texnansi.dat256
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/generate-converters.rb117
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/generate-pattern-loaders.rb112
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/generate-tl-files.rb129
-rw-r--r--Master/texmf-dist/source/generic/hyph-utf8/languages.rb775
11 files changed, 2309 insertions, 0 deletions
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/README b/Master/texmf-dist/source/generic/hyph-utf8/README
new file mode 100644
index 00000000000..4b753902921
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/README
@@ -0,0 +1,75 @@
+This is HYPH-UTF8 package which contains:
+
+- hyphenation patterns hyph-foo.tex, semi-manually converted into UTF-8
+ from different sources by different authors
+- pattern loaders loadhyph-foo.tex that recognise the engine
+ and load pattern accordingly
+- converters from UTF-8 into EC/T1, QX, part of ISO Latin 3, T2A & LMC
+- some scripts to assist with auto-generating the files
+
+Modificatios were needed in order to support native UTF-8 engines,
+but functionality (hopefully) didn't change in any way, at least not intentionally.
+The patterns themselves are no longer stand-alone; at least for 8-bit engines
+you need to use loadhyph-foo.tex (which will load the patterns) instead.
+
+Modifications were done by Jonathan Kew, Mojca Miklavec & Arthur Reutenauer
+with help & support from:
+
+- Karl Berry, who gave us free hands and all resources
+- Taco Hoekwater, with useful macros
+- Hans Hagen, who did the unicodifisation of patterns already long before
+ and helped with testing, suggestions and bug reports
+- Norbert Preining, who tested & integrated patterns into TeX Live
+
+However, the "copyright/copyleft" owner of patterns remains the original author.
+
+The original patterns should eventually become obsolete,
+and the author of the original file should preferraby modify
+these files instead (feel free to remove credits added by TUG from those files).
+
+If you want to change the patterns, rather than uploading directly to CTAN,
+we would be grateful if you could send it to us (http://tug.org/tex-hyphen)
+or ask for credentials for SVN repository and commit it yourself;
+we will then upload the whole "package" to CTAN.
+
+Before a new "pattern-revolution" starts,
+please try to follow some guidelines if possible:
+
+- \lccode is *forbidden*, and I really mean it
+- all the patterns should be in UTF-8
+- the only "allowed" TeX commands in patterns are: \patterns, \hyphenation,
+ and if you really cannot do without, also \input and \message
+- in particular, please no \catcode or \lccode changes,
+ they belong to loadhyph-foo.tex,
+ and no \lefthyphenmin and \righthyphenmin,
+ they have no influence here and belong elsewhere
+- \begingroup and/or \endinput is not needed
+- feel free to do whatever you want inside comments
+
+We know that TeX is extremely powerful, but give a stupid parser
+at least a chance to read your patterns.
+
+For more information see
+
+ http://tug.org/tex-hyphen
+
+
+Copyright 2008 by TUG
+
+The licence for hyph-foo.tex: Respect licences of original authors.
+The licence for the rest: Use common sense when modifying and coordinate your work.
+
+Hyphenation patterns are essential to TeX;
+whatever changes you make to these files -
+make sure to coordinate your work with everyone involved.
+(TeX Live, MikTeX, CTAN, patter authors, mailing list, ...)
+
+People come and go, TeX evolves. We are aware of that.
+If we disapear one day and there is a need to change files,
+do whatever needs to be done to make TeX behave in a proper
+(backward compatible) way. And keep things clean & consistent.
+
+Things that might change in near future:
+- support for multiple versions of patterns
+- support for runtime loading of patterns
+- support for preferred hyphenation points and discretionaries (ck -> k-k)
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/ec.dat b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/ec.dat
new file mode 100644
index 00000000000..905d5b1d040
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/ec.dat
@@ -0,0 +1,257 @@
+0x00 U+0060 grave
+0x01 U+00B4 acute
+0x02 U+02C6 circumflex
+0x03 U+02DC tilde
+0x04 U+00A8 dieresis
+0x05 U+02DD hungarumlaut
+0x06 U+02DA ring
+0x07 U+02C7 caron
+0x08 U+02D8 breve
+0x09 U+00AF macron
+0x0A U+02D9 dotaccent
+0x0B U+00B8 cedilla
+0x0C U+02DB ogonek
+0x0D U+201A quotesinglbase
+0x0E U+2039 guilsinglleft
+0x0F U+203A guilsinglright
+0x10 U+201C quotedblleft
+0x11 U+201D quotedblright
+0x12 U+201E quotedblbase
+0x13 U+00AB guillemotleft
+0x14 U+00BB guillemotright
+0x15 U+2013 endash
+0x16 U+2014 emdash
+0x17 U+200B cwm
+0x18 U+.... perthousandzero
+0x19 U+0131 1 dotlessi
+0x1A U+0237 1 dotlessj
+0x1B U+FB00 ff
+0x1C U+FB01 fi
+0x1D U+FB02 fl
+0x1E U+FB03 ffi
+0x1F U+FB04 ffl
+0x20 U+2423 visiblespace
+0x21 U+0021 exclam
+0x22 U+0022 quotedbl
+0x23 U+0023 numbersign
+0x24 U+0024 dollar
+0x25 U+0025 percent
+0x26 U+0026 ampersand
+0x27 U+2019 quoteright
+0x28 U+0028 parenleft
+0x29 U+0029 parenright
+0x2A U+002A asterisk
+0x2B U+002B plus
+0x2C U+002C comma
+0x2D U+002D hyphen
+0x2E U+002E period
+0x2F U+002F slash
+0x30 U+0030 zero
+0x31 U+0031 one
+0x32 U+0032 two
+0x33 U+0033 three
+0x34 U+0034 four
+0x35 U+0035 five
+0x36 U+0036 six
+0x37 U+0037 seven
+0x38 U+0038 eight
+0x39 U+0039 nine
+0x3A U+003A colon
+0x3B U+003B semicolon
+0x3C U+003C less
+0x3D U+003D equal
+0x3E U+003E greater
+0x3F U+003F question
+0x40 U+0040 at
+0x41 U+0041 A
+0x42 U+0042 B
+0x43 U+0043 C
+0x44 U+0044 D
+0x45 U+0045 E
+0x46 U+0046 F
+0x47 U+0047 G
+0x48 U+0048 H
+0x49 U+0049 I
+0x4A U+004A J
+0x4B U+004B K
+0x4C U+004C L
+0x4D U+004D M
+0x4E U+004E N
+0x4F U+004F O
+0x50 U+0050 P
+0x51 U+0051 Q
+0x52 U+0052 R
+0x53 U+0053 S
+0x54 U+0054 T
+0x55 U+0055 U
+0x56 U+0056 V
+0x57 U+0057 W
+0x58 U+0058 X
+0x59 U+0059 Y
+0x5A U+005A Z
+0x5B U+005B bracketleft
+0x5C U+005C backslash
+0x5D U+005D bracketright
+0x5E U+005E asciicircum
+0x5F U+005F underscore
+0x60 U+2018 quoteleft
+0x61 U+0061 a
+0x62 U+0062 b
+0x63 U+0063 c
+0x64 U+0064 d
+0x65 U+0065 e
+0x66 U+0066 f
+0x67 U+0067 g
+0x68 U+0068 h
+0x69 U+0069 i
+0x6A U+006A j
+0x6B U+006B k
+0x6C U+006C l
+0x6D U+006D m
+0x6E U+006E n
+0x6F U+006F o
+0x70 U+0070 p
+0x71 U+0071 q
+0x72 U+0072 r
+0x73 U+0073 s
+0x74 U+0074 t
+0x75 U+0075 u
+0x76 U+0076 v
+0x77 U+0077 w
+0x78 U+0078 x
+0x79 U+0079 y
+0x7A U+007A z
+0x7B U+007B braceleft
+0x7C U+007C bar
+0x7D U+007D braceright
+0x7E U+007E asciitilde
+0x7F U+002D hyphenchar
+0x80 U+0102 Abreve
+0x81 U+0104 Aogonek
+0x82 U+0106 Cacute
+0x83 U+010C Ccaron
+0x84 U+010E Dcaron
+0x85 U+011A Ecaron
+0x86 U+0118 Eogonek
+0x87 U+011E Gbreve
+0x88 U+0139 Lacute
+0x89 U+013D Lcaron
+0x8A U+0141 Lslash
+0x8B U+0143 Nacute
+0x8C U+0147 Ncaron
+0x8D U+014A Eng
+0x8E U+0150 Ohungarumlaut
+0x8F U+0154 Racute
+0x90 U+0158 Rcaron
+0x91 U+015A Sacute
+0x92 U+0160 Scaron
+0x93 U+015E Scedilla
+0x94 U+0164 Tcaron
+0x95 U+0162 Tcedilla
+0x96 U+0170 Uhungarumlaut
+0x97 U+016E Uring
+0x98 U+0178 Ydieresis
+0x99 U+0179 Zacute
+0x9A U+017D Zcaron
+0x9B U+017B Zdotaccent
+0x9C U+0132 IJ
+0x9D U+0130 Idotaccent
+0x9E U+0111 1 dcroat
+0x9F U+00A7 section
+0xA0 U+0103 1 abreve
+0xA1 U+0105 1 aogonek
+0xA2 U+0107 1 cacute
+0xA3 U+010D 1 ccaron
+0xA4 U+010F 1 dcaron
+0xA5 U+011B 1 ecaron
+0xA6 U+0119 1 eogonek
+0xA7 U+011F 1 gbreve
+0xA8 U+013A 1 lacute
+0xA9 U+013E 1 lcaron
+0xAA U+0142 1 lslash
+0xAB U+0144 1 nacute
+0xAC U+0148 1 ncaron
+0xAD U+014B 1 eng
+0xAE U+0151 1 ohungarumlaut
+0xAF U+0155 1 racute
+0xB0 U+0159 1 rcaron
+0xB1 U+015B 1 sacute
+0xB2 U+0161 1 scaron
+0xB3 U+015F 1 scedilla
+0xB3 U+0219 1 scommaaccent
+0xB4 U+0165 1 tcaron
+0xB5 U+021B 1 tcommaaccent
+0xB6 U+0171 1 uhungarumlaut
+0xB7 U+016F 1 uring
+0xB8 U+00FF 1 ydieresis
+0xB9 U+017A 1 zacute
+0xBA U+017E 1 zcaron
+0xBB U+017C 1 zdotaccent
+0xBC U+0133 1 ij
+0xBD U+00A1 exclamdown
+0xBE U+00BF questiondown
+0xBF U+00A3 sterling
+0xC0 U+00C0 Agrave
+0xC1 U+00C1 Aacute
+0xC2 U+00C2 Acircumflex
+0xC3 U+00C3 Atilde
+0xC4 U+00C4 Adieresis
+0xC5 U+00C5 Aring
+0xC6 U+00C6 AE
+0xC7 U+00C7 Ccedilla
+0xC8 U+00C8 Egrave
+0xC9 U+00C9 Eacute
+0xCA U+00CA Ecircumflex
+0xCB U+00CB Edieresis
+0xCC U+00CC Igrave
+0xCD U+00CD Iacute
+0xCE U+00CE Icircumflex
+0xCF U+00CF Idieresis
+0xD0 U+00D0 Eth
+0xD1 U+00D1 Ntilde
+0xD2 U+00D2 Ograve
+0xD3 U+00D3 Oacute
+0xD4 U+00D4 Ocircumflex
+0xD5 U+00D5 Otilde
+0xD6 U+00D6 Odieresis
+0xD7 U+0152 OE
+0xD8 U+00D8 Oslash
+0xD9 U+00D9 Ugrave
+0xDA U+00DA Uacute
+0xDB U+00DB Ucircumflex
+0xDC U+00DC Udieresis
+0xDD U+00DD Yacute
+0xDE U+00DE Thorn
+0xDF U+1E9E Germandbls
+0xE0 U+00E0 1 agrave
+0xE1 U+00E1 1 aacute
+0xE2 U+00E2 1 acircumflex
+0xE3 U+00E3 1 atilde
+0xE4 U+00E4 1 adieresis
+0xE5 U+00E5 1 aring
+0xE6 U+00E6 1 ae
+0xE7 U+00E7 1 ccedilla
+0xE8 U+00E8 1 egrave
+0xE9 U+00E9 1 eacute
+0xEA U+00EA 1 ecircumflex
+0xEB U+00EB 1 edieresis
+0xEC U+00EC 1 igrave
+0xED U+00ED 1 iacute
+0xEE U+00EE 1 icircumflex
+0xEF U+00EF 1 idieresis
+0xF0 U+00F0 1 eth
+0xF1 U+00F1 1 ntilde
+0xF2 U+00F2 1 ograve
+0xF3 U+00F3 1 oacute
+0xF4 U+00F4 1 ocircumflex
+0xF5 U+00F5 1 otilde
+0xF6 U+00F6 1 odieresis
+0xF7 U+0153 1 oe
+0xF8 U+00F8 1 oslash
+0xF9 U+00F9 1 ugrave
+0xFA U+00FA 1 uacute
+0xFB U+00FB 1 ucircumflex
+0xFC U+00FC 1 udieresis
+0xFD U+00FD 1 yacute
+0xFE U+00FE 1 thorn
+0xFF U+00DF 1 germandbls
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/il3.dat b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/il3.dat
new file mode 100644
index 00000000000..7e7d856a192
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/il3.dat
@@ -0,0 +1,6 @@
+0xB6 U+0125 1 hcircumflex
+0xBC U+0135 1 jcircumflex
+0xE6 U+0109 1 ccircumflex
+0xF8 U+011D 1 gcircumflex
+0xFD U+016D 1 ubreve
+0xFE U+015D 1 scircumflex
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/lmc.dat b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/lmc.dat
new file mode 100644
index 00000000000..2fefcb77c30
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/lmc.dat
@@ -0,0 +1,70 @@
+0x41 U+0410 CYRA
+0x42 U+0411 CYRB
+0x43 U+0426 CYRC
+0x44 U+0414 CYRD
+0x45 U+0415 CYRE
+0x46 U+0424 CYRF
+0x47 U+0413 CYRG
+0x49 U+0418 CYRI
+0x4A U+0416 CYRZH
+0x4B U+041A CYRK
+0x4C U+041B CYRL
+0x4D U+041C CYRM
+0x4E U+041D CYRN
+0x4F U+041E CYRO
+0x50 U+041F CYRP
+0x51 U+0427 CYRCH
+0x52 U+0420 CYRR
+0x53 U+0421 CYRS
+0x54 U+0422 CYRT
+0x55 U+0423 CYRU
+0x57 U+0412 CYRV
+0x58 U+0425 CYRH
+0x59 U+042B CYRERY
+0x5A U+0417 CYRZ
+0x61 U+0430 1 cyra
+0x62 U+0431 1 cyrb
+0x63 U+0446 1 cyrc
+0x64 U+0434 1 cyrd
+0x65 U+0435 1 cyre
+0x66 U+0444 1 cyrf
+0x67 U+0433 1 cyrg
+0x69 U+0438 1 cyri
+0x6A U+0436 1 cyrzh
+0x6B U+043A 1 cyrk
+0x6C U+043B 1 cyrl
+0x6D U+043C 1 cyrm
+0x6E U+043D 1 cyrn
+0x6F U+043E 1 cyro
+0x70 U+043F 1 cyrp
+0x71 U+0447 1 cyrch
+0x72 U+0440 1 cyrr
+0x73 U+0441 1 cyrs
+0x74 U+0442 1 cyrt
+0x75 U+0443 1 cyru
+0x77 U+0432 1 cyrv
+0x78 U+0445 1 cyrh
+0x79 U+044B 1 cyrery
+0x7A U+0437 1 cyrz
+0x90 U+0429 CYRSHCH
+0x92 U+0428 CYRSH
+0x93 U+042A CYRHRDSN
+0x94 U+042C CYRSFTSN
+0x95 U+042E CYRYU
+0x96 U+042F CYRYA
+0xB0 U+0449 1 cyrshch
+0xB2 U+0448 1 cyrsh
+0xB3 U+044A 1 cyrhrdsn
+0xB4 U+0131 1 i
+0xB5 U+044E 1 cyryu
+0xB6 U+044F 1 cyrya
+0xC4 U+042D CYREREV
+0xCB U+0401 CYRYO
+0xCF U+0419 CYRISHRT
+0xD6 U+04E8 CYROTLD
+0xDC U+04AE CYRY
+0xE4 U+044D 1 cyrerev
+0xEB U+0451 1 cyryo
+0xEF U+0439 1 cyrishrt
+0xF6 U+04E9 1 cyrotld
+0xFC U+04AF 1 cyry
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/qx.dat b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/qx.dat
new file mode 100644
index 00000000000..57abcac1772
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/qx.dat
@@ -0,0 +1,256 @@
+0x00 U+.... .notdef
+0x01 U+2206 Delta
+0x02 U+.... .notdef
+0x03 U+.... .notdef
+0x04 U+.... .notdef
+0x05 U+03A0 Pi
+0x06 U+03A3 Sigma
+0x07 U+00B5 mu
+0x08 U+2026 ellipsis
+0x09 U+.... f_k
+0x0A U+2126 Omega
+0x0B U+FB00 ff
+0x0C U+FB01 fi
+0x0D U+FB02 fl
+0x0E U+FB03 ffi
+0x0F U+FB04 ffl
+0x10 U+0131 1 dotlessi
+0x11 U+0237 1 dotlessj
+0x12 U+0060 grave
+0x13 U+00B4 acute
+0x14 U+02C7 caron
+0x15 U+02D8 breve
+0x16 U+00AF macron
+0x17 U+02DA ring
+0x18 U+00B8 cedilla
+0x19 U+00DF 1 germandbls
+0x1A U+00E6 1 ae
+0x1B U+0153 1 oe
+0x1C U+00F8 1 oslash
+0x1D U+00C6 AE
+0x1E U+0152 OE
+0x1F U+00D8 Oslash
+0x20 U+0020 space
+0x21 U+0021 exclam
+0x22 U+201D quotedblright
+0x23 U+0023 numbersign
+0x24 U+0024 dollar
+0x25 U+0025 percent
+0x26 U+0026 ampersand
+0x27 U+2019 quoteright
+0x28 U+0028 parenleft
+0x29 U+0029 parenright
+0x2A U+002A asterisk
+0x2B U+002B plus
+0x2C U+002C comma
+0x2D U+002D hyphen
+0x2E U+002E period
+0x2F U+002F slash
+0x30 U+0030 zero
+0x31 U+0031 one
+0x32 U+0032 two
+0x33 U+0033 three
+0x34 U+0034 four
+0x35 U+0035 five
+0x36 U+0036 six
+0x37 U+0037 seven
+0x38 U+0038 eight
+0x39 U+0039 nine
+0x3A U+003A colon
+0x3B U+003B semicolon
+0x3C U+00A1 exclamdown
+0x3D U+003D equal
+0x3E U+00BF questiondown
+0x3F U+003F question
+0x40 U+0040 at
+0x41 U+0041 A
+0x42 U+0042 B
+0x43 U+0043 C
+0x44 U+0044 D
+0x45 U+0045 E
+0x46 U+0046 F
+0x47 U+0047 G
+0x48 U+0048 H
+0x49 U+0049 I
+0x4A U+004A J
+0x4B U+004B K
+0x4C U+004C L
+0x4D U+004D M
+0x4E U+004E N
+0x4F U+004F O
+0x50 U+0050 P
+0x51 U+0051 Q
+0x52 U+0052 R
+0x53 U+0053 S
+0x54 U+0054 T
+0x55 U+0055 U
+0x56 U+0056 V
+0x57 U+0057 W
+0x58 U+0058 X
+0x59 U+0059 Y
+0x5A U+005A Z
+0x5B U+005B bracketleft
+0x5C U+201C quotedblleft
+0x5D U+005D bracketright
+0x5E U+02C6 circumflex
+0x5F U+02D9 dotaccent
+0x60 U+2018 quoteleft
+0x61 U+0061 a
+0x62 U+0062 b
+0x63 U+0063 c
+0x64 U+0064 d
+0x65 U+0065 e
+0x66 U+0066 f
+0x67 U+0067 g
+0x68 U+0068 h
+0x69 U+0069 i
+0x6A U+006A j
+0x6B U+006B k
+0x6C U+006C l
+0x6D U+006D m
+0x6E U+006E n
+0x6F U+006F o
+0x70 U+0070 p
+0x71 U+0071 q
+0x72 U+0072 r
+0x73 U+0073 s
+0x74 U+0074 t
+0x75 U+0075 u
+0x76 U+0076 v
+0x77 U+0077 w
+0x78 U+0078 x
+0x79 U+0079 y
+0x7A U+007A z
+0x7B U+2013 endash
+0x7C U+2014 emdash
+0x7D U+02DD hungarumlaut
+0x7E U+02DC tilde
+0x7F U+00A8 dieresis
+0x80 U+20AC Euro
+0x81 U+0104 Aogonek
+0x82 U+0106 Cacute
+0x83 U+003E greater
+0x84 U+.... .notdef
+0x85 U+.... .notdef
+0x86 U+0118 Eogonek
+0x87 U+012E Iogonek
+0x88 U+003C less
+0x89 U+.... .notdef
+0x8A U+0141 Lslash
+0x8B U+0143 Nacute
+0x8C U+007E asciitilde
+0x8D U+005E asciicircum
+0x8E U+.... .notdef
+0x8F U+2020 dagger
+0x90 U+2021 daggerdbl
+0x91 U+015A Sacute
+0x92 U+0160 Scaron
+0x93 U+0218 Scommaaccent
+0x94 U+00B0 degree
+0x95 U+0162 Tcommaaccent
+0x96 U+02DB ogonek
+0x97 U+0172 Uogonek
+0x98 U+0178 Ydieresis
+0x99 U+0179 Zacute
+0x9A U+017D Zcaron
+0x9B U+017B Zdotaccent
+0x9C U+0132 IJ
+0x9D U+007B braceleft
+0x9E U+007D braceright
+0x9F U+00A7 section
+0xA0 U+.... .notdef
+0xA1 U+0105 1 aogonek
+0xA2 U+0107 1 cacute
+0xA3 U+00AE registered
+0xA4 U+00A9 copyright
+0xA5 U+00F7 divide
+0xA6 U+0119 1 eogonek
+0xA7 U+012F 1 iogonek
+0xA8 U+2212 minus
+0xA9 U+00D7 multiply
+0xAA U+0142 1 lslash
+0xAB U+0144 1 nacute
+0xAC U+00B1 plusminus
+0xAD U+221E infinity
+0xAE U+00AB guillemotleft
+0xAF U+00BB guillemotright
+0xB0 U+00B6 paragraph
+0xB1 U+015B 1 sacute
+0xB2 U+0161 1 scaron
+0xB3 U+0219 1 scommaaccent
+0xB4 U+2022 bullet
+0xB5 U+0163 1 tcommaaccent
+0xB6 U+.... threequartersemdash
+0xB7 U+0173 1 uogonek
+0xB8 U+00FF 1 ydieresis
+0xB9 U+017A 1 zacute
+0xBA U+017E 1 zcaron
+0xBB U+017C 1 zdotaccent
+0xBC U+0133 1 ij
+0xBD U+00B7 periodcentered
+0xBE U+0022 quotedbl
+0xBF U+0027 quotesingle
+0xC0 U+00C0 Agrave
+0xC1 U+00C1 Aacute
+0xC2 U+00C2 Acircumflex
+0xC3 U+00C3 Atilde
+0xC4 U+00C4 Adieresis
+0xC5 U+00C5 Aring
+0xC6 U+005C backslash
+0xC7 U+00C7 Ccedilla
+0xC8 U+00C8 Egrave
+0xC9 U+00C9 Eacute
+0xCA U+00CA Ecircumflex
+0xCB U+00CB Edieresis
+0xCC U+00CC Igrave
+0xCD U+00CD Iacute
+0xCE U+00CE Icircumflex
+0xCF U+00CF Idieresis
+0xD0 U+00D0 Eth
+0xD1 U+00D1 Ntilde
+0xD2 U+00D2 Ograve
+0xD3 U+00D3 Oacute
+0xD4 U+00D4 Ocircumflex
+0xD5 U+00D5 Otilde
+0xD6 U+00D6 Odieresis
+0xD7 U+00A4 currency
+0xD8 U+2030 perthousand
+0xD9 U+00D9 Ugrave
+0xDA U+00DA Uacute
+0xDB U+00DB Ucircumflex
+0xDC U+00DC Udieresis
+0xDD U+00DD Yacute
+0xDE U+00DE Thorn
+0xDF U+007C bar
+0xE0 U+00E0 1 agrave
+0xE1 U+00E1 1 aacute
+0xE2 U+00E2 1 acircumflex
+0xE3 U+00E3 1 atilde
+0xE4 U+00E4 1 adieresis
+0xE5 U+00E5 1 aring
+0xE6 U+005F underscore
+0xE7 U+00E7 1 ccedilla
+0xE8 U+00E8 1 egrave
+0xE9 U+00E9 1 eacute
+0xEA U+00EA 1 ecircumflex
+0xEB U+00EB 1 edieresis
+0xEC U+00EC 1 igrave
+0xED U+00ED 1 iacute
+0xEE U+00EE 1 icircumflex
+0xEF U+00EF 1 idieresis
+0xF0 U+00F0 1 eth
+0xF1 U+00F1 1 ntilde
+0xF2 U+00F2 1 ograve
+0xF3 U+00F3 1 oacute
+0xF4 U+00F4 1 ocircumflex
+0xF5 U+00F5 1 otilde
+0xF6 U+00F6 1 odieresis
+0xF7 U+2222 anglearc
+0xF8 U+2300 diameter
+0xF9 U+00F9 1 ugrave
+0xFA U+00FA 1 uacute
+0xFB U+00FB 1 ucircumflex
+0xFC U+00FC 1 udieresis
+0xFD U+00FD 1 yacute
+0xFE U+00FE 1 thorn
+0xFF U+201E quotedblbase
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/t2a.dat b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/t2a.dat
new file mode 100644
index 00000000000..23b8d8fa4ae
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/t2a.dat
@@ -0,0 +1,256 @@
+0x00 U+0060
+0x01 U+00B4
+0x02 U+02C6
+0x03 U+02DC
+0x04 U+00A8
+0x05 U+02DD
+0x06 U+02DA
+0x07 U+02C7
+0x08 U+02D8
+0x09 U+00AF
+0x0A U+02D9
+0x0B U+00B8
+0x0C U+02DB
+0x0D U+04C0 CYRpalochka
+0x0E U+2329 textlangle
+0x0F U+232A textrangle
+0x10 U+201C textquotedblleft
+0x11 U+201D textquotedblright
+0x12 U+F6D5
+0x13 U+F6D6
+0x14 U+F6D4
+0x15 U+2013 textendash
+0x16 U+2014 textemdash
+# 0x17 U+0x17 Nothing?
+# 0x18 U+0x18 ?
+0x19 U+0131 1 i
+0x1A U+F6BE
+0x1B U+FB00 # Hand-edited!
+0x1C U+FB01 #
+0x1D U+FB02 #
+0x1E U+FB03 #
+0x1F U+FB04 #
+0x20 U+0020
+0x21 U+0021 exclam
+0x22 U+0022 quotedbl
+0x23 U+0023 numbersign
+0x24 U+0024 dollar
+0x25 U+0025 percent
+0x26 U+0026 ampersand
+0x27 U+2019 quoteright
+0x28 U+0028 parenleft
+0x29 U+0029 parenright
+0x2A U+002A asterisk
+0x2B U+002B plus
+0x2C U+002C comma
+0x2D U+002D hyphen
+0x2E U+002E period
+0x2F U+002F slash
+0x30 U+0030 zero
+0x31 U+0031 one
+0x32 U+0032 two
+0x33 U+0033 three
+0x34 U+0034 four
+0x35 U+0035 five
+0x36 U+0036 six
+0x37 U+0037 seven
+0x38 U+0038 eight
+0x39 U+0039 nine
+0x3A U+003A colon
+0x3B U+003B semicolon
+0x3C U+003C less
+0x3D U+003D equal
+0x3E U+003E greater
+0x3F U+003F question
+0x40 U+0040 at
+0x41 U+0041 A
+0x42 U+0042 B
+0x43 U+0043 C
+0x44 U+0044 D
+0x45 U+0045 E
+0x46 U+0046 F
+0x47 U+0047 G
+0x48 U+0048 H
+0x49 U+0406 CYRII
+0x4A U+004A J
+0x4B U+004B K
+0x4C U+004C L
+0x4D U+004D M
+0x4E U+004E N
+0x4F U+004F O
+0x50 U+0050 P
+0x51 U+0051 Q
+0x52 U+0052 R
+0x53 U+0053 S
+0x54 U+0054 T
+0x55 U+0055 U
+0x56 U+0056 V
+0x57 U+0057 W
+0x58 U+0058 X
+0x59 U+0059 Y
+0x5A U+005A Z
+0x5B U+005B bracketleft
+0x5C U+005C backslash
+0x5D U+005D bracketright
+0x5E U+02C6 circumflex
+0x5F U+005F underscore
+0x60 U+2018 quoteleft
+0x61 U+0061 a
+0x62 U+0062 b
+0x63 U+0063 c
+0x64 U+0064 d
+0x65 U+0065 e
+0x66 U+0066 f
+0x67 U+0067 g
+0x68 U+0068 h
+0x69 U+0456 1 cyrii
+# Hum ... (fix for Serbian)
+0x6A U+0458 1 cyrj
+0x6B U+006B k
+0x6C U+006C l
+0x6D U+006D m
+0x6E U+006E n
+0x6F U+006F o
+0x70 U+0070 p
+0x71 U+0071 q
+0x72 U+0072 r
+0x73 U+0073 s
+0x74 U+0074 t
+0x75 U+0075 u
+0x76 U+0076 v
+0x77 U+0077 w
+0x78 U+0078 x
+0x79 U+0079 y
+0x7A U+007A z
+0x7B U+007B braceleft
+0x7C U+007C bar
+0x7D U+007D braceright
+0x7E U+02DC tilde
+0x80 U+0490 CYRGUP
+0x81 U+0492 CYRGHCRS
+0x82 U+0402 CYRDJE
+0x83 U+040B CYRTSHE
+0x84 U+04BA CYRSHHA
+0x85 U+0496 CYRZHDSC
+0x86 U+0498 CYRZDSC
+0x87 U+0409 CYRLJE
+0x88 U+0407 CYRYI
+0x89 U+049A CYRKDSC
+0x8A U+04A0
+0x8B U+049C CYRKVCRS
+0x8C U+04D4 CYRAE
+0x8D U+04A2 CYRNDSC
+0x8E U+04A4 CYRNG
+0x8F U+0405 CYRDZE
+0x90 U+04E8 CYROTLD
+0x91 U+04AA CYRSDSC
+0x92 U+040E CYRUSHRT
+0x93 U+04AE CYRY
+0x94 U+04B0 CYRYHCRS
+0x95 U+04B2 CYRHDSC
+0x96 U+040F CYRDZHE
+0x97 U+04B8 CYRCHVCRS
+0x98 U+04B6 CYRCHRDSC
+0x99 U+0404 CYRIE
+0x9A U+04D8 CYRSCHWA
+0x9B U+040A CYRNJE
+0x9C U+0401 CYRYO
+0x9D U+2116 textnumero
+0x9E U+00A4 textcurrency
+0x9F U+00A7 textsection
+0xA0 U+0491 1 cyrgup
+0xA1 U+0493 1 cyrghcrs
+0xA2 U+0452 1 cyrdje
+0xA3 U+045B 1 cyrtshe
+0xA4 U+04BB 1 cyrshha
+0xA5 U+0497 1 cyrzhdsc
+0xA6 U+0499 1 cyrzdsc
+0xA7 U+0459 1 cyrlje
+0xA8 U+0457 1 cyryi
+0xA9 U+049B 1 cyrkdsc
+0xAA U+04A1 1
+0xAB U+049D 1 cyrkvcrs
+0xAC U+04D5 1 cyrae
+0xAD U+04A3 1 cyrndsc
+0xAE U+04A5 1 cyrng
+0xAF U+0455 1 cyrdze
+0xB0 U+04E9 1 cyrotld
+0xB1 U+04AB 1 cyrsdsc
+0xB2 U+045E 1 cyrushrt
+0xB3 U+04AF 1 cyry
+0xB4 U+04B1 1 cyryhcrs
+0xB5 U+04B3 1 cyrhdsc
+0xB6 U+045F 1 cyrdzhe
+0xB7 U+04B9 1 cyrchvcrs
+0xB8 U+04B7 1 cyrchrdsc
+0xB9 U+0454 1 cyrie
+0xBA U+04D9 1 cyrschwa
+0xBB U+045A 1 cyrnje
+0xBC U+0451 1 cyryo
+0xBD U+201E quotedblbase
+0xBE U+00AB guillemotleft
+0xBF U+00BB guillemotright
+0xC0 U+0410 CYRA
+0xC1 U+0411 CYRB
+0xC2 U+0412 CYRV
+0xC3 U+0413 CYRG
+0xC4 U+0414 CYRD
+0xC5 U+0415 CYRE
+0xC6 U+0416 CYRZH
+0xC7 U+0417 CYRZ
+0xC8 U+0418 CYRI
+0xC9 U+0419 CYRISHRT
+0xCA U+041A CYRK
+0xCB U+041B CYRL
+0xCC U+041C CYRM
+0xCD U+041D CYRN
+0xCE U+041E CYRO
+0xCF U+041F CYRP
+0xD0 U+0420 CYRR
+0xD1 U+0421 CYRS
+0xD2 U+0422 CYRT
+0xD3 U+0423 CYRU
+0xD4 U+0424 CYRF
+0xD5 U+0425 CYRH
+0xD6 U+0426 CYRC
+0xD7 U+0427 CYRCH
+0xD8 U+0428 CYRSH
+0xD9 U+0429 CYRSHCH
+0xDA U+042A CYRHRDSN
+0xDB U+042B CYRERY
+0xDC U+042C CYRSFTSN
+0xDD U+042D CYREREV
+0xDE U+042E CYRYU
+0xDF U+042F CYRYA
+0xE0 U+0430 1 cyra
+0xE1 U+0431 1 cyrb
+0xE2 U+0432 1 cyrv
+0xE3 U+0433 1 cyrg
+0xE4 U+0434 1 cyrd
+0xE5 U+0435 1 cyre
+0xE6 U+0436 1 cyrzh
+0xE7 U+0437 1 cyrz
+0xE8 U+0438 1 cyri
+0xE9 U+0439 1 cyrishrt
+0xEA U+043A 1 cyrk
+0xEB U+043B 1 cyrl
+0xEC U+043C 1 cyrm
+0xED U+043D 1 cyrn
+0xEE U+043E 1 cyro
+0xEF U+043F 1 cyrp
+0xF0 U+0440 1 cyrr
+0xF1 U+0441 1 cyrs
+0xF2 U+0442 1 cyrt
+0xF3 U+0443 1 cyru
+0xF4 U+0444 1 cyrf
+0xF5 U+0445 1 cyrh
+0xF6 U+0446 1 cyrc
+0xF7 U+0447 1 cyrch
+0xF8 U+0448 1 cyrsh
+0xF9 U+0449 1 cyrshch
+0xFA U+044A 1 cyrhrdsn
+0xFB U+044B 1 cyrery
+0xFC U+044C 1 cyrsftsn
+0xFD U+044D 1 cyrerev
+0xFE U+044E 1 cyryu
+0xFF U+044F 1 cyrya
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/texnansi.dat b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/texnansi.dat
new file mode 100644
index 00000000000..96aaea03a04
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/data/encodings/texnansi.dat
@@ -0,0 +1,256 @@
+0x00 U+.... .notdef
+0x01 U+20AC Euro
+0x02 U+.... .notdef
+0x03 U+.... .notdef
+0x04 U+2044 fraction
+0x05 U+02D9 dotaccent
+0x06 U+02DD hungarumlaut
+0x07 U+02DB ogonek
+0x08 U+FB02 fl
+0x09 U+.... .notdef
+0x0A U+200B cwm
+0x0B U+FB00 ff
+0x0C U+FB01 fi
+0x0D U+.... .notdef
+0x0E U+FB03 ffi
+0x0F U+FB04 ffl
+0x10 U+0131 1 dotlessi
+0x11 U+0237 1 dotlessj
+0x12 U+0060 grave
+0x13 U+00B4 acute
+0x14 U+02C7 caron
+0x15 U+02D8 breve
+0x16 U+00AF macron
+0x17 U+02DA ring
+0x18 U+00B8 cedilla
+0x19 U+00DF 1 germandbls
+0x1A U+00E6 1 ae
+0x1B U+0153 1 oe
+0x1C U+00F8 1 oslash
+0x1D U+00C6 AE
+0x1E U+0152 OE
+0x1F U+00D8 Oslash
+0x20 U+0020 space
+0x21 U+0021 exclam
+0x22 U+0022 quotedbl
+0x23 U+0023 numbersign
+0x24 U+0024 dollar
+0x25 U+0025 percent
+0x26 U+0026 ampersand
+0x27 U+2019 quoteright
+0x28 U+0028 parenleft
+0x29 U+0029 parenright
+0x2A U+002A asterisk
+0x2B U+002B plus
+0x2C U+002C comma
+0x2D U+002D hyphen
+0x2E U+002E period
+0x2F U+002F slash
+0x30 U+0030 zero
+0x31 U+0031 one
+0x32 U+0032 two
+0x33 U+0033 three
+0x34 U+0034 four
+0x35 U+0035 five
+0x36 U+0036 six
+0x37 U+0037 seven
+0x38 U+0038 eight
+0x39 U+0039 nine
+0x3A U+003A colon
+0x3B U+003B semicolon
+0x3C U+003C less
+0x3D U+003D equal
+0x3E U+003E greater
+0x3F U+003F question
+0x40 U+0040 at
+0x41 U+0041 A
+0x42 U+0042 B
+0x43 U+0043 C
+0x44 U+0044 D
+0x45 U+0045 E
+0x46 U+0046 F
+0x47 U+0047 G
+0x48 U+0048 H
+0x49 U+0049 I
+0x4A U+004A J
+0x4B U+004B K
+0x4C U+004C L
+0x4D U+004D M
+0x4E U+004E N
+0x4F U+004F O
+0x50 U+0050 P
+0x51 U+0051 Q
+0x52 U+0052 R
+0x53 U+0053 S
+0x54 U+0054 T
+0x55 U+0055 U
+0x56 U+0056 V
+0x57 U+0057 W
+0x58 U+0058 X
+0x59 U+0059 Y
+0x5A U+005A Z
+0x5B U+005B bracketleft
+0x5C U+005C backslash
+0x5D U+005D bracketright
+0x5E U+02C6 circumflex
+0x5F U+005F underscore
+0x60 U+2018 quoteleft
+0x61 U+0061 a
+0x62 U+0062 b
+0x63 U+0063 c
+0x64 U+0064 d
+0x65 U+0065 e
+0x66 U+0066 f
+0x67 U+0067 g
+0x68 U+0068 h
+0x69 U+0069 i
+0x6A U+006A j
+0x6B U+006B k
+0x6C U+006C l
+0x6D U+006D m
+0x6E U+006E n
+0x6F U+006F o
+0x70 U+0070 p
+0x71 U+0071 q
+0x72 U+0072 r
+0x73 U+0073 s
+0x74 U+0074 t
+0x75 U+0075 u
+0x76 U+0076 v
+0x77 U+0077 w
+0x78 U+0078 x
+0x79 U+0079 y
+0x7A U+007A z
+0x7B U+007B braceleft
+0x7C U+007C bar
+0x7D U+007D braceright
+0x7E U+02DC tilde
+0x7F U+00A8 dieresis
+0x80 U+0141 Lslash
+0x81 U+0027 quotesingle
+0x82 U+201A quotesinglbase
+0x83 U+0192 1 florin
+0x84 U+201E quotedblbase
+0x85 U+2026 ellipsis
+0x86 U+2020 dagger
+0x87 U+2021 daggerdbl
+0x88 U+02C6 circumflex
+0x89 U+2030 perthousand
+0x8A U+0160 Scaron
+0x8B U+2039 guilsinglleft
+0x8C U+0152 OE
+0x8D U+017D Zcaron
+0x8E U+005E asciicircum
+0x8F U+2212 minus
+0x90 U+0142 1 lslash
+0x91 U+2018 quoteleft
+0x92 U+2019 quoteright
+0x93 U+201C quotedblleft
+0x94 U+201D quotedblright
+0x95 U+2022 bullet
+0x96 U+2013 endash
+0x97 U+2014 emdash
+0x98 U+02DC tilde
+0x99 U+2122 trademark
+0x9A U+0161 1 scaron
+0x9B U+203A guilsinglright
+0x9C U+0153 1 oe
+0x9D U+017E 1 zcaron
+0x9E U+007E asciitilde
+0x9F U+0178 Ydieresis
+0xA0 U+00A0 nbspace
+0xA1 U+00A1 exclamdown
+0xA2 U+00A2 cent
+0xA3 U+00A3 sterling
+0xA4 U+00A4 currency
+0xA5 U+00A5 yen
+0xA6 U+00A6 brokenbar
+0xA7 U+00A7 section
+0xA8 U+00A8 dieresis
+0xA9 U+00A9 copyright
+0xAA U+00AA 1 ordfeminine
+0xAB U+00AB guillemotleft
+0xAC U+00AC logicalnot
+0xAD U+00AD sfthyphen
+0xAE U+00AE registered
+0xAF U+00AF macron
+0xB0 U+00B0 degree
+0xB1 U+00B1 plusminus
+0xB2 U+00B2 twosuperior
+0xB3 U+00B3 threesuperior
+0xB4 U+00B4 acute
+0xB5 U+00B5 mu
+0xB6 U+00B6 paragraph
+0xB7 U+00B7 periodcentered
+0xB8 U+00B8 cedilla
+0xB9 U+00B9 onesuperior
+0xBA U+00BA 1 ordmasculine
+0xBB U+00BB guillemotright
+0xBC U+00BC onequarter
+0xBD U+00BD onehalf
+0xBE U+00BE threequarters
+0xBF U+00BF questiondown
+0xC0 U+00C0 Agrave
+0xC1 U+00C1 Aacute
+0xC2 U+00C2 Acircumflex
+0xC3 U+00C3 Atilde
+0xC4 U+00C4 Adieresis
+0xC5 U+00C5 Aring
+0xC6 U+00C6 AE
+0xC7 U+00C7 Ccedilla
+0xC8 U+00C8 Egrave
+0xC9 U+00C9 Eacute
+0xCA U+00CA Ecircumflex
+0xCB U+00CB Edieresis
+0xCC U+00CC Igrave
+0xCD U+00CD Iacute
+0xCE U+00CE Icircumflex
+0xCF U+00CF Idieresis
+0xD0 U+00D0 Eth
+0xD1 U+00D1 Ntilde
+0xD2 U+00D2 Ograve
+0xD3 U+00D3 Oacute
+0xD4 U+00D4 Ocircumflex
+0xD5 U+00D5 Otilde
+0xD6 U+00D6 Odieresis
+0xD7 U+00D7 multiply
+0xD8 U+00D8 Oslash
+0xD9 U+00D9 Ugrave
+0xDA U+00DA Uacute
+0xDB U+00DB Ucircumflex
+0xDC U+00DC Udieresis
+0xDD U+00DD Yacute
+0xDE U+00DE Thorn
+0xDF U+00DF 1 germandbls
+0xE0 U+00E0 1 agrave
+0xE1 U+00E1 1 aacute
+0xE2 U+00E2 1 acircumflex
+0xE3 U+00E3 1 atilde
+0xE4 U+00E4 1 adieresis
+0xE5 U+00E5 1 aring
+0xE6 U+00E6 1 ae
+0xE7 U+00E7 1 ccedilla
+0xE8 U+00E8 1 egrave
+0xE9 U+00E9 1 eacute
+0xEA U+00EA 1 ecircumflex
+0xEB U+00EB 1 edieresis
+0xEC U+00EC 1 igrave
+0xED U+00ED 1 iacute
+0xEE U+00EE 1 icircumflex
+0xEF U+00EF 1 idieresis
+0xF0 U+00F0 1 eth
+0xF1 U+00F1 1 ntilde
+0xF2 U+00F2 1 ograve
+0xF3 U+00F3 1 oacute
+0xF4 U+00F4 1 ocircumflex
+0xF5 U+00F5 1 otilde
+0xF6 U+00F6 1 odieresis
+0xF7 U+00F7 divide
+0xF8 U+00F8 1 oslash
+0xF9 U+00F9 1 ugrave
+0xFA U+00FA 1 uacute
+0xFB U+00FB 1 ucircumflex
+0xFC U+00FC 1 udieresis
+0xFD U+00FD 1 yacute
+0xFE U+00FE 1 thorn
+0xFF U+00FF 1 ydieresis
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/generate-converters.rb b/Master/texmf-dist/source/generic/hyph-utf8/generate-converters.rb
new file mode 100644
index 00000000000..9fd0066153d
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/generate-converters.rb
@@ -0,0 +1,117 @@
+#!/usr/bin/env ruby
+
+$encoding_data_dir = "data/encodings"
+$encodings = ["ec", "qx", "t2a", "lmc", "il3"]
+
+$output_data_dir = "../../../tex/generic/hyph-utf8/conversions"
+
+class UnicodeCharacter
+ def initialize(code_uni, code_enc, name)
+ @code_uni = code_uni
+ @code_enc = code_enc
+ # TODO: might be longer or shorter
+ @bytes = [code_uni].pack('U').unpack('H2H2')
+ @name = name
+ end
+
+ attr_reader :code_uni, :code_enc, :bytes, :name
+end
+
+class UnicodeCharacters < Hash
+ def add_new_character(code_uni, code_enc, name)
+ first_byte = [code_uni].pack('U').unpack('H2').first
+ if self[first_byte] == nil then
+ self[first_byte] = Array.new
+ end
+ self[first_byte].push(UnicodeCharacter.new(code_uni, code_enc, name))
+ end
+end
+
+# 0x19; U+0131; 1; dotlessi
+$encodings.each do |encoding|
+ #$utf_combinations = Hash.new
+ $unicode_characters = UnicodeCharacters.new
+
+ # those that need lccode to be set
+ $lowercase_characters = Array.new
+
+ File.open($encoding_data_dir + "/" + encoding + ".dat").grep(/^0x(\w+)\tU\+(\w+)\t(\d*)\t([_a-zA-Z\.]*)$/) do |line|
+ # puts line
+ code_enc = $1.hex
+ code_uni = $2.hex
+ if $3.length > 0
+ type = $3.to_i
+ else
+ type = 0
+ end
+ name = $4
+ if type == 1 then
+ $unicode_characters.add_new_character(code_uni, code_enc, name)
+ $lowercase_characters.push(UnicodeCharacter.new(code_uni, code_enc, name))
+ end
+ end
+
+ $file_out = File.open("#{$output_data_dir}#{File::Separator}conv-utf8-#{encoding}.tex", "w")
+ $file_out.puts "% conv-utf8-#{encoding}.tex"
+ $file_out.puts "%"
+ $file_out.puts "% Conversion from UTF-8 to #{encoding.upcase},"
+ $file_out.puts "% used before loading hyphenation patterns for 8-bit TeX engines."
+ $file_out.puts "%"
+ $file_out.puts "% This file is part of hyph-utf8 package and autogenerated."
+ $file_out.puts "% See http://tug.org/tex-hyphen"
+ $file_out.puts "%"
+ $file_out.puts "% Copyright 2008 TeX Users Group."
+ $file_out.puts "% You may freely use, modify and/or distribute this file."
+ $file_out.puts "% (But consider adapting the scripts if you need modifications.)"
+ $file_out.puts "%"
+
+ $unicode_characters.sort.each do |first_byte|
+ # sorting all the second characters alphabetically
+ first_byte[1].sort!{|x,y| x.code_uni <=> y.code_uni }
+ # make all the possible first characters active
+ # output the definition into file
+ $file_out.puts "\\catcode\"#{first_byte[0].upcase}=\\active"
+ end
+ $file_out.puts "%"
+ $unicode_characters.sort.each do |first_byte|
+ $file_out.puts "\\def^^#{first_byte[0]}#1{%"
+ string_fi = ""
+ for i in 1..(first_byte[1].size)
+ uni_character = first_byte[1][i-1]
+
+ second_byte = uni_character.bytes[1]
+ enc_byte = uni_character.code_enc
+ enc_byte = [ uni_character.code_enc ].pack('c').unpack('H2')
+ ux_code = sprintf("U+%04X", uni_character.code_uni)
+ $file_out.puts "\t\\ifx#1^^#{second_byte}^^#{enc_byte}\\else % #{[uni_character.code_uni].pack('U')} - #{ux_code} - #{uni_character.name}"
+ string_fi = string_fi + "\\fi"
+ end
+ $file_out.puts "\t\\errmessage{Hyphenation pattern file corrupted or #{encoding} encoding not supported!}"
+ $file_out.puts string_fi+"}"
+ end
+ $file_out.puts "%"
+ $file_out.puts "% ensure all the chars above have valid \lccode values"
+ $file_out.puts "%"
+ $lowercase_characters.sort!{|x,y| x.code_enc <=> y.code_enc }.each do |character|
+ code = [ character.code_enc ].pack("c").unpack("H2").first.upcase
+ # \lccode"FF="FF
+ ux_code = sprintf("U+%04X", character.code_uni)
+ $file_out.puts "\\lccode\"#{code}=\"#{code} % #{[character.code_uni].pack('U')} - #{ux_code} - #{character.name}"
+ end
+ $file_out.puts
+
+ # if encoding == 'ec'
+ # $file_out.puts '% TODO: test if needed and the exact syntax'
+ # $file_out.puts '% some patterns use apostrophe and hyphen'
+ # # \lccode`\'=`\'
+ # $file_out.puts "\\lccode`\\'=`\\'"
+ # $file_out.puts "\\catcode`-=11"
+ # $file_out.puts
+ # end
+ # if encoding == 't2a'
+ # $file_out.puts "\\lccode`\\'=`\\'"
+ # end
+
+ $file_out.close
+end
+
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/generate-pattern-loaders.rb b/Master/texmf-dist/source/generic/hyph-utf8/generate-pattern-loaders.rb
new file mode 100644
index 00000000000..f00a19d3e68
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/generate-pattern-loaders.rb
@@ -0,0 +1,112 @@
+#!/usr/bin/env ruby
+
+# this file auto-generates loaders for hyphenation patterns - to be improved
+
+load 'languages.rb'
+
+$package_name="hyph-utf8"
+
+
+# TODO - make this a bit less hard-coded
+$path_tex_generic="../../../tex/generic"
+$path_loadhyph="#{$path_tex_generic}/#{$package_name}/loadhyph"
+
+# TODO: should be singleton
+languages = Languages.new.list
+
+#text_if_native_utf = "\input pattern-loader.tex\n\\ifNativeUtfEightPatterns"
+
+text_if_native_utf = <<EOT
+% Test whether we received one or two arguments
+\\def\\testengine#1#2!{\\def\\secondarg{#2}}
+% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character
+\\testengine Τ!\\relax
+% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument
+\\ifx\\secondarg\\empty
+EOT
+
+languages.each do |language|
+ if language.use_new_loader then
+ filename = "#{$path_loadhyph}/loadhyph-#{language.code}.tex"
+ puts "generating '#{filename}'"
+ File.open(filename, "w") do |file|
+ # a message about auto-generation
+ # TODO: write a more comprehensive one
+ file.puts "% loadhyph-#{language.code}.tex"
+ file.puts "%"
+ file.puts "% Autogenerated loader for hyphenation patterns for \"#{language.name}\""
+ file.puts "% by source/generic/hyph-utf8/generate-pattern-loaders.rb"
+ file.puts "% See also http://tug.org/tex-hyphen"
+ file.puts "%"
+ file.puts "% Copyright 2008 TeX Users Group."
+ file.puts "% You may freely use, modify and/or distribute this file."
+ file.puts "% (But consider adapting the scripts if you need modifications.)"
+ file.puts "%"
+ file.puts "% Once it turns out that more than a simple definition is needed,"
+ file.puts "% these lines may be moved to a separate file."
+ file.puts "%"
+
+ # message (where it exists)
+ # if language.message != nil
+ # file.puts("\\message{#{language.message}}\n%")
+ # end
+
+ # TODO:
+ # \lefthyphenmin=2 \righthyphenmin=2
+ # but probably this needs to reside outside of \begingroup/endgroup
+
+ file.puts('\begingroup')
+ if language.code == 'it' or language.code == 'fr' or language.code == 'uk' or language.code == 'la' or language.code == 'zh-latn' then
+ file.puts("\\lccode`\\'=`\\'")
+ end
+ if language.code == 'pt' then
+ file.puts("\\lccode`\\-=`\\-")
+ end
+
+ if language.encoding == "ascii" then
+ file.puts('% ASCII patterns - no additional support is needed')
+ if language.use_old_patterns then
+ file.puts("\\input #{language.filename_old_patterns}")
+ else
+ file.puts("\\message{ASCII #{language.message}}")
+ file.puts("\\input hyph-#{language.code}.tex")
+ end
+ elsif language.use_old_patterns then
+ file.puts(text_if_native_utf)
+ file.puts(" \\message{UTF-8 #{language.message}}")
+ if language.code == 'grc' or language.code.slice(0,2) == 'el' then
+ file.puts(" \\lccode`'=`'\\lccode`’=`’\\lccode`ʼ=`ʼ\\lccode`᾽=`᾽\\lccode`᾿=`᾿")
+ end
+ file.puts(" \\input hyph-#{language.code}.tex")
+ file.puts('\else')
+ file.puts(" \\message{#{language.message}}")
+ # explain why we are still using the old patterns
+ if language.use_old_patterns_comment != nil then
+ file.puts(" % #{language.use_old_patterns_comment}")
+ else
+ puts "Missing comment for #{language.name}"
+ file.puts(' % we still load old patterns for 8-bit TeX')
+ end
+ file.puts(" \\input #{language.filename_old_patterns}")
+ file.puts('\fi')
+ else
+ file.puts(text_if_native_utf)
+ file.puts(" \\message{UTF-8 #{language.message}}")
+ file.puts('\else')
+ file.puts(" \\message{#{language.encoding.upcase} #{language.message}}")
+ file.puts(" \\input conv-utf8-#{language.encoding}.tex")
+ file.puts('\fi')
+ if language.code == 'sr-latn' then
+ file.puts("% Load Serbo-Croatian patterns")
+ file.puts("\\input hyph-hbs.tex")
+ else
+ file.puts("\\input hyph-#{language.code}.tex")
+ end
+# file.puts("\\loadhyphpatterns{#{language.code}}{#{language.encoding}}%")
+ end
+ file.puts('\endgroup')
+ end
+ end
+end
+
+
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/generate-tl-files.rb b/Master/texmf-dist/source/generic/hyph-utf8/generate-tl-files.rb
new file mode 100644
index 00000000000..e4ba9c4e7ed
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/generate-tl-files.rb
@@ -0,0 +1,129 @@
+#!/usr/bin/env ruby
+
+load 'languages.rb'
+
+# this file auto-generates loaders for hyphenation patterns - to be improved
+
+$package_name="hyph-utf8"
+
+# TODO - make this a bit less hard-coded
+$path_tex_generic="../../../tex/generic"
+$path_TL="../../../../TL"
+$path_language_dat="#{$path_TL}/texmf/tex/generic/config"
+# hyphen-foo.tlpsrc for TeX Live
+$path_tlpsrc="#{$path_TL}/tlpkg/tlpsrc"
+
+$l = Languages.new
+# TODO: should be singleton
+languages = $l.list.sort{|a,b| a.name <=> b.name}
+
+language_grouping = {
+ 'norwegian' => ['nb', 'nn'],
+ 'german' => ['de-1901', 'de-1996'],
+ 'serbian' => ['sr-latn', 'sr-cyrl'],
+ 'mongolian' => ['mn-cyrl', 'mn-cyrl-x-2a'],
+ 'greek' => ['el-monoton', 'el-polyton'],
+ 'ancientgreek' => ['grc', 'grc-x-ibycus'],
+ 'chinese' => ['zh-latn'],
+}
+
+language_used_in_group = Hash.new
+language_grouping.each_value do |group|
+ group.each do |code|
+ language_used_in_group[code] = true
+ end
+end
+
+# a hash with language name as key and array of languages as the value
+language_groups = Hash.new
+# single languages first
+languages.each do |language|
+ if language_used_in_group[language.code] == nil then
+ language_groups[language.name] = [language]
+ end
+end
+# then groups of languages
+language_grouping.each do |name,group|
+ language_groups[name] = []
+ group.each do |code|
+ language_groups[name].push($l[code])
+ end
+end
+
+# languages.each do |language|
+# if language.hyphenmin == nil then
+# lmin = ''
+# rmin = ''
+# else
+# lmin = language.hyphenmin[0]
+# rmin = language.hyphenmin[1]
+# end
+# puts "#{language.name}: #{lmin} #{rmin}"
+# end
+
+#--------#
+# TLPSRC #
+#--------#
+$file_tlpsrc = File.open("#{$path_language_dat}/language.tlpsrc", 'w')
+language_groups.sort.each do |language_name,language_list|
+
+ $file_tlpsrc.puts "name hyphen-#{language_name}"
+ $file_tlpsrc.puts "category TLCore"
+
+ language_list.each do |language|
+ name = " name=#{language.name}"
+
+ # synonyms
+ if language.synonyms != nil and language.synonyms.length > 0 then
+ synonyms=" synonyms=#{language.synonyms.join(',')}"
+ else
+ synonyms=""
+ end
+ # lefthyphenmin/righthyphenmin
+ if language.hyphenmin == nil or language.hyphenmin.length == 0 then
+ lmin = ''
+ rmin = ''
+ elsif language.filename_old_patterns == "zerohyph.tex" then
+ lmin = ''
+ rmin = ''
+ else
+ lmin = language.hyphenmin[0]
+ rmin = language.hyphenmin[1]
+ end
+ hyphenmins = " lefthyphenmin=#{lmin} righthyphenmin=#{rmin}"
+ # which file to use
+ if language.use_new_loader then
+ file = " file=loadhyph-#{language.code}.tex"
+ else
+ file = " file=#{language.filename_old_patterns}"
+ end
+
+ $file_tlpsrc.puts "execute AddHyphen#{name}#{synonyms}#{hyphenmins}#{file}"
+
+ end
+ $file_tlpsrc.puts
+end
+$file_tlpsrc.close
+
+#--------------#
+# language.dat #
+#--------------#
+$file_language_dat = File.open("#{$path_language_dat}/language.dat", "w")
+language_groups.sort.each do |language_name,language_list|
+ language_list.each do |language|
+
+ if language.use_new_loader then
+ $file_language_dat.puts "#{language.name}\tloadhyph-#{language.code}.tex"
+ else
+ $file_language_dat.puts "#{language.name}\t#{language.filename_old_patterns}"
+ end
+
+ # synonyms
+ if language.synonyms != nil then
+ language.synonyms.each do |synonym|
+ $file_language_dat.puts "=#{synonym}"
+ end
+ end
+ end
+end
+$file_language_dat.close
diff --git a/Master/texmf-dist/source/generic/hyph-utf8/languages.rb b/Master/texmf-dist/source/generic/hyph-utf8/languages.rb
new file mode 100644
index 00000000000..ebd0e32cc2c
--- /dev/null
+++ b/Master/texmf-dist/source/generic/hyph-utf8/languages.rb
@@ -0,0 +1,775 @@
+
+class Language
+ def initialize(language_hash)
+ @use_new_loader = language_hash["use_new_loader"]
+ @use_old_patterns = language_hash["use_old_patterns"]
+ @use_old_patterns_comment = language_hash["use_old_patterns_comment"]
+ @filename_old_patterns = language_hash["filename_old_patterns"]
+ @filename_old_patterns_other = language_hash["filename_old_patterns_other"]
+ @filename_xu_loader = language_hash["filename_xu_loader"]
+ @code = language_hash["code"]
+ @name = language_hash["name"]
+ @synonyms = language_hash["synonyms"]
+ @hyphenmin = language_hash["hyphenmin"]
+ @encoding = language_hash["encoding"]
+ @exceptions = language_hash["exceptions"]
+ @message = language_hash["message"]
+ end
+
+ attr_reader :use_new_loader, :use_old_patterns, :use_old_patterns_comment, :filename_old_patterns, :filename_xu_loader, :code, :name, :synonyms, :hyphenmin, :encoding, :exceptions, :message
+end
+
+# "use_new_loader"
+# => true - create a new file and use that one
+# => false - use "filename_old_patterns" in language.dat
+# "filename_old_patterns"
+# => [string] - the name used in language.dat if "use_new_loader" is false
+# "eightbitfilename"
+# => [string] - if set, load another file for 8-bit engines
+# "code"
+# => [string] - used in filenames, needs to conform to the standard
+# "name"
+# => [string] -
+# "synonyms" => [],
+# "hyphenmin" => [],
+# "encoding" => nil,
+# "exceptions" => false,
+# "message" => nil,
+
+class Languages < Hash
+ @@list = []
+
+ def initialize
+ languages = [
+# --------------------------------------
+# languages with no hyphenation patterns
+# --------------------------------------
+# arabic zerohyph.tex
+{
+ "use_new_loader" => false,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "zerohyph.tex",
+# "filename_xu_loader" => nil,
+ "code" => "ar",
+ "name" => "arabic",
+ "synonyms" => [],
+# "hyphenmin" => [], # not needed
+ "encoding" => nil,
+ "exceptions" => false,
+ "message" => nil,
+},
+# farsi zerohyph.tex
+# =persian
+{
+ "use_new_loader" => false,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "zerohyph.tex",
+# "filename_xu_loader" => nil,
+ "code" => "fa",
+ "name" => "farsi",
+ "synonyms" => ["persian"],
+# "hyphenmin" => [], # not needed
+ "encoding" => nil,
+ "exceptions" => false,
+ "message" => nil,
+},
+
+# ----------------------------
+# languages using old patterns
+# ----------------------------
+# coptic xu-copthyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "use_old_patterns_comment" => "TODO: automatic conversion could be done, but was too complicated; leave for later.",
+ "filename_old_patterns" => "copthyph.tex",
+# "filename_xu_loader" => "xu-copthyph.tex",
+ "code" => "cop",
+ "name" => "coptic",
+ "synonyms" => [],
+ "hyphenmin" => [1,1], # polyglosia TODO: no documentation found
+ "encoding" => nil,
+ "exceptions" => false,
+ "message" => "Coptic Hyphenation Patterns",
+},
+# danish dkhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "use_old_patterns_comment" => "Old patterns support both EC & OT1 encodings at the same time.",
+ "filename_old_patterns" => "dkhyph.tex",
+ "filename_old_patterns_other" => ["dkcommon.tex", "dkspecial.tex"],
+# "filename_xu_loader" => nil,
+ "code" => "da",
+ "name" => "danish",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Danish Hyphenation Patterns",
+},
+# german xu-dehypht.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "filename_old_patterns" => "dehypht.tex",
+ "use_old_patterns_comment" => "Kept for the sake of backward compatibility, but newer and better patterns by WL are available.",
+# "filename_xu_loader" => "xu-dehypht.tex",
+ "code" => "de-1901",
+ "name" => "german",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "German Hyphenation Patterns (Traditional Orthography)",
+},
+# ngerman xu-dehyphn.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "use_old_patterns_comment" => "Kept for the sake of backward compatibility, but newer and better patterns by WL are available.",
+ "filename_old_patterns" => "dehyphn.tex",
+# "filename_xu_loader" => "xu-dehyphn.tex",
+ "code" => "de-1996",
+ "name" => "ngerman",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "German Hyphenation Patterns (Reformed Orthography)",
+},
+# french xu-frhyph.tex
+# =patois
+# =francais
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "use_old_patterns_comment" => "Old patterns support both EC & OT1 encodings at the same time.",
+ "filename_old_patterns" => "frhyph.tex",
+# "filename_xu_loader" => "xu-frhyph.tex",
+ "code" => "fr",
+ "name" => "french",
+ "synonyms" => ["patois","francais"],
+# "hyphenmin" => [],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "French hyphenation patterns (V2.12, 2002/12/11)",
+},
+# ----------------------------
+# languages using new patterns
+# ----------------------------
+# czech xu-czhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+# "use_old_patterns_comment" => "Loading new patterns would most probably work, but testing is needed.",
+ "filename_old_patterns" => "czhyph.tex",
+ "filename_old_patterns_other" => ["czhyphen.tex","czhyphen.ex"],
+# "filename_xu_loader" => "xu-czhyph.tex",
+ "code" => "cs",
+ "name" => "czech",
+ "synonyms" => [],
+ # Both Czech and Slovak: \lefthyphenmin=2, \righthyphenmin=3
+ # Typographical rules allow \righthyphenmin=2 when typesetting in a
+ # narrow column (newspapers etc.).
+ # (used to be 2,2)
+ "hyphenmin" => [2,3],
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Czech Hyphenation Patterns (Pavel Sevecek, v3, 1995)",
+},
+# slovak xu-skhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+# "use_old_patterns_comment" => "Loading new patterns would most probably work, but testing is needed.",
+ "filename_old_patterns" => "skhyph.tex",
+ "filename_old_patterns_other" => ["skhyphen.tex","skhyphen.ex"],
+# "filename_xu_loader" => "xu-skhyph.tex",
+ "code" => "sk",
+ "name" => "slovak",
+ "synonyms" => [],
+ # see czech
+ "hyphenmin" => [2,3],
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Slovak Hyphenation Patterns (Jana Chlebikova, 1992)",
+},
+# welsh cyhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "cyhyph.tex",
+# "filename_xu_loader" => nil,
+ "code" => "cy",
+ "name" => "welsh",
+ "synonyms" => [],
+ "hyphenmin" => [2,3],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Welsh Hyphenation Patterns",
+},
+# esperanto xu-eohyph.tex
+# TODO
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "eohyph.tex",
+# "filename_xu_loader" => "xu-eohyph.tex",
+ "code" => "eo",
+ "name" => "esperanto",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "il3", # TODO
+ "exceptions" => false,
+ "message" => "Esperanto Hyphenation Patterns",
+},
+# spanish xu-eshyph.tex
+# =espanol
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "eshyph.tex",
+# "filename_xu_loader" => "xu-eshyph.tex",
+ "code" => "es",
+ "name" => "spanish",
+ "synonyms" => ["espanol"],
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Spanish Hyphenation Patterns",
+},
+# basque xu-bahyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "bahyph.tex",
+# "filename_xu_loader" => "xu-bahyph.tex",
+ "code" => "eu",
+ "name" => "basque",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ascii",
+ "exceptions" => false,
+ "message" => "Basque Hyphenation Patterns",
+},
+# catalan cahyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "cahyph.tex",
+# "filename_xu_loader" => "xu-cahyph.tex",
+ "code" => "ca",
+ "name" => "catalan",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Catalan Hyphenation Patterns",
+},
+# galician xu-glhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "glhyph.tex",
+# "filename_xu_loader" => "xu-glhyph.tex",
+ "code" => "gl",
+ "name" => "galician",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Galician Hyphenation Patterns",
+},
+# estonian xu-ethyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "ethyph.tex",
+# "filename_xu_loader" => "xu-ethyph.tex",
+ "code" => "et",
+ "name" => "estonian",
+ "synonyms" => [],
+ "hyphenmin" => [2,3], # babel
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Estonian Hyphenation Patterns",
+},
+# finnish fihyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "fihyph.tex",
+# "filename_xu_loader" => nil,
+ "code" => "fi",
+ "name" => "finnish",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Finnish Hyphenation Patterns",
+},
+# croatian xu-hrhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "hrhyph.tex",
+ "filename_xu_loader" => "xu-hrhyph.tex",
+ "code" => "hr",
+ "name" => "croatian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Croatian Hyphenation Patterns",
+},
+# hungarian xu-huhyphn.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "huhyphn.tex",
+ "filename_xu_loader" => "xu-huhyphn.tex",
+ "code" => "hu",
+ "name" => "hungarian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # polyglosia
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Hungarian Hyphenation Patterns (v20031107)",
+},
+# interlingua iahyphen.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "iahyphen.tex",
+ "filename_xu_loader" => nil,
+ "code" => "ia",
+ "name" => "interlingua",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ascii",
+ "exceptions" => true,
+ "message" => "Hyphenation Patterns for Interlingua",
+},
+# indonesian inhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "inhyph.tex",
+ "filename_xu_loader" => nil,
+ "code" => "id",
+ "name" => "indonesian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "ascii",
+ "exceptions" => true,
+ "message" => "Indonesian Hyphenation Patterns",
+},
+# icelandic icehyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "icehyph.tex",
+ "filename_xu_loader" => nil,
+ "code" => "is",
+ "name" => "icelandic",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Icelandic Hyphenation Patterns",
+},
+# irish gahyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "gahyph.tex",
+ "filename_xu_loader" => nil,
+ "code" => "ga",
+ "name" => "irish",
+ "synonyms" => [],
+ "hyphenmin" => [2,3], # babel
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Irish Hyphenation Patterns",
+},
+# italian ithyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "ithyph.tex",
+ "filename_xu_loader" => nil,
+ "code" => "it",
+ "name" => "italian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ascii",
+ "exceptions" => false,
+ "message" => "Italian Hyphenation Patterns",
+},
+# latin xu-lahyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "use_old_patterns_comment" => "Old patterns support both EC & OT1 encodings at the same time.",
+ "filename_old_patterns" => "lahyph.tex",
+ "filename_xu_loader" => "xu-lahyph.tex",
+ "code" => "la",
+ "name" => "latin",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Latin Hyphenation Patterns",
+},
+# dutch nehyph96.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "nehyph96.tex",
+ "filename_xu_loader" => nil,
+ "code" => "nl",
+ "name" => "dutch",
+ "synonyms" => [],
+ # quoting Hans Hagen:
+ # patterns generated with 2,2 (so don't go less) but use prefered values 2,3 (educational publishers want 4,5 -)
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Dutch Hyphenation Patterns",
+},
+# norsk xu-nohyphbx.tex
+# =norwegian
+# nynorsk nnhyph.tex
+# bokmal nbhyph.tex
+# polish xu-plhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "plhyph.tex",
+ "filename_xu_loader" => "xu-plhyph.tex",
+ "code" => "pl",
+ "name" => "polish",
+ "synonyms" => [],
+ "hyphenmin" => [1,1],
+ "encoding" => "qx",
+ "exceptions" => true,
+ "message" => "Polish Hyphenation Patterns",
+},
+# portuguese pthyph.tex
+# =portuges
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "pthyph.tex",
+ "filename_xu_loader" => nil,
+ "code" => "pt",
+ "name" => "portuguese",
+ "synonyms" => ["portuges"],
+ "hyphenmin" => [2,3], # babel
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Portuguese Hyphenation Patterns",
+},
+# pinyin xu-pyhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "pyhyph.tex",
+ "filename_xu_loader" => "xu-pyhyph.tex",
+ "code" => "zh-latn",
+ "name" => "pinyin",
+ "synonyms" => [],
+ "hyphenmin" => [1,1],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Hyphenation patterns for unaccented pinyin syllables (CJK 4.8.0)",
+},
+# romanian xu-rohyphen.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "rohyphen.tex",
+ "filename_xu_loader" => "xu-rohyphen.tex",
+ "code" => "ro",
+ "name" => "romanian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Romanian Hyphenation Patterns", # : `rohyphen' 1.1 <29.10.1996>
+},
+# slovenian xu-sihyph.tex
+# =slovene
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "sihyph.tex",
+ "filename_xu_loader" => "xu-sihyph.tex",
+ "code" => "sl",
+ "name" => "slovenian",
+ "synonyms" => ["slovene"],
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Slovenian Hyphenation Patterns",
+},
+# uppersorbian xu-sorhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "sorhyph.tex",
+ "filename_xu_loader" => "xu-sorhyph.tex",
+ "code" => "hsb",
+ "name" => "uppersorbian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Upper Sorbian Hyphenation Patterns (E. Werner)",
+# \message{Hyphenation patterns for Upper Sorbian, E. Werner}
+# \message{Completely new revision 1997, March 22}
+},
+# swedish svhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "svhyph.tex",
+ "filename_xu_loader" => nil,
+ "code" => "sv",
+ "name" => "swedish",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # patters say it could be 1,2; babel says 2,2 - double check
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Swedish hyphenation patterns (Jan Michael Rynning, 1994-03-03)",
+},
+# turkish xu-tkhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "tkhyph.tex",
+ "filename_xu_loader" => "xu-tkhyph.tex",
+ "code" => "tr",
+ "name" => "turkish",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # polyglosia
+ "encoding" => "ec",
+ "exceptions" => false,
+ "message" => "Turkish Hyphenation Patterns",
+},
+# ukenglish ukhyphen.tex
+# TODO - should we rename it or not?
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "filename_old_patterns" => "ukhyphen.tex",
+ "filename_xu_loader" => nil,
+ "code" => "en-gb",
+ "name" => "ukenglish",
+ "synonyms" => [],
+ "hyphenmin" => [2,3], # confirmed, same as what Knuth says
+ "encoding" => "ascii",
+ "exceptions" => true,
+ "message" => nil,
+},
+# serbian xu-srhyphc.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "shhyphl.tex",
+ "filename_xu_loader" => nil,
+ "code" => "sr-latn",
+ "name" => "serbian",
+ "synonyms" => [],
+ # It is allowed to leave one character at the end of the row.
+ # However, if you think that it is graphicaly not very
+ # pleasent these patterns will work well with \lefthyphenmin=2.
+ # \lefthyphenmin=1 \righthyphenmin=2
+ "hyphenmin" => [2,2],
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Serbian hyphenation patterns in Latin script",
+},
+# serbianc
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "srhyphc.tex",
+ "filename_xu_loader" => nil, # "xu-srhyphc.tex"
+ "code" => "sr-cyrl",
+ "name" => "serbianc",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "t2a",
+ "exceptions" => true,
+ "message" => "Serbian hyphenation patterns in Cyrillic script",
+},
+# mongolian xu-mnhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "mnhyph.tex",
+ "filename_xu_loader" => "xu-mnhyph.tex",
+ "code" => "mn-cyrl",
+ "name" => "mongolian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "lmc",
+ "exceptions" => false,
+ "message" => "Mongolian hyphenation patterns",
+},
+# mongolian2a
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "mnhyphn.tex",
+ "filename_xu_loader" => nil,
+ "code" => "mn-cyrl-x-2a",
+ "name" => "mongolian2a",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "t2a",
+ "exceptions" => false,
+ "message" => "(New) Mongolian Hyphenation Patterns",
+},
+# greek xu-grphyph4.tex
+# =polygreek
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "use_old_patterns_comment" => "Old patterns work in a different way, one-to-one conversion from UTF-8 is not possible.",
+ "filename_old_patterns" => "grphyph5.tex",
+ "filename_xu_loader" => "xu-grphyph4.tex", # TODO: beware!
+ "code" => "el-polyton",
+ "name" => "greek",
+ "synonyms" => ["polygreek"],
+ # left/right hyphen min for Greek can be as low as one (1),
+ # but for aesthetic reasons keep them at 2/2.
+ # Dimitrios Filippou
+ "hyphenmin" => [1,1], # polyglosia
+ "encoding" => nil,
+ "exceptions" => true,
+# "message" => "Polytonic Greek Hyphenation Patterns",
+ "message" => "Hyphenation patterns for multi-accent (polytonic) Modern Greek"
+},
+# monogreek xu-grmhyph4.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "use_old_patterns_comment" => "Old patterns work in a different way, one-to-one conversion from UTF-8 is not possible.",
+ "filename_old_patterns" => "grmhyph5.tex",
+ "filename_xu_loader" => "xu-grmhyph4.tex", # TODO: beware!
+ "code" => "el-monoton",
+ "name" => "monogreek",
+ "synonyms" => [],
+ "hyphenmin" => [1,1], # polyglosia
+ "encoding" => nil,
+ "exceptions" => true,
+# "message" => "Monotonic Greek Hyphenation Patterns",
+ "message" => "Hyphenation patterns for uni-accent (monotonic) Modern Greek"
+},
+# ancientgreek xu-grahyph4.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => true,
+ "use_old_patterns_comment" => "Old patterns work in a different way, one-to-one conversion from UTF-8 is not possible.",
+ "filename_old_patterns" => "grmhyph5.tex",
+ "filename_xu_loader" => "xu-grahyph4.tex", # TODO: beware!
+ "code" => "grc",
+ "name" => "ancientgreek",
+ "synonyms" => [],
+ "hyphenmin" => [1,1], # polyglosia
+ "encoding" => nil,
+ "exceptions" => false,
+ "message" => "Hyphenation patterns for Ancient Greek"
+},
+# ibycus ibyhyph.tex
+{
+ "use_new_loader" => false,
+ "use_old_patterns" => true,
+ "filename_old_patterns" => "ibyhyph.tex",
+ "filename_xu_loader" => nil,
+ "code" => "grc-x-ibycus",
+ "name" => "ibycus",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => nil,
+ "exceptions" => false,
+ "message" => "Ancient Greek Hyphenation Patterns for Ibycus encoding (v3.0)",
+},
+# bulgarian xu-bghyphen.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "filename_old_patterns" => "bghyphen.tex",
+ "filename_xu_loader" => "xu-bghyphen.tex",
+ "code" => "bg",
+ "name" => "bulgarian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "t2a",
+ "exceptions" => false,
+ "message" => "Bulgarian Hyphenation Patterns",
+},
+# russian xu-ruhyphen.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "code" => "ru",
+ "name" => "russian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "t2a",
+ "exceptions" => false,
+ "message" => "Russian Hyphenation Patterns",
+},
+# ukrainian xu-ukrhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "code" => "uk",
+ "name" => "ukrainian",
+ "synonyms" => [],
+ "hyphenmin" => [2,2],
+ "encoding" => "t2a",
+ "exceptions" => false,
+ "message" => "Ukrainian Hyphenation Patterns",
+},
+# norsk xu-nohyphbx.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "code" => "nb",
+ "name" => "bokmal",
+ "synonyms" => ["norwegian", "norsk"],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Norwegian Bokmal Hyphenation Patterns",
+},
+# nynorsk nnhyph.tex
+{
+ "use_new_loader" => true,
+ "use_old_patterns" => false,
+ "code" => "nn",
+ "name" => "nynorsk",
+ "synonyms" => [],
+ "hyphenmin" => [2,2], # babel
+ "encoding" => "ec",
+ "exceptions" => true,
+ "message" => "Norwegian Nynorsk Hyphenation Patterns",
+},
+ ]
+
+ languages.each do |l|
+ language = Language.new(l)
+ @@list.push(language)
+ self[language.code] = language
+ end
+ end
+
+ def list
+ return @@list
+ end
+end