diff options
Diffstat (limited to 'Master')
47 files changed, 6678 insertions, 54 deletions
diff --git a/Master/texmf-dist/doc/generic/hyph-utf8/sa/hyphenmin.txt b/Master/texmf-dist/doc/generic/hyph-utf8/sa/hyphenmin.txt new file mode 100644 index 00000000000..5fbd790b657 --- /dev/null +++ b/Master/texmf-dist/doc/generic/hyph-utf8/sa/hyphenmin.txt @@ -0,0 +1,11 @@ +This is the explanation of the values of hyphenmin used in Sanskrit (and possibly Indic languages) as given by Yves Codet in a private mail on March 19, 2010: + +For lefthyphenmin I think 1 is correct: in Sanskrit, if we allow breaks such as "tu-dati" ("tu" = 2 characters) we must allow "pa-tati" ("pa" = 1 character). + +For righthyphenmin I first suggested 5, as François wrote, which should be forgotten (and forgiven :). Then I thought 3 could be good; my idea was to prohibit a break before a consonant at the end of a word, such as "apata-t" (final "t" = 2 characters). But if hyphenation patterns have precedence over hyphenmins, since patterns contain rules prohibiting such breaks, that precaution is useless, and it has a drawback: it forbids breaks such as "pata-ti" ("ti" = 2 characters) or "pata-ta" ("ta" = 1 character). If patterns do have precedence, it now seems to me that the best choice is 1 for both hyphenmins (I'm sorry, François, about my hesitations :). + +But if my reasoning is sound, this would be for Sanskrit and Prakrit, in whatever script they are written. Maybe there would be reasons to choose other values for some other Indian languages, which I don't know. + +Kind regards, + +Yves diff --git a/Master/texmf-dist/doc/latex/hyph-utf8/README b/Master/texmf-dist/doc/latex/hyph-utf8/README new file mode 100644 index 00000000000..4b753902921 --- /dev/null +++ b/Master/texmf-dist/doc/latex/hyph-utf8/README @@ -0,0 +1,75 @@ +This is HYPH-UTF8 package which contains: + +- hyphenation patterns hyph-foo.tex, semi-manually converted into UTF-8 + from different sources by different authors +- pattern loaders loadhyph-foo.tex that recognise the engine + and load pattern accordingly +- converters from UTF-8 into EC/T1, QX, part of ISO Latin 3, T2A & LMC +- some scripts to assist with auto-generating the files + +Modificatios were needed in order to support native UTF-8 engines, +but functionality (hopefully) didn't change in any way, at least not intentionally. +The patterns themselves are no longer stand-alone; at least for 8-bit engines +you need to use loadhyph-foo.tex (which will load the patterns) instead. + +Modifications were done by Jonathan Kew, Mojca Miklavec & Arthur Reutenauer +with help & support from: + +- Karl Berry, who gave us free hands and all resources +- Taco Hoekwater, with useful macros +- Hans Hagen, who did the unicodifisation of patterns already long before + and helped with testing, suggestions and bug reports +- Norbert Preining, who tested & integrated patterns into TeX Live + +However, the "copyright/copyleft" owner of patterns remains the original author. + +The original patterns should eventually become obsolete, +and the author of the original file should preferraby modify +these files instead (feel free to remove credits added by TUG from those files). + +If you want to change the patterns, rather than uploading directly to CTAN, +we would be grateful if you could send it to us (http://tug.org/tex-hyphen) +or ask for credentials for SVN repository and commit it yourself; +we will then upload the whole "package" to CTAN. + +Before a new "pattern-revolution" starts, +please try to follow some guidelines if possible: + +- \lccode is *forbidden*, and I really mean it +- all the patterns should be in UTF-8 +- the only "allowed" TeX commands in patterns are: \patterns, \hyphenation, + and if you really cannot do without, also \input and \message +- in particular, please no \catcode or \lccode changes, + they belong to loadhyph-foo.tex, + and no \lefthyphenmin and \righthyphenmin, + they have no influence here and belong elsewhere +- \begingroup and/or \endinput is not needed +- feel free to do whatever you want inside comments + +We know that TeX is extremely powerful, but give a stupid parser +at least a chance to read your patterns. + +For more information see + + http://tug.org/tex-hyphen + + +Copyright 2008 by TUG + +The licence for hyph-foo.tex: Respect licences of original authors. +The licence for the rest: Use common sense when modifying and coordinate your work. + +Hyphenation patterns are essential to TeX; +whatever changes you make to these files - +make sure to coordinate your work with everyone involved. +(TeX Live, MikTeX, CTAN, patter authors, mailing list, ...) + +People come and go, TeX evolves. We are aware of that. +If we disapear one day and there is a need to change files, +do whatever needs to be done to make TeX behave in a proper +(backward compatible) way. And keep things clean & consistent. + +Things that might change in near future: +- support for multiple versions of patterns +- support for runtime loading of patterns +- support for preferred hyphenation points and discretionaries (ck -> k-k) diff --git a/Master/texmf-dist/source/generic/hyph-utf8/README b/Master/texmf-dist/source/generic/hyph-utf8/README new file mode 100644 index 00000000000..07ba26b49a4 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/README @@ -0,0 +1,61 @@ +generate-converters.rb +====================== +INPUT: +- source/generic/hyph-utf8/data/encodings/*.dat +OUTPUT: +- tex/generic/hyph-utf8/conversions/conv-utf8-*.dat + +Auto-generates conversions from UTF-8 to some particular encoding. + +Needs to be run only in case that a new encoding is added or an old encoding is fixed. + +generate-offo.rb +================ +Temporary; another version maintaned by author in Java; this one will be improved or removed. +Will be rewritten to take raw patterns as input with no need to parse. + +generate-pattern-loaders.rb +=========================== +INPUT: +- language.rb with data +OUTPUT: +- tex/generic/hyph-utf8/loadhyph/loadhyph-*.tex + +Auto-generates pattern loaders for languages that should be used in TeX Live or other distros. + +Needs to be run when a new language is added, a bug needs to be fixed or a strategy is changed. + +generate-tl-files.rb +==================== +Has been used once for the very first TL script generation. +Out-of-date and not maintaned; will probably be removed. + +generate-webpage.rb +=================== +Used to generated webpage with language overview. + +languages.rb +============ +Database with language information used for almost every script. +Needs some clean-up and some updates. + + +languages/eu/generate_patterns_eu.rb +==================================== +Generator for patterns for Basque. +Author: Juan M. Aguirregabiria, adapted by Mojca Miklavec + +languages/gl +============ +Generator for patterns for Galician. +Author: Javier A. Múgica + +languages/tr/generate_patterns_tr.rb +==================================== +Generator for patterns for Turkish. +Author: See file, modified by Mojca Miklavec + + +contributed/make-exhyph.pl +========================== +Imported script for hack related to breaking compound words. diff --git a/Master/texmf-dist/source/generic/hyph-utf8/make-exhyph.pl b/Master/texmf-dist/source/generic/hyph-utf8/contributed/make-exhyph.pl index 3fe44239d21..3fe44239d21 100644..100755 --- a/Master/texmf-dist/source/generic/hyph-utf8/make-exhyph.pl +++ b/Master/texmf-dist/source/generic/hyph-utf8/contributed/make-exhyph.pl diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertLanguageData.java b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertLanguageData.java new file mode 100644 index 00000000000..a4f70616a36 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertLanguageData.java @@ -0,0 +1,108 @@ +/* + * Copyright Simon Pepping 2009 + * + * The copyright owner licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* $Id: ConvertLanguageData.java 304 2009-11-26 07:26:55Z spepping $ */ + +package org.tug.texhyphen; + +import java.io.File; +import java.io.FileInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.net.URISyntaxException; +import java.net.URL; + +import javax.xml.transform.Result; +import javax.xml.transform.Source; +import javax.xml.transform.TransformerException; +import javax.xml.transform.TransformerFactory; +import javax.xml.transform.sax.SAXTransformerFactory; +import javax.xml.transform.sax.TransformerHandler; +import javax.xml.transform.stream.StreamResult; +import javax.xml.transform.stream.StreamSource; + +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; +import org.xml.sax.XMLReader; + +/** + * Convert language data in ruby format to XML format + */ +public final class ConvertLanguageData { + + /** + * @param languageDataPath + * @throws IOException + * @throws TransformerException + * @throws SAXException + * @throws URISyntaxException + */ + public static void convert(String languageDataPath, boolean useStylesheet) + throws IOException, TransformerException, SAXException, URISyntaxException { + + // input + InputStream inis = new FileInputStream(languageDataPath); + InputSource input = new InputSource(inis); + input.setSystemId(languageDataPath); + input.setEncoding("utf-8"); + XMLReader reader = new LanguageDataParser(); + + // output + String outPath = languageDataPath.replaceFirst("\\.rb$", ".xml"); + Result result = new StreamResult(outPath); + + // transformation + TransformerFactory tf = TransformerFactory.newInstance(); + if (!tf.getFeature(SAXTransformerFactory.FEATURE)) { + throw new TransformerException("TransformerFactory is not a SAXTransformerFactory"); + } + SAXTransformerFactory stf = (SAXTransformerFactory) tf; + TransformerHandler th; + if (useStylesheet) { + URL xsltUrl = ConvertTeXPattern.class.getResource("ConvertLanguageData.xsl"); + File xsltFile = new File(xsltUrl.toURI()); + InputStream xsltStream = new FileInputStream(xsltFile); + Source xsltSource = new StreamSource(xsltStream); + xsltSource.setSystemId(xsltFile.getAbsolutePath()); + th = stf.newTransformerHandler(xsltSource); + } else { + th = stf.newTransformerHandler(); + } + + // pipeline + reader.setContentHandler(th); + reader.setProperty("http://xml.org/sax/properties/lexical-handler", th); + th.setResult(result); + reader.parse(input); + } + + /** + * @param args + * @throws IOException + * @throws TransformerException + * @throws SAXException + * @throws URISyntaxException + */ + public static void main(String[] args) + throws IOException, TransformerException, SAXException, URISyntaxException { + if (args[0].endsWith("--debug")) { + convert(args[1], false); + } else { + convert(args[0], true); + } + } + +} diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertLanguageData.xsl b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertLanguageData.xsl new file mode 100644 index 00000000000..98aa733a0c0 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertLanguageData.xsl @@ -0,0 +1,38 @@ +<?xml version="1.0" encoding="utf-8"?> +<xsl:stylesheet version="1.0" + xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:lang="urn:org:tug:texhyphen:languagedata"> + + <xsl:output method="xml" encoding="UTF-8" indent="yes"/> + + <xsl:variable name="language-data" select="document('codemapping.xml')/lang:code-mappings"/> + + <xsl:template match="*|@*|node()"> + <xsl:copy> + <xsl:apply-templates select="*|@*|node()"/> + </xsl:copy> + </xsl:template> + + <xsl:template match="lang:language"> + <xsl:variable name="code" select="$language-data/lang:code-mapping[@code=current()/@code]"/> + <xsl:variable name="fop-code"> + <xsl:choose> + <xsl:when test="$code"> + <xsl:value-of select="$code/@fop-code"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="@code"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:copy> + <xsl:if test="string($fop-code)"> + <xsl:attribute name="fop-code"> + <xsl:value-of select="$fop-code"/> + </xsl:attribute> + </xsl:if> + <xsl:apply-templates select="*|@*|node()"/> + </xsl:copy> + </xsl:template> + +</xsl:stylesheet> diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertTeXPattern.java b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertTeXPattern.java new file mode 100644 index 00000000000..1ec9cee9474 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertTeXPattern.java @@ -0,0 +1,385 @@ +/* + * Copyright Simon Pepping 2009 + * + * The copyright owner licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* $Id: ConvertTeXPattern.java 304 2009-11-26 07:26:55Z spepping $ */ + +package org.tug.texhyphen; + +import java.io.File; +import java.io.FileInputStream; +import java.io.FileNotFoundException; +import java.io.FilenameFilter; +import java.io.IOException; +import java.io.InputStream; +import java.net.MalformedURLException; +import java.net.URI; +import java.net.URISyntaxException; +import java.net.URL; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.HashMap; +import java.util.Map; + +import javax.xml.parsers.ParserConfigurationException; +import javax.xml.parsers.SAXParser; +import javax.xml.parsers.SAXParserFactory; +import javax.xml.transform.Result; +import javax.xml.transform.Source; +import javax.xml.transform.Transformer; +import javax.xml.transform.TransformerException; +import javax.xml.transform.TransformerFactory; +import javax.xml.transform.sax.SAXTransformerFactory; +import javax.xml.transform.sax.TransformerHandler; +import javax.xml.transform.stream.StreamResult; +import javax.xml.transform.stream.StreamSource; + +import org.xml.sax.Attributes; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; +import org.xml.sax.XMLReader; +import org.xml.sax.helpers.DefaultHandler; + +/** + * Convert modern UTF8 TeX hyphenation patterns to XML format + */ +public final class ConvertTeXPattern { + + public static void convert(String[] texPatterns, String outfilePath, boolean useStylesheet, + boolean useLanguagedata) + throws IOException, TransformerException, SAXException, URISyntaxException, + ParserConfigurationException, CodeMappingException { + checkCodeMapping(); + Collection<String> languages = codeMapping.keySet(); + convert(texPatterns, outfilePath, useStylesheet, languages); + } + + public static void convert(String[] texPatterns, String outfilePath, boolean useStylesheet) + throws IOException, TransformerException, SAXException, URISyntaxException, + CodeMappingException { + convert(texPatterns, outfilePath, useStylesheet, null); + } + + /** + * infile outfile + * indir outdir (file protocol only) + * infiles outdir + * file and http protocols allowed + * + * @param texPatternUri + * @param outfilePath + * @param useStylesheet + * @param texcodes filter of requested tex codes; is allowed to be null + * @throws IOException + * @throws TransformerException + * @throws SAXException + * @throws URISyntaxException + * @throws CodeMappingException + */ + public static void convert(String[] texPatterns, String outfilePath, boolean useStylesheet, + Collection<String> texcodes) + throws IOException, TransformerException, SAXException, URISyntaxException, + CodeMappingException { + File outDir = new File(outfilePath); + boolean oneTexcode = (texcodes != null && texcodes.size() == 1); + boolean oneInputfile = (texPatterns.length == 1); + boolean oneFilteredInput = (oneTexcode || oneInputfile); + if (!oneFilteredInput && !outDir.isDirectory()) { + throw new IllegalArgumentException + ("with multiple input files the output path " + outfilePath + " must be a directory"); + } + for (String texPattern : texPatterns) { + URI texPatternUri = makeTexPatternUri(texPattern); + URI[] texPatternUris = makeTexPatternUris(texPatternUri); + oneInputfile = (texPatternUris.length == 1); + oneFilteredInput = (oneTexcode || oneInputfile); + if (!oneFilteredInput && !outDir.isDirectory()) { + throw new IllegalArgumentException + ("with an input directory " + texPattern + " the output path " + outfilePath + " must be a directory"); + } + for (URI t : texPatternUris) { + TransformationData transformationData = makeTransformationData(t, outDir, texcodes); + if (transformationData == null) { + continue; + } + doConvert(t, transformationData, useStylesheet); + } + } + } + + /** + * @param texPattern + * @return + * @throws URISyntaxException + * @throws FileNotFoundException + */ + private static URI makeTexPatternUri(String texPattern) + throws URISyntaxException, FileNotFoundException { + URI texPatternUri; + texPatternUri = new URI(texPattern); + String scheme = texPatternUri.getScheme(); + // see if it is a relative file path + if (scheme == null) { + File f = new File(texPattern); + texPatternUri = new URI("file", null, f.getAbsolutePath(), null, null); + scheme = texPatternUri.getScheme(); + } + if (scheme == null || !(scheme.equals("http") || scheme.equals("file"))) { + throw new FileNotFoundException + ("URI with file or http scheme required for hyphenation pattern file"); + } + return texPatternUri; + } + + /** + * @param outfilePath + * @param outDir + * @param texPatternUri + * @param scheme + * @return + * @throws URISyntaxException + */ + private static URI[] makeTexPatternUris(URI texPatternUri) throws URISyntaxException { + URI[] texPatternUris; + texPatternUris = new URI[] {texPatternUri}; + String scheme = texPatternUri.getScheme(); + if (scheme.equals("file")) { + File dir = new File(texPatternUri); + if (dir.isDirectory()) { + ArrayList<URI> l = new ArrayList<URI>(); + FilenameFilter filter = new FilenameFilter() { + public boolean accept(File dir, String name) { + return name.endsWith(".tex"); + } + }; + for (File f : dir.listFiles(filter)) { + l.add(new URI("file", null, f.getAbsolutePath(), null, null)); + } + texPatternUris = l.toArray(texPatternUris); + } + } + return texPatternUris; + } + + /** + * @param t + * @param outDir + * @param texcodes filter of requested tex codes; is allowed to be null + * @return + * @throws CodeMappingException + */ + private static TransformationData makeTransformationData + (URI t, File outDir, Collection<String> texcodes) throws CodeMappingException { + File outFile; + String path = t.getPath(); + String basename = path.substring(path.lastIndexOf('/') + 1); + String base = basename.substring(0, basename.lastIndexOf('.')); + // xmlCode, texCode + String[] codes = mapCode(base); + // code mapping lists no xmlCode + if (codes[0] == null) { + return null; + } + if (texcodes != null && !texcodes.contains(codes[1])) { + return null; + } + if (!outDir.isDirectory()) { + outFile = outDir; + } else { + outFile = new File(outDir, codes[0] + ".xml"); + } + return new TransformationData(outFile, codes[1]); + } + + private static class TransformationData { + File outFile; + String texCode; + TransformationData(File outFile, String texCode) { + this.outFile = outFile; + this.texCode = texCode; + } + } + + private static class CodeMappingException extends Exception { + public CodeMappingException(Exception e) { + super(e); + } + public CodeMappingException(String m) { + super(m); + } + } + + static Map<String, String> codeMapping; + static CodeMappingException codeMappingException; + static { + try { + codeMapping = readLanguagedata(); + } catch (ParserConfigurationException e) { + codeMappingException = new CodeMappingException(e); + } catch (SAXException e) { + codeMappingException = new CodeMappingException(e); + } catch (IOException e) { + codeMappingException = new CodeMappingException(e); + } + } + + private static String[] mapCode(String texCode) throws CodeMappingException { + checkCodeMapping(); + String hyp = "hyph-"; + String xmlCode = texCode; + if (texCode.startsWith(hyp)) { + texCode = texCode.substring(hyp.length()); + xmlCode = codeMapping.get(texCode); + } + return new String[] {xmlCode, texCode}; + } + + /** + * @throws CodeMappingException + */ + private static void checkCodeMapping() throws CodeMappingException { + if (codeMapping == null) { + if (codeMappingException != null) { + throw codeMappingException; + } else { + throw new CodeMappingException("Failure initializing code mapping"); + } + } + } + + public static Map<String,String> readLanguagedata() + throws ParserConfigurationException, SAXException, IOException { + SAXParserFactory spf = SAXParserFactory.newInstance(); + spf.setNamespaceAware(true); + SAXParser parser = spf.newSAXParser(); + InputStream is = ConvertTeXPattern.class.getResourceAsStream("languages.xml"); + TexcodeReader texcodeReader = new TexcodeReader(); + parser.parse(is, texcodeReader); + return texcodeReader.getTexcodes(); + } + + private static class TexcodeReader extends DefaultHandler { + + private Map<String, String> texcodes = new HashMap<String, String>(); + + /* (non-Javadoc) + * @see org.xml.sax.helpers.DefaultHandler#startElement(java.lang.String, java.lang.String, java.lang.String, org.xml.sax.Attributes) + */ + @Override + public void startElement(String uri, String localName, String qName, + Attributes attributes) throws SAXException { + if (uri.equals(LanguageDataParser.LANG_NAMESPACE) && localName.equals("language")) { + String texcode = attributes.getValue("code"); + String fopcode = attributes.getValue("fop-code"); + if (fopcode != null) { + texcodes.put(texcode, fopcode); + } + } + } + + /** + * @return the texcodes + */ + public Map<String,String> getTexcodes() { + return texcodes; + } + + } + + public static void doConvert(URI texPatternUri, TransformationData outdata, boolean useStylesheet) + throws TransformerException, SAXException, MalformedURLException, IOException, URISyntaxException { + + String scheme = texPatternUri.getScheme(); + InputStream inis = null; + if (scheme.equals("file")) { + File in = new File(texPatternUri); + inis = new FileInputStream(in); + } else if (scheme.equals("http")) { + inis = texPatternUri.toURL().openStream(); + } else { + throw new FileNotFoundException + ("URI with file or http scheme required for hyphenation pattern file"); + } + + InputSource input = new InputSource(inis); + input.setSystemId(texPatternUri.toString()); + input.setEncoding("utf-8"); + XMLReader reader = new TeXPatternParser(); + Result result = new StreamResult(outdata.outFile); + TransformerFactory tf = TransformerFactory.newInstance(); + if (!tf.getFeature(SAXTransformerFactory.FEATURE)) { + throw new TransformerException("TransformerFactory is not a SAXTransformerFactory"); + } + SAXTransformerFactory stf = (SAXTransformerFactory) tf; + TransformerHandler th; + if (useStylesheet) { + URL xsltUrl = ConvertTeXPattern.class.getResource("ConvertTeXPattern.xsl"); + File xsltFile = new File(xsltUrl.toURI()); + InputStream xsltStream = new FileInputStream(xsltFile); + Source xsltSource = new StreamSource(xsltStream); + xsltSource.setSystemId(xsltFile.getAbsolutePath()); + th = stf.newTransformerHandler(xsltSource); + Transformer tr = th.getTransformer(); + tr.setParameter("tex-code", outdata.texCode); + } else { + th = stf.newTransformerHandler(); + } + reader.setContentHandler(th); + reader.setProperty("http://xml.org/sax/properties/lexical-handler", th); + th.setResult(result); + reader.parse(input); + } + + /** + * @param args input URI, output file + * @throws URISyntaxException if the URI is not correct + * @throws IOException if a file is not found, or contains illegal content + * @throws TransformerException + * @throws SAXException + * @throws ParserConfigurationException + * @throws CodeMappingException + */ + public static void main(String[] args) + throws URISyntaxException, IOException, TransformerException, SAXException, + ParserConfigurationException, CodeMappingException { + String prefix = "--"; + int i = 0; + boolean useStylesheet = true; + boolean useLanguagedata = false; + Collection<String> texcodes = null; + while (args[i].startsWith(prefix)) { + String option = args[i].substring(prefix.length()); + if (option.equals("debug")) { + useStylesheet = false; + } else if (option.equals("uselanguagedata") || option.equals("langdata")) { + useLanguagedata = true; + } else if (option.equals("texcodes")) { + texcodes = Arrays.asList(args[++i].split(",")); + } else { + throw new IllegalArgumentException("Unknown option: " + option); + } + ++i; + } + if (texcodes != null) { + convert(Arrays.copyOfRange(args, i, args.length - 1), args[args.length - 1], + useStylesheet, texcodes); + } else { + convert(Arrays.copyOfRange(args, i, args.length - 1), args[args.length - 1], + useStylesheet, useLanguagedata); + } + } + +} diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertTeXPattern.xsl b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertTeXPattern.xsl new file mode 100644 index 00000000000..803962b7d5a --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/ConvertTeXPattern.xsl @@ -0,0 +1,128 @@ +<?xml version="1.0" encoding="UTF-8"?> +<xsl:stylesheet version="1.0" + xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:tex="urn:org:tug:texhyphen" + xmlns:lang="urn:org:tug:texhyphen:languagedata" + exclude-result-prefixes="tex lang"> + + <xsl:output doctype-system="hyphenation.dtd" indent="yes"/> + + <xsl:param name="comment-length" select="72"/> + <xsl:param name="tex-code"/> + <xsl:param name="hyphen-min-before-default" select="2"/> + <xsl:param name="hyphen-min-after-default" select="3"/> + + <xsl:template match="/tex:tex"> + <hyphenation-info> + <xsl:choose> + <xsl:when test="tex:hyphenation"> + <!-- (comment*), (patterns), (comment*, hyphenation, comment*) => (comment*), hyphen-min, (comment*, exceptions, comment*), (patterns) --> + <xsl:variable name="set1" select="node()[following-sibling::tex:patterns]"/> + <xsl:variable name="set2" select="tex:patterns"/> + <xsl:variable name="set3" select="node()[preceding-sibling::tex:patterns]"/> + <xsl:apply-templates select="$set1"/> + <xsl:call-template name="hyphen-min"/> + <xsl:apply-templates select="$set3"/> + <xsl:apply-templates select="$set2"/> + </xsl:when> + <xsl:otherwise> + <!-- (comment*), (patterns, comment*) => (comment*), hyphen-min, (patterns, comment*) --> + <xsl:variable name="set1" select="node()[following-sibling::tex:patterns]"/> + <xsl:variable name="set2" select="node()[preceding-sibling::tex:patterns or self::tex:patterns]"/> + <xsl:apply-templates select="$set1"/> + <xsl:call-template name="hyphen-min"/> + <xsl:apply-templates select="$set2"/> + </xsl:otherwise> + </xsl:choose> + </hyphenation-info> + </xsl:template> + + <xsl:template match="tex:patterns"> + <patterns> + <xsl:apply-templates /> + </patterns> + </xsl:template> + + <xsl:template match="tex:patterns" mode="call-hyphen-min"> + <xsl:call-template name="hyphen-min"/> + <patterns> + <xsl:apply-templates /> + </patterns> + </xsl:template> + + <xsl:template name="hyphen-min"> + <xsl:variable name="hyphen-min" + select="document('languages.xml')/lang:languages/lang:language[@code=$tex-code]/lang:hyphen-min" /> + <xsl:if test="count($hyphen-min)"> + <hyphen-min before="{$hyphen-min/@before}" after="{$hyphen-min/@after}" /> + </xsl:if> + </xsl:template> + + <xsl:template match="tex:message"/> + + <xsl:template match="tex:hyphenation"> + <exceptions> + <xsl:apply-templates /> + </exceptions> + </xsl:template> + + <!-- Comments in TeX contain the trailing new line. --> + <!-- Here we keep the trailing new line if the comment is immediately --> + <!-- preceded or followed by a text node. --> + <!-- Otherwise we strip the new line and pad the comment --> + <!-- to the parameter comment-length. --> + <!-- The XSLT engine uses the same criteria to decide if the comment --> + <!-- should start on a new line or not. --> + <!-- This is not quite correct, because we risk adding spaces to --> + <!-- elements with mixed content. --> + <!-- The following test would be more appropriate: --> + <!-- test="preceding-sibling::text() or following-sibling::text()". --> + <xsl:template match="comment()"> + <xsl:choose> + <xsl:when + test="preceding-sibling::node()[1][self::text()] + or following-sibling::node()[1][self::text()]"> + <xsl:comment> + <xsl:value-of select="." /> + </xsl:comment> + </xsl:when> + <xsl:otherwise> + <xsl:variable name="length" select="string-length(.)" /> + <xsl:comment> + <xsl:value-of select="substring(.,1,$length - 1)" /> + <xsl:text> </xsl:text> + <xsl:call-template name="make-spaces"> + <xsl:with-param name="length" + select="$comment-length - ($length - 1)" /> + </xsl:call-template> + </xsl:comment> + </xsl:otherwise> + </xsl:choose> + </xsl:template> + + <xsl:template name="make-spaces"> + <xsl:param name="length" select="0" /> + <xsl:choose> + <xsl:when test="$length >= 10"> + <xsl:text> </xsl:text> + <xsl:call-template name="make-spaces"> + <xsl:with-param name="length" select="$length - 10" /> + </xsl:call-template> + </xsl:when> + <xsl:when test="$length >= 5"> + <xsl:text> </xsl:text> + <xsl:call-template name="make-spaces"> + <xsl:with-param name="length" select="$length - 5" /> + </xsl:call-template> + </xsl:when> + <xsl:when test="$length >= 1"> + <xsl:text> </xsl:text> + <xsl:call-template name="make-spaces"> + <xsl:with-param name="length" select="$length - 1" /> + </xsl:call-template> + </xsl:when> + </xsl:choose> + <xsl:text></xsl:text> + </xsl:template> + +</xsl:stylesheet>
\ No newline at end of file diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/LanguageDataParser.java b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/LanguageDataParser.java new file mode 100644 index 00000000000..95c484b25ab --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/LanguageDataParser.java @@ -0,0 +1,428 @@ +/* + * Copyright Simon Pepping 2009 + * + * The copyright owner licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* $Id: LanguageDataParser.java 304 2009-11-26 07:26:55Z spepping $ */ + +package org.tug.texhyphen; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileInputStream; +import java.io.FileReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.io.Reader; +import java.net.URI; +import java.net.URISyntaxException; +import java.net.URL; +import java.net.URLConnection; +import java.util.Collection; +import java.util.Iterator; +import java.util.List; +import java.util.Vector; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import org.xml.sax.Attributes; +import org.xml.sax.ContentHandler; +import org.xml.sax.DTDHandler; +import org.xml.sax.EntityResolver; +import org.xml.sax.ErrorHandler; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; +import org.xml.sax.SAXNotRecognizedException; +import org.xml.sax.SAXNotSupportedException; +import org.xml.sax.XMLReader; +import org.xml.sax.ext.LexicalHandler; +import org.xml.sax.helpers.AttributesImpl; + +/** + * The class TeXParser parses TeX hyphenation pattern files and produces SAX events + */ +public class LanguageDataParser implements XMLReader { + + public static final String LANG_NAMESPACE = "urn:org:tug:texhyphen:languagedata"; + public static int lineLength = 72; + private static final int TOP_LEVEL = 3, IN_LANG = 4; + private static final Pattern + comment = Pattern.compile("#.*"), + langStart = Pattern.compile("{", Pattern.LITERAL), + langEnd = Pattern.compile("}", Pattern.LITERAL), + dataline = Pattern.compile("\"([^\"]+)\" ?=> ?\"([^\"]+)\","), + keywordline = Pattern.compile("\"([^\"]+)\" ?=> ?(false|true|nil),"), + listline = Pattern.compile("\"([^\"]+)\" ?=> ?\\[(\"[^\"]+\"(?:,\"[^\"]+\")*)\\],"), + datalistline = Pattern.compile("\"([^\"]+)\" ?=> ?\\[([^,]+(?:,[^,]+)*)\\],"), + space = Pattern.compile("[ \\t]+"); + private static final AttributesImpl emptyAtts = new AttributesImpl(); + + private ContentHandler contentHandler; + private DTDHandler dtdHandler; + private EntityResolver entityResolver; + private ErrorHandler errorHandler; + private LexicalHandler lexicalHandler; + + private void parseLanguageData(BufferedReader inbr) throws SAXException, IOException { + int parseState = TOP_LEVEL; + Language lang = null; + + contentHandler.startDocument(); + contentHandler.startPrefixMapping("", LANG_NAMESPACE); + contentHandler.startElement(LANG_NAMESPACE, "languages", "languages", emptyAtts); + + for (String line = inbr.readLine(); line != null; line = inbr.readLine()) { + Matcher matcher = comment.matcher(line).useAnchoringBounds(true); + int start = 0; + while (start < line.length()) { + if (matcher.usePattern(comment).lookingAt()) { + processComment(matcher.group(), parseState == TOP_LEVEL ? null : lang); + } else if (matcher.usePattern(space).lookingAt()) { + // do nothing + } else if (parseState == TOP_LEVEL && matcher.usePattern(langStart).lookingAt()) { + parseState = IN_LANG; + lang = new Language(); + } else if ((parseState == IN_LANG) && matcher.usePattern(langEnd).lookingAt()) { + pushoutLanguage(lang); + lang = null; + parseState = TOP_LEVEL; + } else if (parseState == IN_LANG + && (matcher.usePattern(dataline).lookingAt() + || matcher.usePattern(keywordline).lookingAt())) { + String key = matcher.group(1); + String value = matcher.group(2); + processDataline(key, value, lang); + } else if (parseState == IN_LANG + && (matcher.usePattern(listline).lookingAt() + || matcher.usePattern(datalistline).lookingAt())) { + String key = matcher.group(1); + String values = matcher.group(2); + processListline(key, values, lang); + } else { + break; + } + start = matcher.end(); + matcher = matcher.region(start, line.length()).useAnchoringBounds(true); + } + } + + contentHandler.endElement(LANG_NAMESPACE, "languages", "languages"); + contentHandler.endPrefixMapping(LANG_NAMESPACE); + contentHandler.endDocument(); + } + + static Collection<String> attributeKeys; + static { + attributeKeys = new Vector<String>(); + attributeKeys.add("code"); + attributeKeys.add("name"); + attributeKeys.add("use-old-patterns"); + attributeKeys.add("use-new-loader"); + attributeKeys.add("encoding"); + attributeKeys.add("exceptions"); + } + + private void processComment(String comment, Language lang) throws SAXException { + comment = comment.replace("--", "––"); + if (!comment.endsWith(" ")) { + comment = comment + " "; + } + if (lang == null) { + char[] textchars = comment.toCharArray(); + if (lexicalHandler != null) { + lexicalHandler.comment(textchars, 1, textchars.length - 1); + } + } else { + lang.elements.add(new Element("comment", comment)); + } + + } + + private void processDataline(String key, String value, Language lang) { + key = key.replace('_', '-'); + if (value.equals("nil")) { + value = ""; + } + if (attributeKeys.contains(key)) { + lang.atts.addAttribute("", key, key, "CDATA", value); + } else { + lang.elements.add(new Element(key, value)); + } + } + + private void processListline(String key, String valuesString, Language lang) { + key = key.replace('_', '-'); + valuesString = valuesString.replace("\"", ""); + String[] values = valuesString.split(",[ \\t]*"); + if (attributeKeys.contains(key)) { + StringBuilder attValue = new StringBuilder(); + for (String value : values) { + if (!value.equals("nil")) { + attValue.append(" " + value); + } + } + lang.atts.addAttribute("", key, key, "CDATA", attValue.toString()); + } else if (key.equals("hyphenmin")) { + key = "hyphen-min"; + AttributesImpl atts = new AttributesImpl(); + atts.addAttribute("", "before", "before", "CDATA", values[0]); + atts.addAttribute("", "after", "after", "CDATA", values[1]); + lang.elements.add(new Element(key, "", atts)); + } else { + key = key.replaceAll("s$", ""); + for (String value : values) { + if (value.equals("nil")) { + value = ""; + } + lang.elements.add(new Element(key, value)); + } + } + } + + private void pushoutLanguage(Language lang) throws SAXException { + contentHandler.startElement(LANG_NAMESPACE, "language", "language", lang.atts); + Iterator<Element> iter = lang.elements.iterator(); + while (iter.hasNext()) { + Element elt = iter.next(); + char[] text = elt.content.toCharArray(); + if (elt.tag.equals("comment")) { + if (lexicalHandler != null) { + lexicalHandler.comment(text, 1, text.length - 1); + } + } else { + contentHandler.startElement(LANG_NAMESPACE, elt.tag, elt.tag, elt.atts); + contentHandler.characters(text, 0, text.length); + contentHandler.endElement(LANG_NAMESPACE, elt.tag, elt.tag); + } + } + contentHandler.endElement(LANG_NAMESPACE, "language", "language"); + } + + public Reader getReaderFromInputSource(InputSource input) throws IOException { + Reader reader = input.getCharacterStream(); + String encoding = null; + if (reader == null) { + encoding = input.getEncoding(); + } + if (reader == null) { + InputStream stream = input.getByteStream(); + if (stream != null) { + if (encoding == null) { + reader = new InputStreamReader(stream); + } else { + reader = new InputStreamReader(stream, encoding); + } + } + } + if (reader == null) { + String systemId = input.getSystemId(); + reader = getReaderFromSystemId(systemId, encoding); + } + return reader; + } + + public Reader getReaderFromSystemId(String systemId, String encoding) throws IOException { + if (systemId == null) { + throw new IOException("Cannot create a reader from a null systemID"); + } + if (encoding.isEmpty()) { + encoding = null; + } + Reader reader = null; + URI uri = null; + File file = null; + try { + uri = new URI(systemId); + } catch (URISyntaxException e) { + // handled below + } + if (uri == null || !uri.isAbsolute()) { + file = new File(systemId); + } + if (file != null) { + if (encoding == null) { + reader = new FileReader(file); + } else { + InputStream stream = new FileInputStream(file); + reader = new InputStreamReader(stream, encoding); + } + } else if (uri != null && uri.getScheme().equals("http")) { + URL url = uri.toURL(); + URLConnection conn = url.openConnection(); + if (encoding == null) { + encoding = conn.getContentEncoding(); + } + InputStream stream = conn.getInputStream(); + reader = new InputStreamReader(stream, encoding); + } + return reader; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getContentHandler() + */ + public ContentHandler getContentHandler() { + return contentHandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getDTDHandler() + */ + public DTDHandler getDTDHandler() { + return dtdHandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getEntityResolver() + */ + public EntityResolver getEntityResolver() { + return entityResolver; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getErrorHandler() + */ + public ErrorHandler getErrorHandler() { + return errorHandler; + } + + + /** + * @return the lexicalHandler + */ + public LexicalHandler getLexicalHandler() { + return lexicalHandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getFeature(java.lang.String) + */ + public boolean getFeature(String arg0) + throws SAXNotRecognizedException, SAXNotSupportedException { + throw new SAXNotSupportedException(); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getProperty(java.lang.String) + */ + public Object getProperty(String arg0) + throws SAXNotRecognizedException, SAXNotSupportedException { + throw new SAXNotSupportedException(); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#parse(org.xml.sax.InputSource) + */ + public void parse(InputSource input) throws IOException, SAXException { + Reader reader = getReaderFromInputSource(input); + if (reader == null) { + throw new IOException("Could not open input source " + input); + } + BufferedReader inbr = new BufferedReader(reader); + parseLanguageData(inbr); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#parse(java.lang.String) + */ + public void parse(String systemId) throws IOException, SAXException { + Reader reader = getReaderFromSystemId(systemId, null); + if (reader == null) { + throw new IOException("Could not open input systemID " + systemId); + } + BufferedReader inbr = new BufferedReader(reader); + parseLanguageData(inbr); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setContentHandler(org.xml.sax.ContentHandler) + */ + public void setContentHandler(ContentHandler contenthandler) { + this.contentHandler = contenthandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setDTDHandler(org.xml.sax.DTDHandler) + */ + public void setDTDHandler(DTDHandler dtdhandler) { + this.dtdHandler = dtdhandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setEntityResolver(org.xml.sax.EntityResolver) + */ + public void setEntityResolver(EntityResolver entityresolver) { + this.entityResolver = entityresolver; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setErrorHandler(org.xml.sax.ErrorHandler) + */ + public void setErrorHandler(ErrorHandler errorHandler) { + this.errorHandler = errorHandler; + } + + + /** + * @param lexicalHandler the lexicalHandler to set + */ + public void setLexicalHandler(LexicalHandler lexicalHandler) { + this.lexicalHandler = lexicalHandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setFeature(java.lang.String, boolean) + */ + public void setFeature(String arg0, boolean arg1) + throws SAXNotRecognizedException, SAXNotSupportedException { + throw new SAXNotSupportedException(); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setProperty(java.lang.String, java.lang.Object) + */ + public void setProperty(String name, Object value) + throws SAXNotRecognizedException, SAXNotSupportedException { + if (name.equals("http://xml.org/sax/properties/lexical-handler")) { + lexicalHandler = (LexicalHandler) value; + } else { + throw new SAXNotSupportedException(); + } + } + + private static class Element { + String tag; + String content; + Attributes atts; + Element(String tag, String content) { + this(tag, content, LanguageDataParser.emptyAtts); + } + Element(String tag, String content, Attributes atts) { + this.tag = tag; + this.content = content; + this.atts = atts; + } + } + + private static class Language { + AttributesImpl atts; + List<Element> elements; + Language() { + atts = new AttributesImpl(); + elements = new Vector<Element>(); + } + } + +} diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/README b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/README new file mode 100644 index 00000000000..aaf4e074ae6 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/README @@ -0,0 +1,38 @@ +This utility consists of two parts. + +Part 1 converts languages.rb to languages.xml. It consists of +ConvertLanguageData.java, LanguageDataParser.java, +ConvertLanguageData.xsl, codemapping.xml. Invocation: + +java org.tug.texhyphen.util.ConvertLanguageData path/to/languages.rb + +output: languages.xml, in the same directory as languages.rb. + +The file codemapping contains the mapping from tex-code to +fop-code. If this file has an entry with non-empty fop-code, it is +inserted in languages.xml as attribute fop-code. If this file has an +entry with empty fop-code, that language gets no attribute +fop-code. Otherwise, that language gets attribute fop-code with the +same value as tex-code (attribute code). + +The current file languages.xml was modified manually after conversion, +to add the three languages at the bottom of the file. + +Part 2 converts tex pattern files to fop pattern files. It consists of +ConvertTeXPattern.java, TeXPatternParser.java, ConvertTeXPattern.xsl, +languages.xml. Invocation: + +java org.tug.texhyphen.util.ConvertTeXPattern [--uselanguagedata] +[--texcodes code1,code2,...] infileuri1 [infileuri2 ...] outfile + +infileuri must be a file or http URI. With a file URI, it may also be +a directory; in that case all pattern files in that directory are +converted. outfile must be a path to an output filename or +directory. If there are more input files, it must be an output +directory. Option 'uselanguagedata' converts all input files which +represent a language in the languages file with a fop-code. Option +'--texcodes code1,code2,...' converts only those input files which +represent a language with one of the given tex codes. Without those +options all input files are converted. In any case, only files are +converted which represent a language in the languages file with a +fop-code. diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/TeXPatternParser.java b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/TeXPatternParser.java new file mode 100644 index 00000000000..0cc5a542f78 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/TeXPatternParser.java @@ -0,0 +1,325 @@ +/* + * Copyright Simon Pepping 2009 + * + * The copyright owner licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* $Id: TeXPatternParser.java 304 2009-11-26 07:26:55Z spepping $ */ + +package org.tug.texhyphen; + +import java.io.BufferedReader; +import java.io.File; +import java.io.FileInputStream; +import java.io.FileReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.io.Reader; +import java.net.URI; +import java.net.URISyntaxException; +import java.net.URL; +import java.net.URLConnection; +import java.util.Stack; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import org.xml.sax.ContentHandler; +import org.xml.sax.DTDHandler; +import org.xml.sax.EntityResolver; +import org.xml.sax.ErrorHandler; +import org.xml.sax.InputSource; +import org.xml.sax.SAXException; +import org.xml.sax.SAXNotRecognizedException; +import org.xml.sax.SAXNotSupportedException; +import org.xml.sax.XMLReader; +import org.xml.sax.ext.LexicalHandler; +import org.xml.sax.helpers.AttributesImpl; + +/** + * The class TeXParser parses TeX hyphenation pattern files and produces SAX events + */ +public class TeXPatternParser implements XMLReader { + + public static final String TEX_NAMESPACE = "urn:org:tug:texhyphen"; + private static final int TOP_LEVEL = 3, IN_COMMAND = 4, AFTER_COMMAND = 5, IN_DATA = 6; + private static final Pattern + comment = Pattern.compile("%.*"), + commandStart = Pattern.compile("\\\\"), + command = Pattern.compile("[a-zA-Z]+"), + space = Pattern.compile(" +"), + argOpen = Pattern.compile("\\{"), + argClose = Pattern.compile("\\}"), + text = Pattern.compile("[^%\\\\\\{\\}]+"); + private static final AttributesImpl emptyAtts = new AttributesImpl(); + + private ContentHandler contentHandler; + private DTDHandler dtdHandler; + private EntityResolver entityResolver; + private ErrorHandler errorHandler; + private LexicalHandler lexicalHandler; + + private void parsePatterns(BufferedReader inbr) throws SAXException, IOException { + int parseState = TOP_LEVEL; + Stack<String> stack = new Stack<String>(); + + contentHandler.startDocument(); + contentHandler.startPrefixMapping("", TEX_NAMESPACE); + contentHandler.startElement(TEX_NAMESPACE, "tex", "tex", emptyAtts); + + for (String line = inbr.readLine(); line != null; line = inbr.readLine()) { + Matcher matcher = comment.matcher(line).useAnchoringBounds(true); + int start = 0; + char[] textchars; + boolean inComment = false; + while (start < line.length()) { + if (matcher.usePattern(comment).lookingAt()) { + String text = matcher.group().replace("--", "––"); + textchars = (text + "\n").toCharArray(); + if (lexicalHandler != null) { + lexicalHandler.comment(textchars, 1, textchars.length - 1); + } + inComment = true; + } else if (parseState == IN_DATA && matcher.usePattern(text).lookingAt()) { + textchars = matcher.group().toCharArray(); + contentHandler.characters(textchars, 0, textchars.length); + } else if (parseState != IN_DATA && matcher.usePattern(space).lookingAt()) { + if (parseState == TOP_LEVEL) { + textchars = matcher.group().toCharArray(); + contentHandler.ignorableWhitespace(textchars, 0, textchars.length); + } + } else if (parseState == TOP_LEVEL && matcher.usePattern(commandStart).lookingAt()) { + parseState = IN_COMMAND; + } else if (parseState == IN_COMMAND && matcher.usePattern(command).lookingAt()) { + String tag = matcher.group(); + contentHandler.startElement(TEX_NAMESPACE, tag, tag, emptyAtts); + stack.push(tag); + parseState = AFTER_COMMAND; + } else if (parseState == AFTER_COMMAND && matcher.usePattern(argOpen).lookingAt()) { + parseState = IN_DATA; + } else if (parseState == IN_DATA && matcher.usePattern(argClose).lookingAt()) { + String tag = stack.pop(); + contentHandler.endElement(TEX_NAMESPACE, tag, tag); + parseState = TOP_LEVEL; + } else { + break; + } + start = matcher.end(); + matcher = matcher.region(start, line.length()).useAnchoringBounds(true); + } + textchars = "\n".toCharArray(); + if (parseState == IN_DATA && !inComment) { + contentHandler.characters(textchars, 0, textchars.length); + } else if (parseState == TOP_LEVEL && !inComment) { + contentHandler.ignorableWhitespace(textchars, 0, textchars.length); + } + } + + contentHandler.endElement(TEX_NAMESPACE, "tex", "tex"); + contentHandler.endPrefixMapping(TEX_NAMESPACE); + contentHandler.endDocument(); + } + + public Reader getReaderFromInputSource(InputSource input) throws IOException { + Reader reader = input.getCharacterStream(); + String encoding = null; + if (reader == null) { + encoding = input.getEncoding(); + } + if (reader == null) { + InputStream stream = input.getByteStream(); + if (stream != null) { + if (encoding == null) { + reader = new InputStreamReader(stream); + } else { + reader = new InputStreamReader(stream, encoding); + } + } + } + if (reader == null) { + String systemId = input.getSystemId(); + reader = getReaderFromSystemId(systemId, encoding); + } + return reader; + } + + public Reader getReaderFromSystemId(String systemId, String encoding) throws IOException { + if (systemId == null) { + throw new IOException("Cannot create a reader from a null systemID"); + } + if (encoding.isEmpty()) { + encoding = null; + } + Reader reader = null; + URI uri = null; + File file = null; + try { + uri = new URI(systemId); + } catch (URISyntaxException e) { + // handled below + } + if (uri == null || !uri.isAbsolute()) { + file = new File(systemId); + } + if (file != null) { + if (encoding == null) { + reader = new FileReader(file); + } else { + InputStream stream = new FileInputStream(file); + reader = new InputStreamReader(stream, encoding); + } + } else if (uri != null && uri.getScheme().equals("http")) { + URL url = uri.toURL(); + URLConnection conn = url.openConnection(); + if (encoding == null) { + encoding = conn.getContentEncoding(); + } + InputStream stream = conn.getInputStream(); + reader = new InputStreamReader(stream, encoding); + } + return reader; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getContentHandler() + */ + public ContentHandler getContentHandler() { + return contentHandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getDTDHandler() + */ + public DTDHandler getDTDHandler() { + return dtdHandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getEntityResolver() + */ + public EntityResolver getEntityResolver() { + return entityResolver; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getErrorHandler() + */ + public ErrorHandler getErrorHandler() { + return errorHandler; + } + + + /** + * @return the lexicalHandler + */ + public LexicalHandler getLexicalHandler() { + return lexicalHandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getFeature(java.lang.String) + */ + public boolean getFeature(String arg0) + throws SAXNotRecognizedException, SAXNotSupportedException { + throw new SAXNotSupportedException(); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#getProperty(java.lang.String) + */ + public Object getProperty(String arg0) + throws SAXNotRecognizedException, SAXNotSupportedException { + throw new SAXNotSupportedException(); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#parse(org.xml.sax.InputSource) + */ + public void parse(InputSource input) throws IOException, SAXException { + Reader reader = getReaderFromInputSource(input); + if (reader == null) { + throw new IOException("Could not open input source " + input); + } + BufferedReader inbr = new BufferedReader(reader); + parsePatterns(inbr); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#parse(java.lang.String) + */ + public void parse(String systemId) throws IOException, SAXException { + Reader reader = getReaderFromSystemId(systemId, null); + if (reader == null) { + throw new IOException("Could not open input systemID " + systemId); + } + BufferedReader inbr = new BufferedReader(reader); + parsePatterns(inbr); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setContentHandler(org.xml.sax.ContentHandler) + */ + public void setContentHandler(ContentHandler contenthandler) { + this.contentHandler = contenthandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setDTDHandler(org.xml.sax.DTDHandler) + */ + public void setDTDHandler(DTDHandler dtdhandler) { + this.dtdHandler = dtdhandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setEntityResolver(org.xml.sax.EntityResolver) + */ + public void setEntityResolver(EntityResolver entityresolver) { + this.entityResolver = entityresolver; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setErrorHandler(org.xml.sax.ErrorHandler) + */ + public void setErrorHandler(ErrorHandler errorHandler) { + this.errorHandler = errorHandler; + } + + + /** + * @param lexicalHandler the lexicalHandler to set + */ + public void setLexicalHandler(LexicalHandler lexicalHandler) { + this.lexicalHandler = lexicalHandler; + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setFeature(java.lang.String, boolean) + */ + public void setFeature(String arg0, boolean arg1) + throws SAXNotRecognizedException, SAXNotSupportedException { + throw new SAXNotSupportedException(); + } + + /* (non-Javadoc) + * @see org.xml.sax.XMLReader#setProperty(java.lang.String, java.lang.Object) + */ + public void setProperty(String name, Object value) + throws SAXNotRecognizedException, SAXNotSupportedException { + if (name.equals("http://xml.org/sax/properties/lexical-handler")) { + lexicalHandler = (LexicalHandler) value; + } else { + throw new SAXNotSupportedException(); + } + } + +} diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/codemapping.xml b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/codemapping.xml new file mode 100644 index 00000000000..c5b64722fa0 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/codemapping.xml @@ -0,0 +1,23 @@ +<?xml version="1.0" encoding="utf-8"?> +<code-mappings xmlns="urn:org:tug:texhyphen:languagedata"> + <code-mapping code="grc-x-ibycus" fop-code="grc_X_ibycus"/> + <code-mapping code="zh-latn" fop-code="zh_Latn"/> + <code-mapping code="de-1901" fop-code="de_1901"/> + <code-mapping code="de-1996" fop-code="de"/> + <code-mapping code="de-ch-1901" fop-code="de_CH_1901"/> + <code-mapping code="el-monoton" fop-code="el"/> + <code-mapping code="el-polyton" fop-code="el_Polyton"/> + <code-mapping code="mn-cyrl" fop-code="mn"/> + <code-mapping code="mn-cyrl-x-2a" fop-code="mn_Cyrl_x_2a"/> + <code-mapping code="sh-cyrl" fop-code="sh_Cyrl"/> + <code-mapping code="sh-latn" fop-code="sh_Latn"/> + <code-mapping code="sr-cyrl" fop-code="sr"/> + <code-mapping code="sr-latn" fop-code="sr_Latn"/> + <code-mapping code="en-gb" fop-code="en_GB"/> + <code-mapping code="en-us" fop-code="en"/> + <code-mapping code="eo" fop-code=""/> + <code-mapping code="nn" fop-code=""/> + <code-mapping code="nb" fop-code=""/> + <!-- hu patterns cause a stack overflow when compiled with FOP --> + <code-mapping code="hu" fop-code=""/> +</code-mappings> diff --git a/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/languages.xml b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/languages.xml new file mode 100644 index 00000000000..2f0fd637d3c --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/conversion-to-xml/org/tug/texhyphen/languages.xml @@ -0,0 +1,434 @@ +<?xml version="1.0" encoding="UTF-8"?> +<languages xmlns="urn:org:tug:texhyphen:languagedata"> + <!-- –––––––––––––––––––––––––––––––––––––– --> + <!-- languages with no hyphenation patterns --> + <!-- –––––––––––––––––––––––––––––––––––––– --> + <!-- arabic zerohyph.tex --> + <language fop-code="ar" code="ar" name="arabic" use-new-loader="false" use-old-patterns="false" encoding="" exceptions="false"> + <filename-old-patterns>zerohyph.tex</filename-old-patterns> + <!-- "hyphenmin" => [], # not needed --> + <message/> + </language> + <!-- farsi zerohyph.tex --> + <!-- =persian --> + <language fop-code="fa" code="fa" name="farsi" use-new-loader="false" use-old-patterns="false" encoding="" exceptions="false"> + <synonym>persian</synonym> + <filename-old-patterns>zerohyph.tex</filename-old-patterns> + <!-- "hyphenmin" => [], # not needed --> + <message/> + </language> + <!-- ––––––––––––––––––––––––––––––- --> + <!-- special patterns, not converted --> + <!-- ––––––––––––––––––––––––––––––- --> + <!-- ibycus ibyhyph.tex --> + <language fop-code="grc_X_ibycus" code="grc-x-ibycus" name="ibycus" use-new-loader="false" use-old-patterns="true" encoding="" exceptions="false"> + <filename-old-patterns>ibyhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Ancient Greek Hyphenation Patterns for Ibycus encoding (v3.0)</message> + <author>peter_heslin</author> + </language> + <!-- –––––––––––––––––––––––––––– --> + <!-- languages using old patterns --> + <!-- –––––––––––––––––––––––––––– --> + <!-- greek xu-grphyph4.tex --> + <!-- =polygreek --> + <language fop-code="el_Polyton" code="el-polyton" name="greek" use-new-loader="true" use-old-patterns="true" encoding="" exceptions="true"> + <synonym>polygreek</synonym> + <use-old-patterns-comment>Old patterns work in a different way, one-to-one conversion from UTF-8 is not possible.</use-old-patterns-comment> + <filename-old-patterns>grphyph5.tex</filename-old-patterns> + <!-- left/right hyphen min for Greek can be as low as one (1), --> + <!-- but for aesthetic reasons keep them at 2/2. --> + <!-- Dimitrios Filippou --> + <hyphen-min before="1" after="1"/> + <!-- polyglosia --> + <!-- "message" => "Polytonic Greek Hyphenation Patterns", --> + <message>Hyphenation patterns for multi-accent (polytonic) Modern Greek</message> + <author>dimitrios_filippou</author> + </language> + <!-- monogreek xu-grmhyph4.tex --> + <language fop-code="el" code="el-monoton" name="monogreek" use-new-loader="true" use-old-patterns="true" encoding="" exceptions="true"> + <use-old-patterns-comment>Old patterns work in a different way, one-to-one conversion from UTF-8 is not possible.</use-old-patterns-comment> + <filename-old-patterns>grmhyph5.tex</filename-old-patterns> + <hyphen-min before="1" after="1"/> + <!-- polyglosia --> + <!-- "message" => "Monotonic Greek Hyphenation Patterns", --> + <message>Hyphenation patterns for uni-accent (monotonic) Modern Greek</message> + <author>dimitrios_filippou</author> + </language> + <!-- ancientgreek xu-grahyph4.tex --> + <language fop-code="grc" code="grc" name="ancientgreek" use-new-loader="true" use-old-patterns="true" encoding="" exceptions="false"> + <use-old-patterns-comment>Old patterns work in a different way, one-to-one conversion from UTF-8 is not possible.</use-old-patterns-comment> + <filename-old-patterns>grahyph5.tex</filename-old-patterns> + <hyphen-min before="1" after="1"/> + <!-- polyglosia --> + <message>Hyphenation patterns for Ancient Greek</message> + <author>dimitrios_filippou</author> + </language> + <!-- coptic xu-copthyph.tex --> + <language fop-code="cop" code="cop" name="coptic" use-new-loader="true" use-old-patterns="true" encoding="" exceptions="false"> + <use-old-patterns-comment>TODO: automatic conversion could be done, but was too complicated; leave for later.</use-old-patterns-comment> + <filename-old-patterns>copthyph.tex</filename-old-patterns> + <hyphen-min before="1" after="1"/> + <!-- polyglosia TODO: no documentation found --> + <message>Coptic Hyphenation Patterns</message> + <author>claudio_beccari</author> + </language> + <!-- german xu-dehypht.tex --> + <language fop-code="de_1901" code="de-1901" name="german" use-new-loader="true" use-old-patterns="true" encoding="ec" exceptions="false"> + <use-old-patterns-comment>Kept for the sake of backward compatibility, but newer and better patterns by WL are available.</use-old-patterns-comment> + <filename-old-patterns>dehypht.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>German Hyphenation Patterns (Traditional Orthography)</message> + </language> + <!-- ngerman xu-dehyphn.tex --> + <language fop-code="de" code="de-1996" name="ngerman" use-new-loader="true" use-old-patterns="true" encoding="ec" exceptions="false"> + <use-old-patterns-comment>Kept for the sake of backward compatibility, but newer and better patterns by WL are available.</use-old-patterns-comment> + <filename-old-patterns>dehyphn.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>German Hyphenation Patterns (Reformed Orthography)</message> + </language> + <!-- swissgerman --> + <language fop-code="de_CH_1901" code="de-ch-1901" name="swissgerman" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <!-- TODO: how is it going to be called --> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Swiss-German Hyphenation Patterns (Traditional Orthography)</message> + </language> + <!-- russian xu-ruhyphen.tex --> + <language fop-code="ru" code="ru" name="russian" use-new-loader="true" use-old-patterns="true" encoding="t2a" exceptions="false"> + <use-old-patterns-comment>The old system allows choosing patterns and encodings manually. That mechanism needs to be implemented first in this package, so we still fall back on old system.</use-old-patterns-comment> + <filename-old-patterns>ruhyphen.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Russian Hyphenation Patterns</message> + </language> + <!-- ukrainian xu-ukrhyph.tex --> + <language fop-code="uk" code="uk" name="ukrainian" use-new-loader="true" use-old-patterns="true" encoding="t2a" exceptions="false"> + <use-old-patterns-comment>The old system allows choosing patterns and encodings manually. That mechanism needs to be implemented first in this package, so we still fall back on old system.</use-old-patterns-comment> + <filename-old-patterns>ukrhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Ukrainian Hyphenation Patterns</message> + </language> + <!-- –––––––––––––––––––––––––––– --> + <!-- languages using new patterns --> + <!-- –––––––––––––––––––––––––––– --> + <!-- catalan cahyph.tex --> + <language fop-code="ca" code="ca" name="catalan" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <filename-old-patterns>cahyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Catalan Hyphenation Patterns</message> + </language> + <!-- czech xu-czhyph.tex --> + <language fop-code="cs" code="cs" name="czech" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <filename-old-patterns>czhyph.tex</filename-old-patterns> + <filename-old-patterns-other>czhyphen.tex</filename-old-patterns-other> + <filename-old-patterns-other>czhyphen.ex</filename-old-patterns-other> + <!-- Both Czech and Slovak: \lefthyphenmin=2, \righthyphenmin=3 --> + <!-- Typographical rules allow \righthyphenmin=2 when typesetting in a --> + <!-- narrow column (newspapers etc.). --> + <!-- (used to be 2,2) --> + <hyphen-min before="2" after="3"/> + <message>Czech Hyphenation Patterns (Pavel Sevecek, v3, 1995)</message> + </language> + <!-- slovak xu-skhyph.tex --> + <language fop-code="sk" code="sk" name="slovak" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <filename-old-patterns>skhyph.tex</filename-old-patterns> + <filename-old-patterns-other>skhyphen.tex</filename-old-patterns-other> + <filename-old-patterns-other>skhyphen.ex</filename-old-patterns-other> + <!-- see czech --> + <hyphen-min before="2" after="3"/> + <message>Slovak Hyphenation Patterns (Jana Chlebikova, 1992)</message> + </language> + <!-- welsh cyhyph.tex --> + <language fop-code="cy" code="cy" name="welsh" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>cyhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="3"/> + <message>Welsh Hyphenation Patterns</message> + </language> + <!-- danish dkhyph.tex --> + <language fop-code="da" code="da" name="danish" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>dkhyph.tex</filename-old-patterns> + <filename-old-patterns-other>dkcommon.tex</filename-old-patterns-other> + <filename-old-patterns-other>dkspecial.tex</filename-old-patterns-other> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Danish Hyphenation Patterns</message> + </language> + <!-- esperanto xu-eohyph.tex --> + <!-- TODO --> + <language code="eo" name="esperanto" use-new-loader="true" use-old-patterns="false" encoding="il3" exceptions="false"> + <filename-old-patterns>eohyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- TODO --> + <message>Esperanto Hyphenation Patterns</message> + </language> + <!-- spanish xu-eshyph.tex --> + <!-- =espanol --> + <language fop-code="es" code="es" name="spanish" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <synonym>espanol</synonym> + <filename-old-patterns>eshyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Spanish Hyphenation Patterns</message> + </language> + <!-- basque xu-bahyph.tex --> + <language fop-code="eu" code="eu" name="basque" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>bahyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Basque Hyphenation Patterns</message> + </language> + <!-- french xu-frhyph.tex --> + <!-- =patois --> + <!-- =francais --> + <language fop-code="fr" code="fr" name="french" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <synonym>patois</synonym> + <synonym>francais</synonym> + <filename-old-patterns>frhyph.tex</filename-old-patterns> + <!-- "hyphenmin" => [], --> + <message>French hyphenation patterns (V2.12, 2002/12/11)</message> + </language> + <!-- galician xu-glhyph.tex --> + <language fop-code="gl" code="gl" name="galician" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>glhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Galician Hyphenation Patterns</message> + </language> + <!-- estonian xu-ethyph.tex --> + <language fop-code="et" code="et" name="estonian" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>ethyph.tex</filename-old-patterns> + <hyphen-min before="2" after="3"/> + <!-- babel --> + <message>Estonian Hyphenation Patterns</message> + </language> + <!-- finnish fihyph.tex --> + <language fop-code="fi" code="fi" name="finnish" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>fihyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Finnish Hyphenation Patterns</message> + </language> + <!-- croatian xu-hrhyph.tex --> + <language fop-code="hr" code="hr" name="croatian" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>hrhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Croatian Hyphenation Patterns</message> + </language> + <!-- hungarian xu-huhyphn.tex --> + <language code="hu" name="hungarian" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>huhyphn.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- polyglosia --> + <message>Hungarian Hyphenation Patterns (v20031107)</message> + </language> + <!-- interlingua iahyphen.tex --> + <language fop-code="ia" code="ia" name="interlingua" use-new-loader="true" use-old-patterns="false" encoding="ascii" exceptions="true"> + <filename-old-patterns>iahyphen.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Hyphenation Patterns for Interlingua</message> + </language> + <!-- indonesian inhyph.tex --> + <language fop-code="id" code="id" name="indonesian" use-new-loader="true" use-old-patterns="false" encoding="ascii" exceptions="true"> + <filename-old-patterns>inhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Indonesian Hyphenation Patterns</message> + </language> + <!-- icelandic icehyph.tex --> + <language fop-code="is" code="is" name="icelandic" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>icehyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Icelandic Hyphenation Patterns</message> + </language> + <!-- irish gahyph.tex --> + <language fop-code="ga" code="ga" name="irish" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <filename-old-patterns>gahyph.tex</filename-old-patterns> + <hyphen-min before="2" after="3"/> + <!-- babel --> + <message>Irish Hyphenation Patterns</message> + </language> + <!-- italian ithyph.tex --> + <language fop-code="it" code="it" name="italian" use-new-loader="true" use-old-patterns="false" encoding="ascii" exceptions="false"> + <filename-old-patterns>ithyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Italian Hyphenation Patterns</message> + </language> + <!-- kurmanji --> + <language fop-code="kmr" code="kmr" name="kurmanji" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>kmrhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Kurmanji Hyphenation Patterns (v. 1.0 2009/06/29 JKn and MSh)</message> + </language> + <!-- latin xu-lahyph.tex --> + <language fop-code="la" code="la" name="latin" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <use-old-patterns-comment>Old patterns support both EC & OT1 encodings at the same time.</use-old-patterns-comment> + <filename-old-patterns>lahyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Latin Hyphenation Patterns</message> + </language> + <!-- lithuanian --> + <language fop-code="lt" code="lt" name="lithuanian" use-new-loader="true" use-old-patterns="false" encoding="l7x" exceptions="false"> + <hyphen-min before="2" after="2"/> + <message>Lithuanian Hyphenation Patterns</message> + </language> + <!-- latvian --> + <language fop-code="lv" code="lv" name="latvian" use-new-loader="true" use-old-patterns="false" encoding="l7x" exceptions="false"> + <hyphen-min before="2" after="2"/> + <message>Latvian Hyphenation Patterns</message> + </language> + <!-- dutch nehyph96.tex --> + <language fop-code="nl" code="nl" name="dutch" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <filename-old-patterns>nehyph96.tex</filename-old-patterns> + <!-- quoting Hans Hagen: --> + <!-- patterns generated with 2,2 (so don't go less) but use prefered values 2,3 (educational publishers want 4,5 -) --> + <hyphen-min before="2" after="2"/> + <message>Dutch Hyphenation Patterns</message> + </language> + <!-- norsk xu-nohyphbx.tex --> + <!-- =norwegian --> + <!-- nynorsk nnhyph.tex --> + <!-- bokmal nbhyph.tex --> + <!-- polish xu-plhyph.tex --> + <language fop-code="pl" code="pl" name="polish" use-new-loader="true" use-old-patterns="false" encoding="qx" exceptions="true"> + <filename-old-patterns>plhyph.tex</filename-old-patterns> + <!--{}"hyphenmin" => [1,1], --> + <hyphen-min before="2" after="2"/> + <message>Polish Hyphenation Patterns</message> + </language> + <!-- portuguese pthyph.tex --> + <!-- =portuges --> + <language fop-code="pt" code="pt" name="portuguese" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <synonym>portuges</synonym> + <filename-old-patterns>pthyph.tex</filename-old-patterns> + <hyphen-min before="2" after="3"/> + <!-- babel --> + <message>Portuguese Hyphenation Patterns</message> + </language> + <!-- pinyin xu-pyhyph.tex --> + <language fop-code="zh_Latn" code="zh-latn" name="pinyin" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>pyhyph.tex</filename-old-patterns> + <hyphen-min before="1" after="1"/> + <message>Hyphenation patterns for unaccented pinyin syllables (CJK 4.8.0)</message> + </language> + <!-- romanian xu-rohyphen.tex --> + <language fop-code="ro" code="ro" name="romanian" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>rohyphen.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Romanian Hyphenation Patterns</message> + <!-- : `rohyphen' 1.1 <29.10.1996> --> + </language> + <!-- slovenian xu-sihyph.tex --> + <!-- =slovene --> + <language fop-code="sl" code="sl" name="slovenian" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <synonym>slovene</synonym> + <filename-old-patterns>sihyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Slovenian Hyphenation Patterns</message> + </language> + <!-- uppersorbian xu-sorhyph.tex --> + <language fop-code="hsb" code="hsb" name="uppersorbian" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <filename-old-patterns>sorhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Upper Sorbian Hyphenation Patterns (E. Werner)</message> + <!-- \message{Hyphenation patterns for Upper Sorbian, E. Werner} --> + <!-- \message{Completely new revision 1997, March 22} --> + </language> + <!-- swedish svhyph.tex --> + <language fop-code="sv" code="sv" name="swedish" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>svhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- patters say it could be 1,2; babel says 2,2 - double check --> + <message>Swedish hyphenation patterns (Jan Michael Rynning, 1994-03-03)</message> + </language> + <!-- turkish xu-tkhyph.tex --> + <language fop-code="tr" code="tr" name="turkish" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="false"> + <filename-old-patterns>tkhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- polyglosia --> + <message>Turkish Hyphenation Patterns</message> + </language> + <!-- ukenglish ukhyphen.tex --> + <!-- TODO - should we rename it or not? --> + <language fop-code="en_GB" code="en-gb" name="ukenglish" use-new-loader="true" use-old-patterns="false" encoding="ascii" exceptions="true"> + <synonym>british</synonym> + <synonym>UKenglish</synonym> + <filename-old-patterns>ukhyphen.tex</filename-old-patterns> + <hyphen-min before="2" after="3"/> + <!-- confirmed, same as what Knuth says --> + <message>Hyphenation Patterns for British English</message> + </language> + <!-- serbian xu-srhyphc.tex --> + <language fop-code="sr_Latn" code="sr-latn" name="serbian" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <filename-old-patterns>shhyphl.tex</filename-old-patterns> + <!-- It is allowed to leave one character at the end of the row. --> + <!-- However, if you think that it is graphicaly not very --> + <!-- pleasent these patterns will work well with \lefthyphenmin=2. --> + <!-- \lefthyphenmin=1 \righthyphenmin=2 --> + <hyphen-min before="2" after="2"/> + <message>Serbian hyphenation patterns in Latin script</message> + </language> + <!-- serbianc --> + <language fop-code="sr" code="sr-cyrl" name="serbianc" use-new-loader="true" use-old-patterns="false" encoding="t2a" exceptions="true"> + <filename-old-patterns>srhyphc.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Serbian hyphenation patterns in Cyrillic script</message> + </language> + <!-- mongolian xu-mnhyph.tex --> + <language fop-code="mn" code="mn-cyrl" name="mongolian" use-new-loader="true" use-old-patterns="false" encoding="lmc" exceptions="false"> + <filename-old-patterns>mnhyph.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>Mongolian hyphenation patterns</message> + </language> + <!-- mongolian2a --> + <language fop-code="mn_Cyrl_x_2a" code="mn-cyrl-x-2a" name="mongolian2a" use-new-loader="true" use-old-patterns="false" encoding="t2a" exceptions="false"> + <filename-old-patterns>mnhyphn.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <message>(New) Mongolian Hyphenation Patterns</message> + </language> + <!-- bulgarian xu-bghyphen.tex --> + <language fop-code="bg" code="bg" name="bulgarian" use-new-loader="true" use-old-patterns="false" encoding="t2a" exceptions="false"> + <filename-old-patterns>bghyphen.tex</filename-old-patterns> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Bulgarian Hyphenation Patterns</message> + </language> + <!-- sanskrit --> + <language fop-code="sa" code="sa" name="sanskrit" use-new-loader="true" use-old-patterns="false" encoding="" exceptions="false"> + <hyphen-min before="1" after="5"/> + <!-- polyglosia --> + <!-- no patterns for 8-bit TeX --> + <message>Sanskrit Hyphenation Patterns (v0.2, 2008/1/3)</message> + </language> + <!-- norsk xu-nohyphbx.tex --> + <language code="nb" name="bokmal" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <synonym>norwegian</synonym> + <synonym>norsk</synonym> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Norwegian Bokmal Hyphenation Patterns</message> + </language> + <!-- nynorsk nnhyph.tex --> + <language code="nn" name="nynorsk" use-new-loader="true" use-old-patterns="false" encoding="ec" exceptions="true"> + <hyphen-min before="2" after="2"/> + <!-- babel --> + <message>Norwegian Nynorsk Hyphenation Patterns</message> + </language> + <!-- The following languages are not (yet) in languages.rb. --> + <!-- I added them after this file was generated from languages.rb. --> + <language fop-code="en" code="en-us" name="US english"> + <hyphen-min before="2" after="3"/> + <filename-old-patterns>ushyphmax.tex</filename-old-patterns> + </language> + <language fop-code="sh_Latn" code="sh-latn" name="serbocroatian"> + <hyphen-min before="1" after="2"/> + </language> + <language fop-code="sh_Cyrl" code="sh-cyrl" name="serbocroatianc"> + <hyphen-min before="1" after="2"/> + </language> + <language fop-code="no" code="no" name="norwegian"> + <filename-old-patterns>nohyphbx.tex</filename-old-patterns> + </language> +</languages> diff --git a/Master/texmf-dist/source/generic/hyph-utf8/generate-offo.rb b/Master/texmf-dist/source/generic/hyph-utf8/generate-offo.rb new file mode 100755 index 00000000000..f164c421aeb --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/generate-offo.rb @@ -0,0 +1,96 @@ +#!/usr/bin/env ruby + +# this file generates FOP XML Hyphenation Patterns + +# use 'gem install unicode' if unicode is missing on your computer +# require 'jcode' +# require 'rubygems' +# require 'unicode' + +load 'languages.rb' + +$path_OFFO="../../../../collaboration/offo" + +$l = Languages.new +# TODO: should be singleton +languages = $l.list.sort{|a,b| a.name <=> b.name} + +# TODO: we should rewrite this +# not using: eo, el +# todo: mn, no!, sa, sh +# codes = ['bg', 'ca', 'cs', 'cy', 'da', 'de-1901', 'de-1996', 'de-ch-1901', 'en-gb', 'en-us', 'es', 'et', 'eu', 'fi', 'fr', 'ga', 'gl', 'hr', 'hsb', 'hu', 'ia', 'id', 'is', 'it', 'kmr', 'la', 'lt', 'lv', 'nl', 'no', 'pl', 'pt', 'ro', 'ru', 'sk', 'sl', 'sr-cyrl', 'sv', 'tr', 'uk'] + +language_codes = Hash.new +languages.each do |language| + language_codes[language.code] = language.code +end +language_codes['de-1901'] = 'de_1901' +language_codes['de-1996'] = 'de' +language_codes['de-ch-1901'] = 'de_CH' +language_codes['en-gb'] = 'en_GB' +language_codes['en-us'] = 'en_US' +language_codes['zh-latn'] = 'zh_Latn' +language_codes['el-monoton'] = 'el' +language_codes['el-polyton'] = 'el_polyton' +language_codes['mn-cyrl'] = 'mn_Cyrl' +language_codes['mn-cyrl-x-2a'] = 'mn' +language_codes['sh-latn'] = 'sr_Latn' +language_codes['sh-cyrl'] = nil +language_codes['sr-cyrl'] = 'sr_Cyrl' + +languages.each do |language| + include_language = language.use_new_loader + code = language_codes[language.code] + if code == nil + include_language = false + end + if code == 'en_US' + include_language = true + end + + if include_language + puts "generating #{code}" + + $file_offo_pattern = File.open("#{$path_OFFO}/#{code}.xml", 'w') + + $file_offo_pattern.puts '<?xml version="1.0" encoding="utf-8"?>' + $file_offo_pattern.puts '<hyphenation-info>' + $file_offo_pattern.puts + + # lefthyphenmin/righthyphenmin + if language.hyphenmin == nil or language.hyphenmin.length == 0 then + lmin = '' + rmin = '' + elsif language.filename_old_patterns == "zerohyph.tex" then + lmin = '' + rmin = '' + else + lmin = language.hyphenmin[0] + rmin = language.hyphenmin[1] + end + patterns = language.get_patterns + exceptions = language.get_exceptions + + if code == 'nn' or code == 'nb' + patterns = "" + patterns = $l['no'].get_patterns + end + + $file_offo_pattern.puts "<hyphen-min before=\"#{lmin}\" after=\"#{rmin}\"/>" + $file_offo_pattern.puts + $file_offo_pattern.puts '<exceptions>' + if exceptions != "" + $file_offo_pattern.puts exceptions + end + $file_offo_pattern.puts '</exceptions>' + $file_offo_pattern.puts + $file_offo_pattern.puts '<patterns>' + patterns.each do |pattern| + $file_offo_pattern.puts pattern.gsub(/'/,"’") + end + $file_offo_pattern.puts '</patterns>' + $file_offo_pattern.puts '</hyphenation-info>' + + $file_offo_pattern.close + end +end diff --git a/Master/texmf-dist/source/generic/hyph-utf8/generate-pattern-loaders.rb b/Master/texmf-dist/source/generic/hyph-utf8/generate-pattern-loaders.rb index 3fcbe9e1f9d..8a1b1044532 100644 --- a/Master/texmf-dist/source/generic/hyph-utf8/generate-pattern-loaders.rb +++ b/Master/texmf-dist/source/generic/hyph-utf8/generate-pattern-loaders.rb @@ -59,7 +59,7 @@ languages.each do |language| if language.code == 'it' or language.code == 'fr' or language.code == 'uk' or language.code == 'la' or language.code == 'zh-latn' then file.puts("\\lccode`\\'=`\\'") end - if language.code == 'pt' then + if language.code == 'pt' or language.code == 'tk' then file.puts("\\lccode`\\-=`\\-") end @@ -67,15 +67,28 @@ languages.each do |language| # # some languages (sanskrit) are useless in 8-bit engines; we only want to load them for UTF engines # TODO - maybe consider doing something similar for ibycus - if language.code == 'sa' then + if language.code == 'sa' or + language.code == 'as' or + language.code == 'bn' or + language.code == 'gu' or + language.code == 'hi' or + language.code == 'kn' or + language.code == 'ml' or + language.code == 'mr' or + language.code == 'or' or + language.code == 'pa' or + language.code == 'ta' or + language.code == 'te' then file.puts(text_if_native_utf) file.puts(" \\message{UTF-8 #{language.message}}") file.puts(' % Set \lccode for ZWNJ and ZWJ.') file.puts(' \lccode"200C="200C') file.puts(' \lccode"200D="200D') - file.puts(' % Set \lccode for KANNADA SIGN JIHVAMULIYA and KANNADA SIGN UPADHMANIYA.') + if language.code == 'sa' + file.puts(' % Set \lccode for KANNADA SIGN JIHVAMULIYA and KANNADA SIGN UPADHMANIYA.') file.puts(' \lccode"0CF1="0CF1') file.puts(' \lccode"0CF2="0CF2') + end file.puts(" \\input hyph-#{language.code}.tex") file.puts('\else') file.puts(" \\message{No #{language.message} - only available with Unicode engines}") diff --git a/Master/texmf-dist/source/generic/hyph-utf8/generate-plain-patterns.rb b/Master/texmf-dist/source/generic/hyph-utf8/generate-plain-patterns.rb new file mode 100755 index 00000000000..59bb491f8a0 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/generate-plain-patterns.rb @@ -0,0 +1,97 @@ +#!/usr/bin/env ruby + +# this file generates plain patterns (one-per-line) out of TeX source + +# use 'gem install unicode' if unicode is missing on your computer +require 'jcode' +require 'rubygems' +require 'unicode' + +load 'languages.rb' + +$path_plain="../../../../plain" + +$l = Languages.new +# TODO: should be singleton +languages = $l.list.sort{|a,b| a.name <=> b.name} + +# TODO: we should rewrite this +# not using: eo, el +# todo: mn, no!, sa, sh +# codes = ['bg', 'ca', 'cs', 'cy', 'da', 'de-1901', 'de-1996', 'de-ch-1901', 'en-gb', 'en-us', 'es', 'et', 'eu', 'fi', 'fr', 'ga', 'gl', 'hr', 'hsb', 'hu', 'ia', 'id', 'is', 'it', 'kmr', 'la', 'lt', 'lv', 'nl', 'no', 'pl', 'pt', 'ro', 'ru', 'sk', 'sl', 'sr-cyrl', 'sv', 'tr', 'uk'] + +language_codes = Hash.new +languages.each do |language| + language_codes[language.code] = language.code +end +# language_codes['de-1901'] = 'de-1901' +# language_codes['de-1996'] = 'de-1996' +language_codes['de-ch-1901'] = 'de-CH-1901' +language_codes['en-gb'] = 'en-GB' +language_codes['en-us'] = 'en-US' +language_codes['zh-latn'] = 'zh-Latn' +# language_codes['el-monoton'] = 'el-monoton' +# language_codes['el-polyton'] = 'el-polyton' +language_codes['mn-cyrl'] = nil +language_codes['mn-cyrl-x-2a'] = 'mn' +language_codes['sh-latn'] = 'sr-Latn' +language_codes['sh-cyrl'] = nil +language_codes['sr-cyrl'] = 'sr-Cyrl' + +languages.sort{|x,y| x.code <=> y.code }.each do |language| + if language.use_new_loader or language.code == 'en-us' then + include_language = true + else + include_language = false + puts "(skipping #{language.code})" + end + + code = language_codes[language.code] + if code == nil + include_language = false + end + if code == 'en_US' + include_language = true + end + + if include_language + puts "generating #{code}" + + $file_pat = File.open("#{$path_plain}/#{code}.pat.txt", 'w') + $file_hyp = File.open("#{$path_plain}/#{code}.hyp.txt", 'w') + $file_let = File.open("#{$path_plain}/#{code}.chr.txt", 'w') + $file_inf = File.open("#{$path_plain}/#{code}.lic.txt", 'w') + + patterns = language.get_patterns + exceptions = language.get_exceptions + + if code == 'nn' or code == 'nb' + patterns = "" + patterns = $l['no'].get_patterns + end + + characters_indexes = patterns.join('').gsub(/[.0-9]/,'').unpack('U*').sort.uniq + + # patterns + patterns.each do |pattern| + $file_pat.puts pattern.gsub(/'/,"’") + end + # exceptions + if exceptions != "" + $file_hyp.puts exceptions + end + # letters + characters_indexes.each do |c| + ch = [c].pack('U') + $file_let.puts ch + Unicode.upcase(ch) + end + # licence and readme + $file_inf.puts "#{language.message}\n\n(more info about the licence to be added later)\n\n" + $file_inf.puts language.get_comments_and_licence + + $file_pat.close + $file_hyp.close + $file_let.close + $file_inf.close + end +end diff --git a/Master/texmf-dist/source/generic/hyph-utf8/generate-tl-files.rb b/Master/texmf-dist/source/generic/hyph-utf8/generate-tl-files.rb index 3a8365f2fa5..233fb60bbcb 100644 --- a/Master/texmf-dist/source/generic/hyph-utf8/generate-tl-files.rb +++ b/Master/texmf-dist/source/generic/hyph-utf8/generate-tl-files.rb @@ -24,9 +24,10 @@ language_grouping = { 'greek' => ['el-monoton', 'el-polyton'], 'ancientgreek' => ['grc', 'grc-x-ibycus'], 'chinese' => ['zh-latn'], - # TODO - until someone tells what to do + 'indic' => ['as', 'bn', 'gu', 'hi', 'kn', 'ml', 'mr', 'or', 'pa', 'ta', 'te'], + # TODO - until someone tells what to do; but this is wrong anyway #'serbian' => ['sr-latn', 'sr-cyrl'], - 'serbian' => ['sr-latn'], + 'serbian' => ['sh-latn'], } language_used_in_group = Hash.new @@ -41,12 +42,17 @@ language_groups = Hash.new # single languages first languages.each do |language| # temporary remove cyrilic serbian until someone explains what is needed - if language.code == 'sr-cyrl' then - languages.delete(language) + if language.code == 'sr-cyrl' or language.code == 'en-us' then + # ignore the language elsif language_used_in_group[language.code] == nil then language_groups[language.name] = [language] end + + if language.code == 'sh-latn' then + language.code = 'sr-latn' + end end + # then groups of languages language_grouping.each do |name,group| language_groups[name] = [] diff --git a/Master/texmf-dist/source/generic/hyph-utf8/generate-webpage.rb b/Master/texmf-dist/source/generic/hyph-utf8/generate-webpage.rb index 0f5c202b379..aa3dec8d0f2 100644 --- a/Master/texmf-dist/source/generic/hyph-utf8/generate-webpage.rb +++ b/Master/texmf-dist/source/generic/hyph-utf8/generate-webpage.rb @@ -23,7 +23,7 @@ language_grouping = { 'chinese' => ['zh-latn'], # TODO - until someone tells what to do #'serbian' => ['sr-latn', 'sr-cyrl'], - 'serbian' => ['sr-latn'], + 'serbian' => ['sh-latn'], } language_used_in_group = Hash.new @@ -40,6 +40,8 @@ languages.each do |language| # temporary remove cyrilic serbian until someone explains what is needed if language.code == 'sr-cyrl' then languages.delete(language) + elsif language.code == 'sh-latn' then + language.code = 'sr-latn' elsif language_used_in_group[language.code] == nil then language_groups[language.name] = [language] end @@ -108,42 +110,5 @@ language_groups.sort.each do |language_name,language_list| puts "\t<td>#{encoding}</td>" puts "</tr>\n" end - # if language_name != "russian" and language_name != "ukrainian" then - # language_list.each do |language| - # if language.use_old_patterns and language.filename_old_patterns != "zerohyph.tex" then - # $file_tlpsrc.puts "runpattern f texmf/tex/generic/hyphen/#{language.filename_old_patterns}" - # end - # end - # end - # if language_name == "greek" then - # $file_tlpsrc.puts "docpattern d texmf/doc/generic/elhyphen" - # elsif language_name == "hungarian" then - # $file_tlpsrc.puts "docpattern d texmf/doc/generic/huhyphen" - # elsif language_name == "german" then - # $file_tlpsrc.puts "runpattern f texmf/tex/generic/hyphen/dehyphtex.tex" - # $file_tlpsrc.puts "runpattern f texmf/tex/generic/hyphen/ghyphen.README" - # end - # $file_tlpsrc.close end -#--------------# -# language.dat # -#--------------# -$file_language_dat = File.open("#{$path_language_dat}/language.dat", "w") -language_groups.sort.each do |language_name,language_list| - language_list.each do |language| - if language.use_new_loader then - $file_language_dat.puts "#{language.name}\tloadhyph-#{language.code}.tex" - else - $file_language_dat.puts "#{language.name}\t#{language.filename_old_patterns}" - end - - # synonyms - if language.synonyms != nil then - language.synonyms.each do |synonym| - $file_language_dat.puts "=#{synonym}" - end - end - end -end -$file_language_dat.close diff --git a/Master/texmf-dist/source/generic/hyph-utf8/languages.rb b/Master/texmf-dist/source/generic/hyph-utf8/languages.rb index 685255c3b0f..c9e3c5c463f 100644 --- a/Master/texmf-dist/source/generic/hyph-utf8/languages.rb +++ b/Master/texmf-dist/source/generic/hyph-utf8/languages.rb @@ -16,8 +16,80 @@ class Language if @synonyms==nil then @synonyms = [] end end - + + # TODO: simplify this (reduce duplication) + + def get_exceptions + if @exceptions1 == nil + filename = "../../../tex/generic/hyph-utf8/patterns/hyph-#{@code}.tex"; + lines = IO.readlines(filename, '.').join("") + exceptions = lines.gsub(/%.*/,''); + if (exceptions.index('\hyphenation') != nil) + @exceptions1 = exceptions.gsub(/.*\\hyphenation\s*\{(.*?)\}.*/m,'\1'). + gsub(/\s+/m,"\n"). + gsub(/^\s*/m,''). + gsub(/\s*$/m,''). + split("\n") + else + @exceptions1 = "" + end + end + + return @exceptions1 + end + + def get_patterns + if @patterns == nil + filename = "../../../tex/generic/hyph-utf8/patterns/hyph-#{@code}.tex"; + lines = IO.readlines(filename, '.').join("") + @patterns = lines.gsub(/%.*/,''). + gsub(/.*\\patterns\s*\{(.*?)\}.*/m,'\1'). + gsub(/\s+/m,"\n"). + gsub(/^\s*/m,''). + gsub(/\s*$/m,''). + gsub(/'/,"’"). + split("\n") + + if @code == 'eo' then + @patterns = lines.gsub(/%.*/,''). + gsub(/.*\\patterns\s*\{(.*)\}.*/m,'\1'). + # + gsub(/\\adj\{(.*?)\}/m,'\1a. \1aj. \1ajn. \1an. \1e.'). + gsub(/\\nom\{(.*?)\}/m,'\1a. \1aj. \1ajn. \1an. \1e. \1o. \1oj. \1ojn. \1on.'). + gsub(/\\ver\{(.*?)\}/m,'\1as. \1i. \1is. \1os. \1u. \1us.'). + # + gsub(/\s+/m,"\n"). + gsub(/^\s*/m,''). + gsub(/\s*$/m,''). + split("\n") + end + end + return @patterns + end + + def get_comments_and_licence + if @comments_and_licence == nil then + filename = "../../../tex/generic/hyph-utf8/patterns/hyph-#{@code}.tex"; + lines = IO.readlines(filename, '.').join("") + @comments_and_licence = lines. + gsub(/(.*)\\patterns.*/m,'\1') + end + return @comments_and_licence + end + + # def lc_characters + # if @lc_characters == nil + # lc_characters = Hash.new + # p = self.patterns + # p.each do |pattern| + # end + # end + # return @lc_characters + # end + attr_reader :use_new_loader, :use_old_patterns, :use_old_patterns_comment, :filename_old_patterns, :code, :name, :synonyms, :hyphenmin, :encoding, :exceptions, :message + # this hack is needed for Serbian + attr_writer :code end @@ -36,7 +108,7 @@ authors = { }, "claudio_beccari" => { "name" => "Claudio", - "email" => "claudio.beccari@polito.it", + "email" => "claudio{dot}beccari{at}polito{dot}it", } } @@ -637,6 +709,18 @@ class Languages < Hash "exceptions" => false, "message" => "Swedish hyphenation patterns (Jan Michael Rynning, 1994-03-03)", }, +# turkmen +{ + "code" => "tk", + "name" => "turkmen", + "use_new_loader" => true, + "use_old_patterns" => false, + "filename_old_patterns" => nil, + "hyphenmin" => [1,2], + "encoding" => "ec", + "exceptions" => false, + "message" => "Turkmen Hyphenation Patterns", +}, # turkish xu-tkhyph.tex { "code" => "tr", @@ -662,9 +746,22 @@ class Languages < Hash "exceptions" => true, "message" => "Hyphenation Patterns for British English", }, +# US english +{ + "code" => "en-us", + "name" => "english", + "use_new_loader" => false, + "use_old_patterns" => false, + "filename_old_patterns" => "ushyphmax.tex", + "hyphenmin" => [2,3], # confirmed, same as what Knuth says + "encoding" => "ascii", + "exceptions" => true, + "message" => "Hyphenation Patterns for American English", +}, +# TODO: FIXME!!! # serbian xu-srhyphc.tex { - "code" => "sr-latn", + "code" => "sh-latn", "name" => "serbian", "use_new_loader" => true, "use_old_patterns" => false, @@ -737,6 +834,17 @@ class Languages < Hash "exceptions" => false, "message" => "Sanskrit Hyphenation Patterns (v0.2, 2008/1/3)", }, +# norwegian nohyph.tex +{ + "code" => "no", + "name" => "norwegian", # TODO: fixme + "use_new_loader" => false, + "use_old_patterns" => false, + "hyphenmin" => [2,2], # babel + "encoding" => "ec", + "exceptions" => false, + "message" => "Norwegian Hyphenation Patterns", +}, # norsk xu-nohyphbx.tex { "code" => "nb", @@ -759,6 +867,139 @@ class Languages < Hash "exceptions" => true, "message" => "Norwegian Nynorsk Hyphenation Patterns", }, +##### +# assamese +{ + "code" => "as", + "name" => "assamese", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Assameze Hyphenation Patterns", +}, +# bengali +{ + "code" => "bn", + "name" => "bengali", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Bengali Hyphenation Patterns", +}, +# guajrati +{ + "code" => "gu", + "name" => "guajrati", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Guajrati Hyphenation Patterns", +}, +# assamese +{ + "code" => "as", + "name" => "assamese", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Assameze Hyphenation Patterns", +}, +# hindi +{ + "code" => "hi", + "name" => "hindi", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Hindi Hyphenation Patterns", +}, +# kannada +{ + "code" => "kn", + "name" => "assamese", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Kannada Hyphenation Patterns", +}, +# malayalam +{ + "code" => "ml", + "name" => "malayalam", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Malayalam Hyphenation Patterns", +}, +# marathi +{ + "code" => "mr", + "name" => "marathi", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Marathi Hyphenation Patterns", +}, +# oriya +{ + "code" => "or", + "name" => "oriya", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Oriya Hyphenation Patterns", +}, +# panjabi +{ + "code" => "pa", + "name" => "panjabi", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Panjabi Hyphenation Patterns", +}, +# tamil +{ + "code" => "ta", + "name" => "tamil", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Tamil Hyphenation Patterns", +}, +# telugu +{ + "code" => "te", + "name" => "telugu", + "use_new_loader" => true, + "use_old_patterns" => false, + "hyphenmin" => [1,1], # TODO + "encoding" => nil, # no patterns for 8-bit TeX + "exceptions" => false, + "message" => "Telugu Hyphenation Patterns", +}, ] languages.each do |l| diff --git a/Master/texmf-dist/source/generic/hyph-utf8/languages/tk/generate_patterns_tk.rb b/Master/texmf-dist/source/generic/hyph-utf8/languages/tk/generate_patterns_tk.rb new file mode 100755 index 00000000000..072ea34de52 --- /dev/null +++ b/Master/texmf-dist/source/generic/hyph-utf8/languages/tk/generate_patterns_tk.rb @@ -0,0 +1,135 @@ +#!/usr/bin/env ruby +# +# This script generates hyphenation patterns for Turkmen +# +# This script has been written by Mojca Miklavec <mojca dot miklavec dot lists at gmail dot com> + +# open file for writing the patterns +# $tr = File.new("hyph-tk.tex", "w") +# in TDS +$tr = File.new("../../../../../tex/generic/hyph-utf8/patterns/hyph-tk.tex", "w") + +# write comments into the file +def add_comment(str) + $tr.puts "% " + str.gsub(/\n/, "\n% ").gsub(/% \n/, "%\n") +end + +# define a class of vowels and consonants +# vowels are split into so that unnecessary permutations are not generated +front_vowels = %w{ä e i ö ü} +back_vowels = %w{a y o u} +consonants = %w{b ç d f g h j k l m n p r s t w ý z ň ž ş} +# This is to eliminate impossible combinations +common_suffix_consonants = %w{b ç d g j k l m n p s t ý z ş} + + +# start the file +add_comment( +"Hyphenation patterns for Turkmen (hyph-tk.tex) + +Author: Nazar Annagurban <nazartm at gmail.com> +License: Public domain +Version: 0.1 +Date: 16 March 2010 + +---------------------------------------------------------------------- + +The file has been auto-generated from generate_patterns_tk.rb +that is part of hyph-utf8. + +For more information about UTF-8 hyphenation patterns for TeX and +links to this file see + http://www.tug.org/tex-hyphen/ +") + +# we have the following comment for Basque: +# +# Some of the patterns below represent combinations that never +# happen in Turkmen. Would they happen, they would be hyphenated +# according to the rules. + +$tr.puts '\patterns{' +add_comment("Some suffixes are added through a hyphen. When hyphenating these words, a hyphen is added before the hyphen so that the line ends with a hyphen and the new line starts with a hyphen.") +$tr.puts "1-4" + +add_comment("Allow hyphen after a vowel if and only if there is a single consonant before next the vowel") +front_vowels.each do |v1| + consonants.each do |c| + front_vowels.each do |v2| + $tr.puts "#{v1}1#{c}#{v2}" + end + end +end + +back_vowels.each do |v1| + consonants.each do |c| + back_vowels.each do |v2| + $tr.puts "#{v1}1#{c}#{v2}" + end + end +end + +add_comment("These combinations occur in words of foreign origin or joined words") +consonants.each do |c| + $tr.puts "a1#{c}i" + $tr.puts "a1#{c}e" + $tr.puts "y1#{c}ä" + $tr.puts "y1#{c}i" + $tr.puts "y1#{c}e" + $tr.puts "o1#{c}i" + $tr.puts "o1#{c}e" + $tr.puts "u1#{c}i" + $tr.puts "u1#{c}e" + $tr.puts "i1#{c}a" + $tr.puts "i1#{c}o" + $tr.puts "e1#{c}a" + $tr.puts "e1#{c}o" + $tr.puts "ä1#{c}o" + $tr.puts "ä1#{c}a" + $tr.puts "ö1#{c}a" +end + +add_comment("Allow hyphen between two consonants (if there is only two of them), except when they are at the begining of the word") +consonants.each do |c1| + consonants.each do |c2| + $tr.puts "#{c1}1#{c2}" + $tr.puts ".#{c1}2#{c2}" + end +end + +add_comment("Patterns for triple consonants. There may be additions to this category, as this list is not exhaustive.") +common_suffix_consonants.each do |c| + $tr.puts "ý2t1#{c}" + $tr.puts "ý2n1#{c}" + $tr.puts "ý2d1#{c}" + $tr.puts "r2t1#{c}" + $tr.puts "ý2p1#{c}" + $tr.puts "l2p1#{c}" + $tr.puts "l2t1#{c}" + $tr.puts "g2t1#{c}" + $tr.puts "n2t1#{c}" + $tr.puts "r2k1#{c}" + $tr.puts "r2p1#{c}" + $tr.puts "k2t1#{c}" + $tr.puts "r2h1#{c}" + $tr.puts "s2t1#{c}" + $tr.puts "l2k1#{c}" + $tr.puts "w2p1#{c}" + $tr.puts "n2s1#{c}" + $tr.puts "r2s1#{c}" + $tr.puts "l2m1#{c}" +end + +add_comment("Exceptions and single word occurence patterns for words of foreign origin i.e. Russian") +$tr.puts "s2k1d" +$tr.puts "l1s2k" +$tr.puts "l1s2t" +$tr.puts "s1t2r" +$tr.puts "n2g1l" +$tr.puts "n1g2r" +$tr.puts "s2k1w" + +# end the file +$tr.puts '}' +$tr.close + diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-as.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-as.tex new file mode 100644 index 00000000000..5c296dfdf9b --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-as.tex @@ -0,0 +1,30 @@ +% loadhyph-as.tex +% +% Autogenerated loader for hyphenation patterns for "assamese" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Assameze Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-as.tex +\else + \message{No Assameze Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-bn.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-bn.tex new file mode 100644 index 00000000000..49fa01c7188 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-bn.tex @@ -0,0 +1,30 @@ +% loadhyph-bn.tex +% +% Autogenerated loader for hyphenation patterns for "bengali" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Bengali Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-bn.tex +\else + \message{No Bengali Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-gu.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-gu.tex new file mode 100644 index 00000000000..0c7e2a45339 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-gu.tex @@ -0,0 +1,30 @@ +% loadhyph-gu.tex +% +% Autogenerated loader for hyphenation patterns for "guajrati" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Guajrati Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-gu.tex +\else + \message{No Guajrati Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-hi.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-hi.tex new file mode 100644 index 00000000000..b702ecdec9c --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-hi.tex @@ -0,0 +1,30 @@ +% loadhyph-hi.tex +% +% Autogenerated loader for hyphenation patterns for "hindi" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Hindi Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-hi.tex +\else + \message{No Hindi Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-kn.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-kn.tex new file mode 100644 index 00000000000..7e54e5e0777 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-kn.tex @@ -0,0 +1,30 @@ +% loadhyph-kn.tex +% +% Autogenerated loader for hyphenation patterns for "assamese" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Kannada Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-kn.tex +\else + \message{No Kannada Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-ml.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-ml.tex new file mode 100644 index 00000000000..838386497f6 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-ml.tex @@ -0,0 +1,30 @@ +% loadhyph-ml.tex +% +% Autogenerated loader for hyphenation patterns for "malayalam" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Malayalam Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-ml.tex +\else + \message{No Malayalam Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-mr.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-mr.tex new file mode 100644 index 00000000000..8ec1961988b --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-mr.tex @@ -0,0 +1,30 @@ +% loadhyph-mr.tex +% +% Autogenerated loader for hyphenation patterns for "marathi" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Marathi Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-mr.tex +\else + \message{No Marathi Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-or.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-or.tex new file mode 100644 index 00000000000..424b659985b --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-or.tex @@ -0,0 +1,30 @@ +% loadhyph-or.tex +% +% Autogenerated loader for hyphenation patterns for "oriya" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Oriya Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-or.tex +\else + \message{No Oriya Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-pa.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-pa.tex new file mode 100644 index 00000000000..7f553b865a4 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-pa.tex @@ -0,0 +1,30 @@ +% loadhyph-pa.tex +% +% Autogenerated loader for hyphenation patterns for "panjabi" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Panjabi Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-pa.tex +\else + \message{No Panjabi Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-ta.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-ta.tex new file mode 100644 index 00000000000..8f8aa20067a --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-ta.tex @@ -0,0 +1,30 @@ +% loadhyph-ta.tex +% +% Autogenerated loader for hyphenation patterns for "tamil" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Tamil Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-ta.tex +\else + \message{No Tamil Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-te.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-te.tex new file mode 100644 index 00000000000..8e96db47144 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-te.tex @@ -0,0 +1,30 @@ +% loadhyph-te.tex +% +% Autogenerated loader for hyphenation patterns for "telugu" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Telugu Hyphenation Patterns} + % Set \lccode for ZWNJ and ZWJ. + \lccode"200C="200C + \lccode"200D="200D + \input hyph-te.tex +\else + \message{No Telugu Hyphenation Patterns - only available with Unicode engines} + \input zerohyph.tex +\fi +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-tk.tex b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-tk.tex new file mode 100644 index 00000000000..69c789563cb --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/loadhyph/loadhyph-tk.tex @@ -0,0 +1,28 @@ +% loadhyph-tk.tex +% +% Autogenerated loader for hyphenation patterns for "turkmen" +% by source/generic/hyph-utf8/generate-pattern-loaders.rb +% See also http://tug.org/tex-hyphen +% +% Copyright 2008 TeX Users Group. +% You may freely use, modify and/or distribute this file. +% (But consider adapting the scripts if you need modifications.) +% +% Once it turns out that more than a simple definition is needed, +% these lines may be moved to a separate file. +% +\begingroup +\lccode`\-=`\- +% Test whether we received one or two arguments +\def\testengine#1#2!{\def\secondarg{#2}} +% That's Tau (as in Taco or ΤΕΧ, Tau-Epsilon-Chi), a 2-byte UTF-8 character +\testengine Τ!\relax +% Unicode-aware engine (such as XeTeX or LuaTeX) only sees a single (2-byte) argument +\ifx\secondarg\empty + \message{UTF-8 Turkmen Hyphenation Patterns} +\else + \message{EC Turkmen Hyphenation Patterns} + \input conv-utf8-ec.tex +\fi +\input hyph-tk.tex +\endgroup diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-as.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-as.tex new file mode 100644 index 00000000000..2a83d21ef01 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-as.tex @@ -0,0 +1,105 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Assamese +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +অ1 +আ1 +ই1 +ঈ1 +উ1 +ঊ1 +ঋ1 +ৠ1 +ঌ1 +ৡ1 +এ1 +ঐ1 +ও1 +ঔ1 +% Break after any dependent vowel, but not before. +া1 +ি1 +ী1 +ু1 +ূ1 +ৃ1 +ৄ1 +ৢ1 +ৣ1 +ে1 +ৈ1 +ো1 +ৌ1 +2়2 +ৗ1 +% Break before or after any consonant. +1ক +1খ +1গ +1ঘ +1ঙ +1চ +1ছ +1জ +1ঝ +1ঞ +1ট +1ঠ +1ড +1ড় +1ঢ +1ঢ় +1ণ +1ত +1থ +1দ +1ধ +1ন +1প +1ফ +1ব +1ভ +1ম +1য +1য় +1র +1ল +1শ +1ষ +1স +1হ +% Do not break after khanda ta. +ৎ1 +% Do not break before chandrabindu, anusvara, visarga, avagraha, +% nukta and au length mark. +2ঃ1 +2ং1 +2ঁ1 +2ঽ1 +% Do not break either side of virama (may be within conjunct). +2্2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-bn.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-bn.tex new file mode 100644 index 00000000000..df093323daa --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-bn.tex @@ -0,0 +1,105 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Bengali +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +অ1 +আ1 +ই1 +ঈ1 +উ1 +ঊ1 +ঋ1 +ৠ1 +ঌ1 +ৡ1 +এ1 +ঐ1 +ও1 +ঔ1 +% Break after any dependent vowel, but not before. +া1 +ি1 +ী1 +ু1 +ূ1 +ৃ1 +ৄ1 +ৢ1 +ৣ1 +ে1 +ৈ1 +ো1 +ৌ1 +2়2 +ৗ1 +% Break before or after any consonant. +1ক +1খ +1গ +1ঘ +1ঙ +1চ +1ছ +1জ +1ঝ +1ঞ +1ট +1ঠ +1ড +1ড় +1ঢ +1ঢ় +1ণ +1ত +1থ +1দ +1ধ +1ন +1প +1ফ +1ব +1ভ +1ম +1য +1য় +1র +1ল +1শ +1ষ +1স +1হ +% Do not break after khanda ta. +ৎ1 +% Do not break before chandrabindu, anusvara, visarga, avagraha, +% nukta and au length mark. +2ঃ1 +2ং1 +2ঁ1 +2ঽ1 +% Do not break either side of virama (may be within conjunct). +2্2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-gu.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-gu.tex new file mode 100644 index 00000000000..42ae3f07699 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-gu.tex @@ -0,0 +1,98 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Guajrati +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +અ1 +આ1 +ઇ1 +ઈ1 +ઉ1 +ઊ1 +ઋ1 +ૠ1 +એ1 +ઐ1 +ઓ1 +ઔ1 +% Break after any dependent vowel but not before. +ા1 +િ1 +ી1 +ુ1 +ૂ1 +ૃ1 +ૄ1 +ૢ1 +ૣ1 +ે1 +ૈ1 +ો1 +ૌ1 +% Break before or after any consonant. +1ક +1ખ +1ગ +1ઘ +1ઙ +1ચ +1છ +1જ +1ઝ +1ઞ +1ટ +1ઠ +1ડ +1ઢ +1ણ +1ત +1થ +1દ +1ધ +1ન +1પ +1ફ +1બ +1ભ +1મ +1ય +1ર +1લ +1ળ +1વ +1શ +1ષ +1સ +1હ +% Do not break before chandrabindu, anusvara, visarga, avagraha +% and accents. +2ઁ1 +2ઃ1 +2ઽ1 +% Do not break either side of virama (may be within conjunct). +2્2 +2ં2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-hi.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-hi.tex new file mode 100644 index 00000000000..7ab1679051e --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-hi.tex @@ -0,0 +1,102 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Hindi +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +अ1 +आ1 +इ1 +ई1 +उ1 +ऊ1 +ऋ1 +ॠ1 +ऌ1 +ॡ1 +ए1 +ऐ1 +ओ1 +औ1 +% Break after any dependent vowel but not before. +ा1 +ि1 +ी1 +ु1 +ू1 +ृ1 +ॄ1 +ॢ1 +ॣ1 +े1 +ै1 +ो1 +ौ1 +% Break before or after any consonant. +1क +1ख +1ग +1घ +1ङ +1च +1छ +1ज +1झ +1ञ +1ट +1ठ +1ड +1ढ +1ण +1त +1थ +1द +1ध +1न +1प +1फ +1ब +1भ +1म +1य +1र +1ल +1ळ +1व +1श +1ष +1स +1ह +% Do not break before chandrabindu, anusvara, visarga, avagraha +% and accents. +2ँ1 +2ं1 +2ः1 +2ऽ1 +2॑1 +2॒1 +% Do not break either side of virama (may be within conjunct). +2्2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-kn.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-kn.tex new file mode 100644 index 00000000000..535c314d1a0 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-kn.tex @@ -0,0 +1,105 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Kannada +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +ಅ1 +ಆ1 +ಇ1 +ಈ1 +ಉ1 +ಊ1 +ಋ1 +ೠ1 +ಌ1 +ೡ1 +ಎ1 +ಏ1 +ಐ1 +ಒ1 +ಓ1 +ಔ1 +% Break after any dependent vowel, but not before. +ಾ1 +ಿ1 +ೀ1 +ು1 +ೂ1 +ೃ1 +ೄ1 +ೆ1 +ೇ1 +ೈ1 +ೊ1 +ೋ1 +ೌ1 +% Break before or after any consonant. +1ಕ +1ಖ +1ಗ +1ಘ +1ಙ +1ಚ +1ಛ +1ಜ +1ಝ +1ಞ +1ಟ +1ಠ +1ಡ +1ಢ +1ಣ +1ತ +1ಥ +1ದ +1ಧ +1ನ +1ಪ +1ಫ +1ಬ +1ಭ +1ಮ +1ಯ +1ರ +1ಱ +1ಲ +1ಳ +1ೞ +1ವ +1ಶ +1ಷ +1ಸ +1ಹ +% Do not break before anusvara, visarga, avagraha, +% length mark and ai length mark. +2ಂ1 +2ಃ1 +2ಽ1 +2ೕ1 +2ೖ1 +% Do not break either side of virama (may be within conjunct). +2್2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-lt.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-lt.tex index f53d095a780..05ca61d07dd 100644 --- a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-lt.tex +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-lt.tex @@ -32,6 +32,17 @@ % http://tug.org/tex-hyphen % %------------------------------------------------------------------------------ +% +% A note about lefthyphenmin/righthyphenmin (by Sigitas, February 2009): +% +% At the moment the lefthyphenmin & righthyphenmin are both set to 2. +% The Lithuanian patterns were build earlier then the latest grammatical rules +% took place in 1997. At that time it was forbidden to leave one character at boundary. +% One may change it to one now, but from typographical point of view there is no need to do +% any changes. Even more, changing lefthyphen and righthyphen to 1 requires +% to review and maybe rebuild the patterns file. +% +%------------------------------------------------------------------------------ \patterns{ .ap1 .api1 diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-ml.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-ml.tex new file mode 100644 index 00000000000..ce8b9ba18ed --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-ml.tex @@ -0,0 +1,120 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Malayalam +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +1അ1 +1ആ1 +1ഇ1 +1ഈ1 +1ഉ1 +1ഊ1 +1ഋ1 +1ൠ1 +1ഌ1 +1ൡ1 +1എ1 +1ഏ1 +1ഐ1 +1ഒ1 +1ഓ1 +1ഔ1 +% Break after any dependent vowel, but not before. +ാ1 +ി1 +ീ1 +ു1 +ൂ1 +ൃ1 +െ1 +േ1 +ൈ1 +ൊ1 +ോ1 +ൌ1 +ൗ1 +% Break before or after any consonant. +1ക +1ഖ +1ഗ +1ഘ +1ങ +1ച +1ഛ +1ജ +1ഝ +1ഞ +1ട +1ഠ +1ഡ +1ഢ +1ണ +1ത +1ഥ +1ദ +1ധ +1ന +1പ +1ഫ +1ബ +1ഭ +1മ +1യ +1ര +1റ +1ല +1ള +1ഴ +1വ +1ശ +1ഷ +1സ +1ഹ +% Do not break before anusvara, visarga +2ഃ1 +2ം1 +% Do not break either side of virama (may be within conjunct). +2്2 +% Do not break left side of chillu +ന്2 +ര്2 +ള്2 +ല്2 +ക്2 +ണ്2 +2ന് +2ല് +2ള് +2ണ് +2ര് +2ക് +2ൺ +2ൻ +2ർ +2ൽ +2ൾ +2ൿ +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-mn-cyrl-x-2a.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-mn-cyrl-x-2a.tex index 2572d750840..6d378007a9a 100644 --- a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-mn-cyrl-x-2a.tex +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-mn-cyrl-x-2a.tex @@ -2,7 +2,7 @@ % semi-manual conversions of hyphenation patterns into UTF-8 in June 2008. % % Source: mnhyphn.tex (2008-03-30) -% Author: D. Batmunkh <bataak at gmail.com> +% Author: Dorjgotov Batmunkh <bataak at gmail.com> % % The above mentioned file should become obsolete, % and the author of the original file should preferaby modify this file instead. diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-mr.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-mr.tex new file mode 100644 index 00000000000..7a0ffb448e2 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-mr.tex @@ -0,0 +1,102 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Marathi +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +अ1 +आ1 +इ1 +ई1 +उ1 +ऊ1 +ऋ1 +ॠ1 +ऌ1 +ॡ1 +ए1 +ऐ1 +ओ1 +औ1 +% Break after any dependent vowel but not before. +ा1 +ि1 +ी1 +ु1 +ू1 +ृ1 +ॄ1 +ॢ1 +ॣ1 +े1 +ै1 +ो1 +ौ1 +% Break before or after any consonant. +1क +1ख +1ग +1घ +1ङ +1च +1छ +1ज +1झ +1ञ +1ट +1ठ +1ड +1ढ +1ण +1त +1थ +1द +1ध +1न +1प +1फ +1ब +1भ +1म +1य +1र +1ल +1ळ +1व +1श +1ष +1स +1ह +% Do not break before chandrabindu, anusvara, visarga, avagraha +% and accents. +2ँ1 +2ं1 +2ः1 +2ऽ1 +2॑1 +2॒1 +% Do not break either side of virama (may be within conjunct). +2्2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-or.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-or.tex new file mode 100644 index 00000000000..6ede6d747c2 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-or.tex @@ -0,0 +1,96 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Oriya +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +ଅ1 +ଆ1 +ଇ1 +ଈ1 +ଉ1 +ଊ1 +ଋ1 +ୠ1 +ଌ1 +ୡ1 +ଏ1 +ଐ1 +ଓ1 +ଔ1 +% Break after any dependent vowel, but not before. +ା1 +ି1 +ୀ1 +ୁ1 +ୂ1 +ୃ1 +େ1 +ୈ1 +ୋ1 +ୌ1 +% Break before or after any consonant. +1କ +1ଖ +1ଗ +1ଘ +1ଙ +1ଚ +1ଛ +1ଜ +1ଝ +1ଞ +1ଟ +1ଠ +1ଡ +1ଢ +1ଣ +1ତ +1ଥ +1ଦ +1ଧ +1ନ +1ପ +1ଫ +1ବ +1ଭ +1ମ +1ଯ +1ର +1ଲ +1ଳ +1ଵ +1ଶ +1ଷ +1ସ +1ହ +% Do not break before anusvara, visarga and length mark. +2ଂ1 +2ଃ1 +2ୗ1 +2ଁ1 +% Do not break either side of virama (may be within conjunct). +2୍2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-pa.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-pa.tex new file mode 100644 index 00000000000..2ecbf0c4858 --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-pa.tex @@ -0,0 +1,92 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Panjabi +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +ਅ1 +ਆ1 +ਇ1 +ਈ1 +ਉ1 +ਊ1 +ਏ1 +ਐ1 +ਓ1 +ਔ1 +% Break after any dependent vowel but not before. +ਾ1 +ਿ1 +ੀ1 +ੁ1 +ੂ1 +ੇ1 +ੈ1 +ੋ1 +ੌ1 +% Break before or after any consonant. +1ਕ +1ਖ +1ਗ +1ਘ +1ਙ +1ਚ +1ਛ +1ਜ +1ਝ +1ਞ +1ਟ +1ਠ +1ਡ +1ਢ +1ਣ +1ਤ +1ਥ +1ਦ +1ਧ +1ਨ +1ਪ +1ਫ +1ਬ +1ਭ +1ਮ +1ਯ +1ਰ +1ਲ +1ਲ਼ +1ਵ +1ਸ਼ +1ਸ +1ਹ +% Do not break before chandrabindu, anusvara, visarga, avagraha +% and accents. +2ਁ1 +2ਂ1 +2ਃ1 +% Do not break either side of virama (may be within conjunct). +2੍2 +2ੰ2 +2ੱ2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-sa.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-sa.tex index 7e63e451d48..cc2b9333c4c 100644 --- a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-sa.tex +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-sa.tex @@ -2,13 +2,13 @@ % % File name: hyph-sa.tex % -% Unicode hyphenation patterns for Sanskrit and Prakrit in Devanagari, -% Bengali, Kannada, Malayalam and Telugu scripts. +% Unicode hyphenation patterns for Sanskrit and Prakrit in transliteration, +% and in Devanagari, Bengali, Kannada, Malayalam and Telugu scripts. % % Created: April 1st, 2005 % First release: June 8th, 2006 -% Revised: April 12th, 2009 -% Version: 0.3 +% Revised: February 26th, 2010 +% Version: 0.4 % % Created by Yves Codet with Jonathan Kew's help. % @@ -22,6 +22,127 @@ 22 22 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% TRANSLITERATION +%%%%%%%%%%%%%%%%%%%% +% Decomposed characters +% Break after any vowel, except inside a diphthong. +a1 +ā1 +i1 +ï1 +ī1 +u1 +ü1 +ū1 +r̥1 +ṛ1 % non standard transliteration of vocalic r +r̥̄1 +ṝ1 % non standard transliteration of vocalic long r +l̥1 +ḷ1 % non standard transliteration of vocalic l +l̥̄1 +ḹ1 % non standard transliteration of vocalic long l +e1 +o1 +a2i1 +a2u1 +% Break between a and i or u in hiatus. +a3ï1 +a3ü1 +% Do not break before a final consonant or consonant cluster. +2k. +2kh. +2g. +2gh. +2ṅ. +2c. +2ch. +2j. +2jh. +2ñ. +2ṭ. +2ṭh. +2ḍ. +2ḍh. +2ṇ. +2t. +2th. +2d. +2dh. +2n. +2p. +2ph. +2b. +2bh. +2m. +2y. +2r. +2l. +2ḷ. +2v. +2ś. +2ṣ. +2s. +2h. +2rk. +2rg. +2rṭ. +2rḍ. +2rt. +2rd. +2rp. +2rb. +% Break after anusvara, anunasika and visarga but not before. +2ṃ1 +2m̐1 +2ḥ1 +% Do not break before a macron, an accent or a diaeresis; break after an accent. +2̄ +2́1 +2̀1 +2̈ +%%%%%%%%%%%%%%%%%%%% +% Precomposed characters +% Break after any vowel, except inside a diphthong. +á1 +à1 +ā1 +í1 +ì1 +ï1 +ī1 +ú1 +ù1 +ü1 +ū1 +é1 +è1 +ó1 +ò1 +a2í1 +a2ì1 +a2ú1 +a2ù1 +% Break between a and i or u in hiatus. +a3ï1 +a3ü1 +% Do not break before a final consonant or consonant cluster. +2ṅ. +2ñ. +2ṭ. +2ṭh. +2ḍ. +2ḍh. +2ṇ. +2ḷ. +2ś. +2ṣ. +2rṭ. +2rḍ. +% Break after anusvara and visarga but not before. +2ṃ1 +2ḥ1 +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % DEVANAGARI SCRIPT % Break before or after any independent vowel. 1अ1 @@ -725,4 +846,3 @@ % Do not break either side of virama (may be within conjunct). 2్2 } - diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-ta.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-ta.tex new file mode 100644 index 00000000000..5fc5d270fcf --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-ta.tex @@ -0,0 +1,103 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Tamil +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +1அ1 +1ஆ1 +1இ1 +1ஈ1 +1உ1 +1ஊ1 +1எ1 +1ஏ1 +1ஐ1 +1ஒ1 +1ஓ1 +1ஔ1 +% Break after any dependent vowel, but not before. +ா1 +ி1 +ீ1 +ு1 +ூ1 +ெ1 +ே1 +ை1 +ொ1 +ோ1 +ௌ1 +% Break before or after any consonant. +1க +1ங +1ச +1ஜ +1ஞ +1ட +1ண +1த +1ந +1ப +1ம +1ய +1ர +1ற +1ல +1ள +1ழ +1வ +1ஷ +1ஸ +1ஹ +% Do not break before any consonant + virama. +2க்1 +2ங்1 +2ச்1 +2ஞ்1 +2ட்1 +2ண்1 +2த்1 +2ன்1 +2ந்1 +2ப்1 +2ம்1 +2ய்1 +2ர்1 +2ற்1 +2ல்1 +2ள்1 +2ழ்1 +2வ்1 +2ஷ்1 +2ஸ்1 +2ஹ்1 +% Do not break before anusvara, visarga and length mark. +2ஂ1 +2ஃ1 +2ௗ1 +% Do not break before virama but break after virama. +2்1 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-te.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-te.tex new file mode 100644 index 00000000000..ba85660c02d --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-te.tex @@ -0,0 +1,104 @@ +% These patterns originate from +% http://git.savannah.gnu.org/cgit/smc.git/tree/hyphenation) +% and have been adapted for hyph-utf8 (for use in TeX). +% +% Hyphenation for Telugu +% Copyright (C) 2008-2010 Santhosh Thottingal <santhosh.thottingal@gmail.com> +% +% This library is free software; you can redistribute it and/or +% modify it under the terms of the GNU Lesser General Public +% License as published by the Free Software Foundation; +% version 3 or later version of the License. +% +% This library is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +% Lesser General Public License for more details. +% +% You should have received a copy of the GNU Lesser General Public +% License along with this library; if not, write to the Free Software +% Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +% +\patterns{ +% GENERAL RULE +% Do not break either side of ZERO-WIDTH JOINER (U+200D) +22 +% Break on both sides of ZERO-WIDTH NON JOINER (U+200C) +11 +% Break before or after any independent vowel. +అ1 +ఆ1 +ఇ1 +ఈ1 +ఉ1 +ఊ1 +ఋ1 +ౠ1 +ఌ1 +ౡ1 +ఎ1 +ఏ1 +ఐ1 +ఒ1 +ఓ1 +ఔ1 +% Break after any dependent vowel, but not before. +ా1 +ి1 +ీ1 +ు1 +ూ1 +ృ1 +ౄ1 +ె1 +ే1 +ై1 +ొ1 +ో1 +ౌ1 +% Break before or after any consonant. +1క +1ఖ +1గ +1ఘ +1ఙ +1చ +1ఛ +1జ +1ఝ +1ఞ +1ట +1ఠ +1డ +1ఢ +1ణ +1త +1థ +1ద +1ధ +1న +1ప +1ఫ +1బ +1భ +1మ +1య +1ర +1ఱ +1ల +1ళ +1వ +1శ +1ష +1స +1హ +% Do not break before chandrabindu, anusvara, visarga, +% length mark and ai length mark. +2ఁ1 +2ం1 +2ః1 +2ౕ1 +2ౖ1 +% Do not break either side of virama (may be within conjunct). +2్2 +} diff --git a/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-tk.tex b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-tk.tex new file mode 100644 index 00000000000..4aeef34defd --- /dev/null +++ b/Master/texmf-dist/tex/generic/hyph-utf8/patterns/hyph-tk.tex @@ -0,0 +1,2396 @@ +% Hyphenation patterns for Turkmen (hyph-tk.tex) +% +% Author: Nazar Annagurban <nazartm at gmail.com> +% License: Public domain +% Version: 0.1 +% Date: 16 March 2010 +% +% ---------------------------------------------------------------------- +% +% The file has been auto-generated from generate_patterns_tk.rb +% that is part of hyph-utf8. +% +% For more information about UTF-8 hyphenation patterns for TeX and +% links to this file see +% http://www.tug.org/tex-hyphen/ +% +\patterns{ +% Some suffixes are added through a hyphen. When hyphenating these words, a hyphen is added before the hyphen so that the line ends with a hyphen and the new line starts with a hyphen. +1-4 +% Allow hyphen after a vowel if and only if there is a single consonant before next the vowel +ä1bä +ä1be +ä1bi +ä1bö +ä1bü +ä1çä +ä1çe +ä1çi +ä1çö +ä1çü +ä1dä +ä1de +ä1di +ä1dö +ä1dü +ä1fä +ä1fe +ä1fi +ä1fö +ä1fü +ä1gä +ä1ge +ä1gi +ä1gö +ä1gü +ä1hä +ä1he +ä1hi +ä1hö +ä1hü +ä1jä +ä1je +ä1ji +ä1jö +ä1jü +ä1kä +ä1ke +ä1ki +ä1kö +ä1kü +ä1lä +ä1le +ä1li +ä1lö +ä1lü +ä1mä +ä1me +ä1mi +ä1mö +ä1mü +ä1nä +ä1ne +ä1ni +ä1nö +ä1nü +ä1pä +ä1pe +ä1pi +ä1pö +ä1pü +ä1rä +ä1re +ä1ri +ä1rö +ä1rü +ä1sä +ä1se +ä1si +ä1sö +ä1sü +ä1tä +ä1te +ä1ti +ä1tö +ä1tü +ä1wä +ä1we +ä1wi +ä1wö +ä1wü +ä1ýä +ä1ýe +ä1ýi +ä1ýö +ä1ýü +ä1zä +ä1ze +ä1zi +ä1zö +ä1zü +ä1ňä +ä1ňe +ä1ňi +ä1ňö +ä1ňü +ä1žä +ä1že +ä1ži +ä1žö +ä1žü +ä1şä +ä1şe +ä1şi +ä1şö +ä1şü +e1bä +e1be +e1bi +e1bö +e1bü +e1çä +e1çe +e1çi +e1çö +e1çü +e1dä +e1de +e1di +e1dö +e1dü +e1fä +e1fe +e1fi +e1fö +e1fü +e1gä +e1ge +e1gi +e1gö +e1gü +e1hä +e1he +e1hi +e1hö +e1hü +e1jä +e1je +e1ji +e1jö +e1jü +e1kä +e1ke +e1ki +e1kö +e1kü +e1lä +e1le +e1li +e1lö +e1lü +e1mä +e1me +e1mi +e1mö +e1mü +e1nä +e1ne +e1ni +e1nö +e1nü +e1pä +e1pe +e1pi +e1pö +e1pü +e1rä +e1re +e1ri +e1rö +e1rü +e1sä +e1se +e1si +e1sö +e1sü +e1tä +e1te +e1ti +e1tö +e1tü +e1wä +e1we +e1wi +e1wö +e1wü +e1ýä +e1ýe +e1ýi +e1ýö +e1ýü +e1zä +e1ze +e1zi +e1zö +e1zü +e1ňä +e1ňe +e1ňi +e1ňö +e1ňü +e1žä +e1že +e1ži +e1žö +e1žü +e1şä +e1şe +e1şi +e1şö +e1şü +i1bä +i1be +i1bi +i1bö +i1bü +i1çä +i1çe +i1çi +i1çö +i1çü +i1dä +i1de +i1di +i1dö +i1dü +i1fä +i1fe +i1fi +i1fö +i1fü +i1gä +i1ge +i1gi +i1gö +i1gü +i1hä +i1he +i1hi +i1hö +i1hü +i1jä +i1je +i1ji +i1jö +i1jü +i1kä +i1ke +i1ki +i1kö +i1kü +i1lä +i1le +i1li +i1lö +i1lü +i1mä +i1me +i1mi +i1mö +i1mü +i1nä +i1ne +i1ni +i1nö +i1nü +i1pä +i1pe +i1pi +i1pö +i1pü +i1rä +i1re +i1ri +i1rö +i1rü +i1sä +i1se +i1si +i1sö +i1sü +i1tä +i1te +i1ti +i1tö +i1tü +i1wä +i1we +i1wi +i1wö +i1wü +i1ýä +i1ýe +i1ýi +i1ýö +i1ýü +i1zä +i1ze +i1zi +i1zö +i1zü +i1ňä +i1ňe +i1ňi +i1ňö +i1ňü +i1žä +i1že +i1ži +i1žö +i1žü +i1şä +i1şe +i1şi +i1şö +i1şü +ö1bä +ö1be +ö1bi +ö1bö +ö1bü +ö1çä +ö1çe +ö1çi +ö1çö +ö1çü +ö1dä +ö1de +ö1di +ö1dö +ö1dü +ö1fä +ö1fe +ö1fi +ö1fö +ö1fü +ö1gä +ö1ge +ö1gi +ö1gö +ö1gü +ö1hä +ö1he +ö1hi +ö1hö +ö1hü +ö1jä +ö1je +ö1ji +ö1jö +ö1jü +ö1kä +ö1ke +ö1ki +ö1kö +ö1kü +ö1lä +ö1le +ö1li +ö1lö +ö1lü +ö1mä +ö1me +ö1mi +ö1mö +ö1mü +ö1nä +ö1ne +ö1ni +ö1nö +ö1nü +ö1pä +ö1pe +ö1pi +ö1pö +ö1pü +ö1rä +ö1re +ö1ri +ö1rö +ö1rü +ö1sä +ö1se +ö1si +ö1sö +ö1sü +ö1tä +ö1te +ö1ti +ö1tö +ö1tü +ö1wä +ö1we +ö1wi +ö1wö +ö1wü +ö1ýä +ö1ýe +ö1ýi +ö1ýö +ö1ýü +ö1zä +ö1ze +ö1zi +ö1zö +ö1zü +ö1ňä +ö1ňe +ö1ňi +ö1ňö +ö1ňü +ö1žä +ö1že +ö1ži +ö1žö +ö1žü +ö1şä +ö1şe +ö1şi +ö1şö +ö1şü +ü1bä +ü1be +ü1bi +ü1bö +ü1bü +ü1çä +ü1çe +ü1çi +ü1çö +ü1çü +ü1dä +ü1de +ü1di +ü1dö +ü1dü +ü1fä +ü1fe +ü1fi +ü1fö +ü1fü +ü1gä +ü1ge +ü1gi +ü1gö +ü1gü +ü1hä +ü1he +ü1hi +ü1hö +ü1hü +ü1jä +ü1je +ü1ji +ü1jö +ü1jü +ü1kä +ü1ke +ü1ki +ü1kö +ü1kü +ü1lä +ü1le +ü1li +ü1lö +ü1lü +ü1mä +ü1me +ü1mi +ü1mö +ü1mü +ü1nä +ü1ne +ü1ni +ü1nö +ü1nü +ü1pä +ü1pe +ü1pi +ü1pö +ü1pü +ü1rä +ü1re +ü1ri +ü1rö +ü1rü +ü1sä +ü1se +ü1si +ü1sö +ü1sü +ü1tä +ü1te +ü1ti +ü1tö +ü1tü +ü1wä +ü1we +ü1wi +ü1wö +ü1wü +ü1ýä +ü1ýe +ü1ýi +ü1ýö +ü1ýü +ü1zä +ü1ze +ü1zi +ü1zö +ü1zü +ü1ňä +ü1ňe +ü1ňi +ü1ňö +ü1ňü +ü1žä +ü1že +ü1ži +ü1žö +ü1žü +ü1şä +ü1şe +ü1şi +ü1şö +ü1şü +a1ba +a1by +a1bo +a1bu +a1ça +a1çy +a1ço +a1çu +a1da +a1dy +a1do +a1du +a1fa +a1fy +a1fo +a1fu +a1ga +a1gy +a1go +a1gu +a1ha +a1hy +a1ho +a1hu +a1ja +a1jy +a1jo +a1ju +a1ka +a1ky +a1ko +a1ku +a1la +a1ly +a1lo +a1lu +a1ma +a1my +a1mo +a1mu +a1na +a1ny +a1no +a1nu +a1pa +a1py +a1po +a1pu +a1ra +a1ry +a1ro +a1ru +a1sa +a1sy +a1so +a1su +a1ta +a1ty +a1to +a1tu +a1wa +a1wy +a1wo +a1wu +a1ýa +a1ýy +a1ýo +a1ýu +a1za +a1zy +a1zo +a1zu +a1ňa +a1ňy +a1ňo +a1ňu +a1ža +a1žy +a1žo +a1žu +a1şa +a1şy +a1şo +a1şu +y1ba +y1by +y1bo +y1bu +y1ça +y1çy +y1ço +y1çu +y1da +y1dy +y1do +y1du +y1fa +y1fy +y1fo +y1fu +y1ga +y1gy +y1go +y1gu +y1ha +y1hy +y1ho +y1hu +y1ja +y1jy +y1jo +y1ju +y1ka +y1ky +y1ko +y1ku +y1la +y1ly +y1lo +y1lu +y1ma +y1my +y1mo +y1mu +y1na +y1ny +y1no +y1nu +y1pa +y1py +y1po +y1pu +y1ra +y1ry +y1ro +y1ru +y1sa +y1sy +y1so +y1su +y1ta +y1ty +y1to +y1tu +y1wa +y1wy +y1wo +y1wu +y1ýa +y1ýy +y1ýo +y1ýu +y1za +y1zy +y1zo +y1zu +y1ňa +y1ňy +y1ňo +y1ňu +y1ža +y1žy +y1žo +y1žu +y1şa +y1şy +y1şo +y1şu +o1ba +o1by +o1bo +o1bu +o1ça +o1çy +o1ço +o1çu +o1da +o1dy +o1do +o1du +o1fa +o1fy +o1fo +o1fu +o1ga +o1gy +o1go +o1gu +o1ha +o1hy +o1ho +o1hu +o1ja +o1jy +o1jo +o1ju +o1ka +o1ky +o1ko +o1ku +o1la +o1ly +o1lo +o1lu +o1ma +o1my +o1mo +o1mu +o1na +o1ny +o1no +o1nu +o1pa +o1py +o1po +o1pu +o1ra +o1ry +o1ro +o1ru +o1sa +o1sy +o1so +o1su +o1ta +o1ty +o1to +o1tu +o1wa +o1wy +o1wo +o1wu +o1ýa +o1ýy +o1ýo +o1ýu +o1za +o1zy +o1zo +o1zu +o1ňa +o1ňy +o1ňo +o1ňu +o1ža +o1žy +o1žo +o1žu +o1şa +o1şy +o1şo +o1şu +u1ba +u1by +u1bo +u1bu +u1ça +u1çy +u1ço +u1çu +u1da +u1dy +u1do +u1du +u1fa +u1fy +u1fo +u1fu +u1ga +u1gy +u1go +u1gu +u1ha +u1hy +u1ho +u1hu +u1ja +u1jy +u1jo +u1ju +u1ka +u1ky +u1ko +u1ku +u1la +u1ly +u1lo +u1lu +u1ma +u1my +u1mo +u1mu +u1na +u1ny +u1no +u1nu +u1pa +u1py +u1po +u1pu +u1ra +u1ry +u1ro +u1ru +u1sa +u1sy +u1so +u1su +u1ta +u1ty +u1to +u1tu +u1wa +u1wy +u1wo +u1wu +u1ýa +u1ýy +u1ýo +u1ýu +u1za +u1zy +u1zo +u1zu +u1ňa +u1ňy +u1ňo +u1ňu +u1ža +u1žy +u1žo +u1žu +u1şa +u1şy +u1şo +u1şu +% These combinations occur in words of foreign origin or joined words +a1bi +a1be +y1bä +y1bi +y1be +o1bi +o1be +u1bi +u1be +i1ba +i1bo +e1ba +e1bo +ä1bo +ä1ba +ö1ba +a1çi +a1çe +y1çä +y1çi +y1çe +o1çi +o1çe +u1çi +u1çe +i1ça +i1ço +e1ça +e1ço +ä1ço +ä1ça +ö1ça +a1di +a1de +y1dä +y1di +y1de +o1di +o1de +u1di +u1de +i1da +i1do +e1da +e1do +ä1do +ä1da +ö1da +a1fi +a1fe +y1fä +y1fi +y1fe +o1fi +o1fe +u1fi +u1fe +i1fa +i1fo +e1fa +e1fo +ä1fo +ä1fa +ö1fa +a1gi +a1ge +y1gä +y1gi +y1ge +o1gi +o1ge +u1gi +u1ge +i1ga +i1go +e1ga +e1go +ä1go +ä1ga +ö1ga +a1hi +a1he +y1hä +y1hi +y1he +o1hi +o1he +u1hi +u1he +i1ha +i1ho +e1ha +e1ho +ä1ho +ä1ha +ö1ha +a1ji +a1je +y1jä +y1ji +y1je +o1ji +o1je +u1ji +u1je +i1ja +i1jo +e1ja +e1jo +ä1jo +ä1ja +ö1ja +a1ki +a1ke +y1kä +y1ki +y1ke +o1ki +o1ke +u1ki +u1ke +i1ka +i1ko +e1ka +e1ko +ä1ko +ä1ka +ö1ka +a1li +a1le +y1lä +y1li +y1le +o1li +o1le +u1li +u1le +i1la +i1lo +e1la +e1lo +ä1lo +ä1la +ö1la +a1mi +a1me +y1mä +y1mi +y1me +o1mi +o1me +u1mi +u1me +i1ma +i1mo +e1ma +e1mo +ä1mo +ä1ma +ö1ma +a1ni +a1ne +y1nä +y1ni +y1ne +o1ni +o1ne +u1ni +u1ne +i1na +i1no +e1na +e1no +ä1no +ä1na +ö1na +a1pi +a1pe +y1pä +y1pi +y1pe +o1pi +o1pe +u1pi +u1pe +i1pa +i1po +e1pa +e1po +ä1po +ä1pa +ö1pa +a1ri +a1re +y1rä +y1ri +y1re +o1ri +o1re +u1ri +u1re +i1ra +i1ro +e1ra +e1ro +ä1ro +ä1ra +ö1ra +a1si +a1se +y1sä +y1si +y1se +o1si +o1se +u1si +u1se +i1sa +i1so +e1sa +e1so +ä1so +ä1sa +ö1sa +a1ti +a1te +y1tä +y1ti +y1te +o1ti +o1te +u1ti +u1te +i1ta +i1to +e1ta +e1to +ä1to +ä1ta +ö1ta +a1wi +a1we +y1wä +y1wi +y1we +o1wi +o1we +u1wi +u1we +i1wa +i1wo +e1wa +e1wo +ä1wo +ä1wa +ö1wa +a1ýi +a1ýe +y1ýä +y1ýi +y1ýe +o1ýi +o1ýe +u1ýi +u1ýe +i1ýa +i1ýo +e1ýa +e1ýo +ä1ýo +ä1ýa +ö1ýa +a1zi +a1ze +y1zä +y1zi +y1ze +o1zi +o1ze +u1zi +u1ze +i1za +i1zo +e1za +e1zo +ä1zo +ä1za +ö1za +a1ňi +a1ňe +y1ňä +y1ňi +y1ňe +o1ňi +o1ňe +u1ňi +u1ňe +i1ňa +i1ňo +e1ňa +e1ňo +ä1ňo +ä1ňa +ö1ňa +a1ži +a1že +y1žä +y1ži +y1že +o1ži +o1že +u1ži +u1že +i1ža +i1žo +e1ža +e1žo +ä1žo +ä1ža +ö1ža +a1şi +a1şe +y1şä +y1şi +y1şe +o1şi +o1şe +u1şi +u1şe +i1şa +i1şo +e1şa +e1şo +ä1şo +ä1şa +ö1şa +% Allow hyphen between two consonants (if there is only two of them), except when they are at the begining of the word +b1b +.b2b +b1ç +.b2ç +b1d +.b2d +b1f +.b2f +b1g +.b2g +b1h +.b2h +b1j +.b2j +b1k +.b2k +b1l +.b2l +b1m +.b2m +b1n +.b2n +b1p +.b2p +b1r +.b2r +b1s +.b2s +b1t +.b2t +b1w +.b2w +b1ý +.b2ý +b1z +.b2z +b1ň +.b2ň +b1ž +.b2ž +b1ş +.b2ş +ç1b +.ç2b +ç1ç +.ç2ç +ç1d +.ç2d +ç1f +.ç2f +ç1g +.ç2g +ç1h +.ç2h +ç1j +.ç2j +ç1k +.ç2k +ç1l +.ç2l +ç1m +.ç2m +ç1n +.ç2n +ç1p +.ç2p +ç1r +.ç2r +ç1s +.ç2s +ç1t +.ç2t +ç1w +.ç2w +ç1ý +.ç2ý +ç1z +.ç2z +ç1ň +.ç2ň +ç1ž +.ç2ž +ç1ş +.ç2ş +d1b +.d2b +d1ç +.d2ç +d1d +.d2d +d1f +.d2f +d1g +.d2g +d1h +.d2h +d1j +.d2j +d1k +.d2k +d1l +.d2l +d1m +.d2m +d1n +.d2n +d1p +.d2p +d1r +.d2r +d1s +.d2s +d1t +.d2t +d1w +.d2w +d1ý +.d2ý +d1z +.d2z +d1ň +.d2ň +d1ž +.d2ž +d1ş +.d2ş +f1b +.f2b +f1ç +.f2ç +f1d +.f2d +f1f +.f2f +f1g +.f2g +f1h +.f2h +f1j +.f2j +f1k +.f2k +f1l +.f2l +f1m +.f2m +f1n +.f2n +f1p +.f2p +f1r +.f2r +f1s +.f2s +f1t +.f2t +f1w +.f2w +f1ý +.f2ý +f1z +.f2z +f1ň +.f2ň +f1ž +.f2ž +f1ş +.f2ş +g1b +.g2b +g1ç +.g2ç +g1d +.g2d +g1f +.g2f +g1g +.g2g +g1h +.g2h +g1j +.g2j +g1k +.g2k +g1l +.g2l +g1m +.g2m +g1n +.g2n +g1p +.g2p +g1r +.g2r +g1s +.g2s +g1t +.g2t +g1w +.g2w +g1ý +.g2ý +g1z +.g2z +g1ň +.g2ň +g1ž +.g2ž +g1ş +.g2ş +h1b +.h2b +h1ç +.h2ç +h1d +.h2d +h1f +.h2f +h1g +.h2g +h1h +.h2h +h1j +.h2j +h1k +.h2k +h1l +.h2l +h1m +.h2m +h1n +.h2n +h1p +.h2p +h1r +.h2r +h1s +.h2s +h1t +.h2t +h1w +.h2w +h1ý +.h2ý +h1z +.h2z +h1ň +.h2ň +h1ž +.h2ž +h1ş +.h2ş +j1b +.j2b +j1ç +.j2ç +j1d +.j2d +j1f +.j2f +j1g +.j2g +j1h +.j2h +j1j +.j2j +j1k +.j2k +j1l +.j2l +j1m +.j2m +j1n +.j2n +j1p +.j2p +j1r +.j2r +j1s +.j2s +j1t +.j2t +j1w +.j2w +j1ý +.j2ý +j1z +.j2z +j1ň +.j2ň +j1ž +.j2ž +j1ş +.j2ş +k1b +.k2b +k1ç +.k2ç +k1d +.k2d +k1f +.k2f +k1g +.k2g +k1h +.k2h +k1j +.k2j +k1k +.k2k +k1l +.k2l +k1m +.k2m +k1n +.k2n +k1p +.k2p +k1r +.k2r +k1s +.k2s +k1t +.k2t +k1w +.k2w +k1ý +.k2ý +k1z +.k2z +k1ň +.k2ň +k1ž +.k2ž +k1ş +.k2ş +l1b +.l2b +l1ç +.l2ç +l1d +.l2d +l1f +.l2f +l1g +.l2g +l1h +.l2h +l1j +.l2j +l1k +.l2k +l1l +.l2l +l1m +.l2m +l1n +.l2n +l1p +.l2p +l1r +.l2r +l1s +.l2s +l1t +.l2t +l1w +.l2w +l1ý +.l2ý +l1z +.l2z +l1ň +.l2ň +l1ž +.l2ž +l1ş +.l2ş +m1b +.m2b +m1ç +.m2ç +m1d +.m2d +m1f +.m2f +m1g +.m2g +m1h +.m2h +m1j +.m2j +m1k +.m2k +m1l +.m2l +m1m +.m2m +m1n +.m2n +m1p +.m2p +m1r +.m2r +m1s +.m2s +m1t +.m2t +m1w +.m2w +m1ý +.m2ý +m1z +.m2z +m1ň +.m2ň +m1ž +.m2ž +m1ş +.m2ş +n1b +.n2b +n1ç +.n2ç +n1d +.n2d +n1f +.n2f +n1g +.n2g +n1h +.n2h +n1j +.n2j +n1k +.n2k +n1l +.n2l +n1m +.n2m +n1n +.n2n +n1p +.n2p +n1r +.n2r +n1s +.n2s +n1t +.n2t +n1w +.n2w +n1ý +.n2ý +n1z +.n2z +n1ň +.n2ň +n1ž +.n2ž +n1ş +.n2ş +p1b +.p2b +p1ç +.p2ç +p1d +.p2d +p1f +.p2f +p1g +.p2g +p1h +.p2h +p1j +.p2j +p1k +.p2k +p1l +.p2l +p1m +.p2m +p1n +.p2n +p1p +.p2p +p1r +.p2r +p1s +.p2s +p1t +.p2t +p1w +.p2w +p1ý +.p2ý +p1z +.p2z +p1ň +.p2ň +p1ž +.p2ž +p1ş +.p2ş +r1b +.r2b +r1ç +.r2ç +r1d +.r2d +r1f +.r2f +r1g +.r2g +r1h +.r2h +r1j +.r2j +r1k +.r2k +r1l +.r2l +r1m +.r2m +r1n +.r2n +r1p +.r2p +r1r +.r2r +r1s +.r2s +r1t +.r2t +r1w +.r2w +r1ý +.r2ý +r1z +.r2z +r1ň +.r2ň +r1ž +.r2ž +r1ş +.r2ş +s1b +.s2b +s1ç +.s2ç +s1d +.s2d +s1f +.s2f +s1g +.s2g +s1h +.s2h +s1j +.s2j +s1k +.s2k +s1l +.s2l +s1m +.s2m +s1n +.s2n +s1p +.s2p +s1r +.s2r +s1s +.s2s +s1t +.s2t +s1w +.s2w +s1ý +.s2ý +s1z +.s2z +s1ň +.s2ň +s1ž +.s2ž +s1ş +.s2ş +t1b +.t2b +t1ç +.t2ç +t1d +.t2d +t1f +.t2f +t1g +.t2g +t1h +.t2h +t1j +.t2j +t1k +.t2k +t1l +.t2l +t1m +.t2m +t1n +.t2n +t1p +.t2p +t1r +.t2r +t1s +.t2s +t1t +.t2t +t1w +.t2w +t1ý +.t2ý +t1z +.t2z +t1ň +.t2ň +t1ž +.t2ž +t1ş +.t2ş +w1b +.w2b +w1ç +.w2ç +w1d +.w2d +w1f +.w2f +w1g +.w2g +w1h +.w2h +w1j +.w2j +w1k +.w2k +w1l +.w2l +w1m +.w2m +w1n +.w2n +w1p +.w2p +w1r +.w2r +w1s +.w2s +w1t +.w2t +w1w +.w2w +w1ý +.w2ý +w1z +.w2z +w1ň +.w2ň +w1ž +.w2ž +w1ş +.w2ş +ý1b +.ý2b +ý1ç +.ý2ç +ý1d +.ý2d +ý1f +.ý2f +ý1g +.ý2g +ý1h +.ý2h +ý1j +.ý2j +ý1k +.ý2k +ý1l +.ý2l +ý1m +.ý2m +ý1n +.ý2n +ý1p +.ý2p +ý1r +.ý2r +ý1s +.ý2s +ý1t +.ý2t +ý1w +.ý2w +ý1ý +.ý2ý +ý1z +.ý2z +ý1ň +.ý2ň +ý1ž +.ý2ž +ý1ş +.ý2ş +z1b +.z2b +z1ç +.z2ç +z1d +.z2d +z1f +.z2f +z1g +.z2g +z1h +.z2h +z1j +.z2j +z1k +.z2k +z1l +.z2l +z1m +.z2m +z1n +.z2n +z1p +.z2p +z1r +.z2r +z1s +.z2s +z1t +.z2t +z1w +.z2w +z1ý +.z2ý +z1z +.z2z +z1ň +.z2ň +z1ž +.z2ž +z1ş +.z2ş +ň1b +.ň2b +ň1ç +.ň2ç +ň1d +.ň2d +ň1f +.ň2f +ň1g +.ň2g +ň1h +.ň2h +ň1j +.ň2j +ň1k +.ň2k +ň1l +.ň2l +ň1m +.ň2m +ň1n +.ň2n +ň1p +.ň2p +ň1r +.ň2r +ň1s +.ň2s +ň1t +.ň2t +ň1w +.ň2w +ň1ý +.ň2ý +ň1z +.ň2z +ň1ň +.ň2ň +ň1ž +.ň2ž +ň1ş +.ň2ş +ž1b +.ž2b +ž1ç +.ž2ç +ž1d +.ž2d +ž1f +.ž2f +ž1g +.ž2g +ž1h +.ž2h +ž1j +.ž2j +ž1k +.ž2k +ž1l +.ž2l +ž1m +.ž2m +ž1n +.ž2n +ž1p +.ž2p +ž1r +.ž2r +ž1s +.ž2s +ž1t +.ž2t +ž1w +.ž2w +ž1ý +.ž2ý +ž1z +.ž2z +ž1ň +.ž2ň +ž1ž +.ž2ž +ž1ş +.ž2ş +ş1b +.ş2b +ş1ç +.ş2ç +ş1d +.ş2d +ş1f +.ş2f +ş1g +.ş2g +ş1h +.ş2h +ş1j +.ş2j +ş1k +.ş2k +ş1l +.ş2l +ş1m +.ş2m +ş1n +.ş2n +ş1p +.ş2p +ş1r +.ş2r +ş1s +.ş2s +ş1t +.ş2t +ş1w +.ş2w +ş1ý +.ş2ý +ş1z +.ş2z +ş1ň +.ş2ň +ş1ž +.ş2ž +ş1ş +.ş2ş +% Patterns for triple consonants. There may be additions to this category, as this list is not exhaustive. +ý2t1b +ý2n1b +ý2d1b +r2t1b +ý2p1b +l2p1b +l2t1b +g2t1b +n2t1b +r2k1b +r2p1b +k2t1b +r2h1b +s2t1b +l2k1b +w2p1b +n2s1b +r2s1b +l2m1b +ý2t1ç +ý2n1ç +ý2d1ç +r2t1ç +ý2p1ç +l2p1ç +l2t1ç +g2t1ç +n2t1ç +r2k1ç +r2p1ç +k2t1ç +r2h1ç +s2t1ç +l2k1ç +w2p1ç +n2s1ç +r2s1ç +l2m1ç +ý2t1d +ý2n1d +ý2d1d +r2t1d +ý2p1d +l2p1d +l2t1d +g2t1d +n2t1d +r2k1d +r2p1d +k2t1d +r2h1d +s2t1d +l2k1d +w2p1d +n2s1d +r2s1d +l2m1d +ý2t1g +ý2n1g +ý2d1g +r2t1g +ý2p1g +l2p1g +l2t1g +g2t1g +n2t1g +r2k1g +r2p1g +k2t1g +r2h1g +s2t1g +l2k1g +w2p1g +n2s1g +r2s1g +l2m1g +ý2t1j +ý2n1j +ý2d1j +r2t1j +ý2p1j +l2p1j +l2t1j +g2t1j +n2t1j +r2k1j +r2p1j +k2t1j +r2h1j +s2t1j +l2k1j +w2p1j +n2s1j +r2s1j +l2m1j +ý2t1k +ý2n1k +ý2d1k +r2t1k +ý2p1k +l2p1k +l2t1k +g2t1k +n2t1k +r2k1k +r2p1k +k2t1k +r2h1k +s2t1k +l2k1k +w2p1k +n2s1k +r2s1k +l2m1k +ý2t1l +ý2n1l +ý2d1l +r2t1l +ý2p1l +l2p1l +l2t1l +g2t1l +n2t1l +r2k1l +r2p1l +k2t1l +r2h1l +s2t1l +l2k1l +w2p1l +n2s1l +r2s1l +l2m1l +ý2t1m +ý2n1m +ý2d1m +r2t1m +ý2p1m +l2p1m +l2t1m +g2t1m +n2t1m +r2k1m +r2p1m +k2t1m +r2h1m +s2t1m +l2k1m +w2p1m +n2s1m +r2s1m +l2m1m +ý2t1n +ý2n1n +ý2d1n +r2t1n +ý2p1n +l2p1n +l2t1n +g2t1n +n2t1n +r2k1n +r2p1n +k2t1n +r2h1n +s2t1n +l2k1n +w2p1n +n2s1n +r2s1n +l2m1n +ý2t1p +ý2n1p +ý2d1p +r2t1p +ý2p1p +l2p1p +l2t1p +g2t1p +n2t1p +r2k1p +r2p1p +k2t1p +r2h1p +s2t1p +l2k1p +w2p1p +n2s1p +r2s1p +l2m1p +ý2t1s +ý2n1s +ý2d1s +r2t1s +ý2p1s +l2p1s +l2t1s +g2t1s +n2t1s +r2k1s +r2p1s +k2t1s +r2h1s +s2t1s +l2k1s +w2p1s +n2s1s +r2s1s +l2m1s +ý2t1t +ý2n1t +ý2d1t +r2t1t +ý2p1t +l2p1t +l2t1t +g2t1t +n2t1t +r2k1t +r2p1t +k2t1t +r2h1t +s2t1t +l2k1t +w2p1t +n2s1t +r2s1t +l2m1t +ý2t1ý +ý2n1ý +ý2d1ý +r2t1ý +ý2p1ý +l2p1ý +l2t1ý +g2t1ý +n2t1ý +r2k1ý +r2p1ý +k2t1ý +r2h1ý +s2t1ý +l2k1ý +w2p1ý +n2s1ý +r2s1ý +l2m1ý +ý2t1z +ý2n1z +ý2d1z +r2t1z +ý2p1z +l2p1z +l2t1z +g2t1z +n2t1z +r2k1z +r2p1z +k2t1z +r2h1z +s2t1z +l2k1z +w2p1z +n2s1z +r2s1z +l2m1z +ý2t1ş +ý2n1ş +ý2d1ş +r2t1ş +ý2p1ş +l2p1ş +l2t1ş +g2t1ş +n2t1ş +r2k1ş +r2p1ş +k2t1ş +r2h1ş +s2t1ş +l2k1ş +w2p1ş +n2s1ş +r2s1ş +l2m1ş +% Exceptions and single word occurence patterns for words of foreign origin i.e. Russian +s2k1d +l1s2k +l1s2t +s1t2r +n2g1l +n1g2r +s2k1w +} |