diff options
Diffstat (limited to 'Master/texmf-dist/source/fonts/malayalam/preproc/unicode/UNICODE.TXT')
-rw-r--r-- | Master/texmf-dist/source/fonts/malayalam/preproc/unicode/UNICODE.TXT | 161 |
1 files changed, 161 insertions, 0 deletions
diff --git a/Master/texmf-dist/source/fonts/malayalam/preproc/unicode/UNICODE.TXT b/Master/texmf-dist/source/fonts/malayalam/preproc/unicode/UNICODE.TXT new file mode 100644 index 00000000000..a86da65dd77 --- /dev/null +++ b/Master/texmf-dist/source/fonts/malayalam/preproc/unicode/UNICODE.TXT @@ -0,0 +1,161 @@ + +Subject: UNICODE-tools/SGML for language tagging in UNICODE + + +Since I am trying to process several languages in TeX at the same +time, and getting a bit fed up of several preprocessors, etc. +I want to make one universal, table driven pre-processor in awaitance of +a TeX that really handles things correctly. I also want to have +it produce Unicode files so that I have a good standard for storage, file +exchange, editing, and so on. + +I think (dream?) of some tools to help me in this work. + +transcribe + translate to/from transcription in ASCII to Unicode + given some transcription tables for the languages/scripts + used in the document. It should do much more than placing 00 in front + of every byte, also in English. It should replace quotes, dashes + etc for the correct signs, interprete TeX commands for accented + letters, and so on. +pretex + pre-process a unicode file for TeX, given some script tables + that describe the scripts used as it is encoded in a TeX-font +sorttext + sort a unicode file according to the rules of a given language, + on a line by line or paragraph by paragraph base. +atou + create `unicode' files by placing 00 in front of every byte. +utoa + create `ascii' files by stripping MS-byte. + + +In the original files I create (in ASCII), I want to tag the text with +SGML-style tags indicating the languages used. Transcribe reads +this and translates my transcription to appropriate Unicode characters. +(Including replacing TeX commands for accented letters for appropriate +accents/accented characters in Unicode.) Given a transcript file for +that language. +A transcript file includes a list of letters and their UNICODE-values +It knows how to handle the syllabic nature of the script, although it is +transcribed in an alphabetic script. +The default language is english (but not english.american). + +Need to decide on file formats and contents of transcript files + +I can sort the resulting files with sorttext, given a sort-key for a +certain language. + +Need to decide on file formats and contents of sort-key files + +I can pre-process the files for TeX with pretex, which will turn the files +into ASCII again, but places codes for glyphs and font changes as +neccessary for non-latin scripts, as indicated in the script file. +For latin languages it could take care of all kinds of conventions, like the +use of ligatures, hypenation, spacing after sentencesm etc. The tags could be +used by spell checkers. + +Need to decide on file formats and contents of script files (defining +context dependent behaviour of characters, locations of glyphs in fonts, +etc.) + +---- + +SGML-style Language tags in UNICODE + +how to? (are the indicators floating marks or block formers?) + +<malayalam>Malayalam text</malayalam> +<block language=malayalam>Malayalam text</block> + +it will keep those language-markers in the UNICODE file. + +known languages: (languages are sometimes known in several dialects or +otherwish variations, I suggest some kind of standard here) + +dutch +english +english.american +english.phonetic +french +german +german.fractur +hindi +hindi.transcription +bahasa-indonesia +malay +malay.arabic +malayalam +malayalam.traditional +malayalam.transcription +marathi +sanskrit +sanskrit.transcription +urdu + +unknown +unknown.transcribed + +---- + +UNICODE representation of Malayalam + +U+0D00 -- U+0D7F + +structure assumed in parsing: + +vowel +consonant +vowel-sign +virama +diacritic +joiner +non-joiner +other + +syntax: + +primary ::== ( <consonant> | <vowel> ) [diacritic] +secondary ::== ( <vowel-sign> | <virama> ) [diacritic] +consonant-cluster ::== <primary> { [<join>|<non-join>] <virama> <primary> } | <other> +modifiers ::== { [<join>] <secondary> } +syllabe ::== <consonant-cluster> [<modifiers>] +text ::== { <syllabe> } + +semantics: + +The algorithm reads a syllabe from the string. + +build-syllabe syllabe = + if prebuild syllabe + glyphs = prebuild syllabe + else + cluster-glyphs, rest-modifiers = build-cluster cluster with modifiers + glyphs = apply rest-modifiers to cluster-glyphs + endif + +build-cluster cluster with modifiers = + if prebuild cluster + glyphs = prebuild syllabe + else + split-off final ya, ra, la, try again + else + split-off front ra (for repham), try again + else + split first letter from + build-syllabe first-part with virama + build-syllabe second-part with modifiers + rest-modifiers = nil + endif + +apply modifiers to glyphs = + apply first-modifier to glyphs + apply rest-of-modifiers to glyphs + + + + +The algorithm ignores joins, non-join will cause virama's to be +used + +<eof> |