Difference between revisions of "User:Yzhang4/Final project"
(→Table) |
(→Implementation Method) |
||
| (76 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
Our group's final project is a orthographic conversion tool from IPA to [https://en.wikipedia.org/wiki/Tibetan_script Tibetan Script] of Japhug based on lexd and twol of apertium. | Our group's final project is a orthographic conversion tool from IPA to [https://en.wikipedia.org/wiki/Tibetan_script Tibetan Script] of Japhug based on lexd and twol of apertium. | ||
| − | Note: in order to display Tibetan glyphs correctly, you should download and install [https://www.thlib.org/tools/scripts/wiki/tibetan%20machine%20uni.html this font]. | + | '''Note: in order to display Tibetan glyphs correctly, you should download and install [https://www.thlib.org/tools/scripts/wiki/tibetan%20machine%20uni.html this font].''' |
| − | =Description= | + | =Project Description= |
Currently, compared to IPA, Tibetan Script-based orthography is more accepted among Japhug speakers. Therefore, we developed the orthographic conversion tool to make our transducer/translator more accessible for the Japhug community. | Currently, compared to IPA, Tibetan Script-based orthography is more accepted among Japhug speakers. Therefore, we developed the orthographic conversion tool to make our transducer/translator more accessible for the Japhug community. | ||
| − | ==Transcription | + | ==Transcription Scheme== |
| − | Our orthographic conversion is based on the orthography proposed by Guillaume Jacques, from [https://www.academia.edu/46628296/A_grammar_of_Japhug A Grammar of Japhug(2021)] p.105 to p.106 | + | Our orthographic conversion is based on the orthography proposed by Guillaume Jacques, from [https://www.academia.edu/46628296/A_grammar_of_Japhug A Grammar of Japhug(2021)] p.105 to p.106. |
| + | We try to follow the conventions for spelling Tibetan as much as possible, like keeping the vowel in the first or second consonant of a syllable. | ||
| + | |||
| + | ===Main differences from Standard Tibetan script=== | ||
This transcription aims to minimize ambiguity, maximizing one-to-one map from sounds to orthography. Because the phonology of Japhug is richer than Standard Tibetan, in order to reduce ambiguity, this transcription uses some archaic Tibetan letters and some Tibetan letters for Sanskrit. For example, the archaic Tibetan diacritic ྀ represents vowel /ɯ/, and the letter ཊ that used to represent Sanskrit sound /ʈ/ represents /tʂ/ in this transcription. | This transcription aims to minimize ambiguity, maximizing one-to-one map from sounds to orthography. Because the phonology of Japhug is richer than Standard Tibetan, in order to reduce ambiguity, this transcription uses some archaic Tibetan letters and some Tibetan letters for Sanskrit. For example, the archaic Tibetan diacritic ྀ represents vowel /ɯ/, and the letter ཊ that used to represent Sanskrit sound /ʈ/ represents /tʂ/ in this transcription. | ||
| − | + | Since vowel /ɤ/ is more common than /a/ in Japhug, we use /ɤ/ as the base vowel of each syllable (a.k.a [https://en.wikipedia.org/wiki/Aksara akshara]) instead of /a/ in Standard Tibetan. On the other hand, we use the long vowel diacritic ཱ to represent /a/. | |
| + | |||
| + | There are no uvular consonants in Standard Tibetan. In order to represent uvular consonants, we use the symbol "༹" to distinguish uvulars from their velar counterparts. For example, "q" is transcribed as "ཀ༹", corresponding to "k" that transcribes to "ཀ". | ||
| + | |||
| + | When continuant velar and uvular consonants are at the end of a syllable, they become "ག". | ||
| + | |||
| + | Japhug consonants can be prenasalized. In this transcription, we use letter འ to represent prenasalization. For example, "znde" should be transcribed as "འསྡེ". | ||
| + | |||
| + | There is no distinction between palatal and alveo-palatal sounds in standard Tibetan. We emailed Jacques, and he suggested us use velar+j to represent palatals, while keep alveo-palatals the same as standard Tibetan, because velar+i/j is very rare in Japhug. For example, palatal consonant "c" is transcribed as "ཀྱ", and its alveo-palatal counterpart "tɕ" is "ཅ" | ||
===Table=== | ===Table=== | ||
| − | The following table tells about the details of the transcription: | + | The following table tells about the details of the transcription for consonants. Colored entries are consonants not in Standard Tibetan: |
{| class="wikitable" style="text-align:center;" | {| class="wikitable" style="text-align:center;" | ||
!colspan=2 rowspan=2| | !colspan=2 rowspan=2| | ||
| Line 63: | Line 74: | ||
|- | |- | ||
!<small>prenasalized</small> | !<small>prenasalized</small> | ||
| − | |ᵐb | + | !style="background:beige;"|ᵐb |
| − | |འཔ | + | !style="background:beige;"|འཔ |
| − | |ⁿd | + | !style="background:beige;"|ⁿd |
| − | |འད | + | !style="background:beige;"|འད |
| − | |ⁿdz | + | !style="background:beige;"|ⁿdz |
| − | |འཛ | + | !style="background:beige;"|འཛ |
| − | |ᶯɖʐ | + | !style="background:beige;"|ᶯɖʐ |
| − | |འཌ | + | !style="background:beige;"|འཌ |
| − | |ᶮdʑ | + | !style="background:beige;"|ᶮdʑ |
| − | |འཇ | + | !style="background:beige;"|འཇ |
| − | |ᶮɟ | + | !style="background:beige;"|ᶮɟ |
| − | |འཇ | + | !style="background:beige;"|འཇ |
| − | |ᵑɡ | + | !style="background:beige;"|ᵑɡ |
| − | |འག | + | !style="background:beige;"|འག |
| − | |ᶰɢ | + | !style="background:beige;"|ᶰɢ |
| − | |འག༹ | + | !style="background:beige;"|འག༹ |
|- | |- | ||
!<small>voiced</small> | !<small>voiced</small> | ||
| Line 87: | Line 98: | ||
|dz | |dz | ||
|ཛ | |ཛ | ||
| − | |dʐ | + | !style="background:beige;"|dʐ |
| − | |ཌ | + | !style="background:beige;"|ཌ |
|dʑ | |dʑ | ||
|ཇ | |ཇ | ||
| − | |ɟ | + | !style="background:beige;"|ɟ |
| − | | | + | !style="background:beige;"|གྱ |
|ɡ | |ɡ | ||
|ག | |ག | ||
| Line 105: | Line 116: | ||
|ts | |ts | ||
|ཙ | |ཙ | ||
| − | |tʂ | + | !style="background:beige;"|tʂ |
| − | |ཊ | + | !style="background:beige;"|ཊ |
|tɕ | |tɕ | ||
|ཅ | |ཅ | ||
| − | |c | + | !style="background:beige;"|c |
| − | | | + | !style="background:beige;"|ཀྱ |
|k | |k | ||
|ཀ | |ཀ | ||
| − | |q | + | !style="background:beige;"|q |
| − | |ཀ༹ | + | !style="background:beige;"|ཀ༹ |
|- | |- | ||
!<small>aspirated</small> | !<small>aspirated</small> | ||
| − | + | |pʰ | |
| − | | | + | |ཕ |
| − | | | + | |tʰ |
| − | | | + | |ཐ |
| − | | | + | |tsʰ |
| − | | | + | |ཚ |
| − | | | + | !style="background:beige;"|tʂʰ |
| − | | | + | !style="background:beige;"|ཋ |
| + | |tɕʰ | ||
| + | |ཆ | ||
| + | !style="background:beige;"|cʰ | ||
| + | !style="background:beige;"|ཁྱ | ||
| + | |kʰ | ||
| + | |ཁ | ||
| + | !style="background:beige;"|qʰ | ||
| + | !style="background:beige;"|ཁ༹ | ||
|- | |- | ||
!rowspan=2|Continuant | !rowspan=2|Continuant | ||
!<small>voiced</small> | !<small>voiced</small> | ||
| − | |w | + | |w/β/f |
| + | |བ༹/བ/ཕ༹ | ||
|l | |l | ||
| + | |ལ | ||
|z | |z | ||
| + | |ཟ | ||
|r | |r | ||
| + | |ར | ||
|ʑ | |ʑ | ||
| + | |ཞ | ||
|j | |j | ||
| − | |ɣ | + | |ཡ |
| − | |ʁ | + | !style="background:beige;"|ɣ |
| + | !style="background:beige;"|ཝ | ||
| + | !style="background:beige;"|ʁ | ||
| + | !style="background:beige;"|ཝ༹/འ/ག | ||
|- | |- | ||
!<small>voiceless</small> | !<small>voiceless</small> | ||
| + | | | ||
| | | | ||
|ɬ | |ɬ | ||
| + | |ལྷ | ||
|s | |s | ||
| + | |ས | ||
|ʂ | |ʂ | ||
| + | |ཥ | ||
|ɕ | |ɕ | ||
| + | |ཤ | ||
| | | | ||
| − | |x | + | | |
| − | |χ | + | !style="background:beige;"|x |
| + | !style="background:beige;"|ཧ/ག | ||
| + | !style="background:beige;"|χ | ||
| + | !style="background:beige;"|ཧ༹/ག | ||
| + | |} | ||
| + | |||
| + | The following table is the scheme for vowel transcription. Colored entries are those different or new to Standard Tibetan: | ||
| + | {| class="wikitable" | ||
| + | |- style="text-align:center;" | ||
| + | !IPA | ||
| + | !Vowel mark | ||
| + | !IPA | ||
| + | !Vowel mark | ||
| + | !IPA | ||
| + | !Vowel mark | ||
| + | !IPA | ||
| + | !Vowel mark | ||
| + | !IPA | ||
| + | !Vowel mark | ||
| + | !IPA | ||
| + | !Vowel mark | ||
| + | !IPA | ||
| + | !Vowel mark | ||
| + | !IPA | ||
| + | !Vowel mark | ||
| + | |- style="text-align:center;" | ||
| + | !style="background:beige;"| ɤ | ||
| + | !style="background:beige;"| N/A | ||
| + | !style="background:beige;"| a | ||
| + | !style="background:beige;"| ཱ | ||
| + | | i | ||
| + | | ི | ||
| + | | u | ||
| + | | ུ | ||
| + | !style="background:beige;"| y | ||
| + | !style="background:beige;"| ུ | ||
| + | !style="background:beige;"| ɯ | ||
| + | !style="background:beige;"| ྀ | ||
| + | | e | ||
| + | | ེ | ||
| + | | o | ||
| + | | ོ | ||
|} | |} | ||
| + | |||
| + | When a vowel is not preceded by any consonants, use འ with the corresponding vowel mark. For example, "andi" is transcribed as "འཱ༌འདི". | ||
| + | |||
| + | Because y is extremely uncommon in Japhug, we choose to represent it in the same way as u. | ||
==Implementation Method== | ==Implementation Method== | ||
| + | |||
| + | We built a mini-transducer based on apertium to implement IPA to Tibetan Script transcriber. Specifically, we hard coded the rules for transcription in lexd and twol files. Moreover, because there are ambiguation in transcribing consonant clusters, we also used an xfst file to deal with that kind of ambiguation. | ||
| + | |||
| + | Currently, we have about 17 patterns with 164 lexicons in the lexd file, 1 twol rule and 63 disambiguation rules in the xfst file. | ||
| + | |||
| + | =Installation= | ||
| + | You can install the orthographical convertor from our [https://github.swarthmore.edu/Ling073-sp23/ling073-jya Japhug language transducer]. | ||
| + | |||
| + | After clone the transducer and make it, you can use the following command to do transcription: | ||
| + | <pre> | ||
| + | apertium -d . jya_Latn-jya_Tibt | ||
| + | </pre> | ||
| + | |||
| + | For example, to transcribe a full Japhug sentence: | ||
| + | <pre> | ||
| + | echo "tɤrʑaβ nɯ pjɤrɤɕphɤt" | apertium -d . jya_Latn-jya_Tibt | ||
| + | </pre> | ||
| + | |||
| + | You'll expect to get | ||
| + | <pre> | ||
| + | ཏ༌རྮཱབ༌ ནྀ༌ པྱ༌ར༌ཤྥཏ། | ||
| + | </pre> | ||
=Evaluation= | =Evaluation= | ||
| + | Since there are not many existing texts of Japhug in this transcription, we'll just transcribe a sample corpus ([https://github.swarthmore.edu/Ling073-sp23/ling073-jya-corpus/blob/main/jya.corpus.basic.txt jya.corpus.basic.txt]) from IPA to Tibetan script. After than, we manually change each transcribed word into the most correct form, to make the target transcription file ([https://github.swarthmore.edu/Ling073-sp23/ling073-jya-corpus/blob/main/jya.corpus.basic.tibt.target.txt jya.corpus.basic.tibt.target.txt]). | ||
| + | |||
| + | Finally, we directly check the word error rate of the transcriber by comparing the target transcription with the transcription generated by the transcriber using [https://github.swarthmore.edu/Ling073-sp23/ling073-jya-corpus/blob/main/WER_calculator.py this script]. The results are shown below: | ||
| + | |||
| + | <pre> | ||
| + | Total tokens = 2503 | ||
| + | Matched tokens = 2478 | ||
| + | Not Matched tokens = 25 | ||
| + | Word Error Rate = %0.9988014382740711 | ||
| + | </pre> | ||
| + | |||
| + | Most of the mis-matches have something to do with consonant clusters. For example, the word "ɲɯɤznɤkharnɯ" should be expected to have transcription "ཉྀ༌འ༌འས་ཁཱ་རྣྀ་", but it generates incorrect form "ཉྀ༌འ༌ཟའཀ༌ཧཱ྇༌རནྀ༌" in our transcriber, in which prenasalization "zn" is not transcribed correctly, and aspiration consonant "ཁ(kh)" is divided into two parts and becomes "ཀ(k)" and "ཧ྇(h)" separately. | ||
| + | When transcribe back from IPA to Tibetan Script, the WER is much higher: | ||
| + | <pre> | ||
| + | Total tokens = 2503 | ||
| + | Matched tokens = 1751 | ||
| + | Not Matched tokens = 752 | ||
| + | Word Error Rate = %30.04394726328406 | ||
| + | </pre> | ||
| + | |||
| + | We also tested the coverage of the multiorthographical transducer in the corpus: | ||
| + | <pre> | ||
| + | TOP UNKNOWN WORDS: | ||
| + | 13 ^ཉྀ༌ངུ།/*ཉྀ༌ངུ།$ | ||
| + | 8 ^རྒྱ༌ལྤུ༌/*རྒྱ༌ལྤུ༌$ | ||
| + | 8 ^རྐྱཱ༌ནྀ༌/*རྐྱཱ༌ནྀ༌$ | ||
| + | 7 ^ཅན༌/*ཅན༌$ | ||
| + | 6 ^ཁེ༹༌/*ཁེ༹༌$ | ||
| + | 5 ^ལ༌འཱ༌རི༌/*ལ༌འཱ༌རི༌$ | ||
| + | 5 ^འད༌/*འད༌$ | ||
| + | 5 ^ཝྀ༌/*ཝྀ༌$ | ||
| + | 5 ^ཏྀ༌ཝཡན༌/*ཏྀ༌ཝཡན༌$ | ||
| + | 5 ^ཁྱོ༌/*ཁྱོ༌$ | ||
| + | 5 ^ཀྀ༌སྠྀ༌ཀྱི༌/*ཀྀ༌སྠྀ༌ཀྱི༌$ | ||
| + | 4 ^ནྀ༌ནཱཡ༌/*ནྀ༌ནཱཡ༌$ | ||
| + | 4 ^ཏུ༌བཟེ༌འཱ༌/*ཏུ༌བཟེ༌འཱ༌$ | ||
| + | 4 ^ཀྀ༌ཀྀ༌ཏུ༌/*ཀྀ༌ཀྀ༌ཏུ༌$ | ||
| + | 3 ^ཤྤྀ༌ནྀ༌ཀྲཱ༌ཏཱ༌/*ཤྤྀ༌ནྀ༌ཀྲཱ༌ཏཱ༌$ | ||
| + | 3 ^ཡ༌ཝེ༌/*ཡ༌ཝེ༌$ | ||
| + | 3 ^མྀ༌ཏོ༌ཁྀ༌/*མྀ༌ཏོ༌ཁྀ༌$ | ||
| + | 3 ^མཉི༌ཀཱ༌/*མཉི༌ཀཱ༌$ | ||
| + | 3 ^ནྀ༌ཕེ༌/*ནྀ༌ཕེ༌$ | ||
| + | 3 ^ཉྀ༌སྀ༌སམ༌/*ཉྀ༌སྀ༌སམ༌$ | ||
| + | coverage: 2344 / 2656 (~0.88253012048192771084) | ||
| + | remaining unknown forms: 312 | ||
| + | ambiguity: 5416 / 2656 (~2.03915662650602409639) | ||
| + | jya.corpus.basic.tibt Mon May 8 07:10:04 PM EDT 2023 2344/2656 ~0.88253012048192771084 \| 5416/2656 ~2.03915662650602409639 | ||
| + | </pre> | ||
| + | |||
| + | =Future Improvements= | ||
| + | |||
| + | For some Japhug expressions with multiple consonants between vowels, there still exists ambiguity. For example, the word "paχɕi" (apple) can be transcribed as either "པཱ༌ཧ༹ཤི" when treating the consonant χ as the first consonant of the second syllable, or "པཱག༌ཤི" if treating χ as the last consonant of the first syllable. We'll try to figure out some ways to standardize that. | ||
| + | |||
| + | We need to develop a keyboard layout for the Japhug Tibetan Script, which is slightly different than that for Central Tibetan. | ||
| + | |||
| + | Also, we need to hear more feedback from the Japhug community, and promote the Japhug Tibetan Script Among Japhug speakers. | ||
=External Resources= | =External Resources= | ||
| − | + | [https://github.swarthmore.edu/Ling073-sp23/ling073-jya-corpus Japhug Corpus and Relating Tools] | |
| + | |||
| + | [https://wikis.swarthmore.edu/ling073/Japhug Japhug Grammar resources] | ||
| + | |||
| + | [https://github.swarthmore.edu/Ling073-sp23/ling073-jya Japhug Transducer] | ||
| + | |||
| + | [https://www.thlib.org/tools/scripts/wiki/tibetan%20machine%20uni.html Tibetan Font] | ||
[[Category:sp23_FinalProjects]] [[Category:Japhug]] | [[Category:sp23_FinalProjects]] [[Category:Japhug]] | ||
Latest revision as of 00:08, 9 May 2023
Our group's final project is a orthographic conversion tool from IPA to Tibetan Script of Japhug based on lexd and twol of apertium.
Note: in order to display Tibetan glyphs correctly, you should download and install this font.
Contents
Project Description
Currently, compared to IPA, Tibetan Script-based orthography is more accepted among Japhug speakers. Therefore, we developed the orthographic conversion tool to make our transducer/translator more accessible for the Japhug community.
Transcription Scheme
Our orthographic conversion is based on the orthography proposed by Guillaume Jacques, from A Grammar of Japhug(2021) p.105 to p.106.
We try to follow the conventions for spelling Tibetan as much as possible, like keeping the vowel in the first or second consonant of a syllable.
Main differences from Standard Tibetan script
This transcription aims to minimize ambiguity, maximizing one-to-one map from sounds to orthography. Because the phonology of Japhug is richer than Standard Tibetan, in order to reduce ambiguity, this transcription uses some archaic Tibetan letters and some Tibetan letters for Sanskrit. For example, the archaic Tibetan diacritic ྀ represents vowel /ɯ/, and the letter ཊ that used to represent Sanskrit sound /ʈ/ represents /tʂ/ in this transcription.
Since vowel /ɤ/ is more common than /a/ in Japhug, we use /ɤ/ as the base vowel of each syllable (a.k.a akshara) instead of /a/ in Standard Tibetan. On the other hand, we use the long vowel diacritic ཱ to represent /a/.
There are no uvular consonants in Standard Tibetan. In order to represent uvular consonants, we use the symbol "༹" to distinguish uvulars from their velar counterparts. For example, "q" is transcribed as "ཀ༹", corresponding to "k" that transcribes to "ཀ".
When continuant velar and uvular consonants are at the end of a syllable, they become "ག".
Japhug consonants can be prenasalized. In this transcription, we use letter འ to represent prenasalization. For example, "znde" should be transcribed as "འསྡེ".
There is no distinction between palatal and alveo-palatal sounds in standard Tibetan. We emailed Jacques, and he suggested us use velar+j to represent palatals, while keep alveo-palatals the same as standard Tibetan, because velar+i/j is very rare in Japhug. For example, palatal consonant "c" is transcribed as "ཀྱ", and its alveo-palatal counterpart "tɕ" is "ཅ"
Table
The following table tells about the details of the transcription for consonants. Colored entries are consonants not in Standard Tibetan:
| Labial | Dental | Alveolar | Retroflex | Alveopalatal | Palatal | Velar | Uvular | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| IPA | Letter | IPA | Letter | IPA | Letter | IPA | Letter | IPA | Letter | IPA | Letter | IPA | Letter | IPA | Letter | ||
| Occlusive | nasal | m | མ | n | ན | ɲ | ཉ | ŋ | ང | ||||||||
| prenasalized | ᵐb | འཔ | ⁿd | འད | ⁿdz | འཛ | ᶯɖʐ | འཌ | ᶮdʑ | འཇ | ᶮɟ | འཇ | ᵑɡ | འག | ᶰɢ | འག༹ | |
| voiced | b | བ | d | ད | dz | ཛ | dʐ | ཌ | dʑ | ཇ | ɟ | གྱ | ɡ | ག | |||
| voiceless | p | པ | t | ཏ | ts | ཙ | tʂ | ཊ | tɕ | ཅ | c | ཀྱ | k | ཀ | q | ཀ༹ | |
| aspirated | pʰ | ཕ | tʰ | ཐ | tsʰ | ཚ | tʂʰ | ཋ | tɕʰ | ཆ | cʰ | ཁྱ | kʰ | ཁ | qʰ | ཁ༹ | |
| Continuant | voiced | w/β/f | བ༹/བ/ཕ༹ | l | ལ | z | ཟ | r | ར | ʑ | ཞ | j | ཡ | ɣ | ཝ | ʁ | ཝ༹/འ/ག |
| voiceless | ɬ | ལྷ | s | ས | ʂ | ཥ | ɕ | ཤ | x | ཧ/ག | χ | ཧ༹/ག | |||||
The following table is the scheme for vowel transcription. Colored entries are those different or new to Standard Tibetan:
| IPA | Vowel mark | IPA | Vowel mark | IPA | Vowel mark | IPA | Vowel mark | IPA | Vowel mark | IPA | Vowel mark | IPA | Vowel mark | IPA | Vowel mark |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ɤ | N/A | a | ཱ | i | ི | u | ུ | y | ུ | ɯ | ྀ | e | ེ | o | ོ |
When a vowel is not preceded by any consonants, use འ with the corresponding vowel mark. For example, "andi" is transcribed as "འཱ༌འདི".
Because y is extremely uncommon in Japhug, we choose to represent it in the same way as u.
Implementation Method
We built a mini-transducer based on apertium to implement IPA to Tibetan Script transcriber. Specifically, we hard coded the rules for transcription in lexd and twol files. Moreover, because there are ambiguation in transcribing consonant clusters, we also used an xfst file to deal with that kind of ambiguation.
Currently, we have about 17 patterns with 164 lexicons in the lexd file, 1 twol rule and 63 disambiguation rules in the xfst file.
Installation
You can install the orthographical convertor from our Japhug language transducer.
After clone the transducer and make it, you can use the following command to do transcription:
apertium -d . jya_Latn-jya_Tibt
For example, to transcribe a full Japhug sentence:
echo "tɤrʑaβ nɯ pjɤrɤɕphɤt" | apertium -d . jya_Latn-jya_Tibt
You'll expect to get
ཏ༌རྮཱབ༌ ནྀ༌ པྱ༌ར༌ཤྥཏ།
Evaluation
Since there are not many existing texts of Japhug in this transcription, we'll just transcribe a sample corpus (jya.corpus.basic.txt) from IPA to Tibetan script. After than, we manually change each transcribed word into the most correct form, to make the target transcription file (jya.corpus.basic.tibt.target.txt).
Finally, we directly check the word error rate of the transcriber by comparing the target transcription with the transcription generated by the transcriber using this script. The results are shown below:
Total tokens = 2503 Matched tokens = 2478 Not Matched tokens = 25 Word Error Rate = %0.9988014382740711
Most of the mis-matches have something to do with consonant clusters. For example, the word "ɲɯɤznɤkharnɯ" should be expected to have transcription "ཉྀ༌འ༌འས་ཁཱ་རྣྀ་", but it generates incorrect form "ཉྀ༌འ༌ཟའཀ༌ཧཱ྇༌རནྀ༌" in our transcriber, in which prenasalization "zn" is not transcribed correctly, and aspiration consonant "ཁ(kh)" is divided into two parts and becomes "ཀ(k)" and "ཧ྇(h)" separately. When transcribe back from IPA to Tibetan Script, the WER is much higher:
Total tokens = 2503 Matched tokens = 1751 Not Matched tokens = 752 Word Error Rate = %30.04394726328406
We also tested the coverage of the multiorthographical transducer in the corpus:
TOP UNKNOWN WORDS:
13 ^ཉྀ༌ངུ།/*ཉྀ༌ངུ།$
8 ^རྒྱ༌ལྤུ༌/*རྒྱ༌ལྤུ༌$
8 ^རྐྱཱ༌ནྀ༌/*རྐྱཱ༌ནྀ༌$
7 ^ཅན༌/*ཅན༌$
6 ^ཁེ༹༌/*ཁེ༹༌$
5 ^ལ༌འཱ༌རི༌/*ལ༌འཱ༌རི༌$
5 ^འད༌/*འད༌$
5 ^ཝྀ༌/*ཝྀ༌$
5 ^ཏྀ༌ཝཡན༌/*ཏྀ༌ཝཡན༌$
5 ^ཁྱོ༌/*ཁྱོ༌$
5 ^ཀྀ༌སྠྀ༌ཀྱི༌/*ཀྀ༌སྠྀ༌ཀྱི༌$
4 ^ནྀ༌ནཱཡ༌/*ནྀ༌ནཱཡ༌$
4 ^ཏུ༌བཟེ༌འཱ༌/*ཏུ༌བཟེ༌འཱ༌$
4 ^ཀྀ༌ཀྀ༌ཏུ༌/*ཀྀ༌ཀྀ༌ཏུ༌$
3 ^ཤྤྀ༌ནྀ༌ཀྲཱ༌ཏཱ༌/*ཤྤྀ༌ནྀ༌ཀྲཱ༌ཏཱ༌$
3 ^ཡ༌ཝེ༌/*ཡ༌ཝེ༌$
3 ^མྀ༌ཏོ༌ཁྀ༌/*མྀ༌ཏོ༌ཁྀ༌$
3 ^མཉི༌ཀཱ༌/*མཉི༌ཀཱ༌$
3 ^ནྀ༌ཕེ༌/*ནྀ༌ཕེ༌$
3 ^ཉྀ༌སྀ༌སམ༌/*ཉྀ༌སྀ༌སམ༌$
coverage: 2344 / 2656 (~0.88253012048192771084)
remaining unknown forms: 312
ambiguity: 5416 / 2656 (~2.03915662650602409639)
jya.corpus.basic.tibt Mon May 8 07:10:04 PM EDT 2023 2344/2656 ~0.88253012048192771084 \| 5416/2656 ~2.03915662650602409639
Future Improvements
For some Japhug expressions with multiple consonants between vowels, there still exists ambiguity. For example, the word "paχɕi" (apple) can be transcribed as either "པཱ༌ཧ༹ཤི" when treating the consonant χ as the first consonant of the second syllable, or "པཱག༌ཤི" if treating χ as the last consonant of the first syllable. We'll try to figure out some ways to standardize that.
We need to develop a keyboard layout for the Japhug Tibetan Script, which is slightly different than that for Central Tibetan.
Also, we need to hear more feedback from the Japhug community, and promote the Japhug Tibetan Script Among Japhug speakers.