Evenki and English/Structural transfer
From LING073
Contents
Pre-Evaluation
Note: Due to the previously mentioned orthography issues with the Evenki transducer, our test results are artificially low. Since the transducer is imported from apertium and we did not created, this currently not a problem we can fix. However, our work could be applied in future work on Evenki if this issue were to be fixed.
------------------------------------------------------- Number of words in reference: 20 Number of words in test: 22 Number of unknown words (marked with a star) in test: 20 Percentage of unknown words: 90.91 %
Results when removing unknown-word marks (stars) ------------------------------------------------------- Edit distance: 22 Word error rate (WER): 110.00 % Number of position-independent correct words: 0 Position-independent word error rate (PER): 110.00 %
Structural Transfer Rules
We added a structural transfer rule to account for the different subject complement verb orders in Evenki and English.
Mode Outputs For Each Sentence
| sentence | tagger | lex | transfer | evn-eng |
|---|---|---|---|---|
| Тадук нунгартын аичимни чатын. | ^*Тадук$ ^*нунгартын$ ^*аичимни$ ^*чатын$. | ^*Тадук/*Тадук$ ^*нунгартын/*нунгартын$ ^*аичимни/*аичимни$ ^*чатын/*чатын$. | ^*Тадук$ ^*нунгартын$ ^*аичимни$ ^*чатын$. | *Тадук *нунгартын *аичимни *чатын. |
Post-Evaluation
------------------------------------------------------- Number of words in reference: 20 Number of words in test: 22 Number of unknown words (marked with a star) in test: 20 Percentage of unknown words: 90.91 %
Results when removing unknown-word marks (stars) ------------------------------------------------------- Edit distance: 22 Word error rate (WER): 110.00 % Number of position-independent correct words: 0 Position-independent word error rate (PER): 110.00 %
Note: Nothing improved when implementing structural transfer but this is likely due to the same issue from the transducer preventing words in both monolingual and bilingual dictionaries from mapping to any translation. Should that error be resolved significant improvement would likely be seen however fixing that issue is currently outside of the scope of our capabilities.
