Difference between revisions of "Evenki and English/Structural transfer"
From LING073
(→Pre-Evaluation) |
|||
| Line 15: | Line 15: | ||
Number of position-independent correct words: 0 | Number of position-independent correct words: 0 | ||
Position-independent word error rate (PER): 110.00 % | Position-independent word error rate (PER): 110.00 % | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
=== Structural Transfer Rules === | === Structural Transfer Rules === | ||
| − | + | We added a structural transfer rule to account for the different subject complement verb orders in Evenki and English. | |
| − | |||
| − | |||
| − | |||
=== Mode Outputs For Each Sentence=== | === Mode Outputs For Each Sentence=== | ||
| Line 38: | Line 29: | ||
! sentence !! tagger !! lex !! transfer !! evn-eng | ! sentence !! tagger !! lex !! transfer !! evn-eng | ||
|- | |- | ||
| − | | | + | | Тадук нунгартын аичимни чатын. || ^*Тадук$ ^*нунгартын$ ^*аичимни$ ^*чатын$. || ^*Тадук/*Тадук$ ^*нунгартын/*нунгартын$ ^*аичимни/*аичимни$ ^*чатын/*чатын$. || ^*Тадук$ ^*нунгартын$ ^*аичимни$ ^*чатын$. || *Тадук *нунгартын *аичимни *чатын. |
| − | |||
| − | |||
| − | |||
| − | |||
|- | |- | ||
|} | |} | ||
| Line 49: | Line 36: | ||
| − | + | ------------------------------------------------------- | |
| − | + | Number of words in reference: 20 | |
| − | + | Number of words in test: 22 | |
| − | + | Number of unknown words (marked with a star) in test: 20 | |
| − | + | Percentage of unknown words: 90.91 % | |
| − | + | ||
| − | + | Results when removing unknown-word marks (stars) | |
| + | ------------------------------------------------------- | ||
| + | Edit distance: 22 | ||
| + | Word error rate (WER): 110.00 % | ||
| + | Number of position-independent correct words: 0 | ||
| + | Position-independent word error rate (PER): 110.00 % | ||
| + | |||
| + | Note: Nothing improved when implementing structural transfer but this is likely due to the same orthographical issue from the transducer. Should | ||
| + | that error be resolved significant improvement would likely be seen. | ||
| + | [[Category:Sp23_StructuralTransfer ]][[Category:evenki]][[Category:english]] | ||
[[Category:Sp23_StructuralTransfer ]][[Category:evenki]][[Category:english]] | [[Category:Sp23_StructuralTransfer ]][[Category:evenki]][[Category:english]] | ||
Revision as of 14:48, 11 May 2023
Contents
Pre-Evaluation
Note: Due to the previously mentioned orthography issues with the Evenki transducer, our test results are artificially low. Since the transducer is imported from apertium and we did not created, this currently not a problem we can fix. However, our work could be applied in future work on Evenki if this issue were to be fixed.
------------------------------------------------------- Number of words in reference: 20 Number of words in test: 22 Number of unknown words (marked with a star) in test: 20 Percentage of unknown words: 90.91 %
Results when removing unknown-word marks (stars) ------------------------------------------------------- Edit distance: 22 Word error rate (WER): 110.00 % Number of position-independent correct words: 0 Position-independent word error rate (PER): 110.00 %
Structural Transfer Rules
We added a structural transfer rule to account for the different subject complement verb orders in Evenki and English.
Mode Outputs For Each Sentence
Add at least one example sentence for each item you implement. Show the outputs of the following modes for your translation system: tagger, lex, transfer, and the pair itself (abc-xyz).
| sentence | tagger | lex | transfer | evn-eng |
|---|---|---|---|---|
| Тадук нунгартын аичимни чатын. | ^*Тадук$ ^*нунгартын$ ^*аичимни$ ^*чатын$. | ^*Тадук/*Тадук$ ^*нунгартын/*нунгартын$ ^*аичимни/*аичимни$ ^*чатын/*чатын$. | ^*Тадук$ ^*нунгартын$ ^*аичимни$ ^*чатын$. | *Тадук *нунгартын *аичимни *чатын. |
Post-Evaluation
------------------------------------------------------- Number of words in reference: 20 Number of words in test: 22 Number of unknown words (marked with a star) in test: 20 Percentage of unknown words: 90.91 %
Results when removing unknown-word marks (stars) ------------------------------------------------------- Edit distance: 22 Word error rate (WER): 110.00 % Number of position-independent correct words: 0 Position-independent word error rate (PER): 110.00 %
Note: Nothing improved when implementing structural transfer but this is likely due to the same orthographical issue from the transducer. Should
that error be resolved significant improvement would likely be seen.
