Difference between revisions of "Nivkh and Russian"
From LING073
(→WER and PER) |
(→See also) |
||
| (15 intermediate revisions by 2 users not shown) | |||
| Line 15: | Line 15: | ||
====WER and PER==== | ====WER and PER==== | ||
| − | Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt | + | Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt. |
| + | |||
| + | Stored in niv-rus.test.0.txt | ||
<pre> | <pre> | ||
Number of words in reference: 170 | Number of words in reference: 170 | ||
| − | Number of words in test: | + | Number of words in test: 217 |
| − | |||
| − | |||
</pre> | </pre> | ||
<pre> | <pre> | ||
Results when unknown-word marks (stars) are not removed | Results when unknown-word marks (stars) are not removed | ||
| − | Edit distance: | + | Edit distance: 217 |
| − | Word error rate (WER): | + | Word error rate (WER): 127.65 % |
Number of position-independent correct words: 0 | Number of position-independent correct words: 0 | ||
| − | Position-independent word error rate (PER): | + | Position-independent word error rate (PER): 127.65 % |
| + | </pre> | ||
| + | |||
| + | == Post-evaluation == | ||
| + | These are the results at the time of our final presentation. | ||
| + | |||
| + | Coverage using bilingual transducer: | ||
| + | <pre> | ||
| + | Total number of tokens in the corpus: 198 | ||
| + | Coverage: 71.72% (niv.sentences.txt) | ||
| + | Coverage: 24.92% (rus.sentences.txt)q | ||
| + | </pre> | ||
| + | |||
| + | === WER and PER === | ||
| + | Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt. | ||
| + | |||
| + | Stored in niv-rus.test.1.txt | ||
| + | <pre> | ||
| + | Number of words in reference: 170 | ||
| + | Number of words in test: 217 | ||
| + | </pre> | ||
| + | <pre> | ||
| + | Results when unknown-word marks (stars) are not removed | ||
| + | |||
| + | Edit distance: 195 | ||
| + | Word error rate (WER): 114.71 % | ||
| + | Number of position-independent correct words: 31 | ||
| + | Position-independent word error rate (PER): 109.41 % | ||
| + | </pre> | ||
| + | |||
| + | == Final evaluation == | ||
| + | These are the final results three days after our final presentation. | ||
| + | === WER and PER === | ||
| + | Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt. | ||
| + | |||
| + | Stored in niv-rus.test.2.txt | ||
| + | <pre> | ||
| + | Number of words in reference: 175 | ||
| + | Number of words in test: 218 | ||
| + | </pre> | ||
| + | <pre> | ||
| + | Results when unknown-word marks (stars) are not removed | ||
| + | |||
| + | Edit distance: 187 | ||
| + | Word error rate (WER): 106.86 % | ||
| + | Number of position-independent correct words: 46 | ||
| + | Position-independent word error rate (PER): 98.29 % | ||
</pre> | </pre> | ||
| + | |||
| + | ==See also== | ||
| + | * [[English and Nivkh]] | ||
| + | * [https://github.com/gleec14/apertium-niv-rus.git translator repository] | ||
| + | * [https://github.com/gleec14/apertium-niv public Nivkh transducer repository] | ||
| + | |||
[[Category:Nivkh]] | [[Category:Nivkh]] | ||
[[Category:Russian]] | [[Category:Russian]] | ||
| + | [[Category:Sp19 FinalProjects]] | ||
Latest revision as of 22:42, 21 May 2019
Contents
Initial Evaluation
niv->rus translator
Coverage using monolingual transducer:
Coverage: 73.23%
Coverage using bilingual transducer:
Total number of tokens in the corpus: 170 Number of tokens not in dictionary: 55 Coverage: 71.21% (niv.sentences.txt) Coverage: 25.23% (rus.sentences.txt)
WER and PER
Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt.
Stored in niv-rus.test.0.txt
Number of words in reference: 170 Number of words in test: 217
Results when unknown-word marks (stars) are not removed Edit distance: 217 Word error rate (WER): 127.65 % Number of position-independent correct words: 0 Position-independent word error rate (PER): 127.65 %
Post-evaluation
These are the results at the time of our final presentation.
Coverage using bilingual transducer:
Total number of tokens in the corpus: 198 Coverage: 71.72% (niv.sentences.txt) Coverage: 24.92% (rus.sentences.txt)q
WER and PER
Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt.
Stored in niv-rus.test.1.txt
Number of words in reference: 170 Number of words in test: 217
Results when unknown-word marks (stars) are not removed Edit distance: 195 Word error rate (WER): 114.71 % Number of position-independent correct words: 31 Position-independent word error rate (PER): 109.41 %
Final evaluation
These are the final results three days after our final presentation.
WER and PER
Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt.
Stored in niv-rus.test.2.txt
Number of words in reference: 175 Number of words in test: 218
Results when unknown-word marks (stars) are not removed Edit distance: 187 Word error rate (WER): 106.86 % Number of position-independent correct words: 46 Position-independent word error rate (PER): 98.29 %
