Nivkh and Russian

From LING073
Jump to: navigation, search

Initial Evaluation

niv->rus translator

Coverage using monolingual transducer:

Coverage: 73.23%

Coverage using bilingual transducer:

Total number of tokens in the corpus: 170
Number of tokens not in dictionary: 55
Coverage: 71.21% (niv.sentences.txt)
Coverage: 25.23% (rus.sentences.txt)

WER and PER

Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt.

Stored in niv-rus.test.0.txt

Number of words in reference: 170
Number of words in test: 217
Results when unknown-word marks (stars) are not removed

Edit distance: 217
Word error rate (WER): 127.65 %
Number of position-independent correct words: 0
Position-independent word error rate (PER): 127.65 %

Post-evaluation

These are the results at the time of our final presentation.

Coverage using bilingual transducer:

Total number of tokens in the corpus: 198
Coverage: 71.72% (niv.sentences.txt)
Coverage: 24.92% (rus.sentences.txt)q

WER and PER

Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt.

Stored in niv-rus.test.1.txt

Number of words in reference: 170
Number of words in test: 217
Results when unknown-word marks (stars) are not removed

Edit distance: 195
Word error rate (WER): 114.71 %
Number of position-independent correct words: 31
Position-independent word error rate (PER): 109.41 %

Final evaluation

These are the final results three days after our final presentation.

WER and PER

Results from running apertium-eval-translator on niv.sentences.txt and checking with rus.sentences.txt.

Stored in niv-rus.test.2.txt

Number of words in reference: 175
Number of words in test: 218
Results when unknown-word marks (stars) are not removed

Edit distance: 187
Word error rate (WER): 106.86 %
Number of position-independent correct words: 46
Position-independent word error rate (PER): 98.29 %

See also