Difference between revisions of "Dhivehi and English/Final project"
(→Evaluation) |
(→Evaluation) |
||
| Line 18: | Line 18: | ||
==Evaluation== | ==Evaluation== | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
{| class="wikitable" | {| class="wikitable" | ||
| Line 34: | Line 27: | ||
! Words in prediction | ! Words in prediction | ||
|- | |- | ||
| − | | 1 | + | | '''1''' |
| 86.68% | | 86.68% | ||
| 40.49% | | 40.49% | ||
| Line 40: | Line 33: | ||
| 9650 | | 9650 | ||
|- | |- | ||
| − | | 2 | + | | '''2''' |
| 90.69% | | 90.69% | ||
| 29.38% | | 29.38% | ||
| Line 46: | Line 39: | ||
| 12070 | | 12070 | ||
|- | |- | ||
| − | | 3 | + | | '''3''' |
| 85.65% | | 85.65% | ||
| 48.29% | | 48.29% | ||
Revision as of 01:13, 15 May 2019
Contents
Dhivehi RBMT and ONMT
Background
Dhivehi is a language with limited technological resources. Currently there are online Dhivehi-English dictionaries and various keyboard layouts. However, the dictionaries are not comprehensive. There is no existing machine translation system for Dhivehi to English. For this project, we explored two approaches to Dhivehi-English machine translation - rule-based machine translation and neural machine translation.
Motivation/Goal
Building a robust Dhivehi-English machine translation system would be beneficial to the Dhivehi language community by connecting them to the English language through technology. In addition, it will be beneficial for language learner of Dhivehi or English. It also promotes Dhivehi. Our goal is to explore rule-based machine translation and neural machine translation for Dhivehi-English. We will examine the advantages and disadvantages of both approaches.
Methods
RBMT
Using Apertium, we studied the grammatical structure of Dhivehi and wrote various rules (phonological, syntactic, etc.) that map Dhivehi to English.
NMT
Neural Machine Translation: using OpenNMT, we trained on Dhivehi-English parallel corpus consisting of mostly biblical text, some modern phrases, and Universal Declaration of Human Rights. The neural net learns the features of Dhivehi based on input and tries to predict the correct English sequences.
Evaluation
| Fold | WER | PER | Words in target | Words in prediction |
|---|---|---|---|---|
| 1 | 86.68% | 40.49% | 12970 | 9650 |
| 2 | 90.69% | 29.38% | 12664 | 12070 |
| 3 | 85.65% | 48.29% | 12292 | 7802 |
Github
You can find the code for this project available at: https://github.com/nav6maini/Dhivehi-RBMT-and-ONMT
References
Apertium - http://wiki.apertium.org/wiki/Main_Page
OpenNMT - opennmt.net
NMT - https://en.wikipedia.org/wiki/Neural_machine_translation
