Revision as of 01:18, 15 May 2019

Dhivehi RBMT and ONMT

Background

Dhivehi is a language with limited technological resources. Currently there are online Dhivehi-English dictionaries and various keyboard layouts. However, the dictionaries are not comprehensive. There is no existing machine translation system for Dhivehi to English. For this project, we explored two approaches to Dhivehi-English machine translation - rule-based machine translation and neural machine translation.

Motivation/Goal

Building a robust Dhivehi-English machine translation system would be beneficial to the Dhivehi language community by connecting them to the English language through technology. In addition, it will be beneficial for language learner of Dhivehi or English. It also promotes Dhivehi. Our goal is to explore rule-based machine translation and neural machine translation for Dhivehi-English. We will examine the advantages and disadvantages of both approaches.

Methods

RBMT

Using Apertium, we studied the grammatical structure of Dhivehi and wrote various rules (phonological, syntactic, etc.) that map Dhivehi to English.

NMT

Neural Machine Translation: using OpenNMT, we trained on Dhivehi-English parallel corpus consisting of mostly biblical text, some modern phrases, and Universal Declaration of Human Rights. The neural net learns the features of Dhivehi based on input and tries to predict the correct English sequences.

Evaluation

Cross Validation

We used three-fold cross validation and found that the second model performed the best against its corresponding validation data set. We deemed it the best because the WER's of all the models were somewhat similar, but model two's PER was significantly lower.

RBMT and NMT

Github

You can find the code for this project available at: https://github.com/nav6maini/Dhivehi-RBMT-and-ONMT

References

Apertium - http://wiki.apertium.org/wiki/Main_Page

OpenNMT - opennmt.net

NMT - https://en.wikipedia.org/wiki/Neural_machine_translation