Lingala and Kikuyu

From LING073
Revision as of 23:07, 8 April 2017 by Apowell1 (talk | contribs)

Jump to: navigation, search

Resources for machine translation between Lingala and Kikuyu. Code is stored in a GitHub Repository.

Developed Resources

Parallel Corpus

A parallel corpus created using two translations of Genesis is available on Github.

Contrastive Grammar

Contrastive Grammar is detailed on the wiki page Lingala and Kikuyu/Contrastive Grammar.

Lexical Selection

Lexical selection is detailed on the wiki page Lingala and Kikuyu/Lexical Selection.

Structural Transfer

Structural transfer is detailed on the wiki page Lingala and Kikuyu/Structural Transfer

Installing Unidecode For MT Evaluation

Most available Lingala text does not mark tones; however, the Lingala transducer has been configured using spellrelax to provide analyses and surface generation with tone markings included as accents on vowels. As a result, generated translations will include tone markings not present in the reference translation and artificially inflate evaluation metrics such as WER and PER. Fortunately, we can take advantage of the fact that accented characters are not present in ASCII to replace accented characters with their unaccented version on the command line.

Unidecode is a Python package that implements a function to convert Unicode strings into their closest ASCII approximation. For simple accented characters, this is precisely the mapping we need to remove the accents for evaluation against a reference translation. Unidecode can also be used on the command line with our machine translation system as follows: echo /path/to/kik-corpus | apertium -d . kik-lin | unidecode. To install unidecode, it is recommended to use pip, as experience suggests that apt-get will not run the necessary installation scripts to make unidecode accessible in Bash.

On the Ling 073 virtual machines, Pip can be installed using sudo apt-get install python-pip. After installing pip, unidecode can be installed using pip install unidecode.

Lexical Selection

This section is part of the Lexical transfer assignment. In addition to adding many one-to-one mappings to our .dix file, we have included a couple of one-to-many mappings for each direction of translation. See Lingala and Kikuyu/Lexical Selection for further details.

From Lingala to Kikuyu, we consider four one-to-many mappings:

  • The Lingala verb kokíma can be translated in English as both "to run" and "to run away." Kikuyu has two distinct verbs corresponding to these English translations, respectively, hanyũk and ũr.
  • The Lingala verb kofánda can be translated in English as both "to sit" and "to live" (as in reside). Kikuyu has two distinct verbs corresponding to these English translations, respectively, ikar and tũũr.
  • The Lingala verb kobánza can be translated in English as both "to think" and "to remember." Kikuyu has two distinct verbs corresponding to these English translations, respectively, ĩcir and ririkan.
  • The Lingala noun mabele can be translated in English as both "sand" and "milk." Kikuyu has two distinct nouns corresponding to these English translations, respectively, thanga and ria.

From Kikuyu to Lingala, we consider three one-to-many-mappings:

  • The Kikuyu verb igu can be translated in English as both "to obey" and "to hear." Lingala has two distinct verbs corresponding to these English translations, respectively, kotosa and koyoka.
  • The Kikuyu verb rut can be translated in English as both "to learn" and "to remove." Lingala has two distinct verbs corresponding to these English translations, respectively, koyekola and kokukula.
  • the Kikuyu noun ria can be translated in English as both "milk" and "pond." Lingala has two distinct nouns corresponding to these English translations, respectively, mabele and etimá.

Initial Evaluation

lin → kik evaluation

  • Evaluation on mini corpus:
    • WER: 87.50%
    • PER: 87.50%
    • Number of words: 9
    • Number of unknown words: 0
    • Proportion of stems translated correctly: 100%
  • Evaluation of corpus:
    • WER: 99.06%
    • PER: 94.34%
    • Number of words: 107
    • Number of unknown words: 71
    • Proportion of stems translated correctly:33.64%

kik → lin evaluation

  • Evaluation on mini corpus:
    • WER: 80%
    • PER: 80%
    • Number of words: 7 (This should really be 8, but the command we are supposed to use to count this outputs 7. The proportion remains unchanged in our case, however.)
    • Number of unknown words: 0
    • Proportion of stems translated correctly: 100%
  • Evaluation of corpus:
    • WER: 98.13%
    • PER: 90.65%
    • Number of words: 106
    • Number of unknown words: 79 (Note: Using the commands on the wiki gives a slightly different count that using the apertium-eval-translator tool. We use the count given by the command, not the tool)
    • Proportion of stems translated correctly: 25.47%

Final Evaluation

lin → kik Implementation Details

kik → lin Implementation Details

This direction of the RBMT system was improved by adding items to the bilingual dictionary, adding additional morphology to the transducer, writing more constraint grammar rules, and writing additional transfer rules. Specific implementation details with examples of improved output are detailed in the sections that follow.

Bilingual Dictionary Expansion

Difference can be seen by looking at the Github commit history.

Morphology Expansion

  • Support for proper nouns
    • Example sentence: "Nzambe" (God)
    • Old output: ^Nzambe/*Nzambe$^./.<sent>$
    • New output: ^Nzambe/Nzambe<n><prop>$^./.<sent>$
  • Support for imperative mood
    • Example sentence: "Kíma!" (Run!)
    • Old output: ^Kíma/*Kíma$^!/!<sent>$^./.<sent>$
    • New output: ^Kíma/kokíma<v><imp>$^!/!<sent>$^./.<sent>$
  • Point 3
    • Example sentence:
    • Old output:
    • New output
  • Point 4
    • Example sentence
    • Old output:
    • New output:
  • Point 5
    • Example sentence
    • Old output:
    • New output:
  • Point 6
    • Example sentence
    • Old output:
    • New output:

Additional Constraint Grammar Rules

  • Rule 1
    • Example sentence
    • Old output (below)
    • New output (below)
  • Rule 2
    • Example sentence
    • Old output (below)
    • New output (below)
  • Rule 3
    • Example sentence
    • Old output (below)
    • New output (below)
  • Rule 4
    • Example sentence
    • Old output (below)
    • New output (below)

Additional Transfer Rules

  • Rule 1
    • Example sentence
    • Old output
      • Tagger:
      • Biltrans:
      • Chunker:
      • Interchunk:
      • Postchunk:
      • Complete translation:
    • New output
      • Tagger:
      • Biltrans:
      • Chunker:
      • Interchunk:
      • Postchunk:
      • Complete translation:
  • Rule 2
    • Example sentence
    • Old output
      • Tagger:
      • Biltrans:
      • Chunker:
      • Interchunk:
      • Postchunk:
      • Complete translation:
    • New output
      • Tagger:
      • Biltrans:
      • Chunker:
      • Interchunk:
      • Postchunk:
      • Complete translation:
  • Rule 3
    • Example sentence
    • Old output
      • Tagger:
      • Biltrans:
      • Chunker:
      • Interchunk:
      • Postchunk:
      • Complete translation:
    • New output
      • Tagger:
      • Biltrans:
      • Chunker:
      • Interchunk:
      • Postchunk:
      • Complete translation:

lin Transducer Evaluation

  • Precision (annotated corpus):
  • Recall (annotated corpus):
  • Coverage (large corpus):
  • Number of words (large corpus):
  • Number of stems in transducer:

kik Transducer Evaluation

  • Precision (annotated corpus):
  • Recall (annotated corpus):
  • Coverage (large corpus):
  • Number of words (large corpus):
  • Number of stems in transducer:

lin → kik MT Evaluation

  • WER (longer corpus):
  • PER (longer corpus):
  • Percentage unknown words:
  • Percentage known words:
  • Trimmed coverage over longer corpus:
  • Trimmed coverage over large corpus:
  • Number of stems in longer corpus:
  • Number of stems in large corpus:

kik → lin MT Evaluation

  • WER (longer corpus):
  • PER (longer corpus):
  • Percentage unknown words:
  • Percentage known words:
  • Trimmed coverage over longer corpus:
  • Trimmed coverage over large corpus:
  • Number of stems in longer corpus:
  • Number of stems in large corpus:
Retrieved from "http://wikis.swarthmore.edu/ling073/index.php?title=Lingala_and_Kikuyu&oldid=4408"