Lingala/Transducer

From LING073
Revision as of 03:47, 16 February 2017 by Apowell1 (talk | contribs)

Jump to: navigation, search

A transducer for Lingala is available on Github.

Evaluation

Coverage

After adding four common words to the transducer, there are 63 stems, giving a coverage of 35.22% of the corpus. The improvement gained by adding additional common words to the transducer is detailed in the Notes section.

Top Unknown Words

The top unknown words are as follows:

  • nsɔ (21)
  • nyɔ (14)
  • mi (12)
  • ete (11)
  • lya (10)
  • wa (10)
  • mwa (10)
  • nkema (9)
  • Nzambe (9)
  • li (8)
  • té (7)
  • ma (7)
  • likolo (7)
  • bwa (7)
  • esalemaki (7)
  • ba (7)
  • Nkolo (7)
  • banzelu (6)
  • tata (6)
  • likoki (6)

Tests

In the main yaml file (lin.yaml), 119 tests pass and 14 fail, for a total of 133 tests. In the common words yaml file, 4 tests pass and 13 fail, for a total of 17 tests (spellrelax caused some tests to be viewed as duplicates, even though aq-covtest reported them as distinct words).


Notes

Addition of Common Words

The unknown words added to the corpus include:

  • o<pr> ↔ o, a preposition whose addition brought the (initial coverage) from 26.06% to 29.23%
  • mpé<cngcoo> ↔ mpé, a conjunction (often translated "and"), which increased the coverage from 29.23% to 32.93%
  • to<cngcoo> ↔ to, a conjunction (translated "or"), which increased coverage from 32.93% to 33.80%
  • te<neg> ↔ te, the Lingala word for "no" which is used generally to indicate negation, which increased coverage from 33.80% to 35.22%

Failing Tests

  • mokonzi<n><pl><cl1><abs><inst> ↔ bakimokonzi fails because I only implemented a single level of prefix stacking. I get the sense that it is quite rare to have more than two prefix (with the grammar book only mentioning the possibility in passing). Implementing another layer of prefix stacking would require a substantial number of additional suffix entries in the lexicon due to the way Apertium handles prefixes. It seemed unproductive to clutter the lexicon in this way for such a rarely used morphological form.
  • Tests for the pronoun baýo wâna and óyo wâna fail because each pronoun essentially requires its own stem and suffix lexicon. A cursory review of Lingala literature suggests that use of this demonstrative is rare (with óyo being preferred), so it seemed unproductive to clutter the lexicon to support this, at least initially.
  • Most of the tests for verb mood fail. This is mainly because support for the subjunctive mood has not yet been implemented. Because of the way prefixes are handled, adding an additional verb prefix possibility requires doubling the number of verb suffix possibilities, of which there are already an extremely large number. Since the subjunctive is denoted by tone only, which is not typically marked in literature, it seemed unproductive to add all of the extra suffix possibilities since the form would likely never be analyzed.
  • Two of the verb present tense tests past. This is because the verb stem in question, kokámwa, takes an irregular ending in the present tense, é instead of í. For the initial version of the transducer, I have limited endings to the present tense to the most commonly found, í. If it turns out there are a large number of verbs that end with é in the present tense, support can be added in the future.