User:Scasada1/Final Project

Final Project Description

For our final project, we decided to work on a two-part expansion of our RBMT-system for our language (Crimean Urum). This work will consist of many parts, but we will outline our base goals here:

• Improve our morphological transducer for Crimean Urum, aiming for a corpus coverage above 80%.
• *Improve our Crimean Urum machine translation accuracy.

We believe that our morphological transducer and machine translation system will be potentially helpful for living Urum language communities with tasks like generating novel sentences, and/or helping with an auto-correction system, MT, text processing, etc.

We have released information for our project publicly on GitHub (apertium-uum) and here you may separately view a digital copy of our final presentation poster as it exists on GitHub.

SP2023 Urum "Polished RBMT System for Crimean Urum" Poster

Previous Work

Nuosu Transducer

Nuosu-Mandarin Machine Translation

Additions

Urum Transducer

• We added 100+ stems to the .lexd file, covering common nouns, verbs, adjectives, proper nouns, etc. as they occurred in terms of frequency in appearance within (Олександр ГАРКАВЕЦЬ УРУМИ НАДАЗОВ’Я)
• Expanded our number of rules in .twol to cover the rest of Urum's vowel and consonant harmony: (Total Rules: 16 → 24)
• We added a large number of additional .lexd PATTERNS including:
  • VFutureDefinite, VFutureIndefinite, VTransgressive, VPastParticiple, NumOrdinal, NumDiscrete, CaseIns, CaseAbe, & CaseTerm

Urum-English Machine Translation

• Added one structural transfer rule to with the formation of NPs in the case of proper nouns (prior not realizing correctly, omitting 'the' and instead having no output, ex: "king #Herod"
• Added 100+ new lexicon items to our bilingual dictionary, making the total ~250.

Corpus Construction

• Constructed small corpus with 30 sentences; the lexicon in this corpus are mostly added to our transducer and the bilingual dictionary, which is useful for evaluateing how the effectiveness of the structural transfer rules when the lexicon are mostly known.

• Constructed large corpus with 100 sentences; the sentences in this corpus are randomly selected from the Bible and the Grammar book, and evaluating over this corpus shows the general accuracy of our machine translation system.

Selected Example Sentences from the Large Corpus

• (iii) ꋌꊋꋠꃅꀥ → (zho) 他用力地跑

  (iii) ꋌ<prn> ꊋꋠ<adj> ꃅ<advl> ꀥ<vblex> → (zho) 他<prn> 用力地<adv> 跑<vblex>

• (iii) ꉪꊇꀃꑍꏦꎫꁧ → (zho) 我们今天去街上

  (iii) ꉪꊇ<prn> ꀃꑍ<n> ꏦꎫ<n> ꁧ<vblex> → (zho) 我们<prn> 今天<n> 去<vblex> 街上<n>

• (iii) ꉢꎧꀋꅝꀐ → (zho) 我没有喝红酒

  (iii) ꉢ<prn> ꎧ<n> ꀋ<neg> ꅝ<vblex> ꀐ<part> → (zho) 我<prn> 没有<neg> 喝<vblex> 红酒<n>

• (iii) ꊾꆹꉬꇮꐯꀋꇊꌡ → (zho) 所有人差不多地像

  (iii) ꊾ<n> ꆹ<part> ꉬꇮ<adj> ꐯ<RECL> ꀋꇊ<adv> ꌡ<vblex> → (zho) 所有<adj> 人<n> 差不多地<adv> 像<vblex>

• (iii) ꉢꃀꌠꀐ → (zho) 我是老人了

  (iii) ꉢ<prn> ꃀꌠ<n> ꀐ<part> → (zho) 我<prn> 现在<adv> 是<pr> 老人<n>

Final Evaluation

Transducer Overview (uum)

• Coverage over the "Urums of North Azov: Collection of Narratives": 60.071%
• Precision: 55.678%
• Recall: 73.077%
• Total Stems: 471
  • N-Roots: 121
  • V-Roots: 61
  • Adv/Adj-Roots: 22

Transfer Overview (uum ‣ eng)

• Urum Bible: "Nativity":
  • WER: 81.29%
  • PER: 57.19%

(The amount of error present here is likely due to the type of language used in the King James translation of the bible, as we used this as gold for the English translation. We believe that the word-error rate, and more specifically PER, would be far lower with a version of the bible in English which is more accurate to the original text).

• Bilingual Dictionary Entries: 260
• Total LRX Rules: 8

RBMT Evalutation

Corpus	Tokens	Coverage	Accuracy
Urum Bible: "Nativity"	170	95.882% (163/170)	91.765%
"Urums of North Azov: Collection of Narratives"	18K	28.487% (53K/186K)	N/A

Future Work

• We currently have a very rich analysis system, but we need to expand our current dictionary significantly (~1000+ words, ideally)
  • In order to solve this, we have a publicly available spreadsheet dictionary for Urum, based off a Ukranian-Urum dictionary translated to English (which is available on GitHub).
• Refine our Apertium (uum ‣ eng) transfer pipeline; i.e, robust expansion of lexical selection and lexical transfer as issues begin to arise.

Links to Public Git Repo and Poster

• Morphological Analyser, generator, and disambiguator for Nuosu
  • Morphological Transducer

• Machine Translation between Nuosu and Mandarin
  • Machine Translation System

• Improved Mandarin Transducer
  • Apertium-zho

• Summary Poster of the Project
  • Poster