Zazaki and English
From LING073
Contents
Resources for machine translation between Zazaki and English
Corpus: https://github.swarthmore.edu/Ling073-sp26/ling073-zza-eng-corpus
External resources
Transducers
Additions
For the polished RBMT phase, the following three expansions were made:
1. Bilingual dictionary expansion
Added 122 new bilingual entries to the bidix, drawn primarily from the top unknown words in a 1.48M-word Zazaki Wikipedia extract. Coverage on the large corpus rose from the midterm baseline (~40%) to 51.66%. Categories:
- 65 masculine nouns
- 20 feminine nouns
- 5 proper nouns
- 4 adjectives
- 12 adverbs
- 2 reflexive/pronoun entries
- 4 new verbs and 4 verbs that existed in the lexd but were missing from the bidix
- Several postpositions and interrogatives that existed in the lexd but were missing from the bidix
- A generic numeral entry so that digits pass through translation correctly
The corresponding entries were added to the monolingual lexd file as well, so all new bidix stems analyse on the Zazaki side.
2. New disambiguation rules (Constraint Grammar)
Added 3 new CG rules to apertium-zza.zza.rlx, bringing the total to 5:
- Rule 5: rê as postposition after a nominal. The token rê is ambiguous between the noun "road" (<n><m>) and the postposition "to/for" (<post>). Postpositions in Zazaki follow their complement, so when rê is preceded by a nominal it is the postposition. ~4,700 tokens disambiguated.
- Example: Homayî rê duayî kerdêne "they prayed to God" — rê selected as <post>.
- Rule 6: xo reflexive as singular by default. The reflexive pronoun xo (~8,600 occurrences in the corpus) has identical surface forms for <ref><sg> and <ref><pl>. The singular reading is overwhelmingly more common.
- Example: kitabê xo "his/her book" — xo selected as <sg>.
- Rule 7: Masculine noun -ê as <dir> (ezafe) before a nominal. Masculine nouns ending in -ê are ambiguous between <sg><dir> (ezafe construction) and <sg><ind> (indefinite). When followed by another nominal, the ezafe reading applies.
- Example: welatê Zazayan "country of the Zazas" — welatê selected as <dir>.
The 5 rules together remove ~45% of ambiguous analyses on the large corpus (1.0121 → 1.0067) and ~42% on the basic parallel corpus (1.0344 → 1.0199).
3. New lexical selection rules
Added 2 new lrx rule sets (4 rules total) to apertium-zza-eng.zza-eng.lrx, bringing the total to 3 sets:
- ser: "head" vs "year". ser primarily means "head" (body part), but in dating contexts (after a numeral) it means "year". A default rule selects "head"; a contextual rule selects "year" after a <num>.
- 1985 ser → "1985 year" (year reading)
- ser ey → "head ..." (head reading)
- ra: "from" vs "to". The postposition ra usually means "from" (separation/source), but with speech verbs like vat "say", it marks the addressee and means "to".
- cîneke ra va → "woman to said" (to reading)
- Dêsım ra heta Çolig → "from Dêsım to Çolig" (from reading)
Initial evaluation
Coverage
| Measure | Tokens | Coverage |
|---|---|---|
| Monolingual transducer (zza.automorf.bin) on zza.sentences.txt | 690 / 1166 | 59.2% |
| Bilingual transducer (zza-eng.automorf.bin) on zza.sentences.txt | 647 / 1123 | 57.6% |
10 Sentence Analysis
| # | Original (Zazaki) | Intended English | biltrans output | Full translation output |
|---|---|---|---|---|
| 1 | Bi silametî şo. | Go in peace! | ^Bi<cnjcoo>/With<pr>$ ^silamet<n><m><pl><dir>/peace<n><pl><dir>$ ^şîy<v><imp><sg>/go<vblex><imp><sg>$^.<sent>/.<sent>$ |
With #peace #go. |
| 2 | Îsayî va "Ti heqdar ê." | "You are right," Jesus said. | ^Îsa<n><m><sg><obl>/Jesus<np><sg><obl>$ ^vat<v><past><p3><m><sg>/say<vblex><past><p3><m><sg>$ "^Tı<prn><pers><p2><sg><dir>/You<prn><pers><p2><sg><dir>$ ^heqdar<adj><m><sg>/right<adj><m><sg>$ ^bîy<v><pres><p2><sg>/be<vbser><pres><p2><sg>$^.<sent>/.<sent>$" |
#Jesus #say "#You #right are." |
| 3 | Şîmûnî va "Kerem bike Seyda." | Simon said "Please, Teacher." | ^Şîmûn<n><m><sg><obl>/Simon<np><sg><obl>$ ^vat<v><past><p3><m><sg>/say<vblex><past><p3><m><sg>$ "^Kerem<n><m><sg><dir>/Please<n><sg><dir>$ ^kerd<v><imp><sg>/do<vblex><imp><sg>$ ^Seyda<n><m><sg><dir>/Teacher<n><sg><dir>$^.<sent>/.<sent>$" |
#Simon #say "#Please #do #Teacher." |
| 4 | Şîmûnî cewab da va | Simon answered and said | ^Şîmûn<n><m><sg><obl>/Simon<np><sg><obl>$ ^cewab<n><m><sg><dir>/answer<n><sg><dir>$ ^day<v><past><p3><m><sg>/give<vblex><past><p3><m><sg>$ ^vat<v><past><p3><m><sg>/say<vblex><past><p3><m><sg>$ |
#Simon #answer #give #say |
| 5 | Îsayî zî cîneke ra va | Jesus also said to the woman | ^Îsa<n><m><sg><obl>/Jesus<np><sg><obl>$ ^zî<post>/also<adv>$ ^cîneke<n><f><sg><dir>/woman<n><sg><dir>$ ^ra<post>/from<pr>$ ^vat<v><past><p3><m><sg>/say<vblex><past><p3><m><sg>$ |
#Jesus also #woman from #say |
| 6 | Îsayî cîneke ra va "Guneyê to ameyî efûkerdiş." | Jesus said to the woman, "Your sins are forgiven." | ^Îsa<n><m><sg><obl>/Jesus<np><sg><obl>$ ^cîneke<n><f><sg><dir>/woman<n><sg><dir>$ ^ra<post>/from<pr>$ ^vat<v><past><p3><m><sg>/say<vblex><past><p3><m><sg>$ "^Guney<n><m><sg><ind>/Sin<n><sg><ind>$ ^tı<prn><pers><p2><sg><obl>/you<prn><pers><p2><sg><obl>$ ^amey<v><past><pl>/come<vblex><past><pl>$ ^efûkerdiş<n><m><sg><dir>/forgiveness<n><sg><dir>$^.<sent>/.<sent>$" |
#Jesus #woman from #say "#Sin #you #come #forgiveness." |
| 7 | Îsayî zî cîneke ra va "Îmanê to ti xelisnaya." | Jesus told the woman, "Your faith has saved you." | ^Îsa<n><m><sg><obl>/Jesus<np><sg><obl>$ ^zî<post>/also<adv>$ ^cîneke<n><f><sg><dir>/woman<n><sg><dir>$ ^ra<post>/from<pr>$ ^vat<v><past><p3><m><sg>/say<vblex><past><p3><m><sg>$ "^Îman<n><m><sg><ind>/Faith<n><sg><ind>$ ^tı<prn><pers><p2><sg><obl>/you<prn><pers><p2><sg><obl>$ ^tı<prn><pers><p2><sg><dir>/you<prn><pers><p2><sg><dir>$ ^xelisna<v><past><p3><f><sg>/save<vblex><past><p3><f><sg>$^.<sent>/.<sent>$" |
#Jesus also #woman from #say "#Faith #you #you #save." |
| 8 | La Îsa miyanê qelebalixî ra xelisîya û uca ra kewt dûrî. | But Jesus slipped through the crowd and got away. | ^La<cnjcoo>/But<cnjcoo>$ ^Îsa<n><m><sg><dir>/Jesus<np><sg><dir>$ ^miyan<n><m><sg><ind>/middle<n><sg><ind>$ ^qelebalix<n><m><pl><dir>/crowd<n><pl><dir>$ ^ra<post>/from<pr>$ ^xelisîy<v><past><p3><m><sg>/escape<vblex><past><p3><m><sg>$ ^û<cnjcoo>/and<cnjcoo>$ ^uca<adv>/there<adv>$ ^ra<post>/from<pr>$ ^kewt<v><past><p3><m><sg>/fall<vblex><past><p3><m><sg>$ ^dûrî<adj><m><sg>/far<adj><m><sg>$^.<sent>/.<sent>$ |
But #Jesus #middle #crowd from #escape and there from #fall #far. |
| 9 | Şagirdan zî o qeyde kerd û her kes da roniştiş. | They did this, and all the people sat down. | ^Şagird<n><m><pl><obl>/Disciple<n><pl><obl>$ ^zî<post>/also<adv>$ ^o<det><dem><m><dir>/that<det><dem><m><dir>$ ^qeyde<n><m><sg><dir>/manner<n><sg><dir>$ ^kerd<v><past><p3><m><sg>/do<vblex><past><p3><m><sg>$ ^û<cnjcoo>/and<cnjcoo>$ ^her<adv>/every<adv>$ ^kes<n><m><sg><dir>/person<n><sg><dir>$ ^day<v><past><p3><m><sg>/give<vblex><past><p3><m><sg>$ ^roniştiş<n><m><sg><dir>/sitting<n><sg><dir>$^.<sent>/.<sent>$ |
#Disciple also #that #manner #do and every #person #give #sitting. |
| 10 | Ew no dano pıro şıno koli. | Then he set off to cut firewood. | ^Ew<prn><pers><p3><pl><dir>/He<prn><pers><p3><pl><dir>$ ^no<det><dem><m><dir>/this<det><dem><m><dir>$ ^day<v><pres><p3><m><sg>/give<vblex><pres><p3><m><sg>$ ^pıro<post>/onto<pr>$ ^şîy<v><pres><p3><m><sg>/go<vblex><pres><p3><m><sg>$ ^koli<n><m><sg><dir>/firewood<n><sg><dir>$^.<sent>/.<sent>$ |
#He #this #give onto #go #firewood. |
Final evaluation
Monolingual transducer
- Stems in transducer: 265
- Words in large corpus (zza.corpus.large.wiki.txt): 1,478,167
- Coverage over large corpus: 51.66% (1,039,506 known / 2,012,031 tokens)
- Precision: 75.16%
- Recall: 46.18%
- F1: 57.21%
(Precision and recall measured against zza.eval.test.txt and zza.eval.gold.txt; 249 cohorts compared.)
Disambiguation
| Corpus | Ambiguity before CG | Ambiguity after CG | Reduction |
|---|---|---|---|
| Large Wikipedia corpus | 1.0121 | 1.0067 | ~45% |
| Basic parallel corpus | 1.0344 | 1.0199 | ~42% |
The 5 disambiguation rules together remove a substantial proportion of ambiguous analyses, particularly in resolving high-frequency forms like rê, xo, and ezafe-marked masculine nouns.
zza → eng
| Measure | Value |
|---|---|
| Tokens in longer corpus (zza.longer.txt) | 944 words / 1,093 tokens |
| Tokens in large corpus (zza.corpus.large.wiki.txt) | 1,478,167 words / 1,919,996 tokens |
| Trimmed coverage over longer corpus | 60.10% (657 / 1,093) |
| Trimmed coverage over large corpus | 49.05% (941,946 / 1,919,996) |
| WER over longer corpus | 94.96% |
| PER over longer corpus | 84.88% |
| Stem translation accuracy over longer corpus | 47.00% (47 of 100 system-produced unique English lemmas appear in the gold reference) |
WER and PER are high because most content words in the longer corpus (Bible passages and folktale fragments) remain unknown stems. Stem translation accuracy of 47% indicates that roughly half of the English lemmas the system produces are present in the gold reference, suggesting the bidix entries that exist are largely correct.
Ambiguity Resolution
Find the Ambiguity Resolution here.
