Latest revision as of 19:42, 5 May 2023

Resources for Machine Translation from Sardinian to English

Developed Resources

• Parallel corpus git repo.

  Lines 1-8 from this website.

• Git repo for language pair
• Git repo for Sardinian
• Forked git repo for English
• Lexical selection wiki
• Contrastive grammar wiki
• Structural transfer wiki

srd → eng evaluation

• coverage of monolingual Sardinian transducer on srd.sentences.txt:

 Number of tokenised words in the corpus: 262
 Coverage: 94.66%

• coverage of bilingual transducer on srd.sentences

 Number of tokenised words in the corpus: 266
 Coverage: 85.71%

Additions for polished RBMT system

• Over 100 new stems to bilingual dictionary

  (now we have 172 total entries in the .dix file)

• 4 sets of lexical selection rules

  (now we have rules that solve 5 problems in total)

• 3 items from contrastive grammar implemented in transfer rules

  (now we have rules that implement 4 total items)

Final evaluation

• For the monolingual transducer:
  • Precision and recall against srd.sentences.txt (1314 char):

 Totals: 145 forms, 417 tp, 0 fp, 0 tn, 11 fn
 Precision: 100.00000%
 Recall: 97.42991%

• • Coverage over the large corpus:
    • 90.4% with monolingual transducer: (mostly simple words that could easily be added)

 TOP UNKNOWN WORDS:
   2336 ^po/*po$
   1275 ^che/*che$
   448 ^i/*i$
   387 ^norte/*norte$
   377 ^The/*The$
   353 ^of/*of$
   297 ^poi/*poi$
   269 ^del/*del$
   241 ^the/*the$
   235 ^da/*da$
   215 ^ch/*ch$
   211 ^Il/*Il$
   205 ^chentza/*chentza$
   200 ^br/*br$
   199 ^Arbaree/*Arbaree$
   197 ^della/*della$
   192 ^stètiu/*stètiu$
   165 ^ddoi/*ddoi$
   154 ^B/*B$
   145 ^Norte/*Norte$
 coverage: 1648134 / 1822907 (~0.90412401729764601266)

• • The number of words in the large corpus: ~1.85M words
  • The number of stems in the transducer: ~85.5k stems

• For MT in the direction we developed (srd-eng):
  • WER and PER over longer corpus:

 -------------------------------------------------------
 Number of words in reference: 315
 Number of words in test: 341
 Number of unknown words (marked with a star) in test: 35
 Percentage of unknown words: 10.26 %
 
 Results when removing unknown-word marks (stars)
 -------------------------------------------------------
 Edit distance: 246
 Word error rate (WER): 78.10 %
 Number of position-independent correct words: 118
 Position-independent word error rate (PER): 70.79 %

• • The proportion of stems translated correctly in the longer corpus.

 118/341 = 34.6%

• • Trimmed coverage over longer and large corpora.
    • Over longer corpus, our coverage with the bilingual transducer is as follows (90.1%):

 TOP UNKNOWN WORDS:
     1 ^volùmene/*volùmene$
     1 ^venit/*venit$
     1 ^tenzeint/*tenzeint$
     1 ^tedescu/*tedescu$
     1 ^sìntesi/*sìntesi$
     1 ^sincasu/*sincasu$
     1 ^sambenados/*sambenados$
     1 ^pressare/*pressare$
     1 ^ponzo/*ponzo$
     1 ^poder/*poder$
     1 ^paráula/*paráula$
     1 ^offerit/*offerit$
     1 ^nd/*nd$
     1 ^matessi/*matessi$
     1 ^lu/*lu$
     1 ^issoro/*issoro$
     1 ^iscrittu/*iscrittu$
     1 ^ischeda/*ischeda$
     1 ^ghiratu/*ghiratu$
     1 ^fundadores/*fundadores$
 coverage: 322 / 357 (~0.90196078431372549020)
 remaining unknown forms: 35

• • • Coverage is 52% for bilingual transducer over large corpus: (also mostly simple words that could easily be added)

 TOP UNKNOWN WORDS:
  12208 ^cun/*cun$
  8207 ^prus/*prus$
  5205 ^o/*o$
  4218 ^comente/*comente$
  4100 ^fintzas/*fintzas$
  3068 ^pustis/*pustis$
  2658 ^annos/*annos$
  2623 ^totu/*totu$
  2589 ^ma/*ma$
  2417 ^sua/*sua$
  2376 ^parte/*parte$
  2336 ^po/*po$
  2250 ^Sardigna/*Sardigna$
  2233 ^intre/*intre$
  2120 ^ca/*ca$
  2070 ^tzitade/*tzitade$
  1964 ^che/*che$
  1888 ^ue/*ue$
  1873 ^bidda/*bidda$
  1739 ^manera/*manera$
 coverage: 963208 / 1863589 (~0.51685645279082458632)

• • The number of tokens in longer and large corpora.
    • longer: 341 tokens
    • large: ~1.85M tokens

10 sentences glosses and outputs

• Note: These are outputs at end of lexical transfer before lexical selection and other weeks' improvements were implemented

expanding lexicon

glosses

Cando so arrivatu a s'istazione, su trenu sink'andaiat

'when I arrived at the station, the train was leaving'

Su trenu sinke fit andande

'The train was leaving"

Su trenu at a arrivare a sas deke

'The train will arrive at ten o'clock'

Su trenu devet arrivare a sas deke

'The train is supposed to arrive at ten o'clock'

T'appo natu ki su trenu diat arrivare a sas deke

'I told you that the train would arrive at ten o'clock'

M'an natu ki su trenu devíat arrivare a sas nove

'They told me that the train was supposed to arrive at nine'

At a próere cras

'It will rain tomorrow'

Devet próere cras

'it is supposed to rain tomorrow'

At a próere cras? Si emmo abbarramus in domo, ma su nono andamus a mare

'Will it rain tomorrow? If so we shall stay at homr, buy if not we will go to the sea'

No'isco si at a próere cras

'I do to know whether it will rain tomorrow'

Una de sas berbekes est maláida

'one of the sheep is sick'

Appo pèrditu duos de sos libros meos

'I have lost two of my books'

Metas de custas pitzinnas sun galanas

'Many of these girls are pretty'

Una pitzinna meta bella

'a very beautiful girl'

Cussas duas bravas pitzinnas

'those two good girls'

unu better cane bellu

'a huge beautiful dog'

un'átteru bette cane

'another huge dog'

Sun andatos a Frantza

'they went to France'

Est ghiratu dae Corsica

'He returned from Corsica'

Connnosco bene sa Frantza

"i know France well'

Custu libru est iscrittu in tedescu

'This book is written in German'

Cussus paráula venit dae su catalanu

'This word comes from Catalan'

Su sardu est una limba antica

'Sardinian is an ancient language'

Appo mandicatu casu

'I ate cheese'

Laura est toddende frores

'Laura is picking flowers'

De frores non nd'appo tódditu

'I did not pick any flowers'

No'amus comporatu pane

'We did not buy bread'

Mandico sa pasta kin casu

'I ate pasta with cheese'

Tres sórrikes an mandicatu tottu su casu

'Three mice ate all the cheese'

semus ponnende vinu in ampullas

'We are putting wine in bottles'

lettu meu esr

'It is my bed'

No'appo lessu libru perunu

'i did not read any book'

Sos k'an furatu si dinari at a èssere punitos

'The people who stole the money will be punished'

Appo invitat sos de Juanne

'I invited Jon's family'

Cussa pinna est sa mea

'That pen is mine'

Cudda mákkina est sa de Juanne

'That car is John's'

Custa ricca at datu mili francos a su vetzu

'This rich woman gave a thousand lira to the old man'

Su guvernu devíat dare ua pensione a dondzi vetzu

'The government should give a pension to every old person'

External Resources

• Github repository for Sardinian transducer (and our clone)
• Github repository for English transducer
• Dictionary pdf
• Online official dictionary
• Web-based machine translator
• Sardinian-Italian web-based dictionary
• 1931 pdf with select translations
• Definiteness effects: evidence from Sardinian
• Webster's Sardinian English Thesaurus Dictionary
• Italian-Sardinian Dictionary
• Old but extensive Sardo-Italian and vice versa dictionary
• nice Sardo-Italiano dictionary pdf
• Conjugations of "to have" in Sardinian
• Whole Sardinian grammar page about conjugations of to have all written in Sardinian language!
• Sardinian grammar:pdf,,