Difference between revisions of "Asturian and English"
| (4 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
== Machine Translation Resources == | == Machine Translation Resources == | ||
| − | [https://github.swarthmore.edu/mparker3/ling073-ast-eng-corpus Github repository] containing the | + | [https://github.swarthmore.edu/mparker3/ling073-ast-eng-corpus Github repository] containing a number of tools for machine translation, including small test corpuses, a corpus of the Asturian Wikipedia, UD-annotated ConLLU files, and UDPipe models for UD annotation |
== Contrastive Grammar == | == Contrastive Grammar == | ||
| Line 12: | Line 12: | ||
[https://wikis.swarthmore.edu/ling073/Asturian_and_English/Lexical_selection Lexical Selection] | [https://wikis.swarthmore.edu/ling073/Asturian_and_English/Lexical_selection Lexical Selection] | ||
| + | |||
| + | == Dependency Annotation == | ||
| + | |||
| + | [https://wikis.swarthmore.edu/ling073/Asturian/Universal_dependencies Dependency Annotation] | ||
== ast -> eng evaluation == | == ast -> eng evaluation == | ||
| Line 21: | Line 25: | ||
Number of words which were free rides: 0 | Number of words which were free rides: 0 | ||
| − | == | + | == Additions == |
Stems: added 100 new stems to the dictionary file | Stems: added 100 new stems to the dictionary file | ||
| + | |||
Lexical selection: Implemented 3 new lexical selection rules | Lexical selection: Implemented 3 new lexical selection rules | ||
| + | |||
Structural Transfer: Implemented 3 new structural transfer rules | Structural Transfer: Implemented 3 new structural transfer rules | ||
| + | |||
| + | == Evaluation == | ||
| + | |||
| + | === Asturian Transducer === | ||
| + | |||
| + | Coverage against the large corpus wasn't available because of encoding errors. | ||
| + | |||
| + | Measuring precision and recall between the annotated corpus and the sample corpus also wasn't available because of corpus length errors and compound words in Asturian. | ||
| + | |||
| + | There are 39861079 words in the <code>longer</code> corpus. | ||
| + | |||
| + | Total number of stems in transducer could not be calculated because the number of stems in the <code>apertium-ast.ast.dix</code> were too numerous to count by hand and the transducer did not use a <code>lexc</code> file. | ||
| + | |||
| + | === Asturian-English Translation System === | ||
| + | |||
| + | WER over longer corpus: 99.18% | ||
| + | PER over longer corpus: 98.47% | ||
| + | |||
| + | Proportion of stems tranlated correctly in the longer corpus: what script do I use to find this? | ||
| + | |||
| + | |||
| + | |||
[[Category:Sp18_TranslationPairs]] | [[Category:Sp18_TranslationPairs]] | ||
Latest revision as of 19:12, 2 June 2018
Resources for machine translation between Asturian and English
Contents
Machine Translation Resources
Github repository containing a number of tools for machine translation, including small test corpuses, a corpus of the Asturian Wikipedia, UD-annotated ConLLU files, and UDPipe models for UD annotation
Contrastive Grammar
Lexical Selection
Dependency Annotation
ast -> eng evaluation
WER: 100%
PER: 100%
Number of words which were free rides: 0
Additions
Stems: added 100 new stems to the dictionary file
Lexical selection: Implemented 3 new lexical selection rules
Structural Transfer: Implemented 3 new structural transfer rules
Evaluation
Asturian Transducer
Coverage against the large corpus wasn't available because of encoding errors.
Measuring precision and recall between the annotated corpus and the sample corpus also wasn't available because of corpus length errors and compound words in Asturian.
There are 39861079 words in the longer corpus.
Total number of stems in transducer could not be calculated because the number of stems in the apertium-ast.ast.dix were too numerous to count by hand and the transducer did not use a lexc file.
Asturian-English Translation System
WER over longer corpus: 99.18% PER over longer corpus: 98.47%
Proportion of stems tranlated correctly in the longer corpus: what script do I use to find this?
