Wichita/Transducer
Contents
Initial Evaluation of Morphological Generation
https://github.swarthmore.edu/Ling073-sp26/ling073-wic
Evaluation
- 100% of tests pass, besides Adjectives (50%) and Unknown Words (93.75%).
Transducer Size:
| Category | Count |
|---|---|
| Lexicons | 24 |
| Lexicon entries | 107 |
| Patterns | 3 |
| Pattern entries | 12 |
Total Stems:
| Stem lexicon | Count |
|---|---|
| Punctuation | 22 |
| Aspect | 3 |
| Person | 7 |
| VerbRoot | 9 |
| PronounRoot | 10 |
| InterjectionRoot | 7 |
| AdverbRoot | 17 |
| ConjunctionRoot | 6 |
| Adjectives | 2 |
| NounRoot | 6 |
| Cases | 2 |
| Narrative | 1 |
| Quotative | 2 |
| Object | 1 |
| V-Suffixes | 3 |
| Numeral | 2 |
| All anonymous lexicons | 7 |
| Total | 107 |
Coverage:
Coverage before addition: ~31%
Coverage after addition: ~33%
Top Unknown Words:
- hinniˀ → Fixed
- kˀítaːks
- aːkoːkˀa
- wéraˀ
- kíkˀiːs
- hiriwa → Fixed
- niyeːs
- haˀwiːˀ
- híriˀ
- harah
- tiˀi
- naːhissah
- khiˀas
- éːkiriˀ → Fixed
- éːkaːˀ
- wáːcˀarˀa
Notes
12/16 Verb tests currently don't match gold, because the tags don't match between the code and the wiki.
1/5 Noun tests don't match gold because we haven't coded the root change for this verb.
0/3 Noun Incorporation tests match gold, due to formatting issues.
0/10 Pronoun tests match gold, because their tags have not been added to the lexicon.
0/6 Adjective tests match gold. Multiple words mean the same phrase, so we have been unable to make distinct rules for them.
0/3 Conjunction tests match gold, due to their tags not being included in the lexicon.
0/3 Quantifiers and Demonstrative tests match gold, due to their tags not being included in the lexicon.
1/17 Adverb tests don't match gold as it is missing a tag.
13/15 Unknown Words don't match gold, as their tags are not included in the lexicon.
- By adding <cnjcoo> to hinniˀ and hiriwa, as well as <n> to éːkiriˀ, coverage increased by ~2%.
Analyzer Evaluation
Initial Evaluation
Coverage: ~33%
Number of Passing Tests: 85/89 (~96%)
- All test are 100% passing besides Adjectives (3/6 = 50%) and Unknown Words (15/16 = 93.75%).
Number of Tests Matching Gold: 39/85 (~46%)
- Our number of tests matching gold decreased slightly after adding our unknown word tests at the end of the previous lab.
Gold Tests:
- 4/5 Nouns
- 0/3 Noun Incorporation
- 0/10 Pronouns
- 0/3 Adjectives
- 0/3 Quantifiers & Demonstratives
- 7/7 Interjections
- 6/6 Conjunctions
- 16/17 Adverbs
- 4/16 Verbs
- 2/15 Unknown Words
Number of Gold Matches for Generation Test: 42/76 (~55%)
Final Evaluation of Morphological Generation
Updated Coverage: 38.3%
Number of Passing Tests: 89
Number of Tests Matching Gold: 55/89 --> 62%
Gold Tests:
- 0/5 Nouns
- 10/10 Pronouns
- 0/6 Adjectives
- 7/7 Interjections
- 6/6 Conjunctions
- 17/17 Adverbs
- 10/22 Verbs
- 5/16 Unknown Words
Number of Gold Matches for Generation Test: 56/89 (63%)
Twol Rules:
- 2 added
- 0 modified
