Difference between revisions of "Wichita/Transducer"

From LING073
Jump to: navigation, search
(Created page with "==Evaluation== ==Notes== Category:Sp26_Transducers")
 
(→‎Final Evaluation of Morphological Generation)
 
(42 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
==Evaluation==
+
==Initial Evaluation of Morphological Generation==
  
−
==Notes==
+
https://github.swarthmore.edu/Ling073-sp26/ling073-wic
  
 +
===Evaluation===
  
 +
*100% of tests pass, besides Adjectives (50%) and Unknown Words (93.75%).
 +
 +
====Transducer Size:====
 +
 +
{| class="wikitable"
 +
! Category
 +
! Count
 +
|-
 +
| Lexicons
 +
| 24
 +
|-
 +
| Lexicon entries
 +
| 107
 +
|-
 +
| Patterns
 +
| 3
 +
|-
 +
| Pattern entries
 +
| 12
 +
|}
 +
 +
====Total Stems:====
 +
 +
{| class="wikitable"
 +
! Stem lexicon
 +
! Count
 +
|-
 +
| Punctuation
 +
| 22
 +
|-
 +
| Aspect
 +
| 3
 +
|-
 +
| Person
 +
| 7
 +
|-
 +
| VerbRoot
 +
| 9
 +
|-
 +
| PronounRoot
 +
| 10
 +
|-
 +
| InterjectionRoot
 +
| 7
 +
|-
 +
| AdverbRoot
 +
| 17
 +
|-
 +
| ConjunctionRoot
 +
| 6
 +
|-
 +
| Adjectives
 +
| 2
 +
|-
 +
| NounRoot
 +
| 6
 +
|-
 +
| Cases
 +
| 2
 +
|-
 +
| Narrative
 +
| 1
 +
|-
 +
| Quotative
 +
| 2
 +
|-
 +
| Object
 +
| 1
 +
|-
 +
| V-Suffixes
 +
| 3
 +
|-
 +
| Numeral
 +
| 2
 +
|-
 +
| All anonymous lexicons
 +
| 7
 +
|-
 +
! Total
 +
! 107
 +
|}
 +
 +
====Coverage:====
 +
 +
Coverage before addition: ~31%
 +
 +
Coverage after addition: ~33%
 +
 +
Top Unknown Words:
 +
 +
*hinniˀ → Fixed
 +
*kˀítaːks
 +
*aːkoːkˀa
 +
*wéraˀ
 +
*kíkˀiːs
 +
*hiriwa → Fixed
 +
*niyeːs
 +
*haˀwiːˀ
 +
*híriˀ
 +
*harah
 +
*tiˀi
 +
*naːhissah
 +
*khiˀas
 +
*éːkiriˀ → Fixed
 +
*éːkaːˀ
 +
*wáːcˀarˀa
 +
 +
===Notes===
 +
12/16 Verb tests currently don't match gold, because the tags don't match between the code and the wiki.
 +
 +
1/5 Noun tests don't match gold because we haven't coded the root change for this verb.
 +
 +
0/3 Noun Incorporation tests match gold, due to formatting issues.
 +
 +
0/10 Pronoun tests match gold, because their tags have not been added to the lexicon.
 +
 +
0/6 Adjective tests match gold. Multiple words mean the same phrase, so we have been unable to make distinct rules for them.
 +
 +
0/3 Conjunction tests match gold, due to their tags not being included in the lexicon.
 +
 +
0/3 Quantifiers and Demonstrative tests match gold, due to their tags not being included in the lexicon.
 +
 +
1/17 Adverb tests don't match gold as it is missing a tag.
 +
 +
13/15 Unknown Words don't match gold, as their tags are not included in the lexicon.
 +
 +
*By adding <cnjcoo> to hinniˀ and hiriwa, as well as <n> to éːkiriˀ, coverage increased by ~2%.
 +
 +
==Analyzer Evaluation==
 +
===Initial Evaluation===
 +
Coverage: ~33%
 +
 +
Number of Passing Tests: 85/89 (~96%)
 +
*All test are 100% passing besides Adjectives (3/6 = 50%) and Unknown Words (15/16 = 93.75%).
 +
 +
Number of Tests Matching Gold: 39/85 (~46%)
 +
*Our number of tests matching gold decreased slightly after adding our unknown word tests at the end of the previous lab.
 +
 +
Gold Tests:
 +
*4/5 Nouns
 +
*0/3 Noun Incorporation
 +
*0/10 Pronouns
 +
*0/3 Adjectives
 +
*0/3 Quantifiers & Demonstratives
 +
*7/7 Interjections
 +
*6/6 Conjunctions
 +
*16/17 Adverbs
 +
*4/16 Verbs
 +
*2/15 Unknown Words
 +
 +
Number of Gold Matches for Generation Test: 42‌‌/76 (~55%)
 +
 +
===Final Evaluation of Morphological Generation===
 +
 +
Updated Coverage: 38.3%
 +
 +
Number of Passing Tests: 89
 +
 +
Number of Tests Matching Gold: 55/89 --> 62%
 +
 +
 +
Gold Tests:
 +
*0/5 Nouns
 +
*10/10 Pronouns
 +
*0/6 Adjectives
 +
*7/7 Interjections
 +
*6/6 Conjunctions
 +
*17/17 Adverbs
 +
*10/22 Verbs
 +
*5/16 Unknown Words
 +
 +
Number of Gold Matches for Generation Test: 56/89 (63%)
 +
 +
Twol Rules:
 +
*2 added
 +
*0 modified
 +
 
[[Category:Sp26_Transducers]]
 
[[Category:Sp26_Transducers]]

Latest revision as of 00:56, 9 April 2026

Initial Evaluation of Morphological Generation

https://github.swarthmore.edu/Ling073-sp26/ling073-wic

Evaluation

  • 100% of tests pass, besides Adjectives (50%) and Unknown Words (93.75%).

Transducer Size:

Category Count
Lexicons 24
Lexicon entries 107
Patterns 3
Pattern entries 12

Total Stems:

Stem lexicon Count
Punctuation 22
Aspect 3
Person 7
VerbRoot 9
PronounRoot 10
InterjectionRoot 7
AdverbRoot 17
ConjunctionRoot 6
Adjectives 2
NounRoot 6
Cases 2
Narrative 1
Quotative 2
Object 1
V-Suffixes 3
Numeral 2
All anonymous lexicons 7
Total 107

Coverage:

Coverage before addition: ~31%

Coverage after addition: ~33%

Top Unknown Words:

  • hinniˀ → Fixed
  • kˀítaːks
  • aːkoːkˀa
  • wéraˀ
  • kíkˀiːs
  • hiriwa → Fixed
  • niyeːs
  • haˀwiːˀ
  • híriˀ
  • harah
  • tiˀi
  • naːhissah
  • khiˀas
  • éːkiriˀ → Fixed
  • éːkaːˀ
  • wáːcˀarˀa

Notes

12/16 Verb tests currently don't match gold, because the tags don't match between the code and the wiki.

1/5 Noun tests don't match gold because we haven't coded the root change for this verb.

0/3 Noun Incorporation tests match gold, due to formatting issues.

0/10 Pronoun tests match gold, because their tags have not been added to the lexicon.

0/6 Adjective tests match gold. Multiple words mean the same phrase, so we have been unable to make distinct rules for them.

0/3 Conjunction tests match gold, due to their tags not being included in the lexicon.

0/3 Quantifiers and Demonstrative tests match gold, due to their tags not being included in the lexicon.

1/17 Adverb tests don't match gold as it is missing a tag.

13/15 Unknown Words don't match gold, as their tags are not included in the lexicon.

  • By adding <cnjcoo> to hinniˀ and hiriwa, as well as <n> to éːkiriˀ, coverage increased by ~2%.

Analyzer Evaluation

Initial Evaluation

Coverage: ~33%

Number of Passing Tests: 85/89 (~96%)

  • All test are 100% passing besides Adjectives (3/6 = 50%) and Unknown Words (15/16 = 93.75%).

Number of Tests Matching Gold: 39/85 (~46%)

  • Our number of tests matching gold decreased slightly after adding our unknown word tests at the end of the previous lab.

Gold Tests:

  • 4/5 Nouns
  • 0/3 Noun Incorporation
  • 0/10 Pronouns
  • 0/3 Adjectives
  • 0/3 Quantifiers & Demonstratives
  • 7/7 Interjections
  • 6/6 Conjunctions
  • 16/17 Adverbs
  • 4/16 Verbs
  • 2/15 Unknown Words

Number of Gold Matches for Generation Test: 42‌‌/76 (~55%)

Final Evaluation of Morphological Generation

Updated Coverage: 38.3%

Number of Passing Tests: 89

Number of Tests Matching Gold: 55/89 --> 62%


Gold Tests:

  • 0/5 Nouns
  • 10/10 Pronouns
  • 0/6 Adjectives
  • 7/7 Interjections
  • 6/6 Conjunctions
  • 17/17 Adverbs
  • 10/22 Verbs
  • 5/16 Unknown Words

Number of Gold Matches for Generation Test: 56/89 (63%)

Twol Rules:

  • 2 added
  • 0 modified
Retrieved from "http://wikis.swarthmore.edu/ling073/index.php?title=Wichita/Transducer&oldid=28006"