Difference between revisions of "Kickapoo/Transducer"

From LING073
Jump to: navigation, search
(→‎Hi)
(→‎Final Evaluation For Morphological Generation)
 
(37 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
Our code is listed here: https://github.swarthmore.edu/Ling073-sp26/ling073-kic
  
−
== Hi ==
+
== Evaluations ==
−
To do:
+
 
−
add aspect;
+
=== Transducer size ===
−
add corpus sources;
+
 
−
fix code (redstem and more);
+
Using <code>lexd -x apertium-kic.kic.lexd > /dev/null</code>, the current transducer contains:
 +
 
 +
{| class="wikitable"
 +
! Category
 +
! Count
 +
|-
 +
| Lexicons
 +
| 28
 +
|-
 +
| Lexicon entries
 +
| 105
 +
|-
 +
| Patterns
 +
| 14
 +
|-
 +
| Pattern entries
 +
| 26
 +
|}
 +
 
 +
=== Total number of stems in the transducer ===
 +
 
 +
We counted only open-class lexical stem lexicons and excluded punctuation, affixes, person markers, tense/mode markers, and other closed-class/function-word lexicons.
 +
 
 +
{| class="wikitable"
 +
! Stem lexicon
 +
! Count
 +
|-
 +
| A-Stems
 +
| 8
 +
|-
 +
| I-Stems
 +
| 5
 +
|-
 +
| L-Stems
 +
| 1
 +
|-
 +
| Verbstem
 +
| 2
 +
|-
 +
| VStem_AA
 +
| 4
 +
|-
 +
! Total
 +
! 20
 +
|}
 +
 
 +
=== Current coverage over the combined corpus ===
 +
 
 +
Coverage before adding common words:
 +
* <code>83 / 584 = 14.21%</code>
 +
 
 +
Current coverage:
 +
* <code>133 / 584 = 22.77%</code>
 +
 
 +
Improvement in coverage:
 +
* <code>+50</code> analysed tokens
 +
* <code>+8.56 percentage points</code>
 +
 
 +
Additional corpus statistics:
 +
* Remaining unknown forms: <code>451</code>
 +
* Ambiguity: <code>847 / 584 = 1.45</code>
 +
 
 +
=== Current top unknown words ===
 +
 
 +
{| class="wikitable"
 +
! Rank
 +
! Unknown word
 +
! Frequency
 +
|-
 +
| 1
 +
| <code>kwiyatheeha</code>
 +
| 4
 +
|-
 +
| 2
 +
| <code>awiyeehi</code>
 +
| 4
 +
|-
 +
| 3
 +
| <code>yoóh</code>
 +
| 3
 +
|-
 +
| 4
 +
| <code>yaaha</code>
 +
| 3
 +
|-
 +
| 5
 +
| <code>sehe</code>
 +
| 3
 +
|-
 +
| 6
 +
| <code>iskweetheehani</code>
 +
| 3
 +
|-
 +
| 7
 +
| <code>epapakethomehaki</code>
 +
| 3
 +
|-
 +
| 8
 +
| <code>aia</code>
 +
| 3
 +
|-
 +
| 9
 +
| <code>yoohi</code>
 +
| 2
 +
|-
 +
| 10
 +
| <code>yeetoke</code>
 +
| 2
 +
|-
 +
| 11
 +
| <code>witheniikaani</code>
 +
| 2
 +
|-
 +
| 12
 +
| <code>weepenaanwi</code>
 +
| 2
 +
|-
 +
| 13
 +
| <code>tahkiai</code>
 +
| 2
 +
|-
 +
| 14
 +
| <code>seeski</code>
 +
| 2
 +
|-
 +
| 15
 +
| <code>see</code>
 +
| 2
 +
|-
 +
| 16
 +
| <code>poohkamáaki</code>
 +
| 2
 +
|-
 +
| 17
 +
| <code>papakethohahi</code>
 +
| 2
 +
|-
 +
| 18
 +
| <code>nootha</code>
 +
| 2
 +
|-
 +
| 19
 +
| <code>niihpya</code>
 +
| 2
 +
|-
 +
| 20
 +
| <code>nekoti</code>
 +
| 2
 +
|}
 +
 
 +
 
 +
===Unimplemented===
 +
 
 +
# Vaux in Aspect not yet added; we haven't figured out a way to correctly analyze it;
 +
# Negation also not yet added due to the same reason;
 +
# Mode not yet added, may require further discussion;
 +
 
 +
 
 +
=== Regtest results ===
  
−
== Evaluations ==
+
The following table shows the number of tests that pass in each corpus.
 +
 
 +
{| class="wikitable"
 +
! Corpus
 +
! Tests passed
 +
! Match gold
 +
|-
 +
| analysis
 +
| 2 / 2
 +
| 0 / 2
 +
|-
 +
| generation
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| Animate Nouns-morph
 +
| 2 / 2
 +
| 2 / 2
 +
|-
 +
| Animate Nouns-gen
 +
| 2 / 2
 +
| 1 / 2
 +
|-
 +
| Inanimate Nouns-morph
 +
| 2 / 2
 +
| 1 / 2
 +
|-
 +
| Inanimate Nouns-gen
 +
| 2 / 2
 +
| 1 / 2
 +
|-
 +
| Noun Inflection (Posession)-morph
 +
| 5 / 5
 +
| 5 / 5
 +
|-
 +
| Noun Inflection (Posession)-gen
 +
| 5 / 5
 +
| 1 / 5
 +
|-
 +
| Person-morph
 +
| 10 / 10
 +
| 9 / 10
 +
|-
 +
| Person-gen
 +
| 10 / 10
 +
| 0 / 10
 +
|-
 +
| Future Tense-morph
 +
| 2 / 2
 +
| 0 / 2
 +
|-
 +
| Future Tense-gen
 +
| 2 / 2
 +
| 0 / 2
 +
|-
 +
| First Aorist-morph
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| First Aorist-gen
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| Second Aorist-morph
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| Second Aorist-gen
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| Third Aorist-morph
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| Third Aorist-gen
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| Aspect-morph
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| Aspect-gen
 +
| 1 / 1
 +
| 0 / 1
 +
|-
 +
| Mode-morph
 +
| 6 / 6
 +
| 0 / 6
 +
|-
 +
| Mode-gen
 +
| 6 / 6
 +
| 0 / 6
 +
|-
 +
| Negation-morph
 +
| 6 / 6
 +
| 0 / 6
 +
|-
 +
| Negation-gen
 +
| 7 / 7
 +
| 0 / 7
 +
|-
 +
| Locative-morph
 +
| 1 / 1
 +
| 1 / 1
 +
|-
 +
| Locative-gen
 +
| 1 / 1
 +
| 1 / 1
 +
|-
 +
| commonwords-morph
 +
| 4 / 4
 +
| 4 / 4
 +
|}
 +
 
 +
=== Test status summary ===
 +
 
 +
All 27 corpora currently pass all tests:
 +
* '''79 / 79 tests pass (100%)'''
 +
 
 +
=== Commonwords evaluation ===
 +
 
 +
We added a new <code>commonwords-morph</code> test file for frequent unknown words from the corpus.
  
 +
{| class="wikitable"
 +
! Corpus
 +
! Tests passed
 +
! Match gold
 +
|-
 +
| commonwords-morph
 +
| 4 / 4
 +
| 4 / 4
 +
|}
  
 
== Notes ==
 
== Notes ==
  
 +
===Unresolved Tests===
 +
 +
 +
== Analyser Evaluation ==
 +
 +
===Twol Rules===
 +
 +
No Twol rules are added; No verification has been noticed in Kickapoo grammar in the new script.
 +
 +
===Updated Stem Counts===
 +
 +
Lexicons: 29
 +
Lexicon entries: 106
 +
Patterns: 13
 +
Pattern entries: 26
 +
 +
Counts for individual lexicons:
 +
 +
Punctuation: 22
 +
 +
Demonstrative: 2
 +
 +
Particle: 1
 +
 +
A-Stems: 8
 +
 +
A-Number: 2
 +
 +
A-Poss: 5
 +
 +
I-Stems: 5
 +
 +
I-Number: 2
 +
 +
I-Poss: 5
 +
 +
L-Stems: 1
 +
 +
Loc: 1
 +
 +
Adverb: 1
 +
 +
Conj: 1
 +
 +
PrefPerson: 2
 +
 +
Vaux: 15
 +
 +
Verbstem: 2
 +
 +
VerbAnimacy: 2
 +
 +
SuffPerson: 8
 +
 +
Tense: 3
 +
 +
VStem_AA: 4
 +
 +
V-Mode: 5
 +
 +
Numeral: 3
 +
 +
All anonymous lexicons: 6
 +
 +
===Generator Evaluation===
 +
 +
Corpus 1 of 12: Animate Nouns-morph
 +
  2/2 (100.0%) tests pass (2/2 (100.0%) match gold)
 +
 +
Corpus 2 of 12: Inanimate Nouns-morph
 +
  2/2 (100.0%) tests pass (1/2 (50.0%) match gold)
 +
 +
Corpus 3 of 12: Noun Inflection (Posession)-morph
 +
  5/5 (100.0%) tests pass (5/5 (100.0%) match gold)
 +
 +
Corpus 4 of 12: Person-morph
 +
  10/10 (100.0%) tests pass (3/10 (30.0%) match gold)
 +
 +
Corpus 5 of 12: Future Tense-morph
 +
  2/2 (100.0%) tests pass (0/2 (0.0%) match gold)
 +
 +
Corpus 6 of 12: First Aorist-morph
 +
  1/1 (100.0%) tests pass (0/1 (0.0%) match gold)
 +
 +
Corpus 7 of 12: Second Aorist-morph
 +
  1/1 (100.0%) tests pass (0/1 (0.0%) match gold)
 +
 +
Corpus 8 of 12: Third Aorist-morph
 +
  1/1 (100.0%) tests pass (0/1 (0.0%) match gold)
 +
 +
Corpus 9 of 12: Aspect-morph
 +
  1/1 (100.0%) tests pass (0/1 (0.0%) match gold)
 +
 +
Corpus 10 of 12: Mode-morph
 +
  6/6 (100.0%) tests pass (0/6 (0.0%) match gold)
 +
 +
Corpus 11 of 12: Negation-morph
 +
  6/6 (100.0%) tests pass (0/6 (0.0%) match gold)
 +
 +
Corpus 12 of 12: Locative-morph
 +
  1/1 (100.0%) tests pass (1/1 (100.0%) match gold)
 +
 +
All tests pass.
 +
 +
 +
== Final Evaluation For Morphological Generation ==
  
 +
The morphological generation component now passes '''95 out of 95 tests''', with '''0 failing tests'''.
  
 +
Overall, the system achieves a '''match gold rate of 72/95 = 75.8%'''.
  
 +
For this part of the assignment, we added '''2''' new twol rules.
  
 +
When we reran the coverage test on the largest corpus, the new coverage was '''24.01%'''.
  
 
[[Category:Sp26_Transducers]]
 
[[Category:Sp26_Transducers]]
 
[[Category: Kickapoo]]
 
[[Category: Kickapoo]]

Latest revision as of 21:01, 29 March 2026

Our code is listed here: https://github.swarthmore.edu/Ling073-sp26/ling073-kic

Evaluations

Transducer size

Using lexd -x apertium-kic.kic.lexd > /dev/null, the current transducer contains:

Category Count
Lexicons 28
Lexicon entries 105
Patterns 14
Pattern entries 26

Total number of stems in the transducer

We counted only open-class lexical stem lexicons and excluded punctuation, affixes, person markers, tense/mode markers, and other closed-class/function-word lexicons.

Stem lexicon Count
A-Stems 8
I-Stems 5
L-Stems 1
Verbstem 2
VStem_AA 4
Total 20

Current coverage over the combined corpus

Coverage before adding common words:

  • 83 / 584 = 14.21%

Current coverage:

  • 133 / 584 = 22.77%

Improvement in coverage:

  • +50 analysed tokens
  • +8.56 percentage points

Additional corpus statistics:

  • Remaining unknown forms: 451
  • Ambiguity: 847 / 584 = 1.45

Current top unknown words

Rank Unknown word Frequency
1 kwiyatheeha 4
2 awiyeehi 4
3 yoóh 3
4 yaaha 3
5 sehe 3
6 iskweetheehani 3
7 epapakethomehaki 3
8 aia 3
9 yoohi 2
10 yeetoke 2
11 witheniikaani 2
12 weepenaanwi 2
13 tahkiai 2
14 seeski 2
15 see 2
16 poohkamáaki 2
17 papakethohahi 2
18 nootha 2
19 niihpya 2
20 nekoti 2


Unimplemented

  1. Vaux in Aspect not yet added; we haven't figured out a way to correctly analyze it;
  2. Negation also not yet added due to the same reason;
  3. Mode not yet added, may require further discussion;


Regtest results

The following table shows the number of tests that pass in each corpus.

Corpus Tests passed Match gold
analysis 2 / 2 0 / 2
generation 1 / 1 0 / 1
Animate Nouns-morph 2 / 2 2 / 2
Animate Nouns-gen 2 / 2 1 / 2
Inanimate Nouns-morph 2 / 2 1 / 2
Inanimate Nouns-gen 2 / 2 1 / 2
Noun Inflection (Posession)-morph 5 / 5 5 / 5
Noun Inflection (Posession)-gen 5 / 5 1 / 5
Person-morph 10 / 10 9 / 10
Person-gen 10 / 10 0 / 10
Future Tense-morph 2 / 2 0 / 2
Future Tense-gen 2 / 2 0 / 2
First Aorist-morph 1 / 1 0 / 1
First Aorist-gen 1 / 1 0 / 1
Second Aorist-morph 1 / 1 0 / 1
Second Aorist-gen 1 / 1 0 / 1
Third Aorist-morph 1 / 1 0 / 1
Third Aorist-gen 1 / 1 0 / 1
Aspect-morph 1 / 1 0 / 1
Aspect-gen 1 / 1 0 / 1
Mode-morph 6 / 6 0 / 6
Mode-gen 6 / 6 0 / 6
Negation-morph 6 / 6 0 / 6
Negation-gen 7 / 7 0 / 7
Locative-morph 1 / 1 1 / 1
Locative-gen 1 / 1 1 / 1
commonwords-morph 4 / 4 4 / 4

Test status summary

All 27 corpora currently pass all tests:

  • 79 / 79 tests pass (100%)

Commonwords evaluation

We added a new commonwords-morph test file for frequent unknown words from the corpus.

Corpus Tests passed Match gold
commonwords-morph 4 / 4 4 / 4

Notes

Unresolved Tests

Analyser Evaluation

Twol Rules

No Twol rules are added; No verification has been noticed in Kickapoo grammar in the new script.

Updated Stem Counts

Lexicons: 29 Lexicon entries: 106 Patterns: 13 Pattern entries: 26

Counts for individual lexicons:

Punctuation: 22

Demonstrative: 2

Particle: 1

A-Stems: 8

A-Number: 2

A-Poss: 5

I-Stems: 5

I-Number: 2

I-Poss: 5

L-Stems: 1

Loc: 1

Adverb: 1

Conj: 1

PrefPerson: 2

Vaux: 15

Verbstem: 2

VerbAnimacy: 2

SuffPerson: 8

Tense: 3

VStem_AA: 4

V-Mode: 5

Numeral: 3

All anonymous lexicons: 6

Generator Evaluation

Corpus 1 of 12: Animate Nouns-morph 

 2/2 (100.0%) tests pass (2/2 (100.0%) match gold)

Corpus 2 of 12: Inanimate Nouns-morph 

 2/2 (100.0%) tests pass (1/2 (50.0%) match gold)

Corpus 3 of 12: Noun Inflection (Posession)-morph 

 5/5 (100.0%) tests pass (5/5 (100.0%) match gold)

Corpus 4 of 12: Person-morph 

 10/10 (100.0%) tests pass (3/10 (30.0%) match gold)

Corpus 5 of 12: Future Tense-morph 

 2/2 (100.0%) tests pass (0/2 (0.0%) match gold)

Corpus 6 of 12: First Aorist-morph 

 1/1 (100.0%) tests pass (0/1 (0.0%) match gold)

Corpus 7 of 12: Second Aorist-morph 

 1/1 (100.0%) tests pass (0/1 (0.0%) match gold)

Corpus 8 of 12: Third Aorist-morph 

 1/1 (100.0%) tests pass (0/1 (0.0%) match gold)

Corpus 9 of 12: Aspect-morph 

 1/1 (100.0%) tests pass (0/1 (0.0%) match gold)

Corpus 10 of 12: Mode-morph 

 6/6 (100.0%) tests pass (0/6 (0.0%) match gold)

Corpus 11 of 12: Negation-morph 

 6/6 (100.0%) tests pass (0/6 (0.0%) match gold)

Corpus 12 of 12: Locative-morph 

 1/1 (100.0%) tests pass (1/1 (100.0%) match gold)

All tests pass.


Final Evaluation For Morphological Generation

The morphological generation component now passes 95 out of 95 tests, with 0 failing tests.

Overall, the system achieves a match gold rate of 72/95 = 75.8%.

For this part of the assignment, we added 2 new twol rules.

When we reran the coverage test on the largest corpus, the new coverage was 24.01%.

Retrieved from "http://wikis.swarthmore.edu/ling073/index.php?title=Kickapoo/Transducer&oldid=27937"