Ainu/Transducer

Generator

Initial evaluation of morphological generation

There are 79 stems in the transducer. 75/115 ain.yaml tests pass (40 fail). 2/20 commonwords.yaml tests pass (18 fail). Generation test has 77/164 ain.yaml tests pass (87 fail).

Final evaluation of morphological generation

After changing the grammar dramatically, 78/110 ain.yaml tests pass (32 fail). I added 9 new .twol rules, although only 7 are working. I did not have any before (I only had prefix/suffix tag-matching .twoc rules). Somehow my corpus coverage has gone down to 18.27% of the 925 tokenised words in the corpus.

Notes

Adding the .twoc prefix fix for getting rid of paths with odd numbers of feature tags fixed a lot. 77/133 ain.yaml tests pass, 56 fail.

Analyser

Evaluation

The most common unknown words originally were

Common Words

Word	Times Encountered
ネ	60
カ	50
アン	44
タ	30
ワ	29
コ	25
イ	20
ヒ	18
ピ	16
ルウェ	16
パ	16
ㇺ	14
ㇷ	14
ウタ	14
セコ	13
ハウェアン	13
キ	12
クニ	10
ニ	9
ヤㇰ	9

Original coverage of these words is 11.19%. Now the coverage is 22.76%.

There are 79 stems in the transducer. 75/115 ain.yaml tests pass (40 fail). 2/20 commonwords.yaml tests pass (18 fail).

Now the most common words are

Common Words

Word	Times Encountered
カ	50
アン	44
タ	30
ワ	29
イ	20
ヒ	18
ピ	16
ルウェ	16
パ	16
ㇺ	14
ㇷ	14
ウタ	14
セコ	13
ハウェアン	13
キ	12
クニ	10
ニ	9
ヤㇰ	9
チュㇷ	8
カノ	8

Notes

After adding "ネ<vbser> ↔ ネ" and "コ<cnjsub> ↔ コ" to the transducer, coverage went from 11.19% to 21.21%.

The tests I am struggling with are for adjectivity-- I need a new grammar book to confirm that there are really so many differing adjective endings because the current book I am using as a reference was written over 100 years ago. I have not yet had time to add pronoun or noun affinity tests. The remaining failing tests are irregular cases at which I will have to look more closely.

Transducer

Ainu transducer available on my GitHub.