Revision as of 07:17, 14 March 2017

Analyser

Evaluation

My current coverage is horrendous (0.48%) due to a tokenization error, I believe. It reports many single-character words (such as な, え, お, and friends) but my corpus does not seem to have the ones that aq-covtest claims to see. However, there are several real words that it is (correctly) identifying as commonly unknown. These tend to be discourse markers, such as なあ, また, as a lot of my corpora are transcripts of conversations. Adding these analyses to the transducer increases coverage to 4.43%.

There are currently 27 stems, according to lexccounter.

Number of tokenised words in the corpus: 835 Coverage: 4.43% Top unknown words in the corpus: About 28/50 tests currently pass.

37 え 17 ち 16 ん 14 う 14 さ 13 い 11 ら 11 とぅ 10 うり 10 あ 10 や 10 お 9 ぬ 8 か 7 み 7 やい 7 うまんか 7 ちゅ 6 あんち 6 さい Translation time: 0.004895448684692383 seconds

Notes

The tests that don't yet pass are either some extra nominal suffixive tests that I haven't gotten around to yet, or involve verbs. Verbs are a bit more complicated than the nominals, since they retain the verb stem and overwrite the rest.

Generator

Initial evaluation of morphological generator

27 passes; 51 fails. Coverage: 4.43%

Post evaluation