Okinawan/Transducer
From LING073
Revision as of 14:42, 2 March 2017 by Aconant1 (talk | contribs) (Created page with "== Evaluation == My current coverage is horrendous (0.48%) due to a tokenization error, I believe. It reports many single-character words, but my corpus does not seem to have...")
Evaluation
My current coverage is horrendous (0.48%) due to a tokenization error, I believe. It reports many single-character words, but my corpus does not seem to have the ones that aq-covtest claims to see. However, there are several real words that it is (correctly) identifying as commonly unknown. These tend to be discourse markers, such as なあ, また, as a lot of my corpora are transcripts of conversations. Adding these analyses to the transducer increases coverage to 4.43%.
Notes
The tests that don't yet pass are either some extra nominal suffixive tests that I haven't gotten around to yet, or involve verbs. Verbs are a bit more complicated than the nominals, since they retain the verb stem and overwrite the rest.
