Difference between revisions of "Okinawan/Transducer"
(Created page with "== Evaluation == My current coverage is horrendous (0.48%) due to a tokenization error, I believe. It reports many single-character words, but my corpus does not seem to have...") |
m (→Evaluation) |
||
| Line 1: | Line 1: | ||
== Evaluation == | == Evaluation == | ||
| − | My current coverage is horrendous (0.48%) due to a tokenization error, I believe. It reports many single-character words, but my corpus does not seem to have the ones that aq-covtest claims to see. However, there are several | + | My current coverage is horrendous (0.48%) due to a tokenization error, I believe. It reports many single-character words (such as な, え, お, and friends) but my corpus does not seem to have the ones that aq-covtest claims to see. However, there are several real words that it is (correctly) identifying as commonly unknown. These tend to be discourse markers, such as |
| − | real words that it is (correctly) identifying as commonly unknown. These tend to be discourse markers, such as | ||
なあ, また, as a lot of my corpora are transcripts of conversations. Adding these analyses to the transducer increases coverage to 4.43%. | なあ, また, as a lot of my corpora are transcripts of conversations. Adding these analyses to the transducer increases coverage to 4.43%. | ||
| + | There are currently 27 stems, according to lexccounter. | ||
| + | |||
| + | About 28/50 tests currently pass. | ||
== Notes == | == Notes == | ||
Revision as of 06:08, 6 March 2017
Evaluation
My current coverage is horrendous (0.48%) due to a tokenization error, I believe. It reports many single-character words (such as な, え, お, and friends) but my corpus does not seem to have the ones that aq-covtest claims to see. However, there are several real words that it is (correctly) identifying as commonly unknown. These tend to be discourse markers, such as なあ, また, as a lot of my corpora are transcripts of conversations. Adding these analyses to the transducer increases coverage to 4.43%.
There are currently 27 stems, according to lexccounter.
About 28/50 tests currently pass.
Notes
The tests that don't yet pass are either some extra nominal suffixive tests that I haven't gotten around to yet, or involve verbs. Verbs are a bit more complicated than the nominals, since they retain the verb stem and overwrite the rest.
