Difference between revisions of "Wolof/Disambiguator"
From LING073
(Created page with "======Evaluation of disambiguator======") |
|||
| (10 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| + | [https://github.swarthmore.edu/Ling073-sp23/ling073-wol link to git repo with transducer] | ||
======Evaluation of disambiguator====== | ======Evaluation of disambiguator====== | ||
| + | *Final ambiguity as of 3/22: 7907 / 7335 (~1.07798227675528289025) | ||
| + | (obtained by running the script below:) | ||
| + | coverage-hfst /home/rorlija1/ling073-wol-corpus/wol.corpus.basic.txt /home/rorlija1/ling073-wol/wol.automorf.hfst | ||
| + | *Number of tokens: 7344 | ||
| + | (Obtained by running the script below:) | ||
| + | cat /home/rorlija1/ling073-wol-corpus/wol.corpus.basic.txt | apertium-destxt | lt-proc /home/rorlija1/ling073-wol/wol.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | wc -l | ||
| + | |||
| + | *Total number of analyses: 7708 | ||
| + | (Obtained by running the script below:) | ||
| + | cat /home/rorlija1/ling073-wol-corpus/wol.corpus.basic.txt | apertium-destxt | lt-proc /home/rorlija1/ling073-wol/wol.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l | ||
| + | |||
| + | *Total number of analyses after disambiguation: 7705 | ||
| + | (Obtained by running the script below:) | ||
| + | cat /home/rorlija1/ling073-wol-corpus/wol.corpus.basic.txt | apertium-destxt | lt-proc /home/rorlija1/ling073-wol/wol.automorf.bin | apertium-retxt | cg-proc /home/rorlija1/ling073-wol/wol.rlx.bin | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l | ||
| + | |||
| + | *Examples: | ||
| + | (1) "lañu ne ci seen xarit" (they said/say to their friend) -> should be seen<p3> | ||
| + | ^lañu/lañu<prn><pers><p3><pl><ofc>$ ^ne/ne<prep>$ ^ci/ci<prep>$ ^'''seen/seen<det><pers><p3><sg>'''/seen<det><pers><p2><sg>$ ^xarit/xarit<n>$^./.<sent>$ | ||
| + | (2) "leen ne ci seen xaj" (you (pl) said/say to your dog) -> should be seen<p2> | ||
| + | ^leen/lañu<prn><pers><p3><pl><obj>/ngeen<prn><pers><p2><pl><obj>$ ^ne/ne<prep>$ ^ci/ci<prep>$ ^seen/seen<det><pers><p3><sg>/'''seen<det><pers><p2><sg>'''$ ^xaj/xaj<n>$^./.<sent>$ | ||
| + | [[Category:Sp23_Disambiguation]],[[Category:Wolof]]] | ||
Latest revision as of 20:02, 22 March 2023
link to git repo with transducer
Evaluation of disambiguator
- Final ambiguity as of 3/22: 7907 / 7335 (~1.07798227675528289025)
(obtained by running the script below:)
coverage-hfst /home/rorlija1/ling073-wol-corpus/wol.corpus.basic.txt /home/rorlija1/ling073-wol/wol.automorf.hfst
- Number of tokens: 7344
(Obtained by running the script below:)
cat /home/rorlija1/ling073-wol-corpus/wol.corpus.basic.txt | apertium-destxt | lt-proc /home/rorlija1/ling073-wol/wol.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | wc -l
- Total number of analyses: 7708
(Obtained by running the script below:)
cat /home/rorlija1/ling073-wol-corpus/wol.corpus.basic.txt | apertium-destxt | lt-proc /home/rorlija1/ling073-wol/wol.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l
- Total number of analyses after disambiguation: 7705
(Obtained by running the script below:)
cat /home/rorlija1/ling073-wol-corpus/wol.corpus.basic.txt | apertium-destxt | lt-proc /home/rorlija1/ling073-wol/wol.automorf.bin | apertium-retxt | cg-proc /home/rorlija1/ling073-wol/wol.rlx.bin | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l
- Examples:
(1) "lañu ne ci seen xarit" (they said/say to their friend) -> should be seen<p3>
^lañu/lañu<prn><pers><p3><pl><ofc>$ ^ne/ne<prep>$ ^ci/ci<prep>$ ^seen/seen<det><pers><p3><sg>/seen<det><pers><p2><sg>$ ^xarit/xarit<n>$^./.<sent>$
(2) "leen ne ci seen xaj" (you (pl) said/say to your dog) -> should be seen<p2>
^leen/lañu<prn><pers><p3><pl><obj>/ngeen<prn><pers><p2><pl><obj>$ ^ne/ne<prep>$ ^ci/ci<prep>$ ^seen/seen<det><pers><p3><sg>/seen<det><pers><p2><sg>$ ^xaj/xaj<n>$^./.<sent>$,]
