Difference between revisions of "Misc tools"
(→lexccounter: apertium-eval-translator) |
(→apertium-eval-translator) |
||
| Line 28: | Line 28: | ||
The reference translation is the parallel text in your corpus, e.g. <code>abc.tests.txt</code>. To get a test translation, run the source text (<code>xyz.tests.txt</code>) through apertium and direct the output into a new file, e.g. | The reference translation is the parallel text in your corpus, e.g. <code>abc.tests.txt</code>. To get a test translation, run the source text (<code>xyz.tests.txt</code>) through apertium and direct the output into a new file, e.g. | ||
cat xyz.tests.txt | apertium -d . xyz-abc > xyz-abc.tests.txt | cat xyz.tests.txt | apertium -d . xyz-abc > xyz-abc.tests.txt | ||
| + | |||
| + | == scrapeTransferTests == | ||
| + | You can scrape the transferTests from your contrastive grammar page into a small parallel corpus. E.g., | ||
| + | scrapeTransferTests -p abc-xyz "Language1_and_Language2/Contrastive_Grammar" | ||
| + | will result in an <code>abc.tests.txt</code> and <code>xyz.tests.txt</code> file that contain the respective sides of any transferTests on your contrastive grammar page specified as being for abc to xyz translation. | ||
Revision as of 03:44, 29 March 2018
You can install miscellaneous tools from this repo, like morphTests2yaml and lexccounter, as follows:
cd ~/ling073 git clone git@github.swarthmore.edu:Ling073-sp18/tools.git cd tools make install
morphTests2yaml
You can scrape {{morphTest}}s from any page on this wiki like this:
morphTests2yaml "Language/Grammar" -l xyz
Put the file in a tests/ directory in your transducer repo and commit the file.
lexccounter
This script counts the number of unique entries in a lexc file.
$ lexccounter apertium-xyz.xyz.lexc Unique entries: 127
apertium-eval-translator
You need two files:
- one test translation, and
- one reference translation
The reference translation is the parallel text in your corpus, e.g. abc.tests.txt. To get a test translation, run the source text (xyz.tests.txt) through apertium and direct the output into a new file, e.g.
cat xyz.tests.txt | apertium -d . xyz-abc > xyz-abc.tests.txt
scrapeTransferTests
You can scrape the transferTests from your contrastive grammar page into a small parallel corpus. E.g.,
scrapeTransferTests -p abc-xyz "Language1_and_Language2/Contrastive_Grammar"
will result in an abc.tests.txt and xyz.tests.txt file that contain the respective sides of any transferTests on your contrastive grammar page specified as being for abc to xyz translation.
