Difference between revisions of "Misc tools"
(→Installing on your own system) |
|||
| (15 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | You can install ''' | + | == Installing on your own system == |
| + | |||
| + | {{notice|These tools should be installed on the lab computers already; this section is just for those of you who would like to use the tools on your own system.}} | ||
| + | |||
| + | You can install miscellaneous tools from [https://github.swarthmore.edu/Ling073-sp26/tools this repo], like '''morphTests2regtest''' and '''lexccounter''', as follows: | ||
<pre> | <pre> | ||
cd ~/ling073 | cd ~/ling073 | ||
| − | git clone git@github.swarthmore.edu:Ling073- | + | git clone git@github.swarthmore.edu:Ling073-sp26/tools.git |
| + | cd tools | ||
| + | make install | ||
| + | </pre> | ||
| + | |||
| + | After you've already cloned the repo, you can update to the most recent version like this: | ||
| + | <pre> | ||
cd tools | cd tools | ||
| + | git pull | ||
make install | make install | ||
</pre> | </pre> | ||
| − | You can scrape <code><NOWIKI>{{morphTest}}</NOWIKI></code>s from any page on this wiki like this: | + | == morphTests2regtest == |
| + | |||
| + | You can scrape <code><NOWIKI>{{morphTest}}</NOWIKI></code>s from any page on this wiki for use with <code>[[apertium-regtest]]</code> like this: | ||
| + | morphTests2regtest "Language/Grammar" -l xyz | ||
| + | |||
| + | Run this from inside your transducer repository. It populates the <code>test/</code> directory in your transducer. Be sure to commit the files. | ||
| + | |||
| + | == morphTests2yaml == | ||
| + | |||
| + | {{notice|We're using [[#morphTests2regtest|morphTests2regtest]] (see above) now.}} | ||
| + | |||
| + | You can scrape <code><NOWIKI>{{morphTest}}</NOWIKI></code>s from any page on this wiki for use with <code>morph-test</code> or <code>aq-morphtest</code> like this: | ||
morphTests2yaml "Language/Grammar" -l xyz | morphTests2yaml "Language/Grammar" -l xyz | ||
Put the file in a <code>tests/</code> directory in your transducer repo and commit the file. | Put the file in a <code>tests/</code> directory in your transducer repo and commit the file. | ||
| − | == | + | == lexccounter == |
| − | + | This script counts the number of unique entries in a lexc file. | |
| + | |||
| + | $ lexccounter apertium-xyz.xyz.lexc | ||
| + | Unique entries: 127 | ||
| + | |||
| + | == apertium-eval-translator == | ||
| + | |||
| + | You need two files: | ||
| + | * one test translation, and | ||
| + | * one reference translation | ||
| + | |||
| + | The reference translation is the parallel text in your corpus, e.g. <code>abc.tests.txt</code>. To get a test translation, run the source text (<code>xyz.tests.txt</code>) through apertium and direct the output into a new file, e.g. | ||
| + | cat xyz.tests.txt | apertium -d . xyz-abc > xyz-abc.tests.txt | ||
| + | |||
| + | == scrapeTransferTests == | ||
| + | You can scrape the transferTests from your contrastive grammar page into a small parallel corpus. E.g., | ||
| + | scrapeTransferTests -p abc-xyz "Language1_and_Language2/Contrastive_Grammar" | ||
| + | will result in an <code>abc.tests.txt</code> and <code>xyz.tests.txt</code> file that contain the respective sides of any transferTests on your contrastive grammar page specified as being for abc to xyz translation. | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
[[Spring 2017/MorphTests2yaml|2017 version of this page]] | [[Spring 2017/MorphTests2yaml|2017 version of this page]] | ||
[[Category:Tools]] | [[Category:Tools]] | ||
Latest revision as of 20:30, 17 May 2026
Contents
Installing on your own system
NOTICE: These tools should be installed on the lab computers already; this section is just for those of you who would like to use the tools on your own system.
You can install miscellaneous tools from this repo, like morphTests2regtest and lexccounter, as follows:
cd ~/ling073 git clone git@github.swarthmore.edu:Ling073-sp26/tools.git cd tools make install
After you've already cloned the repo, you can update to the most recent version like this:
cd tools git pull make install
morphTests2regtest
You can scrape {{morphTest}}s from any page on this wiki for use with apertium-regtest like this:
morphTests2regtest "Language/Grammar" -l xyz
Run this from inside your transducer repository. It populates the test/ directory in your transducer. Be sure to commit the files.
morphTests2yaml
NOTICE: We're using morphTests2regtest (see above) now.
You can scrape {{morphTest}}s from any page on this wiki for use with morph-test or aq-morphtest like this:
morphTests2yaml "Language/Grammar" -l xyz
Put the file in a tests/ directory in your transducer repo and commit the file.
lexccounter
This script counts the number of unique entries in a lexc file.
$ lexccounter apertium-xyz.xyz.lexc Unique entries: 127
apertium-eval-translator
You need two files:
- one test translation, and
- one reference translation
The reference translation is the parallel text in your corpus, e.g. abc.tests.txt. To get a test translation, run the source text (xyz.tests.txt) through apertium and direct the output into a new file, e.g.
cat xyz.tests.txt | apertium -d . xyz-abc > xyz-abc.tests.txt
scrapeTransferTests
You can scrape the transferTests from your contrastive grammar page into a small parallel corpus. E.g.,
scrapeTransferTests -p abc-xyz "Language1_and_Language2/Contrastive_Grammar"
will result in an abc.tests.txt and xyz.tests.txt file that contain the respective sides of any transferTests on your contrastive grammar page specified as being for abc to xyz translation.
