Difference between revisions of "Apertium-regtest"

Revision as of 14:48, 21 February 2023

The tool apertium-regtest is a "regression test" tool that helps to test large numbers of inputs simultaneously and to keep track of what has changed and what is still wrong.

General premise and structure

Apertium-regtest runs as a web client, a command-line interface, or simply checks tests on the command line.

When run from a transducer or translation directory, it checks the test/ directory for tests.

Each test consists of a list of inputs and a list of "gold" (i.e., correct) forms provided by the user, as well as a list of output forms generated by the software upon request and a list of expected forms, generated as the user interacts with the software

Commands

• To run apertium-regtest to access in a web browser:

  $ apertium-regtest web

• To set up an ssh tunnel to a lab machine, if remote

  $ ssh -NL 3000:localhost:3000 machinename.cs.swarthmore.edu (with the name of the machine you're running apertium-regtest on in place of machinename)

  • Then you should be able to access http://localhost:3000 from the computer this command is run on.
• To check the number of morph tests matching gold:

  $ apertium-regtest test -c .-morph

Using the web interface

1. First you'll want to click the button to run all tests.
2. Then you want to accept all tests as expected (even if incorrect / not matching gold). You'll probably have to click on the button for each test.
3. Then you can filter for e.g. tests not matching gold.

When you update something in your transducer or translator and recompile, you'll want to:

1. Rerun all tests.
2. The software will show you what's changed. You'll want to accept the updated forms as expected (again, even if incorrect / not matching gold).

Note:

• There are a few known issues with the web interface.
• Apertium-regtest is fairly new and the lore for how to use it hasn't been well established. So you get to help with that!