User:Aconant1/Final project

Here I detail the beginnings of a user-friendly pipeline for learned lexical selection rules. While the architecture already exists, it is unwieldy and non-trivial to set up. Although my project does not solve every problem with the process, it makes it more approachable, and I hope to expand on it in the future. The code can be found here, on GitHub.

Designing the configuration

The desire was for a single input file to govern the runtime of the program. This meant designing a (very) simply yaml format in which users could input their configuration information. The example file that I included with my submission can be seen below:

This can be contrasted with the hard-coded parameters in the current Makefile architecture by noting the absence of the SCRIPTS parameter. Because this parameter varied computer-to-computer rather than pair-to-pair, it made sense to separate the two.

Main Features

My program can be divided into logical, discrete units:

Dependency validation

First off, one of the most difficult-to-manage parts of the Makefile architecture for lexical selection rule learning is its dependency on dependencies, so to speak. Therefore, the first order of business is to ensure that the executables that the pipeline eventually relies on being there are actually there. This means checking for apertium, apertium-destxt, apertium-pretransfer, multitrans, lrx-comp, among others, as well as the various scripts normally found in apertium-lex-tools that would be tragic to miss.

Dealing with yaml

Next, the program reads the user-supplied yaml file, checking both its integrity and the sanity of its contents. This means that the user must pass a valid source language corpus name, translation pair direction, path to a language pair, name of a bilingual dictionary binary, and the path to a target language binary language model (compiled with IRSTLM).

Evaluation

Because this project is more qualitative than quantitative, evaluation is a difficult thing to talk about. One way to do so might be to compare monolearner with what we had before and with what we might still desire. Although this project does not make it *easier* to

Previous work

I want to quickly mention some resources that I relied on for this project:

• A non-native Python library known as PyYAML trivializes parsing yaml files.
• English and Spanish corpora were borrowed from here.
• The bidirectional English/Spanish dictionary was borrowed from Apertium's trunk.