Difference between revisions of "Ambiguity resolution"

From LING073
Jump to: navigation, search
Line 67: Line 67:
 
# (kir) кара → (eng) black; look at; watch
 
# (kir) кара → (eng) black; look at; watch
 
# (kir) жыл → (eng) year; move
 
# (kir) жыл → (eng) year; move
 +
 +
== Morphological disambiguation ==
 +
 +
The goal of a morphological disambiguator is to choose the correct analysis based on the surrounding words.
 +
 +
Constraint Grammar (CG) is a formalism that can be used to formalise context-sensitive constraints to select or remove analyses from the list of possible analyses.
 +
 +
=== The structure of a CG file ===
 +
 +
At the top you define delimiters.  You can then add lists (which are kind of like sets in twol).  After that you can have any number of constraints.  You can also put lists and constraints in sections. In the original specification for constraint grammar, all the constraints in each section are applied simultaneously, but the sections are applied in order and lists are always global. In the version we use (VISL CG3), the constraints are applied in strict order within the sections.
 +
 +
=== Constraints ===
 +
 +
Think of constraints as a series of heuristics to whittle down the number of analyses until you have as little ambiguity as possible.
 +
 +
The most commonly used constraints are to either '''remove''' a reading under a certain condition or '''select''' a reading under a certain condition.  The syntax is e.g., <code>REMOVE <target> [contextual tests] ;</code>.
 +
 +
Targets can be defined as list names—like <code>Nominals</code>—or individual tag names, in parentheses—e.g., <code>(n pl)</code>.  This target identifies the reading to select or remove.
 +
 +
Contextual tests come together in ()s.  All tests must match in order for the constraint to apply.  Inside the parentheses, the first definition is a position: <code>0</code> means the current cohort, <code>-1</code> means the previous cohort, etc.  The rest of the test is what should be matched.  If you want to match any member of the <code>Nominals</code> set in the previous position, it would be <code>(-1 Nominals)</code>  You can match tag sequences by adding an extra set of parentheses, e.g. <code>(1 (n pl))</code> would match a plural noun in the following position.  You can match specific baseforms with quotation marks, e.g. <code>(1 ("house" n))</code> would match any form of the noun "house" in the following position.
 +
 +
'''Classroom activity''': implement "houses this" example (at top of page).  [https://github.swarthmore.edu/Ling073-sp25/ling073-eng/blob/master/apertium-eng.eng.rlx Solution].
 +
 +
=== Useful commands ===
 +
 +
The file that your disambiguation constraints are defined in is the <code>apertium-xyz.xyz.rlx</code> file.
 +
 +
==== Getting output before disambiguation ====
 +
example command:
 +
echo "The motel houses this dog" | apertium -d . eng-morph
 +
example output:
 +
^The/the<det><def>$ ^motel/motel<n><sg>$ ^houses/house<n><pl>/house<v><tv><p3><sg>$ ^this/this<det><dem><sg>/this<prn><dem><sg>$ ^dog/dog<n><sg>$^./.<sent>$
 +
 +
==== Getting output after disambiguation ====
 +
example command:
 +
echo "The motel houses this dog" | apertium -d . eng-tagger
 +
example output:
 +
^The<det><def>$ ^motel<n><sg>$ ^house<v><tv><p3><sg>$ ^this<det><dem><sg>$ ^dog<n><sg>$^.<sent>$
 +
 +
==== Seeing which constraints are doing what ====
 +
example command:
 +
echo "The motel houses this dog" | apertium -d . eng-disam
 +
example output:
 +
<pre>
 +
"<The>"
 +
"the" det def
 +
"<motel>"
 +
"motel" n sg
 +
"<houses>"
 +
"house" v tv p3 sg
 +
; "house" n pl REMOVE:26
 +
"<this>"
 +
"this" det dem sg
 +
; "this" prn dem sg REMOVE:29
 +
"<dog>"
 +
"dog" n sg
 +
"<.>"
 +
"." sent
 +
</pre>
 +
 +
 +
==== Calculating ambiguity ====
 +
''Note: `lt-proc` and `.bin` can be replaced with `hfst-proc` and `.hfst` throughout if there are problems with tokenisation or similar''
 +
 +
The ambiguity of a corpus is the average number of analyses provided per token.
 +
You can get the number of tokens like this:
 +
cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | wc -l
 +
You can get the total number of analyses in your corpus like this:
 +
cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l
 +
You can get the total number of analyses in your corpus after disambiguation like this:
 +
cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | cg-proc /path/to/xyz.rlx.bin | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l
 +
 +
The commands <code>apertium-destxt</code> and <code>apertium-retxt</code> strip and readd (respectively) characters that could be parsed in problematic ways.
 +
 +
Automating it a little further, you can put something like the following in a file and run it as a shell script.  The first three lines need to be replaced with the path to your corpus, your compiled analyser, and your compiled disambiguator, respectively.
 +
<pre>
 +
CORPUS=/path/to/xyz.corpus.txt
 +
MORPH=/path/to/xyz.automorf.bin
 +
RLX=/path/to/xyz.rlx.bin # you might need to remove ".bin"
 +
TOKENS=`apertium-destxt ${CORPUS} | lt-proc ${MORPH} | apertium-retxt | sed 's/$\W*\^/$\n^/g' | wc -l`
 +
ANALYSES=`apertium-destxt ${CORPUS} | lt-proc ${MORPH} | apertium-retxt | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l`
 +
DISAMB=`apertium-destxt ${CORPUS} | lt-proc ${MORPH} | cg-proc ${RLX} | apertium-retxt | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l`
 +
AMBIGPRE=`calc $ANALYSES/$TOKENS`
 +
AMBIGPOST=`calc $DISAMB/$TOKENS`
 +
echo "Ambiguity before disambiguation: ${AMBIGPRE}"
 +
echo "Ambiguity after disambiguation: ${AMBIGPOST}"
 +
</pre>
 +
 +
It's recommended that you put this script in <code>tests/</code> and name it <code>disambiguation-test.sh</code>.  Don't forget to <code>git add</code> it, <code>git commit</code>, and <code>git push</code>.
 +
 +
==== Checking what forms are ambiguous in the corpus and transducer ====
 +
You can get all of the words in the corpus along with their analyses (or lack thereof) like this:
 +
cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g'
 +
You can get all of the words in the corpus that have more than one analysis like this:
 +
cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | grep '\/.*\/'
 +
 +
You can find all forms in your transducer like this:
 +
hfst-expand -c0 xyz.automorf.hfst
 +
 +
You can count ambiguity in your transducer by comparing the number of all forms to the number of unique forms.
 +
 +
Number of all forms:
 +
hfst-expand -c0 xyz.automorf.hfst | wc -l
 +
 +
Number of unique forms:
 +
hfst-expand -c0 xyz.automorf.hfst | cut -f1 -d':' | sort -u | wc -l
 +
 +
If the number's different you'll want to compare the lists:
 +
<pre>
 +
hfst-expand -c0 xyz.automorf.hfst | cut -f1 -d':' | sort > /tmp/totalforms
 +
hfst-expand -c0 xyz.automorf.hfst | cut -f1 -d':' | sort -u > /tmp/uniqforms
 +
diff /tmp/totalforms /tmp/uniqforms
 +
</pre>
 +
 +
Then you'll want to see what the multiple analyses are of any forms that show up there.
 +
 +
 +
== Lexical selection ==
 +
 +
== The assignment ==

Revision as of 03:08, 24 March 2025

There are two primary types of ambiguity that an Apertium pipeline can deal with:

  • Morphological ambiguity — when a single form has multiple analyses. The solution to this is #Morphological disambiguation.
  • Lexical ambiguity — when a single lexical item (lemma+POS+subcategory) has multiple translations. The solution to this is #Lexical selection.

Distinguishing between the two

Morphological ambiguity

Imagine you have a word that has two different tagsets, e.g. 

^houses/house<n><pl>/house<v><tv><p3><sg>$
^this/this<det><dem><sg>/this<prn><dem><sg>$

Normally your analyser will randomly output ("choose") one of these. Now imagine you have a sentence where the wrong analysis is "chosen" (in this case, twice!): 

^The/The<det><def>$ ^motel/motel<n><sg>$ ^houses/house<n><pl>$ ^this/this<prn><dem><sg>$ ^dog/dog<n><sg>$^./.<sent>$

In both of these examples, there is a different lemma+POS+subcategory. However, the following is also an example of morphological ambiguity: 

^deer/deer<n><sg>/deer<n><pl>$

As with the previous examples, both analyses are correct output from the transducer, but which one is correct in a given context will vary. (The assumption is that one and only one analysis will be correct for a given context.)

The goal of morphological disambiguation is to choose the correct analysis in cases like these based on the surrounding words.

Lexical ambiguity

An example of lexical ambiguity is the following:

  • yard (a unit of length)
  • yard (an enclosed area)

What's different about these words? Spelling, pronunciation, part of speech? What [different] contexts might each occur in?

Can we decide which translation is right from the surrounding words?

In both cases, these two words would receive the same analysis: ^yard/yard<n><sg>$. There would likely even be only one entry in a transducer that covers both nouns. But in a bilingual dictionary, depending on the target language, you could easily need two different entries to translate these correctly. Without further mechanisms, one of the options would be "chosen" randomly during translation.

To demonstrate this point, let's take an example from Spanish-to-English translation ("Estoy en la estación de tren" = "I'm at the train station"): 

^Estar<vblex><pri><p1><sg>/Be<vbser><pri><p1><sg>$ ^en<pr>/in<pr>/at<pr>/on<pr>$ ^el<det><def><f><sg>/the<det><def><f><sg>$ ^estación<n><f><sg>/season<n><sg>/station<n><sg>$ ^de<pr>/of<pr>/from<pr>$ ^tren<n><m><sg>/train<n><sg>$^.<sent>/.<sent>$

Given the output of the bilingual dictionary, this sentence could end up translated as "I'm in the train season". Given the surrounding context, however, it is possilbe to decide the correct translation alternatives.

There are two main scenarios when lexical ambiguity may occur:

  • There are two words in Language A that happen to have the same lemma and POS tags
    • (kir) жаш → (eng) age; tear (eye water)
    • (kir) ат → (eng) horse; name
    • (eng) right → (kir) оң (side); туура (correct)
    • (eng) lie → (kir) жат (lie down); калп айт (state a falsehood)
    • (eng) bat → (kir) жарганат (animal); таякча (stick for hitting a ball)
  • There is a semantic distinction in Language B that is not expressed in Language A
    • (kir) кол → (eng) hand; arm
    • (kir) күн → (eng) sun; day
    • (kir) бир → (eng) one; a/an
    • (kir) кара → (eng) look at; watch
    • (eng) paper → (kir) кагаз (material); доклад (written work)
    • (eng) horse → (kir) ат (for riding, working); жылкы (for herding, eating)
    • (eng) sister → (kir) сиңди (younger, of female); карындаш (younger, of male); эже (older)
    • (eng) move → (kir) жыл/жылдыр (location iv/tv); кыймылда/кыймылдат (movement iv/tv); көч/көчүр (residence, files iv/tv)
    • (eng) fall → (kir) түш (drop from a height / fall down); кула (to the side / fall over); жыгыл (due to loss of footing)

Choosing between the two

Based on the above, which problem does each of the following example represent?

  1. (eng) tear → (kir) жаш (eye water); айыр (rip)
  2. (eng) plant → (kir) өсүмдүк (growing thing); отургуз (put in the ground to grow)
  3. (eng) fall → (kir) күз (season, <n>); күзгү (relating to the season, <adj>); [the verbs above]
  4. (kir) күзгү → (eng) fall (<adj>), mirror
  5. (kir) өч → (eng) revenge; be extinguished
  6. (kir) кыл → (eng) strand of hair; string (of instrument); do; make
  7. (kir) кара → (eng) black; look at; watch
  8. (kir) жыл → (eng) year; move

Morphological disambiguation

The goal of a morphological disambiguator is to choose the correct analysis based on the surrounding words.

Constraint Grammar (CG) is a formalism that can be used to formalise context-sensitive constraints to select or remove analyses from the list of possible analyses.

The structure of a CG file

At the top you define delimiters. You can then add lists (which are kind of like sets in twol). After that you can have any number of constraints. You can also put lists and constraints in sections. In the original specification for constraint grammar, all the constraints in each section are applied simultaneously, but the sections are applied in order and lists are always global. In the version we use (VISL CG3), the constraints are applied in strict order within the sections.

Constraints

Think of constraints as a series of heuristics to whittle down the number of analyses until you have as little ambiguity as possible.

The most commonly used constraints are to either remove a reading under a certain condition or select a reading under a certain condition. The syntax is e.g., REMOVE <target> [contextual tests] ;.

Targets can be defined as list names—like Nominals—or individual tag names, in parentheses—e.g., (n pl). This target identifies the reading to select or remove.

Contextual tests come together in ()s. All tests must match in order for the constraint to apply. Inside the parentheses, the first definition is a position: 0 means the current cohort, -1 means the previous cohort, etc. The rest of the test is what should be matched. If you want to match any member of the Nominals set in the previous position, it would be (-1 Nominals) You can match tag sequences by adding an extra set of parentheses, e.g. (1 (n pl)) would match a plural noun in the following position. You can match specific baseforms with quotation marks, e.g. (1 ("house" n)) would match any form of the noun "house" in the following position.

Classroom activity: implement "houses this" example (at top of page). Solution.

Useful commands

The file that your disambiguation constraints are defined in is the apertium-xyz.xyz.rlx file.

Getting output before disambiguation

example command: 

echo "The motel houses this dog" | apertium -d . eng-morph

example output: 

^The/the<det><def>$ ^motel/motel<n><sg>$ ^houses/house<n><pl>/house<v><tv><p3><sg>$ ^this/this<det><dem><sg>/this<prn><dem><sg>$ ^dog/dog<n><sg>$^./.<sent>$

Getting output after disambiguation

example command: 

echo "The motel houses this dog" | apertium -d . eng-tagger

example output: 

^The<det><def>$ ^motel<n><sg>$ ^house<v><tv><p3><sg>$ ^this<det><dem><sg>$ ^dog<n><sg>$^.<sent>$

Seeing which constraints are doing what

example command: 

echo "The motel houses this dog" | apertium -d . eng-disam

example output: 

"<The>"
	"the" det def
"<motel>"
	"motel" n sg
"<houses>"
	"house" v tv p3 sg
;	"house" n pl REMOVE:26
"<this>"
	"this" det dem sg
;	"this" prn dem sg REMOVE:29
"<dog>"
	"dog" n sg
"<.>"
	"." sent


Calculating ambiguity

Note: `lt-proc` and `.bin` can be replaced with `hfst-proc` and `.hfst` throughout if there are problems with tokenisation or similar

The ambiguity of a corpus is the average number of analyses provided per token. You can get the number of tokens like this: 

cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | wc -l

You can get the total number of analyses in your corpus like this: 

cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l

You can get the total number of analyses in your corpus after disambiguation like this: 

cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | cg-proc /path/to/xyz.rlx.bin | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l

The commands apertium-destxt and apertium-retxt strip and readd (respectively) characters that could be parsed in problematic ways.

Automating it a little further, you can put something like the following in a file and run it as a shell script. The first three lines need to be replaced with the path to your corpus, your compiled analyser, and your compiled disambiguator, respectively. 

CORPUS=/path/to/xyz.corpus.txt
MORPH=/path/to/xyz.automorf.bin
RLX=/path/to/xyz.rlx.bin # you might need to remove ".bin"
TOKENS=`apertium-destxt ${CORPUS} | lt-proc ${MORPH} | apertium-retxt | sed 's/$\W*\^/$\n^/g' | wc -l`
ANALYSES=`apertium-destxt ${CORPUS} | lt-proc ${MORPH} | apertium-retxt | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l`
DISAMB=`apertium-destxt ${CORPUS} | lt-proc ${MORPH} | cg-proc ${RLX} | apertium-retxt | sed 's/$\W*\^/$\n^/g' | cut -f2- -d'/' | sed 's/\//\n/g' | wc -l`
AMBIGPRE=`calc $ANALYSES/$TOKENS`
AMBIGPOST=`calc $DISAMB/$TOKENS`
echo "Ambiguity before disambiguation: ${AMBIGPRE}"
echo "Ambiguity after disambiguation: ${AMBIGPOST}"

It's recommended that you put this script in tests/ and name it disambiguation-test.sh. Don't forget to git add it, git commit, and git push.

Checking what forms are ambiguous in the corpus and transducer

You can get all of the words in the corpus along with their analyses (or lack thereof) like this: 

cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g'

You can get all of the words in the corpus that have more than one analysis like this: 

cat corpus.txt | apertium-destxt | lt-proc /path/to/xyz.automorf.bin | apertium-retxt | sed 's/$\W*\^/$\n^/g' | grep '\/.*\/'

You can find all forms in your transducer like this: 

hfst-expand -c0 xyz.automorf.hfst

You can count ambiguity in your transducer by comparing the number of all forms to the number of unique forms.

Number of all forms: 

hfst-expand -c0 xyz.automorf.hfst | wc -l

Number of unique forms: 

hfst-expand -c0 xyz.automorf.hfst | cut -f1 -d':' | sort -u | wc -l

If the number's different you'll want to compare the lists: 

hfst-expand -c0 xyz.automorf.hfst | cut -f1 -d':' | sort > /tmp/totalforms
hfst-expand -c0 xyz.automorf.hfst | cut -f1 -d':' | sort -u > /tmp/uniqforms
diff /tmp/totalforms /tmp/uniqforms

Then you'll want to see what the multiple analyses are of any forms that show up there.


Lexical selection

The assignment

Retrieved from "http://wikis.swarthmore.edu/ling073/index.php?title=Ambiguity_resolution&oldid=25130"