Difference between revisions of "Dealing with multi-word expressions"

Revision as of 04:16, 29 April 2026

There are a number of considerations and strategies when dealing with multi-word expressions (MWEs).

Monolingual dictionary

Imagine you wanted to have analyses for single tokens that include spaces, such as ^fur coats/fur coat<n><pl>$ or even just ^a lot/a lot<adv>. In lexd, you can simply add these to the appropriate lexicon, being sure to escape the space (i.e., \ ):

LEXICON NounStems

...
coat:coat             # "пальто"
fur:fur               # "жүн"
fur\ coat:fur\ coat   # "тон"
...

Any stems like spaces will act like any other stem, and get any morphology specified by the patterns that reference the NounStems lexicon.

Now imagine that you have examples like ^took away/take away<v><past>$, ^shows up/show up<v><pres><p3><sg>$. These are harder to implement in lexd, because if you used the above approach, you'd get verbal morphology at the end (on the second word) instead of on the first word.

You could technically make individual lexicons for each verb (take, show, throw, go, be, etc.) and each set of things each one patterns with (for take, a simplified list might include: after, apart, away, back, down, in, it, off, on, out, over, to, up). However, that would get really unwieldy really fast.

A better approach for this kind of MWE is to use apertium-separable.

Bilingual dictionary

Use <b/> for spaces in entries.

 <e><p><l>break<b/>a<b/>leg<s n="vblex"/></l>  <r>tener<b/>suerte<s n="vblex"/></r></p></e>

However, this doesn't fix the problem that the form on the right side needs to be implemented in the transducer in a way where the first elements is conjugated.

In combination with monodix approach

If you use the monodix approach discussed above, you can use <g/> for the space after tener instead of <b/>.

Apertium-separable

Apertium-separable allows you to treat MWEs as single entries in a bilingual dictionary, but as individual components at the monolingual level.

Some use cases follow.

MWEs as translations of a single concept to complex concepts

In the example of "fur coat", this is a single concept and word in some languages (Kyrgyz) but a complex concept using multiple words in others (English). Other common examples include "younger sister", "older sister", etc., or even just expressions like "a lot" in English which happen to take up two apparent words but could be thought of as one.

For these cases it is often easiest to include a multi-word entry in your monolingual dictionary that's just used for translation. However, it's also possible to use apertium-separable to separate these expressions out into two words when translating into the language where they're two words (e.g., "younger sister" in English), or to combine them into a single expression when translating from the language where they're two words. In this case, you would want the following:

1. separate words in the monolingual dictionary ("young" and "sister")
2. a single translation in the bilingual dictionary

   <e><p><l>younger<b/>sister<s n="n"/></l> <r>сиңди<s n="n"/></r></p></e>

3. an entry in ….xyz-abc.lsx, where xyz is the language with the MWE:

      <e lm="younger sister" c="сиңди">
         <p>
            <l>young<s n="adj"/><s n="sint"/><s n="cmp"/><d/>sister<s n="n"/></l>
            <r>younger<b/>sister<s n="n"/></r>
         </p>
         <i><t/><d/></i>
      </e>

This will result in a translation chain like the following if translating from English to Kyrgyz:

And the following translation chain if translating from Kyrgyz to English:

MWEs where inflection applies differently than if it were a single element

In the case of "take away", "show up", "get over", etc., the main challenge is that the first element is inflected, and the second element isn't a simple set list that can combine with all (or even some) verbs.

...

MWEs where the parts can be separated

An additional challenge with English verbs like "take away" "throw out", "send up", etc. is that the final element doesn't always immediately follow the first. Often a noun phrase (which might also be a pronoun) intervenes between the elements. E.g., "takes the toy away", "threw it out", "sending several big weather balloons up".

Fortunately, apertium-separable was designed exactly for this sort of situation.

You will need (with example of "throw out" = "ташта"):

1. an entry in ….xyz-abc.lsx, where xyz is the language with the MWE:

                   <e lm="throw out" c="ташта">
                           <p>
                                   <l>throw<s n="vblex"/></l>
                                   <r>throw<b/>out<s n="vblex"/></r>
                           </p>
                           <i><t/><d/></i>
                           <par n="SN"/>
                           <p>
                                   <l>out<s n="adv"/><d/></l>
                                   <r></r>
                           </p>
                   </e>
   

   the "SN" paradigm stands in for "noun phrase" (SN = "sintagma nominal")

2. Any paradigms will have to be defined in the <pardefs>...</pardefs> section of the file:

                   <pardef n="SN">
                           <e><par n="prn"/></e>
                           <e><par n="n"/></e>
                           <e><par n="adj"/><par n="n"/></e>
                           <e><par n="adj"/><par n="adj"/><par n="n"/></e>
                           <e><par n="det"/><par n="n"/></e>
                           <e><par n="det"/><par n="adj"/><par n="n"/></e>
                           <e><par n="det"/><par n="adj"/><par n="adj"/><par n="n"/></e>
                   </pardef>

   In this case, the SN is defined as a pronoun, a noun, an adjective + a noun, two adjectives plus a noun, a determiner + a noun, a determiner + an adjective + a noun, or a determiner + an adjective + an adjective + a noun

3. Any paradigms used in these paradigms will also need to be defined, e.g.:

                   <pardef n="adj">
                           <e><i><w/><s n="adj"/><d/></i></e>
                           <e><i><w/><s n="adj"/><t/><d/></i></e>
                   </pardef>

   In this case, the adj paradigm is defined as a lemma (<w/>) followed by an adjective tag followed by a word boundary (<d/>, or a lemma (<w/>) followed by an adjective tag followed by another tag (<t/>, such as comparative) followed by a word boundary (<d/>.

4. The tags will also have to be defined in the <sdefs>...</sdefs> section:

                   <sdef n="adj"/>

5. A single translation in your bilingual dictionary:

   <e><p><l>throw<b/>out<s n="vblex"/></l> <r>ташта<s n="v"/><s n="tv"/></r></p></e>

6. Your monolingual dictionary (for the MWE language) will have the various components as separate words
   • In this case, "throw" will be a verb (throw<vblex>) and "out" will be an adverb (out<adv>).

This will result in a translation chain like the following if translating from English to Kyrgyz:

And the following translation chain if translating from Kyrgyz to English: