Difference between revisions of "Dealing with multi-word expressions"

Revision as of 03:30, 29 April 2026

There are a number of considerations and strategies when dealing with multi-word expressions (MWEs).

Monolingual dictionary

Imagine you wanted to have analyses for single tokens that include spaces, such as ^fur coats/fur coat<n><pl>$ or even just ^a lot/a lot<adv>. In lexd, you can simply add these to the appropriate lexicon, being sure to escape the space (i.e., \ ):

LEXICON NounStems

...
coat:coat             # "пальто"
fur:fur               # "жүн"
fur\ coat:fur\ coat   # "тон"
...

Any stems like spaces will act like any other stem, and get any morphology specified by the patterns that reference the NounStems lexicon.

Now imagine that you have examples like ^took away/take away<v><past>$, ^shows up/show up<v><pres><p3><sg>$. These are harder to implement in lexd, because if you used the above approach, you'd get verbal morphology at the end (on the second word) instead of on the first word.

You could technically make individual lexicons for each verb (take, show, throw, go, be, etc.) and each set of things each one patterns with (for take, a simplified list might include: after, apart, away, back, down, in, it, off, on, out, over, to, up). However, that would get really unwieldy really fast.

A better approach for this kind of MWE is to use apertium-separable.

Bilingual dictionary

 <e><p><l>break<b/>a<b/>leg<s n="vblex"/></l>  <r>tener<b/>suerte<s n="vblex"/></r></p></e>

Apertium-separable

Apertium-separable allows you to treat MWEs as single entries in a bilingual dictionary, but as individual components at the monolingual level.

Some use cases follow.

MWEs as translations of a single concept to complex concepts

In the example of "fur coat", this is a single concept and word in some languages (Kyrgyz) but a complex concept using multiple words in others (English). Other common examples include "younger sister", "older sister", etc., or even just expressions like "a lot" in English which happen to take up two apparent words but could be thought of as one.

For these cases it is often easiest to include a multi-word entry in your monolingual dictionary that's just used for translation. However, it's also possible to use apertium-separable to separate these expressions out into two words when translating into the language where they're two words (e.g., "younger sister" in English), or to combine them into a single expression when translating from the language where they're two words. In this case, you would want the following:

1. separate words in the monolingual dictionary ("young" and "sister")
2. a single translation in the bilingual dictionary

   <e><p><l>younger<b/>sister<s n&equals;"n"/></l> <r>сиңди<s n&equals;"n"/></r></p></e>

3. an entry in ….xyz-abc.lsx, where xyz is the language with the MWE:

   <e lm="younger sister" c="сиңди">
      <p>
         <l>young<s n="adj"/><s n="sint"/><s n="cmp"/><d/>sister<s n="n"/></l>
         <r>younger<b/>sister<s n="n"/></r>
      </p>
      <i><t/><d/></i>
   </e>

MWEs where inflection applies differently than if it were a single element

In the case of "take away", "show up", "get over", etc., the main challenge is that the first element is inflected, and the second element isn't a simple set list that can combine with all (or even some) verbs.

...

MWEs where the parts can be separated

An additional challenge with English verbs like "take away" "throw out", "send up", etc. is that the final element doesn't always immediately follow the first. Often a noun phrase (which might also be a pronoun) intervenes between the elements. E.g., "takes the toy away", "threw it out", "sending several big weather balloons up".

...