Difference between revisions of "User:Tjones5/Final project"

From LING073
Jump to: navigation, search
(→‎Presentation)
(→‎Approaches)
Line 47: Line 47:
  
 
*Mostly have worked within lexc and twol files
 
*Mostly have worked within lexc and twol files
−
**Examples??
+
 
 +
*Pronouns (optional dash not included in table):
 +
{|class="wikitable sortable"
 +
! Meaning !! Word !! Form
 +
|-
 +
! I, me
 +
|  ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}}<br />{{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
 +
|-
 +
! you
 +
|  nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} <br />{{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
 +
|-
 +
! he/she/it; to him/her/it
 +
|  nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
 +
|-
 +
! we (you & me)
 +
|  ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} <br /> {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
 +
|-
 +
! we (him/her/it & me)
 +
|  ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
 +
|-
 +
! we (you & me & other(s))
 +
|  ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
 +
|-
 +
! we (them & me)
 +
|  nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
 +
|-
 +
! you (both/two)
 +
|  nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
 +
|-
 +
! you (more than 2)
 +
|  nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
 +
|-
 +
! they/them (both/two)
 +
|  nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
 +
|-
 +
! they/them (more than 2)
 +
|  nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} <br /> {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
 +
|-
 +
|}
 +
 
 +
**Noun_Number 2v
 +
{|class="wikitable sortable"
 +
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
 +
|-
 +
! absolutive
 +
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
 +
|-
 +
! dative
 +
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
 +
|-
 +
! ergative
 +
| "agent of a transitive verb" || {{tag|erg}} || -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
 +
|-
 +
! allative
 +
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
 +
|-
 +
! comitative
 +
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
 +
|-
 +
! elative
 +
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
 +
|-
 +
! locative
 +
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
 +
|}
 +
 
 +
 
 +
***wati-ngku
 +
***wati-jarra-rlu
 +
**adding (optional) dashes between everything; selecting only one analysis by using ! Dir/LR
 
*Disambiguation is a challenge because of flexible word order
 
*Disambiguation is a challenge because of flexible word order
 
**Numerals are one thing to deal with
 
**Numerals are one thing to deal with
−
*Need to build a scraper to test on a larger corpus
+
*Need to write scraper to test on a larger corpus
  
 
===Evaluation===
 
===Evaluation===

Revision as of 05:20, 5 May 2017

Placeholder page for final project

Notes

  • To implement:
    • Reduplication
  • Get a big corpus:
    • Build a scraper? to get corpus up to 25K
    • Use regular expressions to clean up scraped (or copied/pasted) text
  • For evaluation:
    • Go back and look at coverage
  • Other ling to do: UD pipeline

Presentation

Description of the problem

  • Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
  • A morphological transducer connects forms with analyses.
  • Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. foxes
  • Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-fox + PLURAL-es
  • Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "foxes"

1000px

Previous work

  • Many languages have working Apertium transducers:
  • Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer
  • Other work?

Benefits of this project

  • Why is a transducer useful to a community of Warlpiri speakers?
    • Search engine
    • Spell checker
    • Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
      • eng → wbp MT system could be used to look up how to say/write a certain word or phrase
      • wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri phrases, or give alternate translations
    • All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

Approaches

  • Mostly have worked within lexc and twol files
  • Pronouns (optional dash not included in table):
Meaning Word Form
I, me ngaju(lu) ngaju<prn><pers><p1><sg> ↔ ngaju
ngaju<prn><pers><p1><sg> ↔ ngajulu
you nyuntu(lu) nyuntu<prn><pers><p2><sg> ↔ nyuntu
nyuntu<prn><pers><p2><sg> ↔ nyuntulu
he/she/it; to him/her/it nyanungu nyanungu<prn><pers><p3><sg> ↔ nyanungu
we (you & me) ngali(jarra) ngali<prn><pers><p1><du><incl> ↔ ngali
ngali<prn><pers><p1><du><incl> ↔ ngalijarra
we (him/her/it & me) ngajarra ngajarra<prn><pers><p1><du><excl> ↔ ngajarra
we (you & me & other(s)) ngalipa ngalipa<prn><pers><p1><pl><incl> ↔ ngalipa
we (them & me) nganimpa nganimpa<prn><pers><p1><pl><excl> ↔ nganimpa
you (both/two) nyumpala nyumpala<prn><pers><p2><du> ↔ nyumpala
you (more than 2) nyurrarla nyurrarla<prn><pers><p2><pl> ↔ nyurrarla
they/them (both/two) nyanungu-jarra nyanungu-jarra<prn><pers><p3><du> ↔ nyanungu-jarra
they/them (more than 2) nyanungu-rra/nyanungu-patu nyanungu<prn><pers><p3><pl> ↔ nyanungu-rra
nyanungu<prn><pers><p3><pl> ↔ nyanungu-patu
    • Noun_Number 2v
case name ~meaning tag possible forms pirli "rock" (2 vowels) warlkurru (>2 vowels) "axe"
absolutive subject of intransitive verbs and object of transitive verbs <abs> — pirli<n><sg><abs> ↔ pirli warlkurru<n><sg><abs> ↔ warlkurru
dative "to" <dat> -ku pirli<n><sg><dat> ↔ pirli-ku warlkurru<n><sg><dat> ↔ warlkurru-ku
ergative "agent of a transitive verb" <erg> -ngku, -rlu pirli<n><sg><erg> ↔ pirli-ngku warlkurru<n><sg><erg> ↔ warlkurru-rlu
allative "onto" <all> -kurra pirli<n><sg><all> ↔ pirli-kurra warlkurru<n><sg><all> ↔ warlkurru-kurra
comitative "with" <com> -ngkajinta, -rlajinta pirli<n><sg><com> ↔ pirli-ngkajinta warlkurru<n><sg><com> ↔ warlkurru-rlajinta
elative "out of" <ela> -ngurlu pirli<n><sg><ela> ↔ pirli-ngurlu warlkurru<n><sg><ela> ↔ warlkurru-ngurlu
locative "at, in on" <loc> -ngka, -rla pirli<n><sg><loc> ↔ pirli-ngka warlkurru<n><sg><loc> ↔ warlkurru-rla


      • wati-ngku
      • wati-jarra-rlu
    • adding (optional) dashes between everything; selecting only one analysis by using ! Dir/LR
  • Disambiguation is a challenge because of flexible word order
    • Numerals are one thing to deal with
  • Need to write scraper to test on a larger corpus

Evaluation

  • Coverage over a large corpus
  • Precision and recall against a hand-annotated randomly selected forms
  • Coverage over large corpus
    • Number of tokenised words in the corpus: 18427
    • Coverage: 51.26%
  • Precision/Recall on wbp.annotated.basic.txt from Polished RMBT:
    • Precision: 97.54902%
    • Recall: 97.07317%
  • Number of stems: ~250


status description stems coverage
prototype language module that has not received heavy development <1,000 <60%
development language module under development ≥1,000 ≥60%
working language module with near-production-quality performance ≥8,000 ≥80%
production language module used in a released pair ≥10,000 ≥90%
Retrieved from "http://wikis.swarthmore.edu/ling073/index.php?title=User:Tjones5/Final_project&oldid=5060"