LING073 - User contributions [en]

Warlpiri/Universal Dependencies

2017-05-12T07:58:26Z

Tjones5: /* obj */

=Evaluation=
==Corpora==
{| class="wikitable"
|+Information
|-
|
|Number of sentences
|Number of forms
|-
|<code>wbp.annotated.ud.conllu</code>
| 34
| 353
|-
|<code>wbp.annotated2.ud.conllu</code>
| 8
| 74
|}

==Withmorph==
Results from using <code>wbp.withmorph.udpipe</code> on the two conllu files:

{| class="wikitable"
|+Results
|-
|
|UAS
|LAS
|-
|<code>wbp.annotated.ud.conllu</code>
| 93.75%
| 88.12%
|-
|<code>wbp.annotated2.ud.conllu</code>
| 48.72%
| 28.21%
|}

==Nomorph==
Results from using <code>wbp.nomorph.udpipe</code> on the two conllu files:

{| class="wikitable"
|+Results
|-
|
|UAS
|LAS
|-
|<code>wbp.annotated.ud.conllu</code>
| 71.25%
| 65.00%
|-
|<code>wbp.annotated2.ud.conllu</code>
| 35.90%
| 23.08%
|}

=Dependency Relations=
==aux==
*An auxiliary is a defined as a "function word associated with a verbal predicate that expresses categories such as tense, mood, aspect, voice or evidentiality" ([http://universaldependencies.org/u/dep/aux_.html]). In Warlpiri, auxiliaries are used in every sentence, and they carry suffixes that signify tense and the number/person of subjects and objects.
*Yani ''karna'' ngajulu ("I'm going") is glossed as:
go + ''aux'' (present + I) + I
*Yani ''kanpa'' nyuntulu ("You're going") is glossed as:
go + ''aux'' (present + you) + you
==nsubj==
*A nominal subject is a "nominal which is the syntactic subject and the proto-agent of a clause" ([http://universaldependencies.org/u/dep/nsubj_.html]). In Warlpiri, the subject is often but not always included as its own word -- for example, pronouns are often omitted.
*Yani karna ''ngajulu'' ("I'm going"):
go + (present + I) + ''nsubj'' (I)
*Yani ka ("He/she/it's going"):
go + present
==obj==
*An object of a verb is usually "the noun phrase that denotes the entity acted upon or which undergoes a change of state or motion (the proto-patient)" ([http://universaldependencies.org/u/dep/obj_.html]). In Warlpiri, this noun is typically in the absolutive case when a transitive verb is used.
*Nyanyi kanpa ''wawirri'' ("You can see a kangaroo"):
see + (present + you) + ''obj'' (kangaroo)
*Nyanyi kangku wawirrirli ''nyuntulu'' ("The kangaroo can see you"):
see + (present + you) + kangaroo (ergative) + ''obj'' (you)

==iobj==
*An indirect object of a verb is "any nominal phrase that is a core argument of the verb but is not its subject or (direct) object" ([http://universaldependencies.org/u/dep/iobj_.html]). In Warlpiri, the dative case is used.
*Nangala-rlu rla yungu pipa ''Jangala-ku''. ("Nangala gave the book to Jangala").
Nangala + (dative subj3sg) + give + book + ''iobj'' (Jangala).
*Karnta-patu-rlu-lu-jana ''kurdu-kurdu'' miyi yinyi. ("The women gave the food to the children")
women + (past + subj3pl obj3pl) + ''iobj'' (children) + food + give

==obl==
*"The obl relation is used for a nominal (noun, pronoun, noun phrase) functioning as a non-core (oblique) argument or adjunct" ([http://universaldependencies.org/u/dep/obl_.html]). In Warlpiri, the ergative case is used when an instrument is being used to perform a task. The allative case is used when an action is being performed into another object.
*Jakamarra-rlu ka Jupurrula luwarni ''karli-kirlirli'' ("Jakamarra is hitting Jupurrula with a boomerang").
Jakamarra + pres + Jupurrula + hit + ''obl'' (boomerang + ergative)
*Wakati ka-rnalu lurlurl-pi-nyi ''parraja-kurra'' ngurlu. ("We shake the seeds of the pigweed into a coolamon.")
pigweed + present (subj1plexcl) + shake + ''obl'' (coolamon + allative) + seeds

[[Category:sp17_UD]]

Warlpiri/Universal Dependencies

2017-05-12T07:57:20Z

Tjones5: /* Dependency Relations */

=Evaluation=
==Corpora==
{| class="wikitable"
|+Information
|-
|
|Number of sentences
|Number of forms
|-
|<code>wbp.annotated.ud.conllu</code>
| 34
| 353
|-
|<code>wbp.annotated2.ud.conllu</code>
| 8
| 74
|}

==Withmorph==
Results from using <code>wbp.withmorph.udpipe</code> on the two conllu files:

{| class="wikitable"
|+Results
|-
|
|UAS
|LAS
|-
|<code>wbp.annotated.ud.conllu</code>
| 93.75%
| 88.12%
|-
|<code>wbp.annotated2.ud.conllu</code>
| 48.72%
| 28.21%
|}

==Nomorph==
Results from using <code>wbp.nomorph.udpipe</code> on the two conllu files:

{| class="wikitable"
|+Results
|-
|
|UAS
|LAS
|-
|<code>wbp.annotated.ud.conllu</code>
| 71.25%
| 65.00%
|-
|<code>wbp.annotated2.ud.conllu</code>
| 35.90%
| 23.08%
|}

=Dependency Relations=
==aux==
*An auxiliary is a defined as a "function word associated with a verbal predicate that expresses categories such as tense, mood, aspect, voice or evidentiality" ([http://universaldependencies.org/u/dep/aux_.html]). In Warlpiri, auxiliaries are used in every sentence, and they carry suffixes that signify tense and the number/person of subjects and objects.
*Yani ''karna'' ngajulu ("I'm going") is glossed as:
go + ''aux'' (present + I) + I
*Yani ''kanpa'' nyuntulu ("You're going") is glossed as:
go + ''aux'' (present + you) + you
==nsubj==
*A nominal subject is a "nominal which is the syntactic subject and the proto-agent of a clause" ([http://universaldependencies.org/u/dep/nsubj_.html]). In Warlpiri, the subject is often but not always included as its own word -- for example, pronouns are often omitted.
*Yani karna ''ngajulu'' ("I'm going"):
go + (present + I) + ''nsubj'' (I)
*Yani ka ("He/she/it's going"):
go + present
==obj==
*An object of a verb is usually "the noun phrase that denotes the entity acted upon or which undergoes a change of state or motion (the proto-patient)" ([http://universaldependencies.org/u/dep/obj_.html]). In Warlpiri, this noun is typically in the absolutive case when a transitive verb is used.
*Nyanyi kanpa ''wawirri'' ("You can see a kangaroo):
see + (present + you) + ''obj'' (kangaroo)
*Nyanyi kangku wawirrirli ''nyuntulu'' ("The kangaroo can see you"):
see + (present + you) + kangaroo (ergative) + ''obj'' (you)
==iobj==
*An indirect object of a verb is "any nominal phrase that is a core argument of the verb but is not its subject or (direct) object" ([http://universaldependencies.org/u/dep/iobj_.html]). In Warlpiri, the dative case is used.
*Nangala-rlu rla yungu pipa ''Jangala-ku''. ("Nangala gave the book to Jangala").
Nangala + (dative subj3sg) + give + book + ''iobj'' (Jangala).
*Karnta-patu-rlu-lu-jana ''kurdu-kurdu'' miyi yinyi. ("The women gave the food to the children")
women + (past + subj3pl obj3pl) + ''iobj'' (children) + food + give

==obl==
*"The obl relation is used for a nominal (noun, pronoun, noun phrase) functioning as a non-core (oblique) argument or adjunct" ([http://universaldependencies.org/u/dep/obl_.html]). In Warlpiri, the ergative case is used when an instrument is being used to perform a task. The allative case is used when an action is being performed into another object.
*Jakamarra-rlu ka Jupurrula luwarni ''karli-kirlirli'' ("Jakamarra is hitting Jupurrula with a boomerang").
Jakamarra + pres + Jupurrula + hit + ''obl'' (boomerang + ergative)
*Wakati ka-rnalu lurlurl-pi-nyi ''parraja-kurra'' ngurlu. ("We shake the seeds of the pigweed into a coolamon.")
pigweed + present (subj1plexcl) + shake + ''obl'' (coolamon + allative) + seeds

[[Category:sp17_UD]]

Warlpiri/Universal Dependencies

2017-05-12T07:41:44Z

Tjones5: /* Dependency Relations */

Warlpiri/Universal Dependencies

2017-05-12T07:11:49Z

Tjones5:

Warlpiri/Universal Dependencies

2017-05-12T01:16:55Z

Tjones5:

=Evaluation=
==Corpora==
{| class="wikitable"
|+Information
|-
|
|Number of sentences
|Number of forms
|-
|<code>wbp.annotated.ud.conllu</code>
| 33
| 158
|-
|<code>wbp.annotated2.ud.conllu</code>
| 23
| 103
|}

==Withmorph==
Results from using <code>wbp.withmorph.udpipe</code> on the two conllu files:

{| class="wikitable"
|+Results
|-
|
|UAS
|LAS
|-
|<code>wbp.annotated.ud.conllu</code>
| 91.14%
| 78.48%
|-
|<code>wbp.annotated2.ud.conllu</code>
| 60.19%
| 47.57%
|}

==Nomorph==
Results from using <code>wbp.nomorph.udpipe</code> on the two conllu files:

{| class="wikitable"
|+Results
|-
|
|UAS
|LAS
|-
|<code>wbp.annotated.ud.conllu</code>
| 60.13%
| 50.00%
|-
|<code>wbp.annotated2.ud.conllu</code>
| 33.01%
| 26.21%
|}

=Dependency Relations=

[[Category:sp17_UD]]

Warlpiri/Universal Dependencies

2017-05-12T01:16:20Z

Tjones5:

=Evaluation=
==Corpora==
{| class="wikitable"
|+Information
|-
|
|Number of sentences
|Number of forms
|-
|<code>wbp.annotated.ud.conllu</code>
| 33
| 158
|-
|<code>wbp.annotated2.ud.conllu</code>
| 23
| 103
|}

==Withmorph==
Here are the results from using <code>wbp.withmorph.udpipe</code> on the two conllu files:

{| class="wikitable"
|+Results
|-
|
|UAS
|LAS
|-
|<code>wbp.annotated.ud.conllu</code>
| 91.14%
| 78.48%
|-
|<code>wbp.annotated2.ud.conllu</code>
| 60.19%
| 47.57%
|}

==Nomorph==
Here are the results from using <code>wbp.nomorph.udpipe</code> on the two conllu files:

{| class="wikitable"
|+Results
|-
|
|UAS
|LAS
|-
|<code>wbp.annotated.ud.conllu</code>
| 60.13%
| 50.00%
|-
|<code>wbp.annotated2.ud.conllu</code>
| 33.01%
| 26.21%
|}

=Dependency Relations=

[[Category:sp17_UD]]

Warlpiri

2017-05-12T01:14:07Z

Tjones5:

== Developed Resources ==
*Keyboard layout wiki page: [https://wikis.swarthmore.edu/ling073/Warlpiri/Keyboard]
*Keyboard layout repository: [https://github.swarthmore.edu/tjones5/ling073-wbp-keyboard]
*Repository containing corpus of Warlpiri and English [https://github.swarthmore.edu/tjones5/ling073-wbp-corpus]
*Grammar documentation page: [https://wikis.swarthmore.edu/ling073/Warlpiri/Grammar]
*Morphological Transducer Code: [https://github.swarthmore.edu/tjones5/ling073-wbp]
*Morphological Transducer Documentation: [https://wikis.swarthmore.edu/ling073/Warlpiri/Transducer]
*Morphological Disambiguator: [https://wikis.swarthmore.edu/ling073/Warlpiri/Disambiguation]
*Resources for machine translation between Guarani and Warlpiri [https://wikis.swarthmore.edu/ling073/Guarani_and_Warlpiri]
*Dependency syntax: [https://wikis.swarthmore.edu/ling073/Warlpiri/Universal_Dependencies]
*Link to final project: [https://wikis.swarthmore.edu/ling073/User:Tjones5/Final_project]

== External Resources ==

Kinds of resources:
*computational resources (spell checkers, orthography converters, speech recognition software),
*dictionaries/phrasebooks/glossaries (multilingual and monolingual, online and paper),
*grammatical descriptions (theoretical and pædagogical),
*scientific works (papers, books, websites), and
*corpora (any collection of authentic text, linguistically annotated or not)

===Computational Resources===
*Kirrkirr is computer software allowing for transformation of lexical databases to provide information languages target at indigenous languages, created by Kevin Jansz, Christopher Manning, and Nitin Indurkhya. [https://web.archive.org/web/20090521110425/http://www-nlp.stanford.edu/kirrkirr] It's available for download on Linux. The specifics of the authors' work on Warlpiri is detailed here: [https://web.archive.org/web/20090506054726/http://www.sultry.arts.usyd.edu.au/SULTRY/warlpiri.html]
*Christopher Horsethief has developed a Warlpiri Font Keyboard available on iOS devices. Available for purchase. [https://appadvice.com/app/warlpiri-font-keyboard/962527420]
===Dictionaries/Phrasebooks===
*The AuSIL (Australian Society for Indigenous Languages) has an interactive Warlpiri-English dictionary [http://ausil.org/Dictionary/Warlpiri/lexicon/index.htm] that includes a Warlpiri lexicon and English to Warlpiri translations. It appears to be free to download, yet download is not available on the current server.
*The Glosbe Dictionary contains 55 phrases translated in English and Warlpiri. The phrases contain usage examples, and anyone can edit their translations [https://glosbe.com/en/wbp/]. Note about licenses: "Some of data we present is licensed with CC-BY-SA, some is FDL, some comes with custom license. Data source is always indicated next to data if it is needed due to the license." [https://glosbe.com/about] There is also a free API that can be used [https://glosbe.com/a-api].
*A list of Warlpiri words, including English translations: [http://catalogue.aiatsis.gov.au/client/en_AU/external/search/detailnonmodal/ent:$002f$002fSD_ILS$002f0$002fSD_ILS:401933/ada?qu=warlpiri&te=ILS] No license necessary.
*A concise and comprehensive edition of the Warlpiri dictionaries used to build KirrKirr for Warlpiri: [https://web.archive.org/web/20090506054726/http://www.sultry.arts.usyd.edu.au/SULTRY/warlpiri.html] Available for free download.
===Grammatical Descriptions===
*The Omniglot encyclopedia includes a short grammatical description of Warlpiri: [http://www.omniglot.com/writing/warlpiri.htm] No license necessary.
*The AuSIL has a description of Warlpiri, including its sounds and abbreviations:[http://ausil.org/Dictionary/Warlpiri/aboutwarlpiri.htm] No license necessary.
*''Syntax - Theory and and Analysis'' (Kiss, 2015) includes a chapter about the grammatical structures of Walpiri [https://books.google.com/books?id=HABfCAAAQBAJ&pg=PA1677&lpg=PA1677&dq=steve+swartz+warlpiri+bible&source=bl&ots=fFrQBJrOAJ&sig=TGGWn3GfUpGRA60w3OLzDq_Yaak&hl=en&sa=X&ved=0ahUKEwjMn_Gp69jRAhWBeyYKHY0aBE0Q6AEIPDAI#v=onepage&q=steve%20swartz%20warlpiri%20bible&f=false]
*''Edinburgh Handbook of Evaluative Morphology'' (Bowler, 2015) [https://books.google.com/books?id=nDckDQAAQBAJ&pg=PA438&lpg=PA438&dq=nominal+warlpiri&source=bl&ots=4vQ5MdRO8z&sig=K3POmlFL24SSi_gCy3cR6rMwEqo&hl=en&sa=X&ved=0ahUKEwiH9b-GwP7RAhVkslQKHWqQBKUQ6AEIOzAF#v=onepage&q=nominal%20warlpiri&f=false] All rights reserved. Available for viewing on Google Scholar.
===Scientific Works===
*The AuSIL also contains links to four papers by Stephen Swartz (1982), regarding Warlpiri clauses, propositional particles, verb roots, and verbal clauses, in photocopied PDF form [http://ausil.org/Dictionary/Warlpiri/aboutwarlpiri.htm] No license is necessary to download it.
*Jane Simpson is a researcher at the Australian National University who has published many papers on Warlpiri, including her PhD thesis "Aspects of Warlpiri Morphology and Syntax" [https://dspace.mit.edu/handle/1721.1/15468] No license is necessary to download it.
*Jane Simpson's book ''Warlpiri Morpho-Syntax'' (1991) is available in McCabe: [https://catalog.tricolib.brynmawr.edu/find/Record/.b2064178]
===Corpora===
*Sample paragraph of Warlpiri from Omniglot Encyclopedia, including the English translation. Copyrighted, but usable as long as reference is given and it's not used commercially. [http://www.omniglot.com/writing/warlpiri.htm]
*New testament in Warlpiri, available in other translations including English. All rights reserved, © Wycliffe Bible Translators, Inc. [https://www.bible.com/bible/1355/jhn.1]
*A series of short stories by AIATSIS Australian Indigenous Languages Collection Informants. Open access, available for download: [http://catalogue.aiatsis.gov.au/client/en_AU/external/search/detailnonmodal/ent:$002f$002fSD_ILS$002f0$002fSD_ILS:400826/ada?qu=warlpiri&te=ILS]
===Other===
*David Nash has compiled a list of Warlpiri references, dictionaries, and literature: [http://www.anu.edu.au/linguistics/nash/aust/wlp/wlp-lx-ref.html]
*A 4-minute news program ''Indigenous Language News Radio'', available online in audio form: [http://www.abc.net.au/news/indigenous/]
*Midterm overview notes: https://wikis.swarthmore.edu/ling073/Warlpiri/Midterm

[[Category:warlpiri]]

[[Category:sp17_ResourceDocumentation]]

Warlpiri/Universal Dependencies

2017-05-12T01:13:55Z

Tjones5: Created page with "Placeholder for UD"

Placeholder for UD

User:Tjones5/Final project

2017-05-11T00:40:41Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.com/tkjones117/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/8cd682e70a81378184b6a90f381f16e6d130459c
*.lexc:
**added a few more stems

==Improvements Made to Disambiguation==
Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/6edc1d225ba43f369468448b83d009616a9c6c41

*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Number of stems: ~340

Final coverage: 61.48%

Against new_corpus.txt:
*Precision: 84.21053%
*Recall: 32.98969%

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T23:58:16Z

Tjones5:

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/8cd682e70a81378184b6a90f381f16e6d130459c
*.lexc:
**added a few more stems

==Improvements Made to Disambiguation==
Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/6edc1d225ba43f369468448b83d009616a9c6c41

*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Number of stems: ~340

Final coverage: 61.48%

Against new_corpus.txt:
*Precision: 84.21053%
*Recall: 32.98969%

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T23:52:55Z

Tjones5: /* Final Evaluation */

User:Tjones5/Final project

2017-05-10T23:52:31Z

Tjones5: /* Final Evaluation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/8cd682e70a81378184b6a90f381f16e6d130459c
*.lexc:
**added a few more stems

==Improvements Made to Disambiguation==
Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/6edc1d225ba43f369468448b83d009616a9c6c41

*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
# stems: ~340
Final coverage: 61.48%

Against new_corpus.txt:
*Precision: 84.21053%
*Recall: 32.98969%

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T23:50:54Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/8cd682e70a81378184b6a90f381f16e6d130459c
*.lexc:
**added a few more stems

==Improvements Made to Disambiguation==
Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/6edc1d225ba43f369468448b83d009616a9c6c41

*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Final coverage: 61.48%

Against new_corpus.txt:
*Precision: 84.21053%
*Recall: 32.98969%

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T23:50:38Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/8cd682e70a81378184b6a90f381f16e6d130459c
.lexc:
**added a few more stems

==Improvements Made to Disambiguation==
Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/6edc1d225ba43f369468448b83d009616a9c6c41

*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Final coverage: 61.48%

Against new_corpus.txt:
*Precision: 84.21053%
*Recall: 32.98969%

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T23:46:27Z

Tjones5: /* Final Evaluation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

==Improvements Made to Disambiguation==
Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/6edc1d225ba43f369468448b83d009616a9c6c41

*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Final coverage: 61.48%

Against new_corpus.txt:
*Precision: 84.21053%
*Recall: 32.98969%

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T07:18:38Z

Tjones5: /* Final Evaluation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

==Improvements Made to Disambiguation==
Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/6edc1d225ba43f369468448b83d009616a9c6c41

*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Final coverage: 61.33%
TODO
-precision/recall against annotated corpus

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T07:16:16Z

Tjones5: /* Improvements Made to Disambiguation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

==Improvements Made to Disambiguation==
Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/6edc1d225ba43f369468448b83d009616a9c6c41

*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Final coverage: 61.33%
TODO
-precision/recall against annotated corpus
-Author file, etc

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T07:14:25Z

Tjones5: /* Improvements Made to Disambiguation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

==Improvements Made to Disambiguation==
*Ambiguity before disambiguation: ~1.41077302631578947368
*Ambiguity after disambiguation (Pre-final project): ~1.35831766917293233083
*Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Final coverage: 61.33%
TODO
-precision/recall against annotated corpus
-Author file, etc

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T07:13:48Z

Tjones5:

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

==Improvements Made to Disambiguation==
*Initial ambiguity:
**Ambiguity before disambiguation: ~1.41077302631578947368
**Ambiguity after disambiguation (Initial): ~1.35831766917293233083
**Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Final Evaluation==
Final coverage: 61.33%
TODO
-precision/recall against annotated corpus
-Author file, etc

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T07:11:52Z

Tjones5: /* Improvements Made to Disambiguation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

Final coverage: 61.33%

TODO:
-add more verb stems
-evaluate coverage, precision/recall against annotated corpus
-remember author, etc

==Improvements Made to Disambiguation==
*Initial ambiguity:
**Ambiguity before disambiguation: ~1.41077302631578947368
**Ambiguity after disambiguation (Initial): ~1.35831766917293233083
**Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T07:11:14Z

Tjones5:

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

Final coverage: 61.33%

TODO:
-add more verb stems
-evaluate coverage, precision/recall against annotated corpus
-remember author, etc

==Improvements Made to Disambiguation==
*Initial ambiguity:
**Ambiguity before disambiguation: ~1.41077302631578947368
**Ambiguity after disambiguation (Initial): ~1.35831766917293233083
**Ambiguity after disambiguation (Final): ~1.34727443609022556391

# If there is a transitive verb two words to the left, I don't take a subject.
REMOVE (n sg prf) IF (-2 Vtv) ;

# If there is a transitive verb two words to the left, I'm in the absolutive case.
SELECT (n sg abs) IF (-2 Vtv) ;

# If there is an intransitive verb two words to the left, I'm the subject.
SELECT (n sg abs) IF (-2 Viv) ;

# If there is an auxiliary to the right or left, I cannot be a numeral.
REMOVE (num) IF ((1 Aux) OR (-1 Aux)) ;

# If there is an auxiliary to the left, I cannot be an auxiliary.
REMOVE (vaux) IF (-1 Aux) ;

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T05:52:56Z

Tjones5:

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

Final coverage: 61.33%

TODO:
-add more verb stems
-evaluate coverage, precision/recall against annotated corpus
-remember author, etc

==Improvements Made to Disambiguation==
*Initial ambiguity:
**Ambiguity before disambiguation: ~1.41077302631578947368
**Ambiguity after disambiguation: ~1.35831766917293233083

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T05:15:49Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/2857cffb76e0014b8ea90b21f40e4cc957ae280f
*.lexc:
**added "yungu" as a variation of the auxiliary "yunga"
**added more optional dashes in the auxiliary suffixes
**added ability to append another object suffixes to auxiliary verb
**added auxiliary suffixes to adverbs

TODO:
-add more verb stems
-evaluate coverage, precision/recall against annotated corpus
-remember author, etc

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T04:40:22Z

Tjones5:

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

TODO:
-add more verb stems
-evaluate coverage, precision/recall against annotated corpus
-remember author, etc

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T04:37:31Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit
-add more verb stems
-remember files

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:49:59Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: nyina = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:48:37Z

Tjones5: /* Current Evaluation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:48:22Z

Tjones5:

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf






[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:48:00Z

Tjones5: /* Notes */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf





<--
==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
-->
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:47:47Z

Tjones5: /* Current Evaluation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf





==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
-->
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:47:31Z

Tjones5: /* Current Evaluation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf





<--
==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
-->
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:47:03Z

Tjones5: /* Current Evaluation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

-->
{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf



<!--
*Pre Final Project
** Coverage over large corpus
***Number of tokenised words in the corpus: 18427
***Coverage: 51.26%
** Precision/Recall on wbp.annotated.basic.txt from Polished RMBT:
***Precision: 97.54902%
***Recall: 97.07317%
** Number of stems: ~250

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:46:32Z

Tjones5: /* Current Evaluation */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms





{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:45:58Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

Commit: next commit

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

Initial Eval:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18481
**Coverage: 49.67%

Checkpoint at presentation:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Number of stems: ~250





{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-10T02:38:23Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: next commit https://github.swarthmore.edu/tjones5/ling073-wbp/commit/f8841c64db1c248815bc5c5a3ce4a23b8d30f641
*.lexc:
**added present verb analysis (ex: piyani = present form of "to be", not piyanimi)
**added verb tense and suffix analysis (apparently sometimes verbs can object suffixes; previously I thought it was only auxiliaries that could do this)
**added "kirra" as another possible allative object ending
**added clitic analysis and several clitics -- something needs to be changed so that they increase coverage
**added 6 noun stems, ~55 verb stems, 3 adverb stems
*.twol:
**eliminated bi-directionality of some rules

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

Initial Eval:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18481
**Coverage: 49.67%

Checkpoint at presentation:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Number of stems: ~250





{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-09T00:01:19Z

Tjones5: /* Improvements Made to the Transducer */

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
*.lexc:
**optional dashes throughout (verb suffixes, pronouns)
**improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
**added several noun stems
*.twol:
**added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: next commit?

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

Initial Eval:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18481
**Coverage: 49.67%

Checkpoint at presentation:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Number of stems: ~250





{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-09T00:00:48Z

Tjones5:

==Improvements Made to the Transducer==
Final version:
https://github.swarthmore.edu/tjones5/ling073-wbp

Commit: https://github.swarthmore.edu/tjones5/ling073-wbp/commit/125bb63f33f5f50caf682b746b972c549b4da7a0
.lexc:
*optional dashes throughout (verb suffixes, pronouns)
*improved organization of pronoun analysis: separation of 1st person from 2nd/3rd person
*added several noun stems

.twol:
*added %{U%} archiphoneme to appear in suffixes on nouns in dative case

Commit: next commit?

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

Initial Eval:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18481
**Coverage: 49.67%

Checkpoint at presentation:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Number of stems: ~250





{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-08T02:21:26Z

Tjones5: /* Current Evaluation */

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

Initial Eval:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18481
**Coverage: 49.67%

Checkpoint at presentation:
* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Number of stems: ~250





{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

Guarani and Warlpiri

2017-05-08T01:42:48Z

Tjones5: /* gug → wbp */

Resources for machine translation between [https://wikis.swarthmore.edu/ling073/Guarani Guarani] and [https://wikis.swarthmore.edu/ling073/Warlpiri Warlpiri]

==Corpora==
*[https://www.bible.com/bible/1355/rev.1 Warlpiri Bible]

*[https://www.bible.com/bible/66/rev.1 Guarani Bible]

*[https://www.bible.com/bible/111/rev.1 English Bible]

*[http://unicode.org/udhr/d/udhr_gug.txt Universal Declaration of Human Rights in Guarani]

==Developed Resources==
*[https://github.swarthmore.edu/tjones5/ling073-gug-wbp-corpus Corpus Repository]
*[https://wikis.swarthmore.edu/ling073/Guarani_and_Warlpiri/Contrastive_Grammar Contrastive Grammar]
*[https://github.swarthmore.edu/tjones5/ling073-gug-wbp Guarani to Warlpiri analyzer repository]
**This repo will hold the fully functioning rule-based machine translation system.
*[https://github.swarthmore.edu/cpillsb1/ling073-wbp-gug Warlpiri to Guarani analyzer repository]
**This repo will not be developed further.
*[https://wikis.swarthmore.edu/ling073/Guarani_and_Warlpiri/Lexical_selection Lexical selection Wiki page]
*[https://wikis.swarthmore.edu/ling073/Guarani_and_Warlpiri/Structural_transfer Structural transfer Wiki page]

==Lexical Selection==
* See Lexical Selection page linked above for more detail

* (gug) ahy'o → (wbp) ngukarna "throat"; lirra "voice"
* (gug) ha'anga → (wbp) japirdimi "threaten"; walaparrirni "imitate"
* (gug) ajúra → (wbp) japirdimi "collar"; waninja "neck"
* (wbp) rdaka → (gug) avatisoka "hand"; po "five"
* (wbp) kirdirrpa → (gug) itakua "cave"; ka'irãi "jail"

==Initial Evaluation==
===wbp → gug evaluation===
====Results when running on tests====
*Total number of words: 14
*Total number of unknown words: 4
*Percentage of unknown words: 28.57%
*Percentage of stems translated correctly: 71.43%

When removing unknown words:
*WER = 78.57%
*PER = 78.57%
When unknown words are not removed:
*WER = 100%
*PER = 100%

====Results when running on sentences (corpus) ====
*Total number of words: 58
*Total number of unknown words: 68
*Percentage of unknown words: 84.45%
*Percentage of stems translate correctly: 15.52%

When removing unknown words:
*WER = 22.41%
*PER = 22.41%
When unknown words are not removed:
*WER = 100%
*PER = 100%

===gug → wbp evaluation===
====Results when running on tests====
*Total number of words: 13
*Total number of unknown words: 0
*Percentage of unknown words: 0%
*Percentage of stems translated correctly: 100%

When removing unknown words:
*WER = 92.86%
*PER = 92.86%
When unknown words are not removed:
*WER = 92.86%
*PER = 92.86%

====Results when running on sentences (corpus) ====
*Total number of words: 66
*Total number of unknown words: 60
*Percentage of unknown words: 90.9%
*Percentage of stems translated correctly: 9.09%

When removing unknown words:
*WER = 0.0%
*PER = 0.0%
When unknown words are not removed:
*WER = 100%
*PER = 100%

==Final Evaluation==

===gug → wbp and wbp → gug===
====100 more stems====
Added 100 more stems (mostly nouns, verbs, and adjectives) to the bilingual and monolingual dictionaries.

===gug → wbp===

====6 new lexical selection rules====
* (gug) akã → (wbp) jurru "head"; payarrpa "hill"
** Rule: Select "jurru" (Eng: head) as the translation of "akã" when it is preceded by "ha'e" (Eng: his/her). Otherwise, select "payarrpa" as the default.

* (gug) asu → (wbp) jampu "left"; jampu-karra "left-handed"
** Rule: Select "jampu-karra" (Eng: left-handed) as the translation of "asu" when it is followed by person words such as "ava" (Eng: person) or "kuña" (Eng: woman) or "mitã" (Eng: child). Otherwise, select "jampu" (Eng: left) as the default.

* (gug) henyhẽ → (wbp) jaka-ngalya "full"; ngayarrka "pregnant"
** Rule: Select "ngayarrka" (Eng: pregnant) as the translation of "henyhẽ" when it is followed/preceded by "kuña" (Eng: woman) or other words describing female people. Otherwise, select "jaka-ngalya" (Eng: full) as the default.

* (gug) mongúi → (wbp) kijirni "to throw"; pata-karrimi "to fall"
** Rule: Select "kijirni" (Eng: to throw) as the translation of "mopẽ" when it is followed by "ita" (Eng: rock) or "apu'a" (Eng: ball). Otherwise, select "pata-karrimi" as the default.

* (gug) moha'anga → (wbp) pantirni "to draw"; jarntirni "to carve"
** Rule: Select "pantirni" (Eng: to draw) as the translation of "moha'anga" when it is followed by "ha'anga" (Eng: drawing). Otherwise, select "jarntirni" as the default.

* (gug) mbogyapy → (wbp) pipa-kurra-mani "to write"; nyinanja-wantimi "to sit down"
** Rule: Select "pipa-kurra-mani" (Eng: to write) as the translation of "mbogyapy" when it is followed by "aranduka" (Eng: book). Otherwise, select "nyinanja-wantimi" as the default.

====6 additions to the morphology====
*Added numeral analysis:
**Several number words in Warlpiri also have meanings as nouns (e.g. rdaka can mean "hand" or "five")
**the -pala ending distinguishes them as number words, but it is not always necessary
**"rdaka-pala": ^rdaka-pala/rdaka<num>$^./.<sent>$
**"rdaka": ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

*Added adverb analysis:
**"nyurruwiyi: ^nyurruwiyi/nyurruwiyi<adv>$^./.<sent>$

*Added elative/source case for verb auxiliaries :
**"kajangka": ^kajangka/ka<vaux><src><subj3sg>$^./.<sent>$

*Added inceptive aspect for verbs:
**"pakarnunjunu": ^pakarnunjunu/pakarni<v><tv><past><inc>$^./.<sent>$

*Added possessive marker "nyanu" for nouns:
**"kaja-nyanu-rlu": ^kaja-nyanu-rlu/kaja<n><pl><poss><erg>$^./.<sent>$

*Added the topicalizing particle grammatical marker for nouns. -ja is used to mark known information or to indicate the topic, although sometimes it has no function.
**echo "kurdukuju": ^kurdukuju/kurdu<n><sg><dat><top>$^./.<sent>$





===wbp → gug===
====6 new lexical selection rules====
* (wbp) nyurltu-nyurltu → (gug) apañuãi "confusing, garbled"; apokytã "tangled"
** Rule: select "apokytã" if "nyurltu-nyurltu" is followed by the Warlpiri word "marnilpa" (hair). In other cases, select "apañuãi" as the default.

* (wbp) warlu → (gug) tini "hot"; aku "angry"
** Rule: select "aku" if "warlu" is followed by the Warlpiri word "miparrpa" (face) or "yapa" (person). In other cases, select "tini" as the default.

* (wbp) yajarni → (gug) henoi "fetch/invite (someone)"; kakuaa "grow (a plant)"
** Rule: select "kakuaa" if "yajarni" is followed by the Warlpiri word "watiya" (plant). In other cases, select "henoi" as the default.

* (wbp) paarr-paarr-jankami → (gug) hapy "to burn, singe (hair or fur)"; jahéi "to hurt"
** Rule: select "hapy" if "paarr-paarr-jankami" is followed by the Warlpiri word "marnilpa" (hair). In other cases, select "jahéi" as the default.

* (wbp) pajirni → (gug) su'u "to bite"; hupi "to pick/gather"
** Rule: select "hupi" if "pajirni" is followed by the Warlpiri word "mangarri" (fruit). In other cases, select "su'u" as the default.

* (wbp) pakarli → (gug) kuatia "headdress"; kuatia "paper"
** Rule: select "kuatia" if "pakarli" is preceded by the Warlpiri word "yangkarni" (to wear). In other cases, select "kuatia" as the default.

====4 new twol rules====
*Nasal harmony on dative suffix -pe:
**-pe now properly changes to -me when before a nasal vowel.
**Ex: havõ (soap). Suffix is regularly -pe, but now becomes -me
echo "havõme" | apertium -d . gug-morph
^havõme/havõ<n><sg><dat>$^./.<sent>$

*Nasal harmony on accusative suffix -ve:
**-ve now properly changes to -me on accusative forms of pronouns
**Ex: peẽ (second person plural pronoun). Suffix is regularly -ve, but now becomes -me
echo "peẽme" | apertium -d . gug-morph
^peẽme/peẽ<prn><pers><p2><sg><acc>$^./.<sent>$

*Stress on accusative forms of pronouns:
**The e at the end of pronouns should be stressed (é) when followed by the acc suffix -ve
**Ex: che (first person singular pronoun). With suffix -ve, becomes chéve
echo "chéve" | apertium -d . gug-morph
^chéve/che<prn><pers><p1><sg><acc>$^./.<sent>$

*Nasal harmony on reflexive prefix je- :
**je- now properly changes to -ñe when before a nasal.
**Ex: ñeñapytĩ (“to tie”).
echo "oñeñapytĩ" | apertium -d . gug-morph
^oñeñapytĩ/ñapytĩ<v><tv><ar><pres><p3><sp><nowiki><ref></nowiki>$^./.<sent>$

===Tests results===

====gug====
*Precision and recall:
**Results:
precisionRecall ../corpus/ling073-gug-corpus/ling073-gug-corpus/gug.annotated.basic.txt ../bootstrap/ling073-gug/corpus.out.txt
Totals: 89 tp, 87 fp, 0 tn, 19 fn
Precision: 82.40741%
Recall: 50.56818%

*Coverage over large corpus:
**Results:
aq-covtest ~/Source/corpus/ling073-gug-corpus/ling073-gug-corpus/gug.corpus.large.txt gug.automorf.bin
Number of tokenised words in the corpus: 522284
Coverage: 28.96%

*Number of stems in transducer: ~270 (didn't count by hand)

====wbp====
*Precision and recall:
**Results:
precisionRecall ../ling073-wbp-corpus/wbp.annotated.basic.txt corpus.out.txt
Totals: 199 tp, 5 fp, 0 tn, 6 fn
Precision: 97.54902%
Recall: 97.07317%

*Coverage over large corpus:
**Results:
aq-covtest ../ling073-wbp-corpus/wbp.corpus.large.txt wbp.automorf.bin
Number of tokenised words in the corpus: 18427
Coverage: 51.26%

*Number of stems in transducer: ~250

====wbp → gug====
*WER and PER
Number of words in reference: 76
Number of words in test: 69
Number of unknown words (marked with a star) in test: 45
Percentage of unknown words: 65.22 %
Word error rate (WER): 100.00 %
Position-independent word error rate (PER): 94.74 %
*Proportion of stems translate correctly in longer: 31/71 = 40.8%

Unsure... Need wbp.longer.txt to test in longer and large

====gug → wbp====
*WER and PER over gug.longer.txt
Number of words in reference: 72
Number of words in test: 81
Number of unknown words (marked with a star) in test: 44
Percentage of unknown words: 54.32 %
Word error rate (WER): 112.50.00 %
Position-independent word error rate (PER): 108.33 %
*Proportion of stems translate correctly in longer: (81-44 / 81) = 37/81 = 45.7%
*Trimmed coverage and number of tokens in longer and large corpora
aq-covtest ../ling073-gug-wbp-corpus/gug.longer.txt gug-wbp.automorf.bin
**Number of tokenised words in the longer corpus: 97
**Coverage: 50.52%

aq-covtest ../ling073-gug-wbp-corpus/gug.corpus.large.txt gug-wbp.automorf.bin
**Number of tokenised words in the large corpus: 501127
**Coverage: 24.74%

[[Category:Sp17_TranslationPairs]]

User:Tjones5/Final project

2017-05-05T06:08:53Z

Tjones5: /* Presentation */

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Precision/Recall on wbp.annotated.basic.txt from Polished RBMT:
**Precision: 97.54902%
**Recall: 97.07317%
* Number of stems: ~250


{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-05T06:07:06Z

Tjones5: /* Current Evaluation */

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against hand-annotated randomly selected forms

* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Precision/Recall on wbp.annotated.basic.txt from Polished RBMT:
**Precision: 97.54902%
**Recall: 97.07317%
* Number of stems: ~250


{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-05T05:57:55Z

Tjones5:

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri stories, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

*Next to implement: perhaps reduplication?
echo "kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><sg><abs>$^./.<sent>$

echo "kurdu-kurdu" | apertium -d . wbp-morph: ^kurdu/kurdu<n><pl><abs>$^./.<sent>$

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against a hand-annotated randomly selected forms

* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Precision/Recall on wbp.annotated.basic.txt from Polished RBMT:
**Precision: 97.54902%
**Recall: 97.07317%
* Number of stems: ~250


{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline
[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-05T05:45:49Z

Tjones5:

Placeholder page for final project

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri phrases, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Current Evaluation===

* Coverage over a large corpus
* Precision and recall against a hand-annotated randomly selected forms

* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Precision/Recall on wbp.annotated.basic.txt from Polished RMBT:
**Precision: 97.54902%
**Recall: 97.07317%
* Number of stems: ~250


{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-05T05:44:53Z

Tjones5: /* Description of the problem */

Placeholder page for final project

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. wolves
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-wolf + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "wolves"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri phrases, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against a hand-annotated randomly selected forms

* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Precision/Recall on wbp.annotated.basic.txt from Polished RMBT:
**Precision: 97.54902%
**Recall: 97.07317%
* Number of stems: ~250


{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-05T05:44:03Z

Tjones5:

Placeholder page for final project

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. foxes
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-fox + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "foxes"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri phrases, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

*pirli (a noun with just 2 vowels) has different endings than warlkurru (which has more than two vowels)
**however when suffixes such as -jarra are added, pirli gets the endings of nouns with >2 vowels: pirli-ngku → pirli-jarra-rlu

{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
|"agent of a transitive verb" || {{tag|erg}} || style="background-color: teal;" | -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}



*Disambiguation is a challenge because of flexible word order
echo "rdaka" | apertium -d . wbp-morph: ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

echo "rdaka-pala" | apertium -d . wbp-morph: ^rdaka-pala/rdaka<num>$^./.<sent>$

*Trying to write the following disambiguation rule: # If there is a noun to the right or left, I cannot be a numeral-like noun.
*Examples of numeral-like nouns, sometimes distinguished by adding the number marker -pala.
**mirdi = "four" or "elbow"
**rdaka = "five" or "hand"
**wirlki = "seven" or "boomerang"
**narntirnki = "nine" or "curled"

*Need to write scraper to test on a larger corpus (current corpus is ~13,000 words)

===Evaluation===

* Coverage over a large corpus
* Precision and recall against a hand-annotated randomly selected forms

* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Precision/Recall on wbp.annotated.basic.txt from Polished RMBT:
**Precision: 97.54902%
**Recall: 97.07317%
* Number of stems: ~250


{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-05T05:23:15Z

Tjones5:

Placeholder page for final project

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. foxes
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-fox + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "foxes"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer
*Other work?

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri phrases, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

**Noun_Number 2v
{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
!style="background-color: teal;"| ergative
| "agent of a transitive verb" || {{tag|erg}} || -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}

***wati-ngku
***wati-jarra-rlu
**adding (optional) dashes between everything; selecting only one analysis by using ! Dir/LR
*Disambiguation is a challenge because of flexible word order
**Numerals are one thing to deal with
*Need to write scraper to test on a larger corpus

===Evaluation===

* Coverage over a large corpus
* Precision and recall against a hand-annotated randomly selected forms

* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Precision/Recall on wbp.annotated.basic.txt from Polished RMBT:
**Precision: 97.54902%
**Recall: 97.07317%
* Number of stems: ~250


{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

[[Category:sp17_FinalProjects]]

User:Tjones5/Final project

2017-05-05T05:20:51Z

Tjones5: /* Approaches */

Placeholder page for final project

==Notes==

*To implement:
**Reduplication

*Get a big corpus:
**Build a scraper? to get corpus up to 25K
**Use regular expressions to clean up scraped (or copied/pasted) text

*For evaluation:
**Go back and look at coverage

*Other ling to do: UD pipeline

==Presentation==

===Description of the problem===
* Word structure is analyzed by composition of morphemes (the smallest units with grammatical meaning)
* A morphological transducer connects forms with analyses.
* Surface level -- represents the concatenation of letters which make up the actual spelling of the word, e.g. foxes
* Lexical level -- represents the concatenation of morphemes making up a word, e.g. NOUN-fox + PLURAL-es
* Morphological parsing -- the problem of recognizing that a word breaks down into component morphemes, as above with "foxes"

[[Image:Simple transducer.png|1000px]]

===Previous work===

*Many languages have working [http://wiki.apertium.org/wiki/Languages Apertium transducers]:
*Warlpiri doesn't have a working transducer -- in fact, no languages from the Pama–Nyungan family have a transducer
*Other work?

===Benefits of this project===

*Why is a transducer useful to a community of Warlpiri speakers?
**Search engine
**Spell checker
**Machine Translation, especially to/from English (the primary language of most Warlpiri speakers). Many are trying to revive the use of Warlpiri, so there are many potentially useful applications.
***eng → wbp MT system could be used to look up how to say/write a certain word or phrase
***wbp → eng MT system could be used to analyze (and then translate) untranslated Warlpiri phrases, or give alternate translations
**All of these applications might also be helpful in starting an analyzer for other indigenous languages that have the same orthography, are highly agglutinative, and have other grammatical similarities

===Approaches===


*Mostly have worked within lexc and twol files

*Pronouns (optional dash not included in table):
{|class="wikitable sortable"
! Meaning !! Word !! Form
|-
! I, me
| ngaju(lu) || {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngaju}} {{morphTest|ngaju{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|sg}}|ngajulu}}
|-
! you
| nyuntu(lu) || {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntu}} {{morphTest|nyuntu{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|sg}}|nyuntulu}}
|-
! he/she/it; to him/her/it
| nyanungu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|sg}}|nyanungu}}
|-
! we (you & me)
| ngali(jarra) || {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngali}} {{morphTest|ngali{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|incl}}|ngalijarra}}
|-
! we (him/her/it & me)
| ngajarra || {{morphTest|ngajarra{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|du}}{{tag|excl}}|ngajarra}}
|-
! we (you & me & other(s))
| ngalipa || {{morphTest|ngalipa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|incl}}|ngalipa}}
|-
! we (them & me)
| nganimpa || {{morphTest|nganimpa{{tag|prn}}{{tag|pers}}{{tag|p1}}{{tag|pl}}{{tag|excl}}|nganimpa}}
|-
! you (both/two)
| nyumpala || {{morphTest|nyumpala{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|du}}|nyumpala}}
|-
! you (more than 2)
| nyurrarla || {{morphTest|nyurrarla{{tag|prn}}{{tag|pers}}{{tag|p2}}{{tag|pl}}|nyurrarla}}
|-
! they/them (both/two)
| nyanungu-jarra || {{morphTest|nyanungu-jarra{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|du}}|nyanungu-jarra}}
|-
! they/them (more than 2)
| nyanungu-rra/nyanungu-patu || {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-rra}} {{morphTest|nyanungu{{tag|prn}}{{tag|pers}}{{tag|p3}}{{tag|pl}}|nyanungu-patu}}
|-
|}

**Noun_Number 2v
{|class="wikitable sortable"
! case name !! ~meaning !! tag !! possible forms !! pirli "rock" (2 vowels) !! warlkurru (>2 vowels) "axe"
|-
! absolutive
| subject of intransitive verbs and object of transitive verbs || {{tag|abs}} || — || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|abs}}|pirli}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|abs}}|warlkurru}}
|-
! dative
| "to" || {{tag|dat}} || -ku || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|dat}}|pirli-ku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|dat}}|warlkurru-ku}}
|-
! ergative
| "agent of a transitive verb" || {{tag|erg}} || -ngku, -rlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|erg}}|pirli-ngku}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|erg}}|warlkurru-rlu}}
|-
! allative
| "onto" || {{tag|all}} || -kurra || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|all}}|pirli-kurra}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|all}}|warlkurru-kurra}}
|-
! comitative
| "with" || {{tag|com}} || -ngkajinta, -rlajinta || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|com}}|pirli-ngkajinta}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|com}}|warlkurru-rlajinta}}
|-
! elative
| "out of" || {{tag|ela}} || -ngurlu || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|ela}}|pirli-ngurlu}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|ela}}|warlkurru-ngurlu}}
|-
! locative
| "at, in on" || {{tag|loc}} || -ngka, -rla || {{morphTest|pirli{{tag|n}}{{tag|sg}}{{tag|loc}}|pirli-ngka}} || {{morphTest|warlkurru{{tag|n}}{{tag|sg}}{{tag|loc}}|warlkurru-rla}}
|}

***wati-ngku
***wati-jarra-rlu
**adding (optional) dashes between everything; selecting only one analysis by using ! Dir/LR
*Disambiguation is a challenge because of flexible word order
**Numerals are one thing to deal with
*Need to write scraper to test on a larger corpus

===Evaluation===

* Coverage over a large corpus
* Precision and recall against a hand-annotated randomly selected forms

* Coverage over large corpus
**Number of tokenised words in the corpus: 18427
**Coverage: 51.26%
* Precision/Recall on wbp.annotated.basic.txt from Polished RMBT:
**Precision: 97.54902%
**Recall: 97.07317%
* Number of stems: ~250


{|class="wikitable"
|-
! status !! description !! stems !! coverage
|-
!style="background-color: red;"| '''prototype'''
| language module that has not received heavy development
| <1,000
| <60%
|-
!style="background-color: orange;"| '''development'''
| language module under development
| ≥1,000
| ≥60%
|-
!style="background-color: yellow;"| '''working'''
| language module with near-production-quality performance
| ≥8,000
| ≥80%
|-
!style="background-color: green;"| '''production'''
| language module used in a released pair
| ≥10,000
| ≥90%
|}

*Table Source: http://wiki.apertium.org/wiki/Languages
*Background on analyzers: http://www.phil.uu.nl/tst/2012/Slides/SLP_Lecture2.pdf

[[Category:sp17_FinalProjects]]

Guarani and Warlpiri

2017-05-05T05:00:44Z

Tjones5: /* wbp */

Resources for machine translation between [https://wikis.swarthmore.edu/ling073/Guarani Guarani] and [https://wikis.swarthmore.edu/ling073/Warlpiri Warlpiri]

==Corpora==
*[https://www.bible.com/bible/1355/rev.1 Warlpiri Bible]

*[https://www.bible.com/bible/66/rev.1 Guarani Bible]

*[https://www.bible.com/bible/111/rev.1 English Bible]

*[http://unicode.org/udhr/d/udhr_gug.txt Universal Declaration of Human Rights in Guarani]

==Developed Resources==
*[https://github.swarthmore.edu/tjones5/ling073-gug-wbp-corpus Corpus Repository]
*[https://wikis.swarthmore.edu/ling073/Guarani_and_Warlpiri/Contrastive_Grammar Contrastive Grammar]
*[https://github.swarthmore.edu/tjones5/ling073-gug-wbp Guarani to Warlpiri analyzer repository]
**This repo will hold the fully functioning rule-based machine translation system.
*[https://github.swarthmore.edu/cpillsb1/ling073-wbp-gug Warlpiri to Guarani analyzer repository]
**This repo will not be developed further.
*[https://wikis.swarthmore.edu/ling073/Guarani_and_Warlpiri/Lexical_selection Lexical selection Wiki page]
*[https://wikis.swarthmore.edu/ling073/Guarani_and_Warlpiri/Structural_transfer Structural transfer Wiki page]

==Lexical Selection==
* See Lexical Selection page linked above for more detail

* (gug) ahy'o → (wbp) ngukarna "throat"; lirra "voice"
* (gug) ha'anga → (wbp) japirdimi "threaten"; walaparrirni "imitate"
* (gug) ajúra → (wbp) japirdimi "collar"; waninja "neck"
* (wbp) rdaka → (gug) avatisoka "hand"; po "five"
* (wbp) kirdirrpa → (gug) itakua "cave"; ka'irãi "jail"

==Initial Evaluation==
===wbp → gug evaluation===
====Results when running on tests====
*Total number of words: 14
*Total number of unknown words: 4
*Percentage of unknown words: 28.57%
*Percentage of stems translated correctly: 71.43%

When removing unknown words:
*WER = 78.57%
*PER = 78.57%
When unknown words are not removed:
*WER = 100%
*PER = 100%

====Results when running on sentences (corpus) ====
*Total number of words: 58
*Total number of unknown words: 68
*Percentage of unknown words: 84.45%
*Percentage of stems translate correctly: 15.52%

When removing unknown words:
*WER = 22.41%
*PER = 22.41%
When unknown words are not removed:
*WER = 100%
*PER = 100%

===gug → wbp evaluation===
====Results when running on tests====
*Total number of words: 13
*Total number of unknown words: 0
*Percentage of unknown words: 0%
*Percentage of stems translated correctly: 100%

When removing unknown words:
*WER = 92.86%
*PER = 92.86%
When unknown words are not removed:
*WER = 92.86%
*PER = 92.86%

====Results when running on sentences (corpus) ====
*Total number of words: 66
*Total number of unknown words: 60
*Percentage of unknown words: 90.9%
*Percentage of stems translated correctly: 9.09%

When removing unknown words:
*WER = 0.0%
*PER = 0.0%
When unknown words are not removed:
*WER = 100%
*PER = 100%

==Final Evaluation==

===gug → wbp and wbp → gug===
====100 more stems====
Added 100 more stems (mostly nouns, verbs, and adjectives) to the bilingual and monolingual dictionaries.

===gug → wbp===

====6 new lexical selection rules====
* (gug) akã → (wbp) jurru "head"; payarrpa "hill"
** Rule: Select "jurru" (Eng: head) as the translation of "akã" when it is preceded by "ha'e" (Eng: his/her). Otherwise, select "payarrpa" as the default.

* (gug) asu → (wbp) jampu "left"; jampu-karra "left-handed"
** Rule: Select "jampu-karra" (Eng: left-handed) as the translation of "asu" when it is followed by person words such as "ava" (Eng: person) or "kuña" (Eng: woman) or "mitã" (Eng: child). Otherwise, select "jampu" (Eng: left) as the default.

* (gug) henyhẽ → (wbp) jaka-ngalya "full"; ngayarrka "pregnant"
** Rule: Select "ngayarrka" (Eng: pregnant) as the translation of "henyhẽ" when it is followed/preceded by "kuña" (Eng: woman) or other words describing female people. Otherwise, select "jaka-ngalya" (Eng: full) as the default.

* (gug) mongúi → (wbp) kijirni "to throw"; pata-karrimi "to fall"
** Rule: Select "kijirni" (Eng: to throw) as the translation of "mopẽ" when it is followed by "ita" (Eng: rock) or "apu'a" (Eng: ball). Otherwise, select "pata-karrimi" as the default.

* (gug) moha'anga → (wbp) pantirni "to draw"; jarntirni "to carve"
** Rule: Select "pantirni" (Eng: to draw) as the translation of "moha'anga" when it is followed by "ha'anga" (Eng: drawing). Otherwise, select "jarntirni" as the default.

* (gug) mbogyapy → (wbp) pipa-kurra-mani "to write"; nyinanja-wantimi "to sit down"
** Rule: Select "pipa-kurra-mani" (Eng: to write) as the translation of "mbogyapy" when it is followed by "aranduka" (Eng: book). Otherwise, select "nyinanja-wantimi" as the default.

====6 additions to the morphology====
*Added numeral analysis:
**Several number words in Warlpiri also have meanings as nouns (e.g. rdaka can mean "hand" or "five")
**the -pala ending distinguishes them as number words, but it is not always necessary
**"rdaka-pala": ^rdaka-pala/rdaka<num>$^./.<sent>$
**"rdaka": ^rdaka/rdaka<num>/rdaka<n><sg><abs>$^./.<sent>$

*Added adverb analysis:
**"nyurruwiyi: ^nyurruwiyi/nyurruwiyi<adv>$^./.<sent>$

*Added elative/source case for verb auxiliaries :
**"kajangka": ^kajangka/ka<vaux><src><subj3sg>$^./.<sent>$

*Added inceptive aspect for verbs:
**"pakarnunjunu": ^pakarnunjunu/pakarni<v><tv><past><inc>$^./.<sent>$

*Added possessive marker "nyanu" for nouns:
**"kaja-nyanu-rlu": ^kaja-nyanu-rlu/kaja<n><pl><poss><erg>$^./.<sent>$

*Added the topicalizing particle grammatical marker for nouns. -ja is used to mark known information or to indicate the topic, although sometimes it has no function.
**echo "kurdukuju": ^kurdukuju/kurdu<n><sg><dat><top>$^./.<sent>$





===wbp → gug===
====6 new lexical selection rules====
* (wbp) nyurltu-nyurltu → (gug) apañuãi "confusing, garbled"; apokytã "tangled"
** Rule: select "apokytã" if "nyurltu-nyurltu" is followed by the Warlpiri word "marnilpa" (hair). In other cases, select "apañuãi" as the default.

* (wbp) warlu → (gug) tini "hot"; aku "angry"
** Rule: select "aku" if "warlu" is followed by the Warlpiri word "miparrpa" (face) or "yapa" (person). In other cases, select "tini" as the default.

* (wbp) yajarni → (gug) henoi "fetch/invite (someone)"; kakuaa "grow (a plant)"
** Rule: select "kakuaa" if "yajarni" is followed by the Warlpiri word "watiya" (plant). In other cases, select "henoi" as the default.

* (wbp) paarr-paarr-jankami → (gug) hapy "to burn, singe (hair or fur)"; jahéi "to hurt"
** Rule: select "hapy" if "paarr-paarr-jankami" is followed by the Warlpiri word "marnilpa" (hair). In other cases, select "jahéi" as the default.

* (wbp) pajirni → (gug) su'u "to bite"; hupi "to pick/gather"
** Rule: select "hupi" if "pajirni" is followed by the Warlpiri word "mangarri" (fruit). In other cases, select "su'u" as the default.

* (wbp) pakarli → (gug) kuatia "headdress"; kuatia "paper"
** Rule: select "kuatia" if "pakarli" is preceded by the Warlpiri word "yangkarni" (to wear). In other cases, select "kuatia" as the default.

====4 new twol rules====
*Nasal harmony on dative suffix -pe:
**-pe now properly changes to -me when before a nasal vowel.
**Ex: havõ (soap). Suffix is regularly -pe, but now becomes -me
echo "havõme" | apertium -d . gug-morph
^havõme/havõ<n><sg><dat>$^./.<sent>$

*Nasal harmony on accusative suffix -ve:
**-ve now properly changes to -me on accusative forms of pronouns
**Ex: peẽ (second person plural pronoun). Suffix is regularly -ve, but now becomes -me
echo "peẽme" | apertium -d . gug-morph
^peẽme/peẽ<prn><pers><p2><sg><acc>$^./.<sent>$

*Stress on accusative forms of pronouns:
**The e at the end of pronouns should be stressed (é) when followed by the acc suffix -ve
**Ex: che (first person singular pronoun). With suffix -ve, becomes chéve
echo "chéve" | apertium -d . gug-morph
^chéve/che<prn><pers><p1><sg><acc>$^./.<sent>$

*Nasal harmony on reflexive prefix je- :
**je- now properly changes to -ñe when before a nasal.
**Ex: ñeñapytĩ (“to tie”).
echo "oñeñapytĩ" | apertium -d . gug-morph
^oñeñapytĩ/ñapytĩ<v><tv><ar><pres><p3><sp><nowiki><ref></nowiki>$^./.<sent>$

===Tests results===

====gug====
*Precision and recall:
**Results:
precisionRecall ../corpus/ling073-gug-corpus/ling073-gug-corpus/gug.annotated.basic.txt ../bootstrap/ling073-gug/corpus.out.txt
Totals: 89 tp, 87 fp, 0 tn, 19 fn
Precision: 82.40741%
Recall: 50.56818%

*Coverage over large corpus:
**Results:
aq-covtest ~/Source/corpus/ling073-gug-corpus/ling073-gug-corpus/gug.corpus.large.txt gug.automorf.bin
Number of tokenised words in the corpus: 522284
Coverage: 28.96%

*Number of stems in transducer: ~270 (didn't count by hand)

====wbp====
*Precision and recall:
**Results:
precisionRecall ../ling073-wbp-corpus/wbp.annotated.basic.txt corpus.out.txt
Totals: 199 tp, 5 fp, 0 tn, 6 fn
Precision: 97.54902%
Recall: 97.07317%

*Coverage over large corpus:
**Results:
aq-covtest ../ling073-wbp-corpus/wbp.corpus.large.txt wbp.automorf.bin
Number of tokenised words in the corpus: 18427
Coverage: 51.26%

*Number of stems in transducer: ~250

====wbp → gug====
*WER and PER
Number of words in reference: 76
Number of words in test: 69
Number of unknown words (marked with a star) in test: 45
Percentage of unknown words: 65.22 %
Word error rate (WER): 100.00 %
Position-independent word error rate (PER): 94.74 %
*Proportion of stems translate correctly in longer: 31/71 = 40.8%

Unsure... Need wbp.longer.txt to test in longer and large

====gug → wbp====
*WER and PER over gug.longer.txt
Number of words in reference: 72
Number of words in test: 81
Number of unknown words (marked with a star) in test: 44
Percentage of unknown words: 54.32 %
Word error rate (WER): 112.50.00 %
Position-independent word error rate (PER): 108.33 %
*Proportion of stems translate correctly in longer: (81-44 / 81) = 37/81 = 45.7%
*Trimmed coverage and number of tokens in longer and large corpora
aq-covtest ../ling073-gug-wbp-corpus/gug.longer.txt gug-wbp.automorf.bin
**Number of tokenised words in the longer corpus: 97
**Coverage: 50.52%

**Number of tokenised words in the LARGE corpus: ?
**Coverage: ?

[[Category:Sp17_TranslationPairs]]

User:Tjones5/Final project

2017-05-05T04:54:39Z

Tjones5: /* Presentation */

User:Tjones5/Final project

2017-05-05T04:50:15Z

Tjones5: **