Difference between revisions of "User:Eortici1/Final project"

Latest revision as of 19:21, 10 May 2017

This page documents my final project, a larger Universal Dependencies corpus and annotation guidelines for Bikol.

The repository for this project is modeled closely after the repositories used by Universal Dependencies (example here).

Introduction to UD

The Universal Dependencies project was created to "provide a universal inventory of categories and guidelines to facilitate consistent annotation of similar constructions across languages". UD uses a set of typed dependency relations between words, where one word is the head of the sentence and all other words are dependent on another word in the sentence. These dependencies are generally universal and are meant to bring out cross-lingual typological parallels among the world's languages.

For examples of the relations used in the Bikol treebank, see the Annotation guidelines for Bikol.

For examples of annotated Bikol sentences, see the Bikol corpus.

Corpus Information

The annotated corpus can be found in .txt and .conllu (with and without morphological information) in the project repository.

The corpus contains extracts from the following sources:

• Various Bikol Wikipedia articles, accessed from 01/05/2017 to 08/05/2017
• Universal Declaration of Human Rights (used English translation for glosses)

Process

In the following section, I address solutions to specific problems I encountered while annotating my corpus.

Pluralization of nouns with 'mga'

To make nouns plural in Bikol, you add the word 'mga' before it.

Halion mo an mga papers sa lamesa ta pigkorehiran na iyan.

When deciding which relation to use for this scenario, my initial thought was to use the @nummod (numeric modifier) relation, because this is a separate word that modifies the quantity of a noun. However, 'mga' has no isolated meaning in Bikol and is therefore not a numeral. So, I decided to use the @det (determiner) relation. In UD's documentation of what constitutes a determiner, it says that quantity determiners (quantifiers) qualify for this relation.

Linker in Modification Structures: 'na'

The biggest challenge of this project (besides the time constraint) was deciding on the relation to use for the linking word, 'na'. In the grammar books I have, this word is simply described as a linking word in modification structures. The word is used very frequently in Bikol in a variety of different modification structures. For this word, I decided to create four language-specific relations. These are outlined below.

amod

May turista na puti nagduman sa San Jose.
There is a white tourist that visited San Jose.

This sentence demonstrates an @amod relation, in which an adjectival modifier (puti, or white) modifies the meaning of a nominal (turista, or tourist). The linking word 'na' is assigned an aux:amod relation dependent on the modifier. Language-specific relations are, by convention, composed in this manner (universal:extension). I chose to make this relation an extension of the @aux (auxiliary) relation because the linking word serves as an auxiliary word to the modifier.

For more documentation on this relation, see the aux:amod documentation.

nmod

kapitolyo na ekonomiya nin Albay
economic capital of Albay

This sentence demonstrates an @nmod relation, in which a nominal modifier (ekonomiya) acts upon another nominal (kapitolyo). The linking word 'na' is assigned an aux:nmod relation dependent on the modifier.

For more documentation on this relation, see the aux:nmod documentation.

advmod

Matuga na naman siguro an bulkan Mayon.
The Mayon volcano will surely erupt again.

This sentence demonstrates an @advmod relation, in which an adverbial modifier (naman, or again) acts upon the predicate, matuga (will erupt). The linking word 'na' is assigned an aux:advmod relation dependent on the adverbial modifier.

Note that siguro (surely), is also an adverbial modifier and also has an @advmod relation to matuga. In this case of multiple adverbial modifiers, only one linking word is necessary. Therefore, the linking word is given the aux:advmod relation to the first adverb because that is the one it is physically linking to the predicate.

For more documentation on this relation, see the aux:advmod documentation.

nummod

Ini may apat na eskalon.
This has four floors.

This sentence demonstrates a @nummod relation, in which an numeric modifier (apat, four) acts upon a nominal (eskalon, or floor). The linking word 'na' is assigned an aux:nummod relation dependent on the adjectival modifier.

Note the order here. Typically, modifiers come after the thing that they modify. However, for @nummod relations, it is typically the opposite, with the numeric modifier coming first. The linking word thus comes after the modifier in order to remain in between the modifier and the word it modifies.

For more documentation on this relation, see the aux:nummod documentation.

UD Annotation Guidelines

The final component of the project was to create annotation guidelines for the Bikol UD Treebank. This documentation would be useful to anyone who wished to contribute to the Treebank.

Again, I used the same format that is used by the Universal Dependencies Project, using the repository of Swedish guidelines as a guide.

The annotation guidelines can be found in the bcl-dep directory of the project's repository, and feature documentation on 25 of the most commonly used relations I used in my corpus, including the four language-specific relations I mentioned earlier in the Process section.

Evaluation

Parsers are typically evaluated with the LAS (Label Attachment Score) and UAS (Unlabeled Attachment Score) obtained by testing on a separate corpus. However, making an additional, separate corpus would be time consuming. To get around this, I divided the corpus into 10 roughly equal parts (10-11 sentences each). Then, I used each part as a test corpus for two parsers (with and without morphological information) which were trained on the remaining corpus.

The partial corpora, conllu and udpipe files can all be found in the bcl-eval directory of the repository. The data from all the individual tests can be found in the Appendix.

To evaluate the parsers trained on the entire corpus, I took the average of all the UAS/LAS scores obtained on the 10 test sets, ± the standard deviation. The results are displayed below.

Further Work

Further work on this project would mainly involve increasing the size of the annotated corpus. My corpus of ~100 sentences and ~1200 tokens is very small in comparison to completed UD Treebanks, due to the time constraints of the project and the nature of the material I was working with (Wikipedia articles, no glosses). The most fully developed UD treebanks have 100-200K+ tokens annotated, which would yield much, much more accurate parsers.

Appendix