User:Eortici1/Final project

This page documents my final project, a larger UD corpus and annotation guidelines for Bikol.

Issues

• links
• mga

Re-evaluate NA !!!

• aux:amod, aux:nmod, aux:advmod, aux:nummod
• check @dep and anything labeled !*

Corpus Information

Corpus Size

Sentences	Forms
103	1163

This corpus contains extracts from the following sources:

• Universal Declaration of Human Rights (used English translation for glosses)
• Various Bikol Wikipedia articles, accessed from 01/05/2017 to 08/05/2017

Evaluation

Parsers are typically evaluated with the LAS (Label Attachment Score) and UAS (Unlabeled Attachment Score) obtained by testing on a separate corpus. However, making an additional, separate corpus would be time consuming. To get around this, I divided the corpus into 10 roughly equal parts (10-11 sentences each). Then, I used each part as a test corpus for two parsers (with and without morphological information) which were trained on the remaining corpus.

The partial corpora, conllu and udpipe files can all be found in the bcl-eval directory of the repository. The data from all the individual tests can be found in the Appendix of this Wiki page.

To evaluate the parsers trained on the entire corpus, I took the average of all the UAS/LAS scores obtained on the 10 test sets, ± the standard deviation. The results are displayed below.

Evaluation of bcl-withmorph.udpipe

UAS	LAS
60.87 ± 5.67	50.00 ± 4.22

Evaluation of bcl-nomorph.udpipe

UAS	LAS
50.51 ± 5.53	36.01 ± 6.59

Appendix

Evaluation

UD Parser Evaluation: Tests on 10 Subsets of Corpus

	UAS with morphology	LAS with morphology	UAS without morphology	LAS without morphology
Test 1	55.37	45.45	48.76	31.40
Test 2	56.34	46.48	50.70	38.03
Test 3	64.29	54.46	57.14	43.75
Test 4	65.00	49.17	53.33	36.67
Test 5	56.44	47.52	40.59	23.76
Test 6	65.52	52.41	56.55	38.62
Test 7	52.89	44.63	46.28	31.40
Test 8	57.45	48.94	45.74	34.04
Test 9	67.65	53.92	49.02	35.29
Test 10	67.77	57.02	57.02	47.11