Difference between revisions of "User:Lchien1/Final project"
(→What I Did) |
(→What I Did) |
||
| Line 5: | Line 5: | ||
The first step was to assemble a list of morphemes. Luckily, our lexd file already had lots of morphemes listed out for me to use. Entries in lexd file look something like this: | The first step was to assemble a list of morphemes. Luckily, our lexd file already had lots of morphemes listed out for me to use. Entries in lexd file look something like this: | ||
| + | aku:aku <pers><sg><p1>: | ||
| + | isu:isu <pers><sg><p2>: | ||
| + | ingra:ingra <pers><sg><p3>: | ||
| + | ira:ira <pers><sg><p3>: # Possible fix to pronoun spelling issue | ||
| + | ita:ita <pers><pl><p1><incl>: | ||
| + | ami:ami <pers><pl><p1><excl>: | ||
| + | amu:amu <pers><pl><p2>: | ||
| + | angra:angra <pers><pl><p3>: | ||
| + | Each line has multiple instances of the morpheme, several tags, and sometimes even comments. This is unfortunately not so useful for me. I need to be able to identify the actual Amis roots, the tags, and get rid of any comments. I wrote a Python program, text_formatter.py, which takes care of this for me. The resulting text looks something like this: | ||
| + | aku,pers,sg,p1 | ||
| + | isu,pers,sg,p2 | ||
| + | ingra,pers,sg,p3 | ||
| + | ira,pers,sg,p3 | ||
| + | ita,pers,pl,p1,incl | ||
| + | ami,pers,pl,p1,excl | ||
| + | amu,pers,pl,p2 | ||
| + | angra,pers,pl,p3 | ||
| + | We've gotten rid of any extra white space, extra characters, comments, and we ended up with a more predictable format where it is clear that the Amis text comes first, followed by comma separated tags. This format will be much easier to use to populate a Python data structure. | ||
| − | + | Next, I created a data structure in Python that would be able to organize this text. My data structure separately stores the bit of Amis at the front, a list of tags, and it keeps track of the root so we can add prefixes and suffixes. I stored several lists of these data structures as variables in a dictionary, with the keys being category of morphemes, such as noun roots, verb tenses, or case markers. Finally, I implemented patterns to stitch morphemes together, while also concatenating any associated tags. The basic architecture of a pattern is as follows: | |
| − | + | #List the desired roots on their own. | |
| − | + | #Identify which types of morphemes are optional and compile a list of them. | |
| − | + | #For each of these options, append to all items in the first list a copy of itself with the new morpheme attached. | |
| − | |||
| − | |||
| − | |||
| − | |||
=Links to my Code= | =Links to my Code= | ||
Revision as of 07:16, 4 May 2023
What I Did
Throughout this semester, I was introduced to several tools we used in class to develop language technologies like hfst, lexd, and Apertium. I did not understand what was happening behind the simple files we created in class and how they were converted to output, and using them felt difficult or not intuitive. I decided to use Python to build my own analyser for Amis, the language I worked with all semester long. By building a tool myself in Python, I have full control over all parts of the tool and its functionality.
I wanted a program that, like the .lexd file I created earlier in the semester, would use Amis grammar patterns to put together morphemes to form words and show tags.
The first step was to assemble a list of morphemes. Luckily, our lexd file already had lots of morphemes listed out for me to use. Entries in lexd file look something like this:
aku:aku <pers><sg><p1>: isu:isu <pers><sg><p2>: ingra:ingra <pers><sg><p3>: ira:ira <pers><sg><p3>: # Possible fix to pronoun spelling issue ita:ita <pers><pl><p1><incl>: ami:ami <pers><pl><p1><excl>: amu:amu <pers><pl><p2>: angra:angra <pers><pl><p3>:
Each line has multiple instances of the morpheme, several tags, and sometimes even comments. This is unfortunately not so useful for me. I need to be able to identify the actual Amis roots, the tags, and get rid of any comments. I wrote a Python program, text_formatter.py, which takes care of this for me. The resulting text looks something like this:
aku,pers,sg,p1 isu,pers,sg,p2 ingra,pers,sg,p3 ira,pers,sg,p3 ita,pers,pl,p1,incl ami,pers,pl,p1,excl amu,pers,pl,p2 angra,pers,pl,p3
We've gotten rid of any extra white space, extra characters, comments, and we ended up with a more predictable format where it is clear that the Amis text comes first, followed by comma separated tags. This format will be much easier to use to populate a Python data structure.
Next, I created a data structure in Python that would be able to organize this text. My data structure separately stores the bit of Amis at the front, a list of tags, and it keeps track of the root so we can add prefixes and suffixes. I stored several lists of these data structures as variables in a dictionary, with the keys being category of morphemes, such as noun roots, verb tenses, or case markers. Finally, I implemented patterns to stitch morphemes together, while also concatenating any associated tags. The basic architecture of a pattern is as follows:
- List the desired roots on their own.
- Identify which types of morphemes are optional and compile a list of them.
- For each of these options, append to all items in the first list a copy of itself with the new morpheme attached.
Links to my Code
https://github.com/lchien1/chien-ami-analysis
