Revision as of 03:45, 9 May 2017

This page provides an overview and an evaluation of Korean Finite State Transducer I developed.

Motivation

For the final project, I decided to work on Finite State Transducer for Korean. Although there are several open-sourced POS tagger(Kkoma, HanNanum, and Komoran), none of them are purely rule-based; HanNanum and Komoran uses Hidden Markov Models(statistical method for making inferences about sequential data), and Kkoma requires the usage of a custom relational database.

Although the techniques employed in the aforementioned systems are crucial in achieving state-of-the-art performance, it has certain drawbacks. First, even after achieving extremely high performance, it i

Overview

In this implementation of Korean POS analyzer, I was able to analyze most of noun morphology correctly. However, as there were multiple level of suffixes that attaches to verbs in Korean, the analysis and construction of Verb alteration was not easy.

Evaluation

Coverage of corpora

Precision and Recall

Code for the transducer

Korean Finite State Transducer