Latest revision as of 16:49, 17 April 2025

Pre-evaluation

Statistics about input files

Number of words in reference: 17

Number of words in test: 109

Number of unknown words (marked with a star) in test: 38

Percentage of unknown words: 34.86 %

Results when removing unknown-word marks (stars)

Edit distance: 107

Word error rate (WER): 629.41 %

Number of position-independent correct words: 2

Position-independent word error rate (PER): 629.41 %

The evaluator didn't work properly without manual tokenisation, as such, the WER is way too big. We didn't realise this until we had finished the evaluation.

Structural Transfer Rules

Direct/Indirect Object Ordering

Shanghainese places the direct object before the indirect object while Mandarin places the indirect object before the direct object.

• (wuu) 拔一把刀其。 → (zho) 给他一把刀。

  (wuu) 拔<vblex> 一<num> 把<cls> 刀<n> 其<prn><pers><p3><sg> → (zho) 给<vblex> 他<prn><pers><p3><sg><m> 一<num> 把<num> 刀<n>

• (wuu) 拔粥吾拉吃。 → (zho) 给我们粥吃。

  (wuu) 拔<vblex> 粥<n> 吾拉<prn><pers><p1><pl> 吃<vblex> → (zho) 给<vblex> 我们<prn><pers><p1><pl> 粥<n> 吃<vblex>

Post-evaluation

Statistics about input files

Number of words in reference: 80

Number of words in test: 91

Number of unknown words (marked with a star) in test: 31

Percentage of unknown words: 34.07 %

Results when removing unknown-word marks (stars)

Edit distance: 57

Word error rate (WER): 71.25 %

Number of position-independent correct words: 33

Position-independent word error rate (PER): 72.50 %