Difference between revisions of "Shanghainese and Mandarin Chinese"

From LING073
Jump to: navigation, search
(→‎Sentences)
(→‎Final Evaluation)
 
(14 intermediate revisions by 2 users not shown)
Line 10: Line 10:
 
*Link to Shanghainese to Mandarin Tatoeba Text: [https://tatoeba.org/en/sentences/search?from=wuu&query=&to=cmn]
 
*Link to Shanghainese to Mandarin Tatoeba Text: [https://tatoeba.org/en/sentences/search?from=wuu&query=&to=cmn]
  
−
==Shanghainese <code>-</code> → Mandarin Chinese <code>zho</code> Evaluation==
+
Initial Evaluation==
 
===Sentences===
 
===Sentences===
 
* (1) <code>wuu</code>: 我爱侬。
 
* (1) <code>wuu</code>: 我爱侬。
 
** <code>zho</code>: 我爱你。
 
** <code>zho</code>: 我爱你。
 
** <code>eng</code>: I love you.
 
** <code>eng</code>: I love you.
−
** lexical transfer output: ^我<prn><pers><p1><sg>/我<prn><pers><p1><sg>$^爱<vblex>/爱<vblex>$^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$。^.<sent>/.<sent>
+
** lexical transfer output: ^我<prn><pers><p1><sg>/我<prn><pers><p1><sg>$ ^爱<vblex>/爱<vblex>$ ^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$ 。 ^.<sent>/.<sent>$
−
** full translation output: 我爱你。
+
** full translation output: 我 爱 你 。
  
 
* (2) <code>wuu</code>: 汏手间勒勒阿里耷？
 
* (2) <code>wuu</code>: 汏手间勒勒阿里耷？
 
** <code>zho</code>: 洗手间在哪里？
 
** <code>zho</code>: 洗手间在哪里？
 
** <code>eng</code>: Where is the restroom?
 
** <code>eng</code>: Where is the restroom?
−
** lexical transfer output: 洗手间在哪里？
+
** lexical transfer output: 洗手间 在 哪里 ？
−
** full translation output: ^汏手间<n>/洗手间<n>$^勒勒<pr>/在<pr>$^阿里耷<adv>/哪里<adv>$？^.<sent>/.<sent>
+
** full translation output: ^汏手间<n>/洗手间<n>$ ^勒勒<pr>/在<pr>$ ^阿里耷<adv>/哪里<adv>$？^.<sent>/.<sent>
  
 
* (3) <code>wuu</code>: 我勿晓得。
 
* (3) <code>wuu</code>: 我勿晓得。
 
** <code>zho</code>: 我不知道。
 
** <code>zho</code>: 我不知道。
 
** <code>eng</code>: I don't know.
 
** <code>eng</code>: I don't know.
−
** lexical transfer output: ^我<prn><pers><p1><sg>/我<prn><pers><p1><sg>$^勿<adv>/不<adv>$^晓得<vblex>/知道<vblex>$。^.<sent>/.<sent>
+
** lexical transfer output: ^我<prn><pers><p1><sg>/我<prn><pers><p1><sg>$ ^勿<adv>/不<adv>$ ^晓得<vblex>/知道<vblex>$。^.<sent>/.<sent>
−
** full translation output: 我不知道。
+
** full translation output: 我 不 知道 。
  
 
* (4) <code>wuu</code>: 伊啥人啊？
 
* (4) <code>wuu</code>: 伊啥人啊？
 
** <code>zho</code>: 他是谁？
 
** <code>zho</code>: 他是谁？
 
** <code>eng</code>: Who are they? (sing.)
 
** <code>eng</code>: Who are they? (sing.)
−
** lexical transfer output: ^伊<prn><pers><p3><sg>/他<prn><pers><p3><sg><m>/她<prn><pers><p3><sg><f>/它<prn><pers><p3><sg><nt>$^啥人<prn><itg>/谁<prn>$^啊<part>/啊<qst>$？^.<sent>/.<sent>$
+
** lexical transfer output: ^伊<prn><pers><p3><sg>/他<prn><pers><p3><sg><m>/她<prn><pers><p3><sg><f>/它<prn><pers><p3><sg><nt>$ ^啥人<prn><itg>/谁<prn>$ ^啊<part>/啊<qst>$？^.<sent>/.<sent>$
−
** full translation output: 他#谁啊？
+
** full translation output: 他 #谁 啊 ？
  
 
* (5) <code>wuu</code>: 侬要吃茶还是吃咖啡啊？
 
* (5) <code>wuu</code>: 侬要吃茶还是吃咖啡啊？
 
** <code>zho</code>: 你想喝茶还是喝咖啡？
 
** <code>zho</code>: 你想喝茶还是喝咖啡？
 
** <code>eng</code>: Do you want to drink tea or coffee?
 
** <code>eng</code>: Do you want to drink tea or coffee?
−
** lexical transfer output: ^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$^要<vaux>/要<vblex>$^吃<vblex>/喝<vblex>/吃<vblex>$^茶<n>/茶<n>$^还是<cnjcoo>/还是<cnjcoo>$^吃<vblex>/喝<vblex>/吃<vblex>$^咖啡<n>/咖啡<n>$^啊<part>/啊<qst>$？^.<sent>/.<sent>
+
** lexical transfer output: ^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$ ^要<vaux>/要<vblex>$ ^吃<vblex>/喝<vblex>/吃<vblex>$ ^茶<n>/茶<n>$ ^还是<cnjcoo>/还是<cnjcoo>$ ^吃<vblex>/喝<vblex>/吃<vblex>$ ^咖啡<n>/咖啡<n>$ ^啊<part>/啊<qst>$？^.<sent>/.<sent>
−
** full translation output: 你要喝茶#还是喝咖啡啊？
+
** full translation output: 你 要 喝 茶 #还是 喝 咖啡 啊 ？
  
 
* (6) <code>wuu</code>: 侬要走啦？
 
* (6) <code>wuu</code>: 侬要走啦？
 
** <code>zho</code>: 你要走了吗？
 
** <code>zho</code>: 你要走了吗？
 
** <code>eng</code>: Are you leaving?
 
** <code>eng</code>: Are you leaving?
−
** lexical transfer output: ^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$^要<vaux>/要<vblex>$^走<vblex>/走<vblex>$^啦<part>/啦<qst>$ ^.<sent>/.<sent>
+
** lexical transfer output: ^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$ ^要<vaux>/要<vblex>$ ^走<vblex>/走<vblex>$ ^啦<part>/啦<qst>$ ^.<sent>/.<sent>
−
** full translation output: 你要走啦？
+
** full translation output: 你 要 走 啦 ？
  
 
* (7) <code>wuu</code>: 勿要紧个！
 
* (7) <code>wuu</code>: 勿要紧个！
 
** <code>zho</code>: 没关系。
 
** <code>zho</code>: 没关系。
 
** <code>eng</code>: Don't worry.
 
** <code>eng</code>: Don't worry.
−
** lexical transfer output: ^勿要紧<ij>/没关系<ij>$^个<part>/的<pr>$！^.<sent>/.<sent>
+
** lexical transfer output: ^勿要紧<ij>/没关系<ij>$ ^个<part>/的<pr>$！^.<sent>/.<sent>
−
** full translation output: #没关系的！
+
** full translation output: #没关系 的 ！
  
−
* (8) <code>wuu</code>: 我倒勿大晓得。
+
* (8) <code>wuu</code>: 我一只也勿要。
−
** <code>zho</code>: 我可不太知道。
+
** <code>zho</code>: 我一个也不要。
−
** <code>eng</code>: I'm not sure.
+
** <code>eng</code>: I don't want even one.
−
** lexical transfer output:  
+
** lexical transfer output: ^我<prn><pers><p1><sg>/我<prn><pers><p1><sg>$ ^一<num>/一<num>$ ^只<cls>/个<num>/只<num>$ ^也<adv>/也<adv>$ ^勿<adv>/不<adv>$ ^要<vaux>/要<vblex>$。^.<sent>/.<sent>
−
** full translation output:  
+
** full translation output: 我 一 个 也 不 要 。
  
 
* (9) <code>wuu</code>: 今朝天好。
 
* (9) <code>wuu</code>: 今朝天好。
 
** <code>zho</code>: 今天好天气。
 
** <code>zho</code>: 今天好天气。
 
** <code>eng</code>: The weather's nice today.
 
** <code>eng</code>: The weather's nice today.
−
** lexical transfer output:  
+
** lexical transfer output: ^今朝<n>/今天<n>$ ^天<n>/天<n>$ ^好<adj>/好<adj>$。^.<sent>/.<sent>
−
** full translation output:  
+
** full translation output: 今天 天 好 。
  
 
* (10) <code>wuu</code>: 一、二、三、四、五、六、七、八、九、十。
 
* (10) <code>wuu</code>: 一、二、三、四、五、六、七、八、九、十。
 
** <code>zho</code>: 一、二、三、四、五、六、七、八、九、十。
 
** <code>zho</code>: 一、二、三、四、五、六、七、八、九、十。
 
** <code>eng</code>: One, two, three, four, five, six, seven, right, nine, ten.
 
** <code>eng</code>: One, two, three, four, five, six, seven, right, nine, ten.
−
** lexical transfer output:  
+
** lexical transfer output: ^一<num>/一<num>$、^二<num>/二<num>$、^三<num>/三<num>$、^四<num>/四<num>$、^五<num>/五<num>$、^六<num>/六<num>$、^七<num>/七<num>$、^八<num>/八<num>$、^九<num>/九<num>$、^十<num>/十<num>$。^.<sent>/.<sent>
−
** full translation output:  
+
** full translation output: 一、二、三、四、五、六、七、八、九、十。
  
 +
===Polished RBMT System===
  
 +
==Additions==
 +
* 2 new lexical selection rules
 +
* 3 new disambiguation rules
 +
* 2 new transfer rules
 +
 +
==Final Evaluation==
 +
*Shanghainese Transducer
 +
**Precision: Coverage hfst is still broken
 +
**Recall: Coverage hfst is still broken
 +
**Number of words in Large Corpus: Coverage hfst is still broken
 +
**Number of stems in Transducer: Coverage hfst is still broken
 +
 +
*<code>wuu</code> → <code>zho</code>
 +
**WER: apertium-eval-translator-line does not work on space-less orthography
 +
**PER: apertium-eval-translator-line does not work on space-less orthography
 +
**Proportion of stems translated correctly in Longer Corpus: apertium-eval-translator-line does not work on space-less orthography
 +
**Trimmed coverage over longer and large corpora: apertium-eval-translator-line does not work on space-less orthography
 +
**The number of tokens in longer and large corpora: apertium-eval-translator-line does not work on space-less orthography
  
 
[[Category:Shanghainese]]
 
[[Category:Shanghainese]]
 
[[Category:Sp25_TranslationPairs]]
 
[[Category:Sp25_TranslationPairs]]

Latest revision as of 16:32, 1 May 2025

Resources for machine translation between Shanghainese and Mandarin Chinese

External Resources

  • Link to repository language pair:[2]
  • Link to Shanghainese repository transducer:[3]
  • Link to Mandarin Chinese repository transducer:[4]
  • Link to Shanghainese to Mandarin Tatoeba Text: [5]

Initial Evaluation==

Sentences

  • (1) wuu: 我爱侬。
    • zho: 我爱你。
    • eng: I love you.
    • lexical transfer output: ^我<prn><pers><p1><sg>/我<prn><pers><p1><sg>$ ^爱<vblex>/爱<vblex>$ ^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$ 。 ^.<sent>/.<sent>$
    • full translation output: 我 爱 你 。
  • (2) wuu: 汏手间勒勒阿里耷？
    • zho: 洗手间在哪里？
    • eng: Where is the restroom?
    • lexical transfer output: 洗手间 在 哪里 ？
    • full translation output: ^汏手间<n>/洗手间<n>$ ^勒勒<pr>/在<pr>$ ^阿里耷<adv>/哪里<adv>$？^.<sent>/.<sent>
  • (3) wuu: 我勿晓得。
    • zho: 我不知道。
    • eng: I don't know.
    • lexical transfer output: ^我<prn><pers><p1><sg>/我<prn><pers><p1><sg>$ ^勿<adv>/不<adv>$ ^晓得<vblex>/知道<vblex>$。^.<sent>/.<sent>
    • full translation output: 我 不 知道 。
  • (4) wuu: 伊啥人啊？
    • zho: 他是谁？
    • eng: Who are they? (sing.)
    • lexical transfer output: ^伊<prn><pers><p3><sg>/他<prn><pers><p3><sg><m>/她<prn><pers><p3><sg><f>/它<prn><pers><p3><sg><nt>$ ^啥人<prn><itg>/谁<prn>$ ^啊<part>/啊<qst>$？^.<sent>/.<sent>$
    • full translation output: 他 #谁 啊 ？
  • (5) wuu: 侬要吃茶还是吃咖啡啊？
    • zho: 你想喝茶还是喝咖啡？
    • eng: Do you want to drink tea or coffee?
    • lexical transfer output: ^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$ ^要<vaux>/要<vblex>$ ^吃<vblex>/喝<vblex>/吃<vblex>$ ^茶<n>/茶<n>$ ^还是<cnjcoo>/还是<cnjcoo>$ ^吃<vblex>/喝<vblex>/吃<vblex>$ ^咖啡<n>/咖啡<n>$ ^啊<part>/啊<qst>$？^.<sent>/.<sent>
    • full translation output: 你 要 喝 茶 #还是 喝 咖啡 啊 ？
  • (6) wuu: 侬要走啦？
    • zho: 你要走了吗？
    • eng: Are you leaving?
    • lexical transfer output: ^侬<prn><pers><p2><sg>/你<prn><pers><p2><sg>$ ^要<vaux>/要<vblex>$ ^走<vblex>/走<vblex>$ ^啦<part>/啦<qst>$ ^.<sent>/.<sent>
    • full translation output: 你 要 走 啦 ？
  • (7) wuu: 勿要紧个！
    • zho: 没关系。
    • eng: Don't worry.
    • lexical transfer output: ^勿要紧<ij>/没关系<ij>$ ^个<part>/的<pr>$！^.<sent>/.<sent>
    • full translation output: #没关系 的 ！
  • (8) wuu: 我一只也勿要。
    • zho: 我一个也不要。
    • eng: I don't want even one.
    • lexical transfer output: ^我<prn><pers><p1><sg>/我<prn><pers><p1><sg>$ ^一<num>/一<num>$ ^只<cls>/个<num>/只<num>$ ^也<adv>/也<adv>$ ^勿<adv>/不<adv>$ ^要<vaux>/要<vblex>$。^.<sent>/.<sent>
    • full translation output: 我 一 个 也 不 要 。
  • (9) wuu: 今朝天好。
    • zho: 今天好天气。
    • eng: The weather's nice today.
    • lexical transfer output: ^今朝<n>/今天<n>$ ^天<n>/天<n>$ ^好<adj>/好<adj>$。^.<sent>/.<sent>
    • full translation output: 今天 天 好 。
  • (10) wuu: 一、二、三、四、五、六、七、八、九、十。
    • zho: 一、二、三、四、五、六、七、八、九、十。
    • eng: One, two, three, four, five, six, seven, right, nine, ten.
    • lexical transfer output: ^一<num>/一<num>$、^二<num>/二<num>$、^三<num>/三<num>$、^四<num>/四<num>$、^五<num>/五<num>$、^六<num>/六<num>$、^七<num>/七<num>$、^八<num>/八<num>$、^九<num>/九<num>$、^十<num>/十<num>$。^.<sent>/.<sent>
    • full translation output: 一、二、三、四、五、六、七、八、九、十。

Polished RBMT System

Additions

  • 2 new lexical selection rules
  • 3 new disambiguation rules
  • 2 new transfer rules

Final Evaluation

  • Shanghainese Transducer
    • Precision: Coverage hfst is still broken
    • Recall: Coverage hfst is still broken
    • Number of words in Large Corpus: Coverage hfst is still broken
    • Number of stems in Transducer: Coverage hfst is still broken
  • wuu → zho
    • WER: apertium-eval-translator-line does not work on space-less orthography
    • PER: apertium-eval-translator-line does not work on space-less orthography
    • Proportion of stems translated correctly in Longer Corpus: apertium-eval-translator-line does not work on space-less orthography
    • Trimmed coverage over longer and large corpora: apertium-eval-translator-line does not work on space-less orthography
    • The number of tokens in longer and large corpora: apertium-eval-translator-line does not work on space-less orthography
Retrieved from "http://wikis.swarthmore.edu/ling073/index.php?title=Shanghainese_and_Mandarin_Chinese&oldid=25952"