Difference between revisions of "Japhug/Transducer"

From LING073
Jump to: navigation, search
 
(32 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
This is the page for our [https://github.swarthmore.edu/Ling073-sp23/ling073-jya Japhug transducer].
+
Git page for our [https://github.swarthmore.edu/Ling073-sp23/ling073-jya Japhug transducer].
  
 +
Git page for our [https://github.swarthmore.edu/Ling073-sp23/ling073-jya-corpus corpus].
 +
 +
[https://wikis.swarthmore.edu/ling073/Category:Japhug Wiki page] to the list of resources.
 +
=Analyser evaluation=
  
 
==Evaluation==
 
==Evaluation==
  
 
===Total number of stems in transducer===
 
===Total number of stems in transducer===
−
 
+
<pre>
−
Lexicons: 21
+
Lexicons: 22
−
 
+
Lexicon entries: 99
−
Lexicon entries: 96
+
Patterns: 10
−
 
+
Pattern entries: 22
−
Patterns: 9
 
−
 
 
−
Pattern entries: 21
 
−
 
 
  
 
Counts for individual lexicons:
 
Counts for individual lexicons:
−
 
 
Punctuation: 22
 
Punctuation: 22
−
 
 
V-Negative: 2
 
V-Negative: 2
−
 
 
N-Possessive: 12
 
N-Possessive: 12
−
 
 
V-Participles: 3
 
V-Participles: 3
−
 
 
V-Stems: 21
 
V-Stems: 21
−
 
 
V-Transitivity: 2
 
V-Transitivity: 2
−
 
 
V-Causative: 1
 
V-Causative: 1
−
 
 
V-Reflexive: 1
 
V-Reflexive: 1
−
 
 
V-Passive: 1
 
V-Passive: 1
−
 
+
N-Stems: 13
−
N-Stems: 11
+
N-Collective: 1
−
 
 
 
Gender: 2
 
Gender: 2
 +
Numeral: 10
 +
All anonymous lexicons: 8
 +
</pre>
  
−
N-Collective: 1
+
===Current coverage===
  
−
Numeral: 10
+
Updated on Mar 20, 2023
 +
We wrote a script and obtained over 6000 word roots from the Japhug dictionary by Guillaume Jacques [https://hal.inria.fr/halshs-01244860/]
 +
As we added the roots to the lexd file, the coverage becomes nearly 75%.
  
−
All anonymous lexicons: 7
+
<pre>
 +
coverage: 2500 / 3341 (~0.74827895839568991320)
 +
remaining unknown forms: 841
 +
ambiguity: 3924 / 3341 (~1.17449865309787488776)
  
  
−
===Current coverage over your combined corpus:===
+
</pre>
  
 
===The current list of top unknown words returned by aq-covtest===
 
===The current list of top unknown words returned by aq-covtest===
−
 
+
<pre>
−
===Number of tests that pass in each yaml file===
+
TOP UNKNOWN WORDS:
 +
    27 ^pɣɤtɕɯ/*pɣɤtɕɯ$
 +
    10 ^tɕu/*tɕu$
 +
      8 ^rɟɤlpu/*rɟɤlpu$
 +
      8 ^rcanɯ/*rcanɯ$
 +
      7 ^ɯrkɯ/*ɯrkɯ$
 +
      7 ^tɤwa/*tɤwa$
 +
      7 ^tɤsno/*tɤsno$
 +
      7 ^tɕɤn/*tɕɤn$
 +
      7 ^ki/*ki$
 +
      6 ^ɯpi/*ɯpi$
 +
      6 ^ɣɯ/*ɣɯ$
 +
      6 ^tɤtɕɯ/*tɤtɕɯ$
 +
      6 ^qhe/*qhe$
 +
      6 ^pɯme/*pɯme$
 +
      6 ^pjɤɕti/*pjɤɕti$
 +
      6 ^pjɤŋu/*pjɤŋu$
 +
      5 ^ɯloʁ/*ɯloʁ$
 +
      5 ^tɯɣjɤn/*tɯɣjɤn$
 +
      5 ^tonɯrdoʁ/*tonɯrdoʁ$
 +
      5 ^pɯŋu/*pɯŋu$
 +
</pre>
  
 
==Notes==
 
==Notes==
−
Tests that completely fail:   
+
'''Tests that completely fail:  '''
  
 
*analysis and generation (Irrelevant tests- they are the "house/houses" example by Jonathan)
 
*analysis and generation (Irrelevant tests- they are the "house/houses" example by Jonathan)
Line 64: Line 83:
  
  
−
Tests that partially fail (We're able to generate all the gold standards in our terminal. Not sure why failing):  
+
'''Tests that partially fail''' (We need to check that we're able to generate all the gold standards in our terminal. Not sure why tests below failing):  
  
 
*Possessive prefixes on nouns-gen (2/25 test not passing) and morph (2/20 test not passing)  
 
*Possessive prefixes on nouns-gen (2/25 test not passing) and morph (2/20 test not passing)  
Line 73: Line 92:
  
 
*Passive-morph (1/3 test not passing)  
 
*Passive-morph (1/3 test not passing)  
 +
 
    
 
    
 +
'''Additional top word(s) added: '''
 +
*lɯlu (We already have lɯlu as 'lɯlu:lɯlɤ', the right part is the sandhi. But to improve coverage, we added 'lɯlu:lɯlu')
 +
 +
*βʑɯ ("mouse")
 +
 +
*tɕendɤre ("then")
 +
 +
*ɕti (affirmative "be")
 +
 +
 +
'''Resulting change in coverage:'''
 +
 +
Coverage went from 24.23% to 31.26%
 +
 +
=Generator evaluation=
 +
 +
==Initial evaluation of morphological generation==
 +
 +
''includes commonwords, excludes first two irreverent tests''
 +
 +
number of passing morphological analysis tests: 16 
 +
 +
number of failing morphological analysis tests: 5
 +
 +
number of gold matches for the generation test: 43
 +
 +
'''Current coverage''': 31.26%
 +
 +
==Final evaluation of morphological generation==
 +
 +
number of passing morphological analysis tests: 17
 +
 +
number of failing morphological analysis tests: 4
 +
 +
number of twol rules that you added: 6
 +
 +
'''New coverage number''': 31.26%
 +
 +
==Precision and Recall==
 +
 +
Precision: 100% (so 100% of the returned analyses were correct)
  
 +
Recall: 65.09% (only 65.09% of the correct analyses were returned)
  
 +
We ran <code>precisionRecall jya.eval.test.txt jya.eval.gold.txt</code>.
  
  
 
[[Category:Sp23_Transducers]][[Category:Japhug]][[Category:Transducers]]
 
[[Category:Sp23_Transducers]][[Category:Japhug]][[Category:Transducers]]

Latest revision as of 23:36, 8 May 2023

Git page for our Japhug transducer.

Git page for our corpus.

Wiki page to the list of resources.

Analyser evaluation

Evaluation

Total number of stems in transducer

Lexicons: 22
Lexicon entries: 99
Patterns: 10
Pattern entries: 22

Counts for individual lexicons:
Punctuation: 22
V-Negative: 2
N-Possessive: 12
V-Participles: 3
V-Stems: 21
V-Transitivity: 2
V-Causative: 1
V-Reflexive: 1
V-Passive: 1
N-Stems: 13
N-Collective: 1
Gender: 2
Numeral: 10
All anonymous lexicons: 8

Current coverage

Updated on Mar 20, 2023 We wrote a script and obtained over 6000 word roots from the Japhug dictionary by Guillaume Jacques [1] As we added the roots to the lexd file, the coverage becomes nearly 75%. 

coverage: 2500 / 3341 (~0.74827895839568991320)
remaining unknown forms: 841
ambiguity: 3924 / 3341 (~1.17449865309787488776)

The current list of top unknown words returned by aq-covtest

TOP UNKNOWN WORDS:
     27 ^pɣɤtɕɯ/*pɣɤtɕɯ$
     10 ^tɕu/*tɕu$
      8 ^rɟɤlpu/*rɟɤlpu$
      8 ^rcanɯ/*rcanɯ$
      7 ^ɯrkɯ/*ɯrkɯ$
      7 ^tɤwa/*tɤwa$
      7 ^tɤsno/*tɤsno$
      7 ^tɕɤn/*tɕɤn$
      7 ^ki/*ki$
      6 ^ɯpi/*ɯpi$
      6 ^ɣɯ/*ɣɯ$
      6 ^tɤtɕɯ/*tɤtɕɯ$
      6 ^qhe/*qhe$
      6 ^pɯme/*pɯme$
      6 ^pjɤɕti/*pjɤɕti$
      6 ^pjɤŋu/*pjɤŋu$
      5 ^ɯloʁ/*ɯloʁ$
      5 ^tɯɣjɤn/*tɯɣjɤn$
      5 ^tonɯrdoʁ/*tonɯrdoʁ$
      5 ^pɯŋu/*pɯŋu$

Notes

Tests that completely fail:

  • analysis and generation (Irrelevant tests- they are the "house/houses" example by Jonathan)
  • Reduplicated Collective-gen and morph (we avoided reduplication and sandhi for this lab)
  • Gerund-gen and morph (gerund contains partial reduplication, so we didn't code it)
  • Sigmatic Causative-gen (0/11 test match gold)


Tests that partially fail (We need to check that we're able to generate all the gold standards in our terminal. Not sure why tests below failing):

  • Possessive prefixes on nouns-gen (2/25 test not passing) and morph (2/20 test not passing)
  • Negation on Verbs-gen (2/4 test do not match gold)
  • Sigmatic Causative-morph (2/11 test match gold)
  • Passive-morph (1/3 test not passing)


Additional top word(s) added:

  • lɯlu (We already have lɯlu as 'lɯlu:lɯlɤ', the right part is the sandhi. But to improve coverage, we added 'lɯlu:lɯlu')
  • βʑɯ ("mouse")
  • tɕendɤre ("then")
  • ɕti (affirmative "be")


Resulting change in coverage:

Coverage went from 24.23% to 31.26%

Generator evaluation

Initial evaluation of morphological generation

includes commonwords, excludes first two irreverent tests

number of passing morphological analysis tests: 16

number of failing morphological analysis tests: 5

number of gold matches for the generation test: 43

Current coverage: 31.26%

Final evaluation of morphological generation

number of passing morphological analysis tests: 17

number of failing morphological analysis tests: 4

number of twol rules that you added: 6

New coverage number: 31.26%

Precision and Recall

Precision: 100% (so 100% of the returned analyses were correct)

Recall: 65.09% (only 65.09% of the correct analyses were returned)

We ran precisionRecall jya.eval.test.txt jya.eval.gold.txt.

Retrieved from "http://wikis.swarthmore.edu/ling073/index.php?title=Japhug/Transducer&oldid=22367"