Final projects/Transducer - Revision history

Jwashin1 at 19:02, 8 May 2017

2017-05-08T19:02:25Z

Jwashin1 at 17:47, 4 May 2017

2017-05-04T17:47:45Z

Jwashin1 at 17:47, 4 May 2017

2017-05-04T17:47:30Z

Jwashin1 at 17:46, 4 May 2017

2017-05-04T17:46:22Z

Jwashin1 at 05:05, 4 May 2017

2017-05-04T05:05:13Z

Jwashin1: Jwashin1 moved page Final project/Transducer to Final projects/Transducer: oops

2017-05-02T20:47:24Z

Jwashin1 moved page Final project/Transducer to Final projects/Transducer: oops

Jwashin1 at 20:46, 2 May 2017

2017-05-02T20:46:11Z

Jwashin1 at 20:43, 2 May 2017

2017-05-02T20:43:38Z

Jwashin1: Created page with "* You can contribute straight to Apertium: ask for commit access. * To get random forms from your corpora for precision/recall measures: *#

ca..."

2017-05-02T20:42:07Z

Created page with "* You can contribute straight to Apertium: ask for commit access. * To get random forms from your corpora for precision/recall measures: *# <pre>ca..."

New page

* You can contribute straight to Apertium: ask for [[:apertium:Commit_access|commit access]].
* To get random forms from your corpora for precision/recall measures:
*# <pre>cat corpus1.txt corpus2.txt | apertium -d . abc-tagger | sed -r 's/\$\s*\^/\$\n\^/g' | cut -f1 -d'/' | sed 's/[\^\s\t]*//' | sort -Ru | head -n 1200 > words.txt</pre>
*# <pre>cat words.txt | apertium -d . abc-disam</pre>
*# Then do this again: [[Polished RBMT system#Hand-annotating corpora]]

@@ Line 1: / Line 1: @@
 For any of the transducer projects:
 * You can contribute straight to Apertium: ask for [[:apertium:Commit_access|commit access]].
-* To get random forms from your corpora for precision/recall measures:
+* To get random forms from your corpora for '''precision/recall''' measures:
 *# get 1200 random words: <pre>cat corpus1.txt corpus2.txt | apertium -d . abc-tagger | sed -r 's/\$\s*\^/\$\n\^/g' | cut -f1 -d'/' | sed 's/[\^\s\t]*//' | sort -Ru | head -n 1200 > words.txt</pre>
 *#: This might not work as expected with your tagger (i.e., you may not get one form per line—there might be some non-form symbols included), so make sure your call to cg-conv in your tagger mode in `modes.xml` doesn't have `-n` and recompile.

@@ Line 6: / Line 6: @@
 *# get analyses from transducer: <pre>cat words.txt | apertium -d . abc-disam</pre>
 *# Then do this again: [[Polished RBMT system#Hand-annotating corpora]]
-* Use the [[:apertium:Wikipedia extractor|wikipedia extractor]] script to get the entire contents of wikipedia in your language.
+* Use the [[:apertium:Wikipedia Extractor|wikipedia extractor]] script to get the entire contents of wikipedia in your language (if it exists).

@@ Line 6: / Line 6: @@
 *# get analyses from transducer: <pre>cat words.txt | apertium -d . abc-disam</pre>
 *# Then do this again: [[Polished RBMT system#Hand-annotating corpora]]
-* Use the [[:apertium:Wikipedia dumps|wikipedia extractor]] script to get the entire contents of wikipedia in your language.
+* Use the [[:apertium:Wikipedia extractor|wikipedia extractor]] script to get the entire contents of wikipedia in your language.

← Older revision		Revision as of 17:46, 4 May 2017
Line 6:		Line 6:
	*# get analyses from transducer: <pre>cat words.txt \| apertium -d . abc-disam</pre>		*# get analyses from transducer: <pre>cat words.txt \| apertium -d . abc-disam</pre>
	*# Then do this again: [[Polished RBMT system#Hand-annotating corpora]]		*# Then do this again: [[Polished RBMT system#Hand-annotating corpora]]
		+	* Use the [[:apertium:Wikipedia dumps\|wikipedia extractor]] script to get the entire contents of wikipedia in your language.

@@ Line 3: / Line 3: @@
 * To get random forms from your corpora for precision/recall measures:
 *# get 1200 random words: <pre>cat corpus1.txt corpus2.txt | apertium -d . abc-tagger | sed -r 's/\$\s*\^/\$\n\^/g' | cut -f1 -d'/' | sed 's/[\^\s\t]*//' | sort -Ru | head -n 1200 > words.txt</pre>
 *# get analyses from transducer: <pre>cat words.txt | apertium -d . abc-disam</pre>
 *# Then do this again: [[Polished RBMT system#Hand-annotating corpora]]

← Older revision	Revision as of 20:47, 2 May 2017
(No difference)

← Older revision		Revision as of 20:46, 2 May 2017
Line 1:		Line 1:
		+	For any of the transducer projects:
	* You can contribute straight to Apertium: ask for [[:apertium:Commit_access\|commit access]].		* You can contribute straight to Apertium: ask for [[:apertium:Commit_access\|commit access]].
	* To get random forms from your corpora for precision/recall measures:		* To get random forms from your corpora for precision/recall measures: