Using flag diacritics for faster compilation

From LING073
Revision as of 03:05, 22 March 2023 by Jwashin1 (talk | contribs) (Created page with "`lexd` is a bit slow to compile dictionaries that make heavy use of filters. To speed this up, you can enable compiling to flag diacritics. == Implementing the change == Fir...")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

`lexd` is a bit slow to compile dictionaries that make heavy use of filters.

To speed this up, you can enable compiling to flag diacritics.

Implementing the change

First thing you want to do is edit your `Makefile.am`:

  • remove `autogen.bin` and `automorf.bin` lines from the TARGETS_COMMON list at the top
  • add `-f` as a switch to the `lexd` commmands
  • add `-F` as a switch to the `hfst-compose` command in the the `automorf.hfst` line

This will all look something like this:

 TARGETS_COMMON =                       \
        $(LANG1).automorf.hfst          \
-       $(LANG1).automorf.bin           \
        $(LANG1).automorf.att.gz        \
        $(LANG1).autogen.hfst           \
-       $(LANG1).autogen.bin            \
        $(LANG1).autogen.att.gz         \
        $(LANG1).autopgen.bin           \
        $(LANG1).rlx.bin            \
        $(LANG1).zhfst

 .deps/$(LANG1).RL.lexd.hfst: .deps/$(LANG1).RL.lexd
-       lexd $< .deps/$(LANG1).RL.lexd.att
+       lexd -f $< .deps/$(LANG1).RL.lexd.att
        hfst-txt2fst .deps/$(LANG1).RL.lexd.att -o $@
 
 .deps/$(LANG1).LR.lexd.hfst: .deps/$(LANG1).LR.lexd
-       lexd $< .deps/$(LANG1).LR.lexd.att
+       lexd -f $< .deps/$(LANG1).LR.lexd.att
        hfst-txt2fst .deps/$(LANG1).LR.lexd.att -o $@
 $(LANG1).automorf.hfst: .deps/$(LANG1).LR.hfst .deps/$(LANG1).spellrelax.hfst
-       hfst-compose -1 $< -2 .deps/$(LANG1).spellrelax.hfst | hfst-invert | hfst-fst2fst -O -o $@
+       hfst-compose -F -1 $< -2 .deps/$(LANG1).spellrelax.hfst | hfst-invert | hfst-fst2fst -O -o $@