Search code examples
pythonspeech-recognitiongstreamercmusphinxpocketsphinx

pocketsphinx setting mandarin/chinese model and dictionary problem using python

(Also asked on sourceforge.net)

I install sphinxbase and pocketsphinx from source(official cmu github, NOT from sourceforge.net) on ubuntu 18

(host: mac Pro; using vmware fusion 11; python:3.6)

I copy the demoapp.py from the offical website.

The original code runs smoothly, but it supports english only.

Then in order to adjust it to recognize Chinese, I do some alteration.

My language model is downloaded from here: https://sourceforge.net/projects/cmusphinx/files/Acoustic%20and%20Language%20Models/Mandarin/zh_broadcastnews_64000_utf8.DMP/download I've converted it to .lm.bin and .lm format.

Here is my init_gst() function ( I only modify this function with other functions unmodified)

def init_gst(self):
    """Initialize the speech components"""

    self.pipeline = gst.parse_launch('autoaudiosrc ! audioconvert !  audioresample '
                                     + '! pocketsphinx name=asr ! fakesink')

    bus = self.pipeline.get_bus()
    bus.add_signal_watch()
    bus.connect('message::element', self.element_message)

    self.pipeline.set_state(gst.State.PAUSED)
    asr = self.pipeline.get_by_name('asr');  # We previously  assigned pocketsphinx element a name asr
    asr.set_property('lm', '/Users/cindy/Documents/pythonworkspace/sphinxenv/chinese.lm')
    asr.set_property('dict', '/Users/cindy/Documents/pythonworkspace/sphinxenv/dict/chinese_dict.dict')

and here is the chinese_dict.dict file:

  • 前進 t ing zh ib
  • 後退 h ou t ui
  • 左轉 z uo zh uan
  • 右轉 y uo zh uan
  • 向左轉 x iang z uo zh uan
  • 向右轉 x iang y uo zh uan
  • 停止 t ing zh ib
  • 加速 j ia s u
  • 減速 j ian s u

The gstreamer GUI shows successfully with a textbox and a button.

However, when I click the button speak and say some chinese words, nothing appears in the textbox.

Would it be that my .lm file too old?(Just some guessing, I am not quite sure.)

Please help.

(Or if you can help me to print some error information, I'll be appreciated; the gstreamer GUI and the pycharm terminal show NOTHING about the bug at all.)

Solution

  • You need to load Chinese acoustic model with -hmm option