I want to develop an speech recognizer in android. I've used this thread and this video to use speech recognition in android device.
Here is my code :
MainActivity.java:
package com.example.pocket_sphinx;
import edu.cmu.pocketsphinx.Hypothesis;
import edu.cmu.pocketsphinx.RecognitionListener;
import android.os.Bundle;
import android.app.Activity;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import static android.widget.Toast.makeText;
import static edu.cmu.pocketsphinx.SpeechRecognizerSetup.defaultSetup;
import android.os.AsyncTask;
import android.util.Log;
import android.widget.TextView;
import android.widget.Toast;
import edu.cmu.pocketsphinx.Assets;
import edu.cmu.pocketsphinx.SpeechRecognizer;
public class MainActivity extends Activity implements RecognitionListener {
private static final String KWS_SEARCH = "wakeup";
private static final String DICTATION_SEARCH = "digits";
private static final String KEYPHRASE = "oh mighty computer";
private SpeechRecognizer recognizer;
private HashMap<String, Integer> captions;
@Override
public void onCreate(Bundle state) {
super.onCreate(state);
// Prepare the data for UI
captions = new HashMap<String, Integer>();
setContentView(R.layout.activity_main);
((TextView) findViewById(R.id.caption_text))
.setText("Preparing the recognizer");
// Recognizer initialization is a time-consuming and it involves IO,
// so we execute it in async task
new AsyncTask<Void, Void, Exception>() {
@Override
protected Exception doInBackground(Void... params) {
try {
Assets assets = new Assets(MainActivity.this);
File assetDir = assets.syncAssets();
setupRecognizer(assetDir);
recognizer.startListening(KWS_SEARCH);
} catch (IOException e) {
return e;
}
return null;
}
@Override
protected void onPostExecute(Exception result) {
if (result != null) {
((TextView) findViewById(R.id.caption_text))
.setText("Failed to init recognizer " + result);
} else {
switchSearch(KWS_SEARCH);
}
}
}.execute();
}
@Override
public void onPartialResult(Hypothesis hypothesis) {
String text = hypothesis.getHypstr();
Log.d("Spoken text",text);
if (text.equals(KEYPHRASE))
switchSearch(DICTATION_SEARCH);
else
((TextView) findViewById(R.id.result_text)).setText(text);
}
@Override
public void onResult(Hypothesis hypothesis) {
((TextView) findViewById(R.id.result_text)).setText("");
if (hypothesis != null) {
String text = hypothesis.getHypstr();
makeText(getApplicationContext(), text, Toast.LENGTH_SHORT).show();
}
}
@Override
public void onBeginningOfSpeech() {
}
@Override
public void onEndOfSpeech() {
Log.d("end","In end of speech");
if (DICTATION_SEARCH.equals(recognizer.getSearchName()))
switchSearch(KWS_SEARCH);
}
private void switchSearch(String searchName) {
recognizer.stop();
recognizer.startListening(searchName);
String caption = getResources().getString(captions.get(searchName));
((TextView) findViewById(R.id.caption_text)).setText(caption);
}
private void setupRecognizer(File assetsDir) {
File modelsDir = new File(assetsDir, "models");
recognizer = defaultSetup()
.setAcousticModel(new File(modelsDir, "hmm/en-us"))
.setDictionary(new File(modelsDir, "dict/cmu07a.dic"))
.setRawLogDir(assetsDir).setKeywordThreshold(1e-20f)
//.setFloat("-beam", 1e-30f)
.getRecognizer();
recognizer.addListener(this);
// Create keyword-activation search.
recognizer.addKeyphraseSearch(KWS_SEARCH, KEYPHRASE);
// Create language model search.
File languageModel = new File(modelsDir, "lm/weather.dmp");
recognizer.addNgramSearch(DICTATION_SEARCH, languageModel);
}
}
Actually I have no idea how to use it.
after running ,it is throwing a runtime exception. I've posted the logcat of my code:
03-08 14:02:53.040: I/cmusphinx(17309): INFO: ngram_search_fwdtree.c(99): 788 unique initial diphones
03-08 14:02:53.230: I/cmusphinx(17309): INFO: ngram_search_fwdtree.c(148): 0 root, 0 non-root channels, 58 single-phone words
03-08 14:02:53.250: I/cmusphinx(17309): INFO: ngram_search_fwdtree.c(186): Creating search tree
03-08 14:02:53.470: I/cmusphinx(17309): INFO: ngram_search_fwdtree.c(192): before: 0 root, 0 non-root channels, 58 single-phone words
03-08 14:02:55.630: I/cmusphinx(17309): INFO: ngram_search_fwdtree.c(326): after: max nonroot chan increased to 2230
03-08 14:02:55.630: I/cmusphinx(17309): INFO: ngram_search_fwdtree.c(339): after: 253 root, 2102 non-root channels, 13 single-phone words
03-08 14:02:55.630: I/cmusphinx(17309): INFO: ngram_search_fwdflat.c(157): fwdflat: min_ef_width = 4, max_sf_win = 25
03-08 14:02:57.130: W/dalvikvm(17309): threadid=2: spin on suspend #1 threadid=11 (pcf=0)
03-08 14:02:57.140: W/dalvikvm(17309): threadid=2: spin on suspend resolved in 1010 msec
03-08 14:02:57.150: D/-heap(17309): GC_CONCURRENT freed 417K, 8% free 7863K/8455K, paused 14ms+1015ms, total 1597ms
03-08 14:02:57.210: I/SpeechRecognizer(17309): Start recognition "wakeup"
03-08 14:02:57.390: I/cmusphinx(17309): INFO: pocketsphinx.c(863): Writing raw audio log file: /mnt/sdcard/Android/data/com.example.pocket_sphinx/files/sync/000000000.raw
03-08 14:02:57.890: I/SpeechRecognizer(17309): Stop recognition
03-08 14:02:57.890: I/SpeechRecognizer(17309): Start recognition "wakeup"
03-08 14:02:57.890: W/dalvikvm(17309): threadid=1: thread exiting with uncaught exception (group=0x41ab9450)
03-08 14:02:57.930: E/AndroidRuntime(17309): FATAL EXCEPTION: main
03-08 14:02:57.930: E/AndroidRuntime(17309): java.lang.NullPointerException
03-08 14:02:57.930: E/AndroidRuntime(17309): at com.example.pocket_sphinx.MainActivity.switchSearch(MainActivity.java:108)
03-08 14:02:57.930: E/AndroidRuntime(17309): at com.example.pocket_sphinx.MainActivity.access$2(MainActivity.java:105)
03-08 14:02:57.930: E/AndroidRuntime(17309): at com.example.pocket_sphinx.MainActivity$1.onPostExecute(MainActivity.java:67)
03-08 14:02:57.930: E/AndroidRuntime(17309): at com.example.pocket_sphinx.MainActivity$1.onPostExecute(MainActivity.java:1)
03-08 14:02:57.930: E/AndroidRuntime(17309): at android.os.AsyncTask.finish(AsyncTask.java:631)
03-08 14:02:57.930: E/AndroidRuntime(17309): at android.os.AsyncTask.access$600(AsyncTask.java:177)
03-08 14:02:57.930: E/AndroidRuntime(17309): at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:644)
03-08 14:02:57.930: E/AndroidRuntime(17309): at android.os.Handler.dispatchMessage(Handler.java:99)
03-08 14:02:57.930: E/AndroidRuntime(17309): at android.os.Looper.loop(Looper.java:137)
03-08 14:02:57.930: E/AndroidRuntime(17309): at android.app.ActivityThread.main(ActivityThread.java:4802)
03-08 14:02:57.930: E/AndroidRuntime(17309): at java.lang.reflect.Method.invokeNative(Native Method)
03-08 14:02:57.930: E/AndroidRuntime(17309): at java.lang.reflect.Method.invoke(Method.java:511)
03-08 14:02:57.930: E/AndroidRuntime(17309): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:813)
03-08 14:02:57.930: E/AndroidRuntime(17309): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:580)
03-08 14:02:57.930: E/AndroidRuntime(17309): at dalvik.system.NativeStart.main(Native Method)
03-08 14:02:57.940: I/cmusphinx(17309): INFO: pocketsphinx.c(863): Writing raw audio log file: /mnt/sdcard/Android/data/com.example.pocket_sphinx/files/sync/000000001.raw
You have two issues with your code:
1) You start search inside doInBackground and start the same search again in onPostExecute, you can just call switchSearch in onPostExecute, that would be enough
2) You do not fill captions array but you use it in switchSearch method. So you get null pointer exception. You can comment out
String caption = getResources().getString(captions.get(searchName));
((TextView) findViewById(R.id.caption_text)).setText(caption);
in switchSearch if you are not going to use captions