Real-time speech recognition from a phone call recorded with Twillio

Question

I'm currently using Twilio to make phone calls and I'd like to add a speech recognition element such that if a user says a specific phrase, my backend can take specific actions. If you're familiar with Twilio, something akin to the Gather verb. It needs to be real-time since if there are issues with recognition, the user would be prompted for clarification.

Answer 1

To add speech recognition to the Twilio Gather verb, add "speech" to the Gather input value, example: input="dtmf speech". After the caller says something and is quiet, the Twilio server translates the speech in text and sends the text to the action URL, then waits for response instructions. Your program can use the text to respond how ever you choose. One choice is to have your program respond with correction instructions (Say verb) and have the caller say something more, which would be processed again by your action URL.

Twilio Gather documentation including the implementation of speech recognition: https://www.twilio.com/docs/api/twiml/gather

Example TwiML with a Gather verb using the speech recognition identifier.

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Gather input="dtmf speech" language="en-us"
          numDigits="1"
          timeout="6"
          action="http://hostname/processUserResponse.py">
        <Say voice="alice" language="en-CA">
            Okay, speech recognition test. Enter any digit or say something.
        </Say>
    </Gather>
    <Say voice="alice" language="en-CA">
        Waited to long to say something. Response canceled ....
    </Say>
</Response>