Is there a way for accepting voice input in waterfall dialog

Question

I am currently creating a Chatbot that has to accept voice inputs from the user. However, using waterfall dialogs to prompt the user for input does not contain a prompt that accepts voice. I'm using Azure Speech services for speech recognition and was wondering is there a way to do it.

I tried converting the speech recognition result to string and send that as user text input but I'm new to coding and feel like I did it wrong. This is a part of the waterfall dialog step.

private  async Task<DialogTurnResult> IntroStep(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            stepContext.Values[StudentInfo] = new BotData();

            SpeechSynthesis.SubjectVoice();

            var promptOptions = new PromptOptions { Prompt = MessageFactory.Text("Hello, how can i help you? \n" +
                "Want to do a Quiz or ask me a Question") };

            SpeechRecognition.HearUser(); // waits for user voice input 
            Model.Answer = (string)stepContext.Result;
            return await stepContext.PromptAsync(nameof(TextPrompt), promptOptions, cancellationToken);
        }

Answer 1

As you are using the Webchat channel (based on your comments), you should process all the speech part on the Webchat, and then your bot will process your messages without any chance on this side.

You have several samples on the official Github repository regarding Speech: see all the samples which number start by 06 here:

• 06.a.cognitive-services-bing-speech-js: Introduces speech-to-text and text-to-speech ability using the (deprecated) Cognitive Services Bing Speech API and JavaScript.
• 06.b.cognitive-services-bing-speech-react: Introduces speech-to-text and text-to-speech ability using the (deprecated) Cognitive Services Bing Speech API and React.
• 06.c.cognitive-services-speech-services-js: Introduces speech-to-text and text-to-speech ability using Cognitive Services Speech Services API.
• 06.d.speech-web-browser: Demonstrates how to implement text-to-speech using Web Chat's browser-based Web Speech API. (link to W3C standard in the sample)
• 06.e.cognitive-services-speech-services-with-lexical-result: Demonstrates how to use lexical result from Cognitive Services Speech Services API.
• 06.f.hybrid-speech: Demonstrates how to use both browser-based Web Speech API for speech-to-text, and Cognitive Services Speech Services API for text-to-speech.

I would suggest to start with 06.d as it does not need many changes.

In a few words, you have to:

• enable speech capability in your webchat, using webSpeechPonyfillFactory: window.WebChat.createBrowserWebSpeechPonyfillFactory()
• in your bot code, ensure that the Speak value of your outgoing activities is set in order to be able to "speak back" to the user. Please note that your bot will "speak" automatically if the previous message was spoken.