botframework azureportal direct-line-botframework

speech services of a bot are not working in "Test in Web chat" option in Azure

I have deployed a web bot app on Azure Portal. This bot uses Speech services to get the user input but after deploying it in the azure, speech services are not working and it is not accepting the spoken information.

Even I click on the audio icon on the web chat window in Azure, it does not

This bot has been tested locally on Bot emulator and worked as required. but in Azure, it does not. No idea whats wrong.

any comment?

Solution

Please double check that you have all the appropriate credentials inserted into Configuration. Test in Web Chat should work out-of-the-box with no issue.

That being said, the Test in Web Chat feature uses the older v3 BotFramework-WebChat (i.e. BotChat) implementation. While it is slated to be updated (no ETA at this point), the underlying code may be an issue for you.

If you wish to test Web Chat, I would recommend setting up a v4 Web Chat environment locally. This is actually fairly easy to do. Here's how, if of interest:

First, enable the Direct Line channel in your Azure bot settings...

...and copy the secret and save somewhere for the moment.

Add this code into your bot's index.js file (can be appended to the end, in full). Store and access your copied direct line secret from a .env file. This is saved as directLineSecret and is passed into the authorization as the Bearer {token}. Also note, the port used is 3500, but you can set this to whatever you choose. This token endpoint will run as your run your bot.

/**
 * Creates token server
 */
const bodyParser = require('body-parser');
const request = require('request');
const corsMiddleware = require('restify-cors-middleware');

const cors = corsMiddleware({
  origins: ['*']
});

// Create server.
let tokenServer = restify.createServer();
tokenServer.pre(cors.preflight);
tokenServer.use(cors.actual);
tokenServer.use(bodyParser.json({
  extended: false
}));
tokenServer.dl_name = 'DirectLine';
tokenServer.listen(process.env.port || process.env.PORT || 3500, function() {
  console.log(`\n${ tokenServer.dl_name } listening to ${ tokenServer.url }.`);
});

// Listen for incoming requests.
tokenServer.post('/directline/token', (req, res) => {
  // userId must start with `dl_`
  const userId = (req.body && req.body.id) ? req.body.id : `dl_${ Date.now() + Math.random().toString(36) }`;
  const options = {
    method: 'POST',
    uri: 'https://directline.botframework.com/v3/directline/tokens/generate',
    headers: {
      'Authorization': `Bearer ${ process.env.directLineSecret}`
    },
    json: {
      User: {
        Id: userId
      }
    }
  };
  request.post(options, (error, response, body) => {
    if (!error && response.statusCode < 300) {
      res.send({
        token: body.token
      });
    } else {
      res.status(500);
      res.send('Call to retrieve token from DirectLine failed');
    }
  });
});

Next, create this simple index.html page that requests a token from your token endpoint and instantiates web chat. This will require a Cognitive Services Speech Service subscription key to enable speech which you can create in Azure. If you changed the port in your token endpoint, match it in the html file in the await fetch.

<!DOCTYPE html>
<html>

  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>WebChat</title>
    <meta name="description" content="">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <style>
      html,
      body {
        height: 100%
      }

      body {
        margin: 0
      }

      #webchat,
      #webchat>* {
        height: 100%;
        width: 100%;
      }
    </style>
  </head>

  <body>
    <div id="webchat" role="main"></div>
    <script type="text/javascript"
      src="https://unpkg.com/markdown-it/dist/markdown-it.min.js"></script>
    <script
      src="https://cdn.botframework.com/botframework-webchat/master/webchat.js"></script>
    <script>
    ( async function () {
      const res = await fetch( 'http://localhost:3500/directline/token', { method: 'POST' } );
      const { token } = await res.json();

      const webSpeechPonyfillFactory = await window.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory( {
        region: '',  // 'westus', for example
        subscriptionKey: ""
      } );

      window.WebChat.renderWebChat( {
        directLine: window.WebChat.createDirectLine( { token } ),
        webSpeechPonyfillFactory
      }, document.getElementById( 'webchat' ) );
      document.querySelector( '#webchat > *' ).focus();
    } )().catch( err => console.error( err ) );
  </script>
  </body>
</html>

Lastly, add the local web server running your html page to your list of trusted URLs in the Direct Line channel in your Azure bot settings, and run your html page locally along side your bot. You now have a local v4 web chat environment you can test from.

Hope of help!!