Search code examples
web-chat

Is there a way to recognize weather the input to Bot is from microphone or is text input?


In my BOT I want to log the times when speech input was provided to the BOT. Is there a way to recognize it in my BOT Framework solution.


Solution

  • When a user speaks to a bot in Web Chat, the generated activity sent to the bot will include a channelData.speech property. You can use this as a marker for determining, in the bot, if the activity came from spoken or text input. The speech property is not appended when the activity is generated from text input. This applies to both Cognitive Services speech (channelId: 'directline') and Direct Line Speech (channelId: 'directlinespeech').

    Example activity

      {
        type: 'message',
        id: '26WpGqt6CCz7M8uRN0ugo9-o|0000002',
        timestamp: 2021-01-28T23:12:10.281Z,
        serviceUrl: 'https://directline.botframework.com/',
        channelId: 'directline',
        from: { id: '<<REDACTED>>', name: '', role: 'user' },
        conversation: { id: '26WpGqt6CCz7M8uRN0ugo9-o' },
        recipient: { id: '<<REDACTED>>', name: '<<REDACTED>>' },    
        textFormat: 'plain',
        locale: 'en-US',
        text: 'Hello.',
        entities: [ [Object] ],
        channelData: {
          speech: {
            alternatives: [
              {
                confidence: 0.51567326,
                transcript: 'Hello.'
              }
            ]
          },
          clientActivityID: '1611875530186v9cv78togd',
          clientTimestamp: '2021-01-28T23:12:10.186Z'
        },
        rawTimestamp: '2021-01-28T23:12:10.2819155Z',
        callerId: 'urn:botframework:azure'
      }
    

    Hope of help!