Does QnA Service Active Learning work with questions from a channel?
I activated the Active Learning option in my QnA Service to improve the answers it gives using feedback from users, where they ask some questions and, if the score is too low, active learning lets them choose among the best rated answers in the Knowledge base or let them choose none of them as the correct one.
The problem is that the feedback the users give should go to my QnA Service for approval but when I look for suggestions in the portal, there's nothing waiting to be approved.
QnA MAker Active learning works with feedback from the emulator or other channels to your bot.
A comment from a related GitHub issue states:
When there is a low confidence score difference between the top answers, we collect weighted implicit and explicit feedback to cluster suggestions for any QnA ID.When enough feedback is collected for any given suggestion, it will show in the KB. More specifically, we cluster similar user queries to generate suggestions. When minimum required feedback is collected, only then will the suggestions show in the KB.
The QnA team wants to avoid publicly divulging the exact logic of what exactly is the "minimum required feedback" and how often suggestions are generated (besides, the team is working on improving and optimizing the logic behind active learning as well) --however to see suggestions appear in the qnamaker.ai portal:not only ensure that you've given the bot enough feedback but also give the back end "some time" to allow for the suggestions to appear in the portal. Again, feedback is collected when your user types in a query that returns answers from QnA that have confidence scores that are close together.
It is also good to note that feedback is not collected in the Test panel in the qnamaker.ai portal as of now. You will need to chat with your bot via emulator or a channel to provide feedback to your bot that it can use for active learning.
Note: If the suggestions are not showing up, then it is probably because the questions users are asking are not generating top N answers that have similar confidence scores.
The latest sample on Active learning in QnA Maker is available here.
Hope this helps.