This article: QnA Maker limits explains that you can have only two published knowledge bases in Free tier.
If I have 3 KBs, is there way to un-publish one of them so I can publish other one?
After posting issue on AzureDocs got this reply, and it somehow works.
You can export the knowledge base which is not required using the export knowledge base option from the settings tab of your KB. This is available for export in .csv or .tsv formats. This should allow you to publish your new KB in the free tier and you can create a new KB with the exported KB without publishing it.