buttons from webhook dialogflow cx

Question

I have a diaglogflow cx assistant and I have my own web front. I am getting the responses from a webhook and there is no problem with texts but I would like to add buttons, cards, images... that, although they cannot be seen in dialogflow cx, I can use the info in my front.

The problem is that dialogflow cx does not resend it in the response to the front that call it. I'm sending this response from the webhook server:

{
    "fulfillment_response": {
        "messages": [
            {
                "text": {
                    "text": [
                        "webhooktext"
                    ]}}],
        "richContent": [[
            {
                    "type": "buttons",
                    "options": [
                        {
                            "text": "button 1"
                        },
                        {
                            "text": "button 2"
                        }]}]]}}

And I am receiving this in the front app:

    "responseId": "fc385ea7-6f8c-4828-9e25-5196916c4028",
    "queryResult": {
        "text": "hey",
        "languageCode": "es",
        "responseMessages": [
            {
                "text": {
                    "text": [
                        "dialogflow text"
                    ]
                }
            },
            {
                "payload": {
                    "kbID": "greeting"
                }
            },
            {
                "text": {
                    "text": [
                        "webhooktext"
                    ]
                }
            }
        ],
        "webhookPayloads": [
            {}
        ],
        "currentPage": {...},
        "intent": {...},
        "intentDetectionConfidence": 1.0,
        "diagnosticInfo": {...},
        "webhookStatuses": [...],
        "match": {...}
    },
    "responseType": "FINAL"
}

I was expecting to receive the button data in the responseMessages array or even in webhookPayloads. I have tried to manage the webhook response structure but sometimes it gives an error message in the front dictionary and other times it shows as above. Should I change richContext key? Where do I place it?

Answer 1

Text along with images and other components can be parsed in Dialogflow CX. You can refer to this documentation for response texts and parse them on your side as per your requirement. For adding visual elements , you can refer to this link.

The most common way to achieve this in Python is to use pf.json_format.MessageToJson method according to this StackOverflow case.