Twilio Autopilot - How can I receive images from Whatsapp?

Question

I'm building a chatbot with Twilio Autopilot and I want to get the images that I send to the bot, how can I get it? How can I get locations as well?

Thanks a lot.

Answer 1

Twilio developer evangelist here.

There isn't a simple way to do this, but there are a few potential work-arounds. One is to have a webhook endpoint that will get the input and if the payload contains elements of an image, then do whatever you want with it, otherwise if it is just text, then maybe send to Autopilot. That is gone over in this blog post on Autopilot enhancements in Node.js.

Another is to make a Twilio Function that would point to a Twilio Studio flow or Assets if it is media in the first message.

Another is to use Twilio Functions or a similar server. You should have an Autopilot task that redirects to that Function with JSON like this:

{
    "actions": [
        {
            "redirect": {
                "uri": "https://REPLACE-WITH-YOUR-FUNCTION-URL.twil.io/auso",
                "method": "POST"
            }
        }
    ]
}

Then your Twilio Function could get the image URL with something like this in Node.js:

const bodyContent = event.MediaUrl0;
const filename = event.MessageSid + '.png';

Now in a Collect Action you can also specify the Twilio.MEDIA as the type for questions expecting Media and currently we support all media format which are supported by Twilio Messaging.

{
                        "question": "Please a take a picture of insurance card?",
                        "name": "insurance_card",
                        "type": "Twilio.MEDIA",
                        "validate": {
                            "allowed_types": {
                                "list": [
                                    "image/jpeg",
                                    "image/gif",
                                    "image/png",
                                    "image/bmp"
                                ]
                            },

Lastly, you may be interested in this blog post on building an image classifier with Autopilot and TensorFlow.

Let me know if this helps at all! :D