How can I optimize a Twilio function?

Question

I'm trying to make a whatsapp bot that provides OpenAI's GPT3.5 with Twilio's whatsapp messaging services via their API. I'm using the following serverless function:

const fetch = require('node-fetch');
exports.handler = async(context, event, callback) => {
    const history = [{"role": "system",
        "content": "Eres un asistente educativo para estudiantes de una escuela primaria"}, {'role': 'user', 'content': event.message}];
    try{
        const response = await fetch('https://api.openai.com/v1/chat/completions', {
            method: "POST",
            headers: {
                "Content-Type": "application/json",
                "Authorization": `Bearer ${context.OPENAI_KEY}`
            },
            body: JSON.stringify({
                "model": "gpt-3.5-turbo",
                "max_tokens": 750,
                "n": 1,
                "temperature": 0,
                "messages": history
            })
        });
        const data = await response.json();
        return callback(null, data.choices[0].message.content);
    }catch(error){
        console.log(error);
        return callback(error);
    }
}

However, the execution usually takes more than 10 seconds and its automatically stopped by Twilio. I need to use a function specifically because I need to send the API key as a header, and the flow HTTP request widget doesn't allow custom headers.

I've already streamlined and removed extra functionality, but the functions still takes a long time, am I doing something wrong or is there a way to further optimize the function?

Answer 1

I'm afraid a serverless environment cannot solve the issue you are describing. Serverless functions usually have a maximum execution time. For example, Twilio serverless functions have a maximum execution time of 10 seconds. Similarly, many others also have a maximum execution time. This inherently is not sufficient for long-running processes like waiting for the OpenAI response.

PS: Even if you found a serverless runtime that has a max execution time that just fits your needs, it's probably unfeasible to use serverless since you pay per execution time.

I'd suggest running this process on a traditional (non-serverless environment). Maybe one day you'll be able to provide a callback URL to OpenAI which points another Twilio serverless function. This way, you could continue having all the Twilio logic on Twilio serverless and only host the other code somewhere else. But again, as of today, this is not possible.