I have an app that is made using Laravel/PHP. It is running on a serverless architecture using AWS API Gateway/Lambda.
I have a job that takes too much time processing, that is why I use SQS for queues. That job calls an external API, it sends some documents, and the API returns the signed documents, but when it sends 5 o more documents at the same time, in the logs I can see this:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
<head>
<title>504 Gateway Timeout</title>
</head>
<body>
<h1>Gateway Timeout</h1>
<p>The gateway did not receive a timely response from the upstream server or application.</p>
</body>
</html>
And I can't get the signed documents, I use Guzzle for the request and it has specified: 'timeout' => 120.
the job take around 48 seconds
REPORT RequestId: xxxx-xx-xxxx-x-xxxDuration: 47.69 ms Billed Duration: 48 ms Memory Size: 1024 MB Max Memory Used: 78 MB Init Duration: 399.67 ms
I don't remember where I read that if even the AWS gateway timeout is 30 seconds, if it adds the job in SQS then it would not be a problem, is it correct?
What can I do?
Edit: I just edited the actual log, it really took more than 30 seconds, but it should not be a problem if that job is in the queue?
I use Laravel Saloon library to make http request because I am building a lot of apis,
This is my job:
<?php
namespace App\Jobs;
use App\Http\Saloon\Requests\CreateModuleJob;
use App\Models\Integration;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Arr;
class CreateModuleJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public function __construct(
public array $information,
public string $module_id,
) {
}
/**
* Execute the job.
*
* @return void
*/
public function handle()
{
$request = new CreateModuleRequest($this->information);
///here takes more than 30 seconds
$response = $request->sendAsync()->then(
function ($response) {
//emit webhook
},
function ($response) {
//emit webhook
}
);
$response->wait();
}
}
I use the spatie webhook library to emit webhooks, those are received by a node app, and in the node app I wrote console.log(data) to see what's going on
and the lambda logs returns this:
2023-07-07T22:19:20.478Z xxxxx-xxx-4fxxxac-xx-xxINFO {
"app_id": "xxxxxxxxxxxxxxxxxxxx",
"event": "signature.failed",
"data": {
"response": "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>504 Gateway Timeout</title>\n</head><body>\n<h1>Gateway Timeout</h1>\n<p>The gateway did not receive a timely response\nfrom the upstream server or application.</p>\n</body></html>\n"
},
"metadata": []
}
in the job I make a request that takes more 30 seconds? that is the problem? even If that job is in SQS?
Edit2: Before making a request to the external api, the lambda function make a request to a redirect server, why? because the external api can accept requests only from a specific ip address, that's why we use the redirect server, is the timeout because the lambda function o the redirect server?
The solution was increasing the timeout in the saloon/guzzle library, I set it before up to 120 seconds, but it required more timeout, so I increased it up to 300 seconds, and now it is working perfect. So my advice is despite it is using AWS Gateway,or your lambda logs shows something strange, you can still increase the timeout as long as you put that job/task in a queue, in my case I used SQS