The challange
I would like to build a simple and fast (< 50ms) API with API gateway that works as an intelligent cache. The request to the API should batch fetch some items from DynamoDB based on their keys. However, if one or more items are not found in dynamoDB I would like to somehow trigger a lambda function, who knows how to populate those missing items for later requests) in dynamoDB.
My idea
The way I though of doing this is by creating any missing items in DynamoDB only with their key and then use DynamoDB steams to invoke a lambda function who reads the records and fill's their blank attributes with values from another external API.
Problem
My problem is however, that as far as I can figure out I can ONLY do ONE request to dynamoDB (either a BatchGetItem or a conditional PutItem) per request to API gateway. Is there some way of achieving what I want.
I would really like to use API gateway as this is needed to scale quite aggressively and I would rather not handle the scaling of servers and applications.
Alternatives
Also an alternative would be to have the client code make the decision of what items were missing from the response and then send a second request to "queue" those for population, however I would really like that logic to NOT go on the client side and and also to avoid two network roundtrips.
I also looked into if dynamoDB would be able to do this "find or create"-behaviour for me - but as it seems, no luck: What event can be triggered to fire a lambda function in DynamoDB?
If your API Gateway endpoint calls a Lambda function (instead of doing a DynamoDB proxy which wouldn't be a clean API anyway) then you can run whatever code you want. You can make as many calls to DynamoDB and other services as you want whenever an API Gateway call triggers the Lambda function. The only restriction is the amount of time your Lambda function can run.