I need that one lambda just start when some file was uploaded on two different S3 prefix.
Example:
I Have file A and File B and i have two different prefix for each file.
I need do some aggregations with file A and FILE B.
For do this i will use a Lambda that will put a message on SNS when this files be uploaded.
I would like config this lambda for be started when this lambda receive this two event notification.
So my lambda just can start when this two files are available on S3 for be processed.
What is the best way for setup this triggers for this work together?
TLDR; Rather than raise "file created" notifications manually in a "posting lambda", have S3 event notifications automatically invoke your "processing lambda" when a target file is created. The lambda runs to completion when both files exist.
docs: Amazon S3 can send an event to a Lambda function when an object is created or deleted.
(1) Create S3 event notifications: Add two event notifications to your bucket. One invokes the processing lambda when a FileA is created on S3. The other invokes it when a FileB is created.
You can set filters so your lambda is notified only on certain S3 actions and file patterns.
(2) The processing lambda checks if the other file exists. The S3 event passed to the lambda has information about the triggering event (e.g. ObjectCreated) and Object (e.g. key name, version, bucket etc.) Use the AWS SDK to check if the other file exists on S3.
If both files exist then invoke the glue routine, else exit.
50% the time the processing lambda will exit early (because only 1 file exists). The other times, the lambda will run all the way through to the glue routine.
(Note: this will work only if you can derive one file name from the other, which is not clear from the OP or comments.)