Stoping all AWS Compute after a certain time period

Question

I've been tasked to crawl urls and abruptly stop the AWS Lambda after 30 seconds after which the number of URLs crawled will be taken as an evaluation metric. In a simple architecture where I have 1 lambda that takes a file and loops through it and writes to the database, I could have simply asked it to timeout after 30 seconds. For the sake of my learning and to meet the other criteria of scaling, the architecture I adopted is this:

So even if I timeout my lambdas, they will run again given the URLs are being sent as events by fargate. The point of fargate is to be able to download a huge file as Lambda has limitations. The use of events will then help me achieve scale by simply allowing more concurrent lambdas. Can I somehow stop eventbus to freeze sending or receiving notifications after 30 seconds? Can I stop all the computes somehow?

I could send errors etc (I only get timeout errors in the past) to a dead queue or an SNS topic and show resiliency in the system to abrupt crashes. I could also demonstrate the number of URLs crawled by showing logs. But assume these measures do not satisfy the evaluator, is there anything I can do?

I can add delays to messages and queues but how would that do anything? I can't add a delay after a certain time period. That would have worked.

Answer 1

One way would be to disable the Event Rule after a set period of time using the cli or SDK from your container:

$ aws events disable-rule --name MyRule --event-bus-name MyEventBus

Another option to stop all further lambda invocation is to set the concurrency limit to 0 as per How to kill/terminate a running AWS Lambda function? :

$ aws lambda put-function-concurrency --function-name my-function --reserved-concurrent-executions 0

This will not stop the executions that are already running - that cannot be done with Lambda.

Your database can then be filtered by the insertion timestamp to remove all writes that happen after the threshold.