I try to write batch to aws DynamoDb. Batch contains 15-20k elements and data is based on 30-40MB txt file. When I send small batches with <14k elements, DynamoDb saves data and everything is OK. When I send 15-20k elements in batch, lambda makes multiple retries. After that lambda sends error: [WARN] ([email protected]:331 errno: No such file or directory) run_dotnet(dotnet_path, &args) failed
I use c# sdk and crash appears on ExecuteAsync method. I tried to use try..catch.. to see something more meaningful, but I don't catch exception. I wonder what message: 'No such file or directory' means. Google doesn't give me answer to this kind of message exception. I wonder that It may be some limits on DynamoDb but it's difficult for me to see what limitation I try to exceed. Crashing-Files are not small and it could be the reason.
Please help.
I tried many things and at the end I found that there was problem with Memory Size in Lambda. I increased MB and it worked. Lambda message is not so clear when it happens.