I have an S3 bucket with millions of objects with many different extensions.
Now I want to create an S. inventory based on extensions (Filter on suffix).
How can I do this when I want to create an S3 inventory? It seems I can filter on prefix (hardcoded path) but not on extension?
I don't want to use aws s3 ls with --include
to copy to other buckets. This will take far too much time.
You can't create S3 inventories filtered by extension.
What you can do is create an S3 inventory of ALL files, then download that inventory file and remove all entries that don't have the extensions you want.
If you want to automate the whole thing you can have a Lambda function do the job when a new inventory file is added to the bucket you choose by using S3 events. Then have the Lambda upload the modified file and use that going further. Watch out for loops, i.e. don't upload the modified file over the old one or using the same prefix, or your Lambda will be executed again.