Search code examples
python-3.xamazon-s3boto3

how to check if a particular directory exists in S3 bucket using python and boto3


How to check if a particular file is present inside a particular directory in my S3? I use Boto3 and tried this code (which doesn't work):

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket')
key = 'dootdoot.jpg'
objs = list(bucket.objects.filter(Prefix=key))
if len(objs) > 0 and objs[0].key == key:
    print("Exists!")
else:
    print("Doesn't exist")

Solution

  • While checking for S3 folder, there are two scenarios:

    Scenario 1

    import boto3
    
    def folder_exists_and_not_empty(bucket:str, path:str) -> bool:
        '''
        Folder should exists. 
        Folder should not be empty.
        '''
        s3 = boto3.client('s3')
        if not path.endswith('/'):
            path = path+'/' 
        resp = s3.list_objects(Bucket=bucket, Prefix=path, Delimiter='/',MaxKeys=1)
        return 'Contents' in resp
    
    • The above code uses MaxKeys=1. This it more efficient. Even if the folder contains lot of files, it quickly responds back with just one of the contents.
    • Observe it checks Contents in response

    Scenario 2

    import boto3
    
    def folder_exists(bucket:str, path:str) -> bool:
        '''
        Folder should exists. 
        Folder could be empty.
        '''
        s3 = boto3.client('s3')
        path = path.rstrip('/') 
        resp = s3.list_objects(Bucket=bucket, Prefix=path, Delimiter='/',MaxKeys=1)
        return 'CommonPrefixes' in resp
    
    • Observe it strips off the last / from path. This prefix will check just that folder and doesn't check within that folder.
    • Observe it checks CommonPrefixes in response and not Contents