Search code examples
pythonamazon-web-servicesamazon-dynamodbdynamodb-queries

Download multiple images from multiple URLS in dictionary python


I am trying to write a function that fetches the URLs of images from dynamodb and download them using the url into a folder in my machine.

Below is my code:-

#reference dynamodb
dynamodb = boto3.resource('dynamodb')
#point at music table
table = dynamodb.Table('Music')
#get image url values from dynamodb
imageUrl = table.scan(AttributesToGet=['img_url'])
#add urls found into dict
urls=[]
urls.append(imageUrl)

#print(urls)

#download all images from their urls
for url in urls:
    url = imageUrl['Items']
    image = url[0]['img_url']
    file_name = image.split('/')[-1]
    response = requests.get(image)
    file = open("/Users/abdotech/Desktop/images/"+file_name, "wb")
    for chunk in response:
      file.write(chunk)


    file.close()

This code downloads only one image from the first url only, although I am iterating through a dictionary list of several image urls. the print(urls) returns all of the image urls found in the database, so thats how i know that my dictionary has all the needed urls.

So basically my bug lies in the for statement and I can't figure out what might be causing the issue.

Thanks in advance.

EDITS:

for url in imageUrl:
    url = imageUrl['Items']
    image = url['img_url']
    file_name = image.split('/')[-1]
    response = requests.get(image)
    file = open("/Users/abdotech/Desktop/images/"+file_name, "wb")
    for chunk in response:
      file.write(chunk)


    file.close()

error:TypeError: list indices must be integers or slices, not str


Solution

  • You aren't properly iterating through the results. The response from the scan contains the list of Items you need to iterate over, you don't need to create another urls list for this. You also are hardcoding the 0 index in your loop so it's always grabbing the first item of the list instead of the current item in the loop.

    Try something like this:

    #reference dynamodb
    dynamodb = boto3.resource('dynamodb')
    #point at music table
    table = dynamodb.Table('Music')
    #get image url values from dynamodb
    imageUrl = table.scan(AttributesToGet=['img_url'])
    
    #print(urls)
    
    #loop through all items returned by dynamodb
    for url in imageUrl['Items']:
        image = url['img_url']
        file_name = image.split('/')[-1]
        response = requests.get(image)
        file = open("/Users/abdotech/Desktop/images/"+file_name, "wb")
        for chunk in response:
          file.write(chunk)
    
    
        file.close()