'NoneType' object is not iterable when trying to loop over GA API result

My final goal is to export in csv the top 500 pages from multiple property within the same account. I have 8 property so I want 8 csv at the end.

Before getting to that, I would like to be able to build multiple lists containing the 500 pages urls for each property.

Since I'm new to python I'm building my script step-by-step.

First I'm building a list of profile ID :

profiles = service.management().profiles().list(
      accountId='XXXXX',
      webPropertyId='UA-XXXXXXX-XX').execute()

  for profile in profiles.get('items', []):
    list.append(profile.get('id'))

Second I'm iterating over this list to get the top 500 pages for each profiles :

  for item in list:
    test = service.data().ga().get(
      ids='ga:' + item,
      start_date='1daysAgo',
      end_date='today',
      metrics='ga:sessions',
      dimensions='ga:pagePath',
      sort='-ga:sessions',
      filters='ga:sessions>500').execute()

Third (It is where things get complicated) I want to loop over the result of the test variable (?) and add to a new array each urls pages :

for row in test.get('rows'):
  rawdata.append(row)

When executing the script at this stage I have the following error :

  File "test.py", line 71, in get_first_profile_id
    for row in test.get('rows'):
TypeError: 'NoneType' object is not iterable

Here is the all code:

  profiles = service.management().profiles().list(
      accountId='5140486',
      webPropertyId='UA-5140486-16').execute()

  for profile in profiles.get('items', []):
    list.append(profile.get('id'))

  for item in list:
    test = service.data().ga().get(
      ids='ga:' + item,
      start_date='1daysAgo',
      end_date='today',
      metrics='ga:sessions',
      dimensions='ga:pagePath',
      sort='-ga:sessions',
      filters='ga:sessions>500').execute()

    for row in test.get('rows'):
      rawdata.append(row)

I hope my goal is clear. thanks for helping !

Solution

For at least one of the values you're running on, the data you load as test doesn't have a rows key. When you call test.get('rows') for that item, it returns None (since you don't give it a different default value). That leads to your exception when you try to iterate over the rows.

You can avoid the exception by using test.get('rows', []) (like you are doing above to the items from profiles). You can also streamline things a bit by using list.extend instead of looping and calling append repeatedly:

profiles = service.management().profiles().list(
    accountId='XXXXX',
    webPropertyId='UA-XXXXXXX-XX').execute()

my_list.extend(profile.get('id') for profile in profiles.get('items', [])) # use extend

for item in my_list:  # it's a really bad idea to have a variable named list, so I renamed it
    test = service.data().ga().get(
        ids='ga:' + item,
        start_date='1daysAgo',
        end_date='today',
        metrics='ga:sessions',
        dimensions='ga:pagePath',
        sort='-ga:sessions',
        filters='ga:sessions>500').execute()

    rawdata.extend(test.get('rows', []))   # here too

Note that this will silently ignore missing data. That could be fine if having missing data is an ordinary thing. However, if it's unusual, you might want want to treat it as an error. An easy way to do that would be to use test['rows'] and catch a KeyError wherever you want to handle the issue.