Search code examples
pythonurlurllib

how to avoid square brackets from url after unquote

i've decided to add a querystring on URL like this

import urllib
import urllib.parse
from urllib.parse import urlencode

url = "https://datausa.io/api/data?Geography=04000US06&drilldowns=Race,Ethnicity&measures=Hispanic%20Population,Hispanic%20Population%20Moe"
parts = urllib.parse.urlparse(url)
query_dict = urllib.parse.parse_qs(parts.query)  
query_dict['Geography'] = '04000US24'
new_parts = list(parts)
new_parts[4] = urlencode(query_dict)
print(urllib.parse.urlunparse(new_parts))

and i got this result

https://datausa.io/api/data?Geography=04000US24&drilldowns=%5B%27Race%2CEthnicity%27%5D&measures=%5B%27Hispanic+Population%2CHispanic+Population+Moe%27%5D

and so this url didn't work anymore. So i used unquote, like this

from urllib.parse import unquote
url = unquote('https://datausa.io/api/data?Geography=04000US24&drilldowns=%5B%27Race%2CEthnicity%27%5D&measures=%5B%27Hispanic+Population%2CHispanic+Population+Moe%27%5D')
print(url)

but got this

https://datausa.io/api/data?Geography=04000US24&drilldowns=['Race,Ethnicity']&measures=['Hispanic+Population,Hispanic+Population+Moe']

and this url doesn't work too.

How can i remove square brackets an come back to original url https://datausa.io/api/data?Geography=04000US24&drilldowns=Race,Ethnicity&measures=Hispanic%20Population,Hispanic%20Population%20Moe ?

Thanks

Solution

  • Use the doseq parameter.

    new_parts[4] = urlencode(query_dict, doseq=True)

    See the docs on urllib.parse.urlencode for more information on the parameter