How to parse parameters in a URL Fragment using Python 3

Question

I need to parse a large number of URLs to retrieve a guid using urllib/python3. Some urls contain a fragment which causes problems with returning the parameters.

import urllib

url = "https://zzz.com/index.html#viewer?guid=6a755e6d-4eae&Link=true&psession=true")
parse_response = urllib.parse.urlsplit(self.url)
self.logger.info("The parsed url components = {}".format(parse_response))

The parsed url components = SplitResult(scheme='https', netloc='abc.com', path='/index.html', query='', fragment='viewer?guid=6a755e6d-4eae&Link=true&psession=true')]

So urllib rightly sees the "#" and stores the rest of the URL as a fragment, and will not return the parameters. What is the best way to process the URL's with and without fragments?

Answer 1

from urllib.parse import urlparse, urldefrag, parse_qs
url = " https://abc.xyz.com/url/with/fragment/query#param1=val1&param2=val2&param3=val3"

print(urlparse(url))
print(urlparse(url).fragment)

pq = parse_qs(urlparse(url).fragment)
print(pq)
print(type(pq))
print("Using urlparse {}".format((pq["access_token"][0])))


f = urldefrag(url).fragment
print(type(f))
print(f)

pq = parse_qs(f)
print(pq)
print(type(pq))
print("Using urldefrag {}".format((pq["access_token"][0])))