Search code examples
pythonpython-3.xurlencode

Why is my url encoder not working for http requests?

My code in Python3:

import requests

URL = "http://xxx.xxx.com/querytext"

try:
    text = urllib.parse.quote_plus(text)
    r = requests.get(URL, params={"text": text}, auth=("guest", "1234"))
    r_json = r.json()
except Exception:
    return None

tag_results = r_json["data"]["result"]

My 'text' may contain some special characters so I want to do url encoding, as shown above. However, for one testing example, if I don't use this line:

text = urllib.parse.quote_plus(text)

I can get the expected result. Otherwise, I can't. So i suspect my way of using url encoding is wrong. What's wrong exactly?

One example of the text could be '#Thisisgreat#, yes!'

Solution

  • As stated in the request docs (https://docs.python-requests.org/en/latest/):

    Requests allows you to send HTTP/1.1 requests extremely easily. There’s no need to manually add query strings to your URLs, or to form-encode your POST data. Keep-alive and HTTP connection pooling are 100% automatic, thanks to urllib3.

    So you do not need to parse the text manually. Doing so may "double parse" it.