I am using Python Requests to access an HTTPS page that contains JSON data. I pass 2 parameters in the get request, they are listed below verify = False cookies = cookie I am aware that the verify = False is to get past SSL certification verification, and the cookies = cookie parameter is to pass cookie values. There are actually 2 cookie values that I have put in a dictionary, and one of the values is very long maybe 300+ characters. But when I run my code I get this error
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2026' in position 602: ordinal not in range(256)
My code is attached below
import requests
test_url = "https://abc/cde"
cookie = {"cookie1":"value1(10 characters)", "cookie2":"value2(300+ characters)"}
response = requests.get(test_URL, verify=False, cookies= cookie)
print(response.content)
\u2026 is the HORIZONTAL ELLISPSIS character which is not a legal character in cookies (see the answers to this Q&A
>>> r = requests.get('https://www.google.com', cookies={'mycookie': '\u2026'})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/me/virtual-envs/so310/lib/python3.10/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/home/me/virtual-envs/so310/lib/python3.10/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
...
File "/usr/local/lib/python3.10/http/client.py", line 1317, in _send_request
self.putheader(hdr, value)
File "/home/me/virtual-envs/so310/lib/python3.10/site-packages/urllib3/connection.py", line 224, in putheader
_HTTPConnection.putheader(self, header, *values)
File "/usr/local/lib/python3.10/http/client.py", line 1249, in putheader
values[i] = one_value.encode('latin-1')
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2026' in position 9: ordinal not in range(256)
The usual solution to this problem is to encode the value with a method that doesn't expose illegal characters, for example base64, or JSON with unicode characters escaped, assuming that the consumer of the cookies will know how to decode the value. Otherwise, remove such characters from the cookie value.