Search code examples
pythonpython-3.xurllib

400 Bad Request HTTPError exception when using urllib


I want to read output/contents of a url requested using python3. It is working perfect, when passing only one string as argument, but not working when passing whole sentences as argument.

For easiness, I have tried to express both cases in two different part in given code below.

what is error? And How can it be fixed?

But, 2nd link is working perfectly, when I am opening in my web browser tab directly.

### code """
 import urllib.request
 def check_curse1():
 content = "hello"  ##### line of attention
    link="http://www.wdylike.appspo
            t.com/?q="+content
    #print(link)
    connection = urllib.request.urlopen(link)
    #print(connection)
    output = connection.read()
    print(output)
    connection.close()
  def check_curse2():
    content = "hi. Need help"  
  #####line of attention
    link="http://www.wdylike.appspo
            t.com/?q="+content
    #print(link)
    connection = urllib.request.urlopen(link)
    #print(connection)
    output = connection.read()
    print(output)
    connection.close()
  print("case 1 working")
  check_curse1()
  print("\n")
  print("case 2 not working")
  check_curse2()

## output part ##
case 1
b'false'
case 2
Traceback (most recent call last):
 File "error.py", line 26, in <module>
  check_curse2()
 File "error.py", line 16, in check_curse2
  connection = urllib.request.urlopen(link)
 File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
  return opener.open(url, data, timeout)
 File "/usr/lib/python3.6/urllib/request.py", line 532, in open
  response = meth(req, response)
 File "/usr/lib/python3.6/urllib/request.py", line 642, in 
   http_response
   'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
  return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in 
  _call_chain
  result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in 
  http_error_default
  raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

Solution

  • You need to encode strings with spaces or special characters before you fire the request.

    import urllib.request
    def check_curse1():
        content = "hello"  ##### line of attention
        link="http://www.wdylike.appspot.com/?q="+content
        #print(link)
        connection = urllib.request.urlopen(link)
        #print(connection)
        output = connection.read()
        print(output)
        connection.close()
    def check_curse2():
        content = {'q': "hi. Need help"}  #####line of attention
        encoded_content = urllib.parse.urlencode(content)
        link="http://www.wdylike.appspot.com/?"+encoded_content
        #print(link)
        connection = urllib.request.urlopen(link)
        #print(connection)
        output = connection.read()
        print(output)
        connection.close()
    
    print("case 1 working")
    check_curse1()
    print("\n")
    print("case 2 not working")
    check_curse2()
    

    Outputs:

    case 1 working
    b'false'
    
    
    case 2 not working
    b'false'