not able to verify ssl-certificate from client side using any of python requests, httplib/httplib2, urllib/urlib2

Question

this question is duplicate. but i am still asking because i am not finding any solution from given answers. so guys don't make it duplicate, instead help me to solve this problem.

i am running my server as https using bottle(paste). when trying to access from java and postman , it's verifying correctly and response coming.

versions used : python 2.7.13

requests.version '2.11.1' import cryptography cryptography.version '2.1.3' import OpenSSL OpenSSL.version '17.3.0'

but unfortunately when trying to access from python modules - it's throwing issue. ?

requests.post(url, verify=_certfile, proxies=proxies) #tried #failed to verify certificate

urllib2.urlopen(url, data=data, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
        cafile=None, capath=None, cadefault=False, context=context)  #tried #failed to verify certificate

import httplib2

http = httplib2.Http(disable_ssl_certificate_validation=True)
headers, content = http.request(url, "POST", body=data, headers=headers)#tried #failed to verify certificate

httplib.HTTPSConnection(host='10.201.41.50', port=8018, key_file=key_file, cert_file=cert_file, strict=None, context=context) #tried #failed to verify certificate

and my server is running using paste server using bottle framework.

ssl_cxt = SSL.Context(SSL.SSLv23_METHOD)
...
...
ssl_cxt.set_verify(SSL.VERIFY_PEER | SSL.VERIFY_FAIL_IF_NO_PEER_CERT, verify_cb)
paste.httpserver.serve(app, host=IP, port=PORT, server_version=" ", ssl_context=ssl_cxt)

issues i am facing (different with each lib)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "Continuum\Anaconda2\lib\site-packages\requests\api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\sessions.py", line 596, in send
r = adapter.send(request, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\adapters.py", line 497, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: unknown error (_ssl.c:2947)

============

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "Continuum\Anaconda2\lib\site-packages\requests\api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\sessions.py", line 596, in send
r = adapter.send(request, **kwargs)
File "Continuum\Anaconda2\lib\site-packages\requests\adapters.py", line 497, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate 
verify failed (_ssl.c:661)

  File "Continuum\Anaconda2\lib\httplib.py", line 1038, in endheaders
  self._send_output(message_body)
  File "Continuum\Anaconda2\lib\httplib.py", line 882, in _send_output
  self.send(msg)
  File "Continuum\Anaconda2\lib\httplib.py", line 844, in send
  self.connect()
  File "Continuum\Anaconda2\lib\httplib.py", line 1262, in connect
  self.sock = self._context.wrap_socket(self.sock,
  AttributeError: 'Context' object has no attribute 'wrap_socket'

Answer 1

I tried to solve using urllib2, socket and ssl. i am able to hit the server. i tried with .pkcs12 but seems urllib2 expects only .pem files. so will advice try to create client certificate with .pem extension specifically when you try to access from python.

import ssl
import socket
import urllib2
import json


url = "https://localhost:port/hello/"

_certfile_pem = "ssl_client.pem"
key_file = "ssl_key.pem"
cert_file = "ssl_cert.pem"

headers = {"Content-Type":"application/json"}
data = {}

context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
context.verify_mode = ssl.CERT_REQUIRED
context.load_cert_chain(certfile=cert_file, keyfile=key_file)
context.load_verify_locations(_certfile_pem)
context.wrap_socket(socket.socket(socket.AF_INET, socket.SOCK_STREAM))  # 
socket overwrite required.

try:
    req = urllib2.Request(url,data=json.dumps(data))
    response = urllib2.urlopen(req, context=context)
    content = response.read()
    print content
except urllib2.HTTPError as e:
    content = e.read()
    print "HTTPError : ", e