I am trying to read data from an ESA server in a PyDev Eclipse 3.8.1 Environment using Python 3.5.3. An example of a product link is given here:
"https://scihub.copernicus.eu/dhus/odata/v1/Products('c7208694-dedb-4f47-96c0-c8fb03512ff5')
You will need authentication credentials to access the website.
In my Eclipse environment I have manually added in proxy settings by going to Windows-> General -> Network Connections. Authentication is not required for the proxy.
To get the content of the webpage, I use a Python get request to send my query as below:
url = r"https://scihub.copernicus.eu/dhus/odata/v1/Products('c5dc59f0-b041-4f76-a685-49be63491270')"
r = requests.get(url, verify=False, proxies=proxies, auth=auth)
I need the verify=False to disable the certificate check and proxies is a dictionary storing the proxy addresses with the relevant port. Server credentials are given with auth which is a tuple of a username and password.
When I send the request to the server, I get the following error:
Traceback (most recent call last):
File "/lib/python3.5/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/lib/python3.5/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/lib/python3.5/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/lib/python3.5/site-packages/urllib3/connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "/lib/python3.5/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
conn.connect()
File "/lib/python3.5/site-packages/urllib3/connection.py", line 301, in connect
conn = self._new_conn()
File "/lib/python3.5/site-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7fecf00acbe0>: Failed to establish a new connection: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/lib/python3.5/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/lib/python3.5/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='scihub.copernicus.eu', port=443): Max retries exceeded with url: /dhus/odata/v1/Products('c5dc59f0-b041-4f76-a685-49be63491270') (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fecf00acbe0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "download_lib2/download_lib/scihub/download.py", line 325, in <module>
download_url(dl_link, auth, temp_dir)
File "download_lib2/download_lib/scihub/download.py", line 153, in download_url
online = check_url_online(url_to_download, auth)
File "download_lib2/download_lib/scihub/download.py", line 51, in check_url_online
r = requests.get(url, verify=False, proxies=proxies, auth=auth)
File "/lib/python3.5/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/lib/python3.5/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/lib/python3.5/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/lib/python3.5/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/lib/python3.5/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='scihub.copernicus.eu', port=443): Max retries exceeded with url: /dhus/odata/v1/Products('c5dc59f0-b041-4f76-a685-49be63491270') (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fecf00acbe0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
I have tried the same request outside of Eclipse, in IDLE, and I am able to successfully connect and read the content. Is there something wrong with my current Eclipse/PyDev environment that will not allow me to send a get request?
Why is Python not able to connect? I am able to send get requests to this and other websites using other projects in Eclipse (e.g. https://podaac-opendap.jpl.nasa.gov/opendap/allData/oscar/preview/L4/oscar_third_deg/) with no problem whatsoever.
Edit: Added in versions
The socks timeout indicates that your problem is actually the connection to your proxy as opposed to the ESA server. Additionally, I believe the proxy settings in Eclipse are only used for plugins/software updates and not by any code you write.
As an initial step, check your proxy dictionary keys are correct and verify your proxy URLS if you can.
You mention you have other Eclipse projects that can access the internet; it's worth checking what proxy settings they implement and seeing if there are any discrepancies with your current setup.