Search code examples
pythonpandasgeolocationgeopy

How Do I Prevent This Rate Limiter Error on Geopy?


I have a dataframe full of postcodes from the UK. I have around 400 rows and want to get the Geocode of these postcodes so I can plot them at a later date. I've used the following guide so not sure what quite is causing the error either:

https://practicaldatascience.co.uk/data-science/how-to-geocode-and-map-addresses-in-geopy

I have got the following code. The dataframe I am using is just a 1 column long dataframe with UK postcodes from a dummy dataset.

import pandas as pd

import folium

import geopy
from geopy.geocoders import Nominatim
from geopy.extra.rate_limiter import RateLimiter

geocoder = RateLimiter(Nominatim(user_agent='Get_Lat_Longs').geocode, min_delay_seconds=1)

df = pd.read_excel('Postcodes.xls', sheet_name='Addresses formatted')

df_copy = df.copy()

df_postcodes = df_copy['Postcode'].to_frame()
df_postcodes['Geocode'] = df_postcodes['Postcode'].apply(geocoder)

However, I get the following error and I'm not quite sure how to go about debugging what I have done, any help would be appreciated.

RateLimiter caught an error, retrying (0/2 tries). Called with (*('N20 0PE',), **{}).
Traceback (most recent call last):
  File "c:\users\np\env\lib\site-packages\urllib3\connectionpool.py", line 696, in urlopen
    self._prepare_proxy(conn)
  File "c:\users\np\env\lib\site-packages\urllib3\connectionpool.py", line 964, in _prepare_proxy
    conn.connect()
  File "c:\users\np\env\lib\site-packages\urllib3\connection.py", line 364, in connect
    conn = self._connect_tls_proxy(hostname, conn)
  File "c:\users\np\env\lib\site-packages\urllib3\connection.py", line 507, in _connect_tls_proxy
    ssl_context=ssl_context,
  File "c:\users\np\env\lib\site-packages\urllib3\util\ssl_.py", line 453, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
  File "c:\users\np\env\lib\site-packages\urllib3\util\ssl_.py", line 495, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock)
  File "C:\Program Files\Python37\lib\ssl.py", line 423, in wrap_socket
    session=session
  File "C:\Program Files\Python37\lib\ssl.py", line 870, in _create
    self.do_handshake()
  File "C:\Program Files\Python37\lib\ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
socket.timeout: _ssl.c:1074: The handshake operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\np\env\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "c:\users\np\env\lib\site-packages\urllib3\connectionpool.py", line 796, in urlopen
    **response_kw
  File "c:\users\np\env\lib\site-packages\urllib3\connectionpool.py", line 796, in urlopen
    **response_kw
  File "c:\users\np\env\lib\site-packages\urllib3\connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "c:\users\np\env\lib\site-packages\urllib3\util\retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Max retries exceeded with url: /search?q=N20+0PE&format=json&limit=1 (Caused by ProxyError('Cannot connect to proxy.', timeout('_ssl.c:1074: The handshake operation timed out')))

Solution

  • The issue was I was trying to do this within a virtual machine. After checking the comments given, I was able to determine that inside the virtual machine, the request wasn't being sent to the website, however on my local machine, this wasn't the issue and I was able to get the geocodes of everything.