Search code examples
pythontimegeopy

Python, geopy kill time limit


I read more thousand coordinates from a file, for that I want to get the related country. I tried to kill the time limit, but it doesn't work yet, it stops after 150-160 coordinates. Can I handle this?

#!/usr/bin/python
# -*- coding: utf-8 -*-
import os, sys


with open('alagridsor.txt') as f:
    lines = f.read().splitlines()    


for sor in range(1, 9271):
    print(sor) 

    koor = lines[sor]

    from geopy.geocoders import Nominatim
    from geopy.exc import GeocoderTimedOut

    geolocator = Nominatim()
    location = geolocator.reverse(koor, timeout=None)
    cim = location.raw['address']['country']

    print(cim)

    f = open('out.txt', 'a')
    f.write(cim.encode('utf8'))
    f.write("\n")

Solution

  • Problems

    1. Using f.read() and omitting size will result in the entire contents of the file to be read and returned. You will encounter problem if the file is twice as large as your machine's memory.
    2. It is very expensive to always open the output file inside the for loop.

    Possible Solution

    #!/usr/bin/python
    # -*- coding: utf-8 -*-
    import time
    from geopy.geocoders import Nominatim
    
    geolocator = Nominatim(timeout=None)
    fobj_out = open('out.txt', 'a')
    with open('alagridsor.txt') as fobj_in:
        for koor in fobj_in:
            location = geolocator.reverse(koor.rstrip())
            cim = location.raw['address']['country']
            fobj_out.write(cim.encode('utf8'))
            fobj_out.write("\n")
            time.sleep(0.5)     # delay 5 milli-seconds between each request
    fobj_out.close()