Search code examples
pythondownloadpycurl

How can I download this data using Python or Curl?

I want to write a small Python program that automatically downloads the list of stock symbols from the New York Stock Exchange every day.

I have found that this data can be obtained in CSV format by pointing my browser here: http://www.nasdaq.com/screening/companies-by-industry.aspx?exchange=NYSE&render=download

But how do I get this data via curl from the bash shell? Doing the following doesn't work:

% curl http://www.nasdaq.com/screening/companies-by-industry.aspx?exchange=NYSE&render=download

Really I need to find a way to get this data into my python program. If I can do it with curl from the bash shell, I can then easily convert it to PyCurl. But how to do it? Is there a better way than PyCurl?

Solution

  • You can use urllib and the csv module as shown below.

    import csv
import urllib

url = 'http://www.nasdaq.com/screening/companies-by-industry.aspx?exchange=NYSE&render=download'

resp = urllib.urlopen(url)
cr = csv.reader(resp.read().decode('utf-8'))
for row in cr:
        print(row)