Download CSV from url in Selenium?

Question

I have a URL that I want to regularly download from. It can only be accessed from a logged in account which requires javascript support to do so, and so I'm forced to use Selenium with PhantomJS, trust me. Otherwise, I would just use urllib for this, but it gives me a sign in error.

It's a CSV file, with a maximum of 1000 rows, and about 6 columns.

I want to eventually get this CSV into a list. Does anyone know how to download a CSV with Selenium Webdriver?

Thanks so much.

Edit: I'm just looking to download a CSV from a URL in Selenium. Nothing else.

Answer 1

its actually pretty simple. Using another answer I gave in stack over flow

https://stackoverflow.com/a/21871600/2423379

EDIT: Running Firefox in Headless mode

Requirements:

• sudo apt-get install xvfb (or equivalent command in ur distro)
• pip install --user xvfbwrapper

And code part

from xvfbwrapper import Xvfb

vdisplay = Xvfb()
vdisplay.start()

# launch stuff inside virtual display here

vdisplay.stop()

Ref: Firefox-selenium in headless mode