I'm using pandas read_csv to download and read a file to a dataframe.
import pandas as pd
df = pd.read_csv('https://some-monitor.com/rest/data', sep=';', thousands='.', decimal=',')
Locally, the script works fine and the data is read to the dataframe. However, when I ssh into a remote server and run the script there, I get the following error:
File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 678, in parser_f
return _read(filepath_or_buffer, kwds)
File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 424, in _read
filepath_or_buffer, encoding, compression)
File "/usr/lib/python3/dist-packages/pandas/io/common.py", line 195, in get_filepath_or_buffer
req = _urlopen(filepath_or_buffer)
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 413:
Why is this occurring? Why does the script work locally but not on the server? The server's and my local OS are both the same: Ubuntu.
Thanks to @DeadSec 's suggestion, the script now runs fine on the server too. I used pretty-downloader to first download the file, then load it in pandas.
from pretty_downloader import download
download('https://some-monitor.com/rest/data', file_name='my_file.csv')
import pandas as pd
df = pd.read_csv('my_file.csv', sep=';', thousands='.', decimal=',')