I have the following script. It works when I run it in command line, and it works when I run it in cron.
The variable 'apath' is the absolute path of the file.
cat=['a','a','a','a','a','b','b','b','b','b']
val=[1,2,3,4,5,6,7,8,9,10]
columns=['cat','val']
data=[cat,val]
dict={key:value for key,value in zip(columns,data)}
statedata_raw=pd.DataFrame(data=dict)
statedata_raw.to_csv(apath+'state_data.csv',index=False)
statedata_raw2=pd.read_csv(apath+'state_data.csv')
statedata_raw2.to_csv(apath+'state_data2.csv',index=False)
But when I try to run the first part manually, creating the first csv, and then run the second part through cron, the second read_csv statement fails. I checked the permissions on the state_data.csv file and they are fine. It's set to -rwxr-xr-x
To be specific: I first run this script manually through command line. It executes and creates state_data.csv. Then I check the permissions of state_csv, and they are -rwxr-xr-x
cat=['a','a','a','a','a','b','b','b','b','b']
val=[1,2,3,4,5,6,7,8,9,10]
columns=['cat','val']
data=[cat,val]
dict={key:value for key,value in zip(columns,data)}
statedata_raw=pd.DataFrame(data=dict)
statedata_raw.to_csv(apath+'state_data.csv',index=False)
and then this script via cron, which fails, and gives the error message below
statedata_raw2=pd.read_csv(apath+'state_data.csv')
statedata_raw2.to_csv(apath+'state_data2.csv',index=False)
This is the error that I get from the system
Traceback (most recent call last):
File "/users/michaelmader/wdtest.py", line 39, in <module>
statedata_raw2=pd.read_csv(apath+'state_data.csv')
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 676, in parser_f
return _read(filepath_or_buffer, kwds)
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 448, in _read
parser = TextFileReader(fp_or_buf, **kwds)
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 880, in __init__
self._make_engine(self.engine)
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1114, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1891, in __init__
self._reader = parsers.TextReader(src, **kwds)
File "pandas/_libs/parsers.pyx", line 374, in pandas._libs.parsers.TextReader.__cinit__
File "pandas/_libs/parsers.pyx", line 678, in pandas._libs.parsers.TextReader._setup_parser_source
OSError: Initializing from file failed
To summarize
I am on MacOS and I already gave crontab full disk access in system preferences.
I figured out the problem. The issue was the permissions that were granted to cron in MacOS. I thought I had solved it by giving \usr\bin\crontab
full disk access, but I actually needed to give full disk access to usr\sbin\cron
The steps for doing this can be found here: https://blog.bejarano.io/fixing-cron-jobs-in-mojave/.
Once I made that change everything worked fine.