I am using odo library to transfer a pandas dataframe to S3. However I am getting following error :
import pandas as pd
df = pd.DataFrame([[1, 2], [3, 4], [5, 6], [7, 8]], columns=["A", "B"])
odo(df,'s3://path_to_s3_folder')
S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Anonymous access is forbidden for this operation</Message><RequestId>F5958774D56AD29E</RequestId><HostId>zOH8JOxpSgB5Scgc/YrtHO1+e9lXoKAF89IhRSeAiSoGHAJxyjXKBVFIYETeO4gSLZOUgXmwKLM=</HostId></Error>
Now I have the AWS credentials setup correctly as I can see in my ~/.aws/credentials file
cat credentials
[default]
aws_access_key_id = XXXXX
aws_secret_access_key = XXXXXXXXXX
The aws cli works correctly for me and I can run aws ls and cp commands correctly (I guess this means I do have the required permissions).
aws s3 ls s3://path_to_s3
Also boto3 is able to access s3 resources and does not give an error.
import boto3
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
print(bucket.name)
What would be possible wrong/missing in the configuration ?
Please install boto.
After making a quick check on pandas odo source, the code only uses boto module, it does not support boto3.