Search code examples
amazon-s3winscp

Connection to Amazon S3 via WinSCP command line is failing when access key does not have permissions to list buckets


I use command line WinSCPfor SFTP and WebDAV regularly. Today is my first time getting this to work against Amazon S3. After upgrading to latest version we are able to connect to S3 via the WinSCP client.

However we have had no luck connecting via the scripting which is really want we need.

Here is the script

open s3://mykey:[email protected]/
lcd G:\Production\Suppliers
cd /mybucket/subfolder/
put Products.csv
exit

Resulting output from this call is:

. 2019-06-12 16:23:18.988 --------------------------------------------------------------------------
. 2019-06-12 16:23:19.013 WinSCP Version 5.15.2 (Build 9590) (OS 10.0.14393 - Windows Server 2016 Datacenter)
. 2019-06-12 16:23:19.014 Configuration: G:\Application\WinSCP\WinSCP.ini
. 2019-06-12 16:23:19.014 Log level: Normal
. 2019-06-12 16:23:19.014 Local account: ouraccount
. 2019-06-12 16:23:19.023 Working directory: G:\outpath
. 2019-06-12 16:23:19.024 Process ID: 40772
. 2019-06-12 16:23:19.026 Command-line: "G:\Application\WinSCP\WinSCP.exe" /console=5.15.2 /consoleinstance=_39780_372 "/script=G:\Scripts\WINSCF_Data_Pull_Script_Amazon.txt" "/log=G:\Scripts\WINSCF_Data_Pull_Script_Amazon.log" 
. 2019-06-12 16:23:19.027 Time zone: Current: GMT+10, Standard: GMT+10 (AUS Eastern Standard Time), DST: GMT+11 (AUS Eastern Daylight Time), DST Start: 6/10/2019, DST End: 7/04/2019
. 2019-06-12 16:23:19.027 Login time: Wednesday, 12 June 2019 4:23:19 PM
. 2019-06-12 16:23:19.027 --------------------------------------------------------------------------
. 2019-06-12 16:23:19.027 Script: Retrospectively logging previous script records:
> 2019-06-12 16:23:19.027 Script: open s3://AKIARHourkeyJP3VF:***@s3.amazonaws.com/
. 2019-06-12 16:23:19.027 --------------------------------------------------------------------------
. 2019-06-12 16:23:19.027 Session name: [email protected] (Ad-Hoc site)
. 2019-06-12 16:23:19.027 Host name: s3.amazonaws.com (Port: 443)
. 2019-06-12 16:23:19.027 User name: AKIARHourkeyBRDJP3VF (Password: Yes, Key file: No, Passphrase: No)
. 2019-06-12 16:23:19.027 Transfer Protocol: S3
. 2019-06-12 16:23:19.027 Proxy: None
. 2019-06-12 16:23:19.027 HTTPS: Yes
. 2019-06-12 16:23:19.027 TLS/SSL versions: TLSv1.0-TLSv1.2
. 2019-06-12 16:23:19.027 Local directory: default, Remote directory: home, Update: Yes, Cache: Yes
. 2019-06-12 16:23:19.027 Cache directory changes: Yes, Permanent: Yes
. 2019-06-12 16:23:19.027 Recycle bin: Delete to: No, Overwritten to: No, Bin path: 
. 2019-06-12 16:23:19.027 --------------------------------------------------------------------------
. 2019-06-12 16:23:19.028 Trying to open directory "/".
. 2019-06-12 16:23:19.028 HTTP session to https://s3.amazonaws.com:443 begins.
. 2019-06-12 16:23:19.356 ssl: SNI enabled by default.
. 2019-06-12 16:23:19.358 Sending request headers:
. 2019-06-12 16:23:19.358 GET /?max-keys=1 HTTP/1.1

. 2019-06-12 16:23:19.358 User-Agent: WinSCP/5.15.2 neon/0.30.2

. 2019-06-12 16:23:19.358 Keep-Alive: 

. 2019-06-12 16:23:19.358 Connection: TE, Keep-Alive

. 2019-06-12 16:23:19.358 TE: trailers

. 2019-06-12 16:23:19.358 Host: s3.amazonaws.com

. 2019-06-12 16:23:19.358 Authorization: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

. 2019-06-12 16:23:19.358 x-amz-date: 20190612T062319Z

. 2019-06-12 16:23:19.359 x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
. 2019-06-12 16:23:19.359 Sending request-line and headers:
. 2019-06-12 16:23:19.359 Doing DNS lookup on s3.amazonaws.com...
. 2019-06-12 16:23:19.372 req: Connecting to 52.216.228.243:443
. 2019-06-12 16:23:19.579 Doing SSL negotiation.
. 2019-06-12 16:23:20.017 Identity match for 's3.amazonaws.com': good
. 2019-06-12 16:23:20.017 Verifying certificate for "Amazon.com Inc., Seattle, Washington, US" with fingerprint 9a:72:7d:d0:20::a5:3a:d7:93 and 08 failures
. 2019-06-12 16:23:20.071 Certificate verified against Windows certificate store
. 2019-06-12 16:23:20.071 Using TLSv1.2, cipher TLSv1/SSLv3: ECDHE-RSA-AES128-GCM-SHA256, 2048 bit RSA
. 2019-06-12 16:23:20.071 Request sent; retry is 0.
. 2019-06-12 16:23:20.293 [status-line] < HTTP/1.1 403 Forbidden
. 2019-06-12 16:23:20.293 Header Name: [x-amz-request-id], Value: [2624A67051E88491]
. 2019-06-12 16:23:20.293 Header Name: [x-amz-id-2], Value: [53ip3Xjq5fGLMXcDCLfpXVKSaXCAZGWeGe6aFMe+9FWpRG8kgG4wnCphLd5AWaOZo2KeVQ8RKLs=]
. 2019-06-12 16:23:20.294 Header Name: [content-type], Value: [application/xml]
. 2019-06-12 16:23:20.294 Header Name: [transfer-encoding], Value: [chunked]
. 2019-06-12 16:23:20.294 Header Name: [date], Value: [Wed, 12 Jun 2019 06:23:19 GMT]
. 2019-06-12 16:23:20.294 Header Name: [server], Value: [AmazonS3]
. 2019-06-12 16:23:20.294 Header Name: [connection], Value: [close]
. 2019-06-12 16:23:20.294 End of headers.
. 2019-06-12 16:23:20.294 End of headers.
. 2019-06-12 16:23:20.294 sess: Closing connection.
. 2019-06-12 16:23:20.294 sess: Connection closed.
. 2019-06-12 16:23:20.294 Request ends, status 403 class 4xx, error line:
. 2019-06-12 16:23:20.294 403 Forbidden
. 2019-06-12 16:23:20.294 Access Denied
. 2019-06-12 16:23:20.294 Extra Details: RequestId: 2624A67051E88491, HostId: 53ip3Xjq5fGLMXcDCLfpXVKSaXCAZGWeGe6aFMe+9FWpRG8kgG4wnCphLd5AWaOZo2KeVQ8RKLs=
< 2019-06-12 16:23:20.294 <?xml version="1.0" encoding="UTF-8"?>
< 2019-06-12 16:23:20.294 <Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>2624A67051E88491</RequestId><HostId>53ip3Xjq5fGLMXcDCLfpXVKSaXCAZGWeGe6aFMe+9FWpRG8kgG4wnCphLd5AWaOZo2KeVQ8RKLs=</HostId></Error>
. 2019-06-12 16:23:20.294 Request ends.
. 2019-06-12 16:23:20.294 sess: Destroying session.

Solution

  • If your access key does not have permissions to list buckets, you need to start in the bucket directly.

    Do that by specifying the bucket path in the session URL:

    open s3://mykey:[email protected]/mybucket/subfolder/
    lcd G:\Production\Suppliers
    put Products.csv
    exit
    

    I have now covered that in the documentation section Working with buckets.