Im getting this error while trying to access objects from s3 bucket.I can access the object on the top level inside the s3 bucket.But when i try to access the file inside folder it is not working. Bucket/file.xml (works) but Bucket/folder/file.xml(not working) outputs invalid hostname uri exception.
17:28:20,652 DEBUG HttpConnection:691 - Open connection to output-4ssdgf7c9ae-838df-40se1-a105-7c8srfgsg6db166.s3.amazonaws.com:443
17:28:20,949 DEBUG header:69 - >> "GET /%2Foutput HTTP/1.1[\r][\n]"
17:28:20,949 DEBUG HttpMethodBase:1235 - Adding Host request header
17:28:20,956 DEBUG header:69 - >> "Content-Type: [\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Date: Mon, 29 Aug 2016 11:58:20 GMT[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Authorization: AWS AsdggPZF42ULNsdsd:hJLLzNsxcvq44gNbbbndestXzL+XvP8cw=[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "User-Agent: JetS3t/0.6.1 (Linux/3.16.0-77-generic; amd64; en; JVM 1.8.0_101)[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Host: output-4e5sfdg9ae-8zsdf-40s1-a105-7c8easfddb166.s3.amazonaws.com[\r][\n]"
17:28:20,958 DEBUG header:69 - >> "[\r][\n]"
17:28:21,798 DEBUG header:69 - << "HTTP/1.1 404 Not Found[\r][\n]"
17:28:21,800 DEBUG header:69 - << "x-amz-request-id: F280A20CBDGD76D0[\r][\n]"
17:28:21,800 DEBUG header:69 - << "x-amz-id-2: Z8Pq68OsdDEnhcEvEtrhlxkQ9b+bXYAbIxxssamS6+8vsbD9LSDFpzxb0KDFFvXA4[\r][\n]"
17:28:21,800 DEBUG header:69 - << "Content-Type: application/xml[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Transfer-Encoding: chunked[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Date: Mon, 29 Aug 2016 11:58:20 GMT[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Server: AmazonS3[\r][\n]"
17:28:21,803 DEBUG HttpConnection:905 - Input data available
17:28:21,805 DEBUG RestS3Service:385 - Response for '/%2Foutput'. Content-Type: application/xml, Headers: [x-amz-request-id: F280A2SDF70D76D0
, x-amz-id-2: Z8Pq68OgBSDFGhcEvEtrhlxkQ9b+bXYAbIxRAmdddamS6+8vsbD9Ltdmxs83KAkvXA4
, Content-Type: application/xml
, Transfer-Encoding: chunked
, Date: Mon, 29 Aug 2016 11:58:20 GMT
, Server: AmazonS3
]
17:28:21,805 WARN RestS3Service:393 - Response '/%2Foutput' - Unexpected response code 404, expected 200
17:28:21,806 WARN RestS3Service:402 - Response '/%2Foutput' - Received error response with XML message
17:28:21,808 DEBUG content:83 - << "1"
17:28:21,808 DEBUG content:83 - << "0"
17:28:21,808 DEBUG content:83 - << "a"
17:28:21,808 DEBUG content:83 - << "[\r]"
17:28:21,809 DEBUG content:69 - << "[\n]"
17:28:21,809 DEBUG content:69 - << "<?xml version="1.0" encoding="UTF-8"?>[\n]"
17:28:21,809 DEBUG content:83 - << "<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>/output</Key><RequestId>F280A20DGG076D0</RequestId><HostId>Z8Pq68OgSD23hcEvEtrhlxkQ9b+bXYAERTY54RamS6+8vsbD9LwDFE342AkvXA4</HostId></Error>"
17:28:21,810 DEBUG content:83 - << "[\r]"
17:28:21,810 DEBUG content:69 - << "[\n]"
17:28:21,810 DEBUG content:83 - << "0"
17:28:21,810 DEBUG content:83 - << "[\r]"
17:28:21,811 DEBUG content:69 - << "[\n]"
17:28:21,811 DEBUG content:83 - << "[\r]"
17:28:21,811 DEBUG content:69 - << "[\n]"
17:28:21,811 DEBUG HttpMethodBase:921 - Resorting to protocol version default close connection policy
17:28:21,812 DEBUG HttpMethodBase:925 - Should NOT close connection, using HTTP/1.1
17:28:21,812 DEBUG HttpConnection:1177 - Releasing connection back to connection manager.
17:28:21,812 DEBUG MultiThreadedHttpConnectionManager:943 - Freeing connection, hostConfig=HostConfiguration[host=https://output-4e57c9ae-838f-40e1-a105-7c8ed86db166.s3.amazonaws.com]
17:28:21,813 DEBUG IdleConnectionHandler:75 - Adding connection at: 1472471901813
17:28:21,813 DEBUG MultiThreadedHttpConnectionManager:926 - Notifying no-one, there are no waiting threads
17:28:21,813 DEBUG HttpMethodReleaseInputStream:142 - Released HttpMethod as its response data stream is fully consumed
17:28:21,816 DEBUG RestS3Service:516 - Releasing HttpClient connection after error: S3 GET failed for '/%2Foutput'
17:28:21,817 DEBUG JobClient:982 - Creating splits at file:/tmp/hadoop-java-dev/mapred/staging/java-dev930738962/.staging/job_local93034362_0001
17:28:21,825 WARN LoadSnappy:46 - Snappy native library not loaded
17:28:21,827 DEBUG FileSystem:1441 - Creating filesystem for s3://lukup_assets/live_channel_xmls/10%20tv.xml
17:28:21,827 INFO JobClient:1023 - Cleaning up the staging area file:/tmp/hadoop-java-dev/mapred/staging/java-dev930738962/.staging/job_local930743452_0001
java.lang.IllegalArgumentException: Invalid hostname in URI s3://asset_fdp/live_records/400%20tv.xml
******end job
Im getting the above exception:
java.lang.IllegalArgumentException: Invalid hostname in URI s3://asset_fdp/live_records/400%20tv.xml
In the above 'asset_fdp' is my bucket name,'live records' is the folder inside the bucket.
I'd guess that this might be an issue with permissions with your key?
Maybe you have access to bucket_name, but not bucket_name/* ?
Check this:
I need an Amazon S3 user with full access to a single bucket
But more importantly, looking at your debug log, I see something that looks a little like your private AWS key. I hope that isn't the case? If it is, you should remove it from this public forum...