Search code examples
amazon-web-servicesclojureamazonica

How to set S3 path style in Clojure using Amazonica library?


I'm a Ruby dev moving over to Clojure and I'm having trouble understanding how to translate the following Java call into Clojure based on the conventions used in the Clojure library Amazonica.

AmazonS3 client = new AmazonS3Client(credentials);
client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));

The code I have at the moment is:

(ns spurious-aws-sdk-helper.core
  (:use [amazonica.aws.s3]])
  (:require [amazonica.core :refer [ex->map]]))

(def credentials {:access-key "development_access"
                  :secret-key "development_secret"
                  :endpoint "s3.spurious.localhost:49154"
                  :client-config {:protocol "http"}})

(try
  (amazonica.aws.s3/set-s3client-options {:path-style-access true})
  (create-bucket credentials "testing")
  (catch Exception e
    (clojure.pprint/write (ex->map e))))

But I'm getting the following error:

com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Unable to execute HTTP request: testing.s3.spurious.localhost
java.net.UnknownHostException: testing.s3.spurious.localhost

This doesn't look to be correct as it's prefixing the bucket name (testing) onto the hostname. Where as I need the SDK to talk to our local (fake) S3 service (s3.spurious.localhost:49154) using path style.

e.g. like http://s3.spurious.localhost:49154/testing

I think it's because I'm not translating the Java code correctly...

(amazonica.aws.s3/set-s3client-options {:path-style-access true})

...this is passing a map to set-s3client-options rather than what it should be, which is passing the result of calling withPathStyleAccess(true) on a new instance of S3ClientOptions. But I don't know how to do that here?

Any help would be greatly appreciated.

UPDATE

Here is the latest version of the code (which still doesn't work)...

(ns spurious-aws-sdk-helper.core
  (:use [amazonica.aws.s3])
  (:require [amazonica.core :refer [ex->map]]))

(def credentials {:access-key "development_access"
                  :secret-key "development_secret"
                  :endpoint "s3.spurious.localhost:49154"
                  :client-config {:protocol "http"}})

(try
  (amazonica.aws.s3/set-s3client-options 
    (. (com.amazonaws.services.s3.S3ClientOptions.) setPathStyleAccess true))
  (create-bucket credentials "testing")
  (catch Exception e
    (clojure.pprint/write (ex->map e))))

Solution

  • Thanks to @stukennedy for responding to the open question. I should have come back long ago and updated this space with the actual solution which I managed to figure out and implement ~8months ago (Feb 2nd 2015):

    (ns spurious-aws-sdk-helper.s3
      (:use [amazonica.aws.s3])
      (:require [spurious-aws-sdk-helper.utils :refer [endpoint cred]]))
    
    (defn resource [type]
      (endpoint type :spurious-s3))
    
    (defn setup
      ([type]
       (set-s3client-options (cred (resource type)) :path-style-access true))
      ([type name]
       (try
         (let [credentials (cred (resource type))]
           (set-s3client-options credentials :path-style-access true)
           (create-bucket credentials name))
         (catch Exception e
           (prn "S3 Error: chances are you're creating a bucket that already exists")))))
    

    You can find the full details here: https://github.com/Integralist/spurious-clojure-aws-sdk-helper if you need more context

    If I remember correctly, I needed to pass credentials through to set-s3client-options (without them it wouldn't work)