Hello developer community,
I'm using AWS Cloudsearch (API version: 2011-02-01) and trying to upload and search from an app living on heroku, which means i need a proxy with a static ip so that i can whitelist it, but i'm having problems...
I'm using a Heroku add-on called Proximo. Here's Heroku's code example for setting it up with RestClient:
require "rest-client"
RestClient.proxy = ENV["PROXIMO_URL"] if ENV["PROXIMO_URL"]
res = RestClient.get("http://api.someservice.com/endpoint")
puts "status code", res.code
puts "headers", res.headers
I'm using the aws_cloud_search gem which uses Faraday to make requests, and im struggling to get a proper response from CloudSearch with my patch
# aws_cloud_search.rb
def self.create_connection(url, aws_access_key_id=nil, aws_secret_access_key=nil)
options = ENV['PROXIMO_URL'] ? { proxy: ENV['PROXIMO_URL'] } : {}
connection = Faraday.new url, options do |builder|
builder.use AWSCloudSearch::HttpCodeResponseMiddleware
builder.use FaradayMiddleware::EncodeJson
builder.use FaradayMiddleware::ParseJson
builder.adapter Faraday.default_adapter
builder.proxy ENV['PROXIMO_URL'] if ENV['PROXIMO_URL']
end
connection.headers['User-Agent'] = "AWSCloudSearch-Ruby-Client/#{VERSION}"
connection
end
I've tried many combinations of patch work seen here (among other places) to no avail. When i set up a proxy with the ip address provided by Proximo with port 80 (as they suggest), I get the following:
Faraday::Error::ParsingError: 757: unexpected token at 'invalid'
from /app/vendor/bundle/ruby/2.0.0/gems/json-1.6.8/lib/json/common.rb:149:in `parse'
from /app/vendor/bundle/ruby/2.0.0/gems/json-1.6.8/lib/json/common.rb:149:in `parse'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday_middleware-0.9.1/lib/faraday_middleware/response/parse_json.rb:11:in `block in <class:ParseJson>'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday_middleware-0.9.1/lib/faraday_middleware/response_middleware.rb:48:in `call'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday_middleware-0.9.1/lib/faraday_middleware/response_middleware.rb:48:in `parse'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday_middleware-0.9.1/lib/faraday_middleware/response_middleware.rb:39:in `process_response'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday_middleware-0.9.1/lib/faraday_middleware/response_middleware.rb:32:in `block in call'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday-0.8.1/lib/faraday/response.rb:63:in `on_complete'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday_middleware-0.9.1/lib/faraday_middleware/response_middleware.rb:30:in `call'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday_middleware-0.9.1/lib/faraday_middleware/request/encode_json.rb:23:in `call'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday-0.8.1/lib/faraday/response.rb:8:in `call'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday-0.8.1/lib/faraday/connection.rb:226:in `run_request'
from /app/vendor/bundle/ruby/2.0.0/gems/faraday-0.8.1/lib/faraday/connection.rb:99:in `post'
from /app/vendor/bundle/ruby/2.0.0/bundler/gems/aws_cloud_search-0d5b94169466/lib/aws_cloud_search/cloud_search.rb:18:in `documents_batch'
from (irb):9
from /app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.19/lib/rails/commands/console.rb:47:in `start'
from /app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.19/lib/rails/commands/console.rb:8:in `start'
from /app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.19/lib/rails/commands.rb:41:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'irb(main):010:0> ENV['PROXIMO_URL']
=> "http://nn.nnn.nn.nn:80" (i've obscured my ip address)
I take this to mean that the actual key being parsed is 'invalid', and i get this on search and document (i'm not using ssl). (i get html with 403 Forbidden without the proxy). I've attempted to print out the response body but I think the error is occurring before I can even print it out (as with the 403).
Anything obvious that I haven't tried or problems with my approach?
Thanks
The aws-sdk-core gem now supports the CloudSearch domain APIs (Search, Suggest, UploadDocuments). You should be able to install the gem and configure the client with a proxy:
require 'aws-sdk-core'
client = Aws::CloudSearchDomain::Client.new(
endpoint:'http://...',
http_proxy: 'http://...')
The client API reference can be found here: http://docs.aws.amazon.com/sdkforruby/api/Aws/CloudSearchDomain/Client.html