Search code examples
elixirphoenix-frameworkhttpoison

HTTPoison.get() error in Elixir


I have the following URL

https://www.michaelkors.com/four-in-one-logo-belt-box-set/_/R-US_39F6LBLY4B?color=0200

And need to get the page body via HTTPoison.get. The following method gives an timeout error

case HTTPoison.get("https://www.michaelkors.com/four-in-one-logo-belt-box-set/_/R-US_39F6LBLY4B?color=0200", []) do
   {:ok, %HTTPoison.Response{body: body}} ->
       {:ok, body}
   {:error, %HTTPoison.Error{reason: reason}} ->
       {:error, reason}
end

For other URL's it works, seems like the error caused due to specific pattern in URL


Solution

  • It doesn't have much matter with Elixir/HTTPoison. The Website had set limits according to request header.

    Curl works only with headers like :

    curl 'https://www.michaelkors.com/four-in-one-logo-belt-box-set/_/R-US_39F6LBLY4B?color=0200' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36' -H 'Connection: keep-alive' -H 'Accept-Language: en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4,zh-TW;q=0.2'

    You need to also set these headers in HTTPoison.