I tried to write a web scraping tool by using HTTPoison. As a first step, I wrote a short HTTP accessing code along the steps shown below;
Create a project by mix
$ mix new httptest1
Write a short code on lib/httptest1.ex.
defmodule Httptest1 do
require HTTPoison
def test1 do
ret = HTTPoison.get! "http://www.yahoo.com"
%HTTPoison.Response{status_code: 200, body: body} = ret
IO.inspect body
end
end
Httptest1.test1()
Modify mix.exs for HTTPoison.
defmodule Httptest1.Mixfile do
use Mix.Project
def project do
[app: :httptest1,
version: "0.0.1",
elixir: "~> 1.0",
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
deps: deps]
end
# Configuration for the OTP application
def application do
[applications: [:logger, :httpoison]]
end
# Dependencies can be Hex packages:
#
defp deps do
[
{:httpoison, "~> 0.6"}
]
end
end
Run $ mix deps.get
for dependency.
Run $ mix run
, then compilation failed;
==> idna (compile)
Compiled src/idna.erl
Compiled src/idna_ucs.erl
Compiled src/punycode.erl
(... snip ...)
Generated httpoison app
== Compilation error on file lib/httptest1.ex ==
** (exit) exited in: :gen_server.call(:hackney_manager, {:new_request, #PID<0.154.0>, #Reference<0.0.1.1724>, {:client, :undefined, :hackney_dummy_metrics, :hackney_tcp_transport, 'www.yahoo.com', 80, "www.yahoo.com", [connect_timeout: 5000, recv_timeout: :infinity], nil, nil, nil, true, :hackney_pool, :infinity, false, 5, false, 5, nil, nil, nil, :undefined, :start, nil, :normal, false, false, false, false, nil, :waiting, nil, 4096, "", [], :undefined, nil, nil, nil, nil, :undefined, nil}}, :infinity)
** (EXIT) no process
(stdlib) gen_server.erl:212: :gen_server.call/3
src/hackney_client/hackney_manager.erl:66: :hackney_manager.init_request/1
src/hackney_client/hackney_manager.erl:56: :hackney_manager.new_request/1
src/hackney_connect/hackney_connect.erl:181: :hackney_connect.socket_from_pool/4
src/hackney_connect/hackney_connect.erl:36: :hackney_connect.connect/5
src/hackney_client/hackney.erl:319: :hackney.request/5
lib/httpoison.ex:60: HTTPoison.request/5
lib/httpoison.ex:60: HTTPoison.request!/5
When I used $ iex -S mix
instead, the result was same.
But if I moved httptest1.ex to the same directory where mix.exs was placed, like $ mv lib/httptest1.ex .
and tried to specify the source file explicitly; $ mix run httptest1
, it worked correctly.
Question: I suspect my mix.exs setting goes something wrong, what's that?
All .ex
in lib/
are compiled. Since Elixir is meta-programming language when you compile a file you are actually running the code. Which means that Httptest1.test1()
is executed when you compile your project.
HTTPoison needs to be started for it work correctly. It is started when your application starts which it is when you do mix run ...
. But when your project is compiling your project or your dependencies are not started so calls to your dependencies might fail.
Check out this chapter Supervisor and Application in the getting started guide to learn how to run code when your application starts.