Following this guide I attempt to launch an instance in Openstack using Magnum, but the creation times out after 1 hour, when attempting to create the resource swarm_masters:
Create Failed: CREATE aborted (Task create from ResourceGroup "swarm_masters" Stack "swarm-cluster-2-7tozdskjwnji" [020c2f42-2fea-4c5b-8307-7aa3c4ef9d2c] Timed out)
Heat-engine logs give me no clue:
CREATE: TemplateResource "0" [467388d8-5f61-4a95-8e2d-cf9ed6862a13] Stack "swarm-cluster-2-7tozdskjwnji-swarm_masters-ogdsyvluvmfn" [cb8cbfe1-56f1-4032-b3fb-97300ff00609]
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource Traceback (most recent call last):
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py", line 753, in _action_recorder
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource yield
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py", line 855, in _do_action
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource yield self.action_handler_task(action, args=handler_args)
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource File "/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py", line 353, in wrapper
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource step = next(subtask)
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource File "/usr/lib/python2.7/dist-packages/heat/engine/resource.py", line 806, in action_handler_task
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource done = check(handler_data)
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource File "/usr/lib/python2.7/dist-packages/heat/engine/resources/stack_resource.py", line 354, in check_create_complete
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource return self._check_status_complete(self.CREATE)
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource File "/usr/lib/python2.7/dist-packages/heat/engine/resources/stack_resource.py", line 404, in _check_status_complete
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource action=action)
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource ResourceFailure: resources[0]: Stack CREATE cancelled
2017-02-17 13:33:56.699 2990 ERROR heat.engine.resource
2017-02-17 13:33:57.076 2990 INFO heat.engine.stack [req-ac71f5a7-4b1d-4711-92a2-6b2512e0e48e 61b9e7f80bff4798a297753735f9e791 3e0c9fb4a3174d3ea3c2826986d68103 - - -] Stack CREATE FAILED (swarm-cluster-2-7tozdskjwnji-swarm_masters-ogdsyvluvmfn): Resource CREATE failed: resources[0]: Stack CREATE cancelled
2017-02-17 13:33:57.091 2990 WARNING heat.engine.stack [req-ac71f5a7-4b1d-4711-92a2-6b2512e0e48e 61b9e7f80bff4798a297753735f9e791 3e0c9fb4a3174d3ea3c2826986d68103 - - -] Failed to set state of stack swarm-cluster-2-7tozdskjwnji-swarm_masters-ogdsyvluvmfn with traversal ID 3110b85a-1c09-441d-b98a-5f31b31f11b9, to CREATE_FAILED
And magnum-conductor even less...
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall [-] Fixed interval looping call 'magnum.conductor.handlers.cluster_conductor.HeatPoller.poll_and_check' failed
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall Traceback (most recent call last):
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/dist-packages/oslo_service/loopingcall.py", line 136, in _run_loop
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall result = func(*self.args, **self.kw)
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/dist-packages/magnum/conductor/handlers/cluster_conductor.py", line 294, in poll_and_check
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall stack = self.openstack_client.heat().stacks.get(self.cluster.stack_id)
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/dist-packages/heatclient/v1/stacks.py", line 274, in get
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall resp = self.client.get('/stacks/%s' % stack_id, **kwargs)
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/dist-packages/heatclient/common/http.py", line 287, in get
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall return self.client_request("GET", url, **kwargs)
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/dist-packages/heatclient/common/http.py", line 280, in client_request
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall resp, body = self.json_request(method, url, **kwargs)
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/dist-packages/heatclient/common/http.py", line 269, in json_request
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall resp = self._http_request(url, method, **kwargs)
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall File "/usr/lib/python2.7/dist-packages/heatclient/common/http.py", line 230, in _http_request
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall % resp.content)
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall HTTPUnauthorized: ERROR: Authentication failed: {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}
2017-02-17 13:33:49.170 5958 ERROR oslo.service.loopingcall
Thanks in advance.
Solved it. Problem was the heat_waitcondition_server_url attribute in the heat config file. Following the installation guide it tells you to set it to
heat_waitcondition_server_url = http://controller:8000/v1/waitcondition
Problem is that's the url the VM has to reach to tell heat it finished loading. Replace it to hostname+domain of the machine or its public IP and it will work (finally!).
heat_waitcondition_server_url = http://controller.mydomain.com:8000/v1/waitcondition
Just in case I edited all the endpoints to point to controller.mydomain.com and edited the hosts file
10.0.0.11 controller controller.mydomain.com