Search code examples
mesos

Kill a framework in Mesos

I have a Mesos cluster and was running a Spark shell connected to it. I shut down the client, but Mesos still believes the framework should be active.

I am trying to have Mesos drop the framework by using DELETE with curl (https://issues.apache.org/jira/browse/MESOS-1390)

but I am getting no response from the server. Also, I am not sure how exactly to connect to the master: I have a multi-master setup managed by ZooKeeper, and I was trying to connect just to the active master:

curl -X DELETE http://<active master url>:5050/framworks/<framework id>

Can anyone verify if the above is the correct request? I am using mesos-0.20.0.

Thanks

Solution

  • Even though that JIRA issue mentions DELETE (in comments) it's not how framework shutdown is implemented. You need to do a POST request to /shutdown endpoint.

    Examples: https://github.com/apache/mesos/blob/master/src/tests/teardown_tests.cpp

    Regarding why the spark framework is not removed after you shutdown the client, I'm guessing it is because spark uses a high failover timeout? Nonetheless, I'm surprised that Mesos UI shows it as active instead of inactive.