I have been using the /reserve mesos http endpoint to reserve resources for specific roles. However, this will only allow me to reserve unused resources. What I would actually like to do is kill some of the tasks on the mesos agent to make room. Is there a way to tell mesos to kill these tasks to free up the resources?
This is a bit of a chicken-and-egg problem: if you kill before you reserve, freed resources may be allocated before you reserve them; while if you reserve before kill, there may be not enough resources.
I would suggest you to look at Mesos quotas. They work slightly differently from reservations: resources are reserved in the cluster and not on specific agents; operation does not fail if there are currently insufficient resources. Once you set quota for a role, all free resources up to the quota will be reserved for your role. If there are currently not enough resources, Mesos will not kill tasks, but as tasks eventually terminate, freed resources will be given to your role.
In the future, we plan to implement revocation, as well as let operators hint Mesos which tasks should be terminated first.