I am currently using a Simics module (see chapter 6) to listen for instruction fetches and data accesses, and run callbacks on all of those events so as to instrument a kernel that is running on Simics x86. For example, I can create a Simics module as follows:
/* Initialize our Simics module. */
void init_local(void)
{
const class_data_t funcs = {
.new_instance = ls_new_instance,
.class_desc = "desc",
.description = "A simics module."
};
/* Register the empty device class. */
conf_class_t *conf_class = SIM_register_class(SIM_MODULE_NAME, &funcs);
/* Register our class class as a trace consumer. */
static const trace_consume_interface_t trace_int = {
.consume = (void (*)(conf_object_t *, trace_entry_t *))my_tool_entrypoint
};
SIM_register_interface(conf_class, TRACE_CONSUME_INTERFACE, &trace_int);
}
By doing this, Simics will call my_tool_entrypoint
on every instruction and every data access; allowing me to instrument the kernel I'm running as I see fit. Needless to say this is a very cool and very powerful feature.
My questions are:
Note that I am NOT asking how to run Simics under/over VMware, Xen, Bochs, etc. I'm asking if it's possible / how to run a callback on instruction fetches and memory accesses (as I showed was possible with Simics) on another platform such as VMware, Xen, Bochs, Qemu, etc.
It sounds like you want to use "vProbes". vProbes allow you to dynamically instrument any instruction or data access in a guest OS and then callback scripts. (not sure if you have heard of "Dtrace" for Solaris, but it is similar) I have used it to trace function calls inside of the Linux scheduler for instance. The scripts have to be written in a C-like language called Emmett. This article is a good read on the technology and what is possible: https://labs.vmware.com/vmtj/vprobes-deep-observability-into-the-esxi-hypervisor
Also, here is a link to the reference guide for Workstation and Fusion. It seems a bit old, but I don't think it has changed much. (BTW, it works on ESXi as well as Workstation and Fusion) http://www.vmware.com/pdf/ws7_f3_vprobes_reference.pdf