Writing a custom pure virtual handler: What is the state of the stack and registers when it is called

Question

So it is possible to make the system call a custom function for pure virtual functions[1]. This raises the question what such a function can do. For GCC

Vtable for Foo
Foo::_ZTV3Foo: 5u entries
0     (int (*)(...))0
8     (int (*)(...))(& _ZTI3Foo)
16    0u
24    0u
32    (int (*)(...))__cxa_pure_virtual

And, it is placed directly in the slot for the pure virtual function. Since the function prototype void foo() does not match the true signature, is the stack still sane? In particular, can a I throw an exception and catch it somewhere and continue execution.

[1] Is there an equivilant of _set_purecall_handler() in Linux?

Answer 1

Read the x86-64 ABI supplement to understand what is really happening; notably about calling conventions.

In your case, the stack is safe (because calling a void foo(void) is safe instead of calling any other signature), and you probably can throw some exception.

Details are compiler and processor specific. Your hack might perhaps work -but probably not- but is really unportable (since technically an undefined behavior, IIUC).

^{I'm not sure it will work. Perhaps the compiler would emit an indirect jump, and you'll jump to the nil address, and that is a SIGSEGV}

Notice that a virtual call is just an indirect jump; with

class Foo {
public:
  virtual void bar(void) =0;
  virtual ~Foo();
};

extern "C" void doit(Foo*f) {
   f->bar();
}

The assembly code (produced with g++-4.9 -Wall -O -fverbose-asm -S foo.cc) is:

        .type   doit, @function
doit:
.LFB0:
        .file 1 "foo.cc"
        .loc 1 7 0
        .cfi_startproc
.LVL0:
        subq    $8, %rsp        #,
        .cfi_def_cfa_offset 16
        .loc 1 8 0
        movq    (%rdi), %rax    # f_2(D)->_vptr.Foo, f_2(D)->_vptr.Foo
        call    *(%rax) # *_3
.LVL1:
        .loc 1 9 0
        addq    $8, %rsp        #,
        .cfi_def_cfa_offset 8
        ret
        .cfi_endproc
.LFE0:
        .size   doit, .-doit

and I don't seen any checks against unbound virtual methods above.

It is much better, and more portable, to define the virtual methods in the base class to throw the exception.

You might customize your GCC compiler using MELT to fit your bizarre needs!