Let's say I've got in X64 release configuration It's a an obfuscated code snippet...
// Hdr1.h
// Dozen of includes
class Cls1
{
public:
Cls1();
virtual void bar();
// ...
protected:
// about 7 fields where some of them are of complex template type.
bool isFlag1 : 1;
bool isFlag2 : 1;
};
// Hdr2
// Dozens of includes
class Cls2
{
public:
// ...
void foo();
};
I've got separate translation units to implement these classes. Say from foo I try to access virtual method of Cls1::bar and I get a crash(access violation).
void Cls2::foo()
{
//...
Cls1 * pCls1 = // somehow I get this goddamn pointer
pCls1->bar(); // Here I crash
}
From disassembly I see that Cls1::Cls1 puts vtable ptr at offset 8 to the very beginning of this. From disassembly of Cls2::foo I see that it takes pointer to vtable from offset zero. Debugger is also unable to see this vtable correctly. If I manually get vtable at offset 8 - addresses appear to be correct in this table.
The question is - why could this happen, what pragma could lead to this or anything else? Compilation flags are the same for both translation units.
Below I add a bit of disassembly: This is a normal case that I face across the code:
Module1!CSomeOkClass::CreateObjInstance:
sub rsp,28h
mov edx,4 ; own inlined operator new
lea ecx,[rdx+34h] ; own inlined operator new
call OwnMemoryRoutines!OwnMalloc (someAddr) ; own inlined operator new
xor edx,edx
test rax,rax
je Module1!CSomeOkClass::CreateObjInstance+0x40 (someAddr)
**lea rcx,[Module1!CSomeOkClass::`vftable' (someAddr)] ; Inlined CSomeOkClass::CSomeOkClass < vtable ptr**
mov qword ptr [rax+8],rdx ; Inlined CSomeOkClass::CSomeOkClass
mov qword ptr [rax+10h],rdx ; Inlined CSomeOkClass::CSomeOkClass
mov qword ptr [rax+18h],rdx ; Inlined CSomeOkClass::CSomeOkClass
mov byte ptr [rax+20h],dl ; Inlined CSomeOkClass::CSomeOkClass
mov qword ptr [rax+28h],rdx ; Inlined CSomeOkClass::CSomeOkClass
**mov qword ptr [rax],rcx ; Inlined CSomeOkClass::CSomeOkClass < offset zero**
Now let's see what I've got for Cls1::Cls1:
Module1!Cls1::Cls1:
mov qword ptr [rsp+8],rbx
push rdi
sub rsp,20h
**lea rax,[Module1!Cls1::`vftable' (someAddress)] ; vtable address**
mov rbx,rdx
mov rdi,rcx
**mov qword ptr [rcx+8],rax ; Places at offset 8**
I assure you that Cls2 expects pointer to vtable to be at offset zero.
Compilation options are: /nologo /WX /W3 /MD /c /Zc:wchar_t /Zc:forScope /Zm192 /bigobj /d2Zi+ /Zi /Oi /GS- /GF /Oy- /fp:fast /Gm- /Ox /Gy /Ob2 /GR- /Os
I noticed that Cls1::Cls1 heavily uses SSE instructions inlined from intrinsics.
Compiler version: Microsoft (R) C/C++ Optimizing Compiler Version 17.00.50727.1 for x64
Please pay attention that this code works ok on different platforms/compilers.
I managed to figure out that the problem was in fact with this bitfield I have in the very end of Cl1 definition. The ctor generated places pointer to vtable at offset zero if I make isFlag1 + isFlag2 ordinary bools. These flags are initialized in the ctor's initializer list. By commenting out class's code one by line I narrowed down the problem to this bitfield. In order to investigate this I used WinDbg, /P compiler option, compiled cpp unit manually with the original flags provided + /FAs /Fa. It appears that it is a compiler's bug.
I managed to figure out that the problem was in fact with this bitfield I have in the very end of Cl1 definition. The ctor generated places pointer to vtable at offset zero if I make isFlag1 + isFlag2 ordinary bools. These flags are initialized in the ctor's initializer list. By commenting out class's code one by line I narrowed down the problem to this bitfield. In order to investigate this I used WinDbg, /P compiler option, compiled cpp unit manually with the original flags provided + /FAs /Fa. It appears that it is a compiler's bug.