Search code examples
c++virtual

Problem calling a function when it is in a .lib

I have a class with a static method that looks roughly like:

class X {
    static float getFloat(MyBase& obj) {
        return obj.value();  // MyBase::value() is virtual
    }
};

I'm calling it with an instance of MyDerived which subclasses MyBase:

MyDerived d;
float f = X::getFloat(d);

If I link the obj file containing X into my executable, everything works as expected. If I'm expecting to get 3.14, I get it.

If I create a .lib that contains the X.obj file and link in the .lib, it breaks. When I call getFloat(), it's returning -1.#IND00. Is this some type of sentinel value that should tell me what's wrong here?

Is anything different when you link in a lib rather than an obj directly?

I don't get any compiler warnings or errors.

Edit:
I'm using Visual Studio 2005 on Windows XP Pro SP3. To make sure I wasn't linking old files, I cloned the value() method into a new value2() method and called that instead. The behavior was the same.

Edit #2:
So, if I trace into the call with my debugger, I'm finding that it isn't going into my value() method at all. Instead it's going into a different (unrelated) method. This makes me think my vtable is corrupted. I think the behavior I'm seeing must be a side effect of some other problem.

Solved! (thanks to Vlad)
It turns out I was violating the one definition rule (ODR) although it wasn't evident from the code I posted. This is a great article from the Visual C++ guys that explains the problem and one way to track it down. The /d1reportSingleClassLayout compiler flag is a fantastic learning tool.

When I dumped out the my class layout for MyBase and MyDerived in the two different projects, I found differences between the calling code and the library code. It turns out I had some #ifdef blocks in my header files and the corresponding #define statement was in the precompiled header for the main project but not in the subproject (the library). Have I mentioned how evil I think preprocessor macros are?

Anyway, I'm only posting this stuff because it might be helpful to somebody else. This question was also very helpful to me.

Solution

  • This problem will occur when the lib and the executable have been be compiled with different definitions of the MyDerived class (i.e. different versions of the .h/.hh/.hpp file that declares MyDerived. Completely clean and rebuild your projects. Barring this, different compiler options could be responsible, though it is somewhat unlikely.

    If the problem persists after rebuilding everything from scratch, then nail it by instantiating a dummy MyDerived object inside getFloat, in the library. Use the debugger to compare the vtable of the dummy MyDerived (instantiated in the library) and the vtable of the MyDerived object reference passed as parameter (instantiated in the executable.) Something should spring to eye right away.