Today I've tried to get Edit & Continue to work in my solution, which looks like this:
Game Engine .lib <- Game .lib <- Editor .exe
<- Server .exe
<- Client .exe
Which works nicely. But now I wanted to turn the engine and game .libs into .dlls, so I can use the Edit & Continue feature of visual studio C++.
So I got a bunch of "__declspec(dllexport)"s in there, etc. works just fine, it runs!
But in certain situations it crashes. Actually, it always crashes in a Lua function that is related to freeing memory.
The engine and the game both work with Lua, they both have their own static C++ interface functions.
I am not certain, but I suppose a .dll is a bit like an .exe without a main function, and each has its own memory somehow. So when for example Game.dll causes Lua to allocate some memory, and Engine.dll causes Lua to free it again, boom! Correct?
Any ideas on how to solve this? Of course Engine.dll should be in charge of Lua, but Game.dll should be able to extend the interface with new static functions.
EDIT: There were no more crashes after I turned Lua itself into a .dll. Before I tried this I also recompiled the statics with the same compiler as all other projects, and I double checked the run time libs, and they are all the same, and I am linking to the right debug/release libs. I am still curious what's going on here.
Have a nice day,
Antoon
P.S. Why don't I have control over returns at Stackoverflow?
You wrote:
The engine and the game both work with Lua, they both have their own static C++ interface functions.
That suggests to me the possibility that the engine and game are each individually statically linked to a copy of Lua.
If that were true, then this is exactly what you would expect to happen if a Lua state created by one copy were passed to the other. The typical result is to mess up the memory state and break the allocator. The root cause is that the implementation of the value nil depends on the address of something inside the Lua engine. A second copy of the engine linked into the same process will have a different address serving as its notion of nil. That way eventually leads to madness.
Of course, if the game and engine share a single copy of the Lua interpreter (say by both using LUA51.DLL) then I'm barking up the wrong tree.