I am trying to design a modular application; one where developer's create plugins as dlls that get linked to the main application using loadlibrary() or dlopen(). My current thoughts on this are:
1: Both the application and plugin module include a core header with a pure virtual class IPlugin with a method run(). The plugin module then implements the class, defining run():
2: The plugin module defines a function IPlugin* GetPlugin() using "extern c" to ensure ABI compatibility
3: The application requires the plugin module using loadlibrary(), retrieves IPlugin from GetPlugin() using getprocaddress()
4: The application calls method run() to run the plugin
This all works, but how do I create a plugin interface that has access to my full application code? If I have a class that creates a window or initializes a button, I want to access those through the plugin interface and be able to manipulate them and receive events, while still having the memory be managed by the main application side.
1. How would I go about doing this?
2. Would the best solution be ABI compatible?
An API either uses C++ or C, there’s no middle ground. If you want to force plugin developers to use an ABI compatible compiler then you can have an interface that deals with classes and virtual methods and there’s no need for any “extern C” brouhaha.
Otherwise, you’re providing a C API and a C API only. Of course you can implement a virtual function table using a struct of function pointers, and internally you can wrap it in a C++ class and whatnot. But that’s your implementation detail and doesn’t figure in the design of the plugin interface itself.
So that’s that, pretty much. There’s no such thing as C++ API compatibility for free on Windows. People use at least 5 incompatible compilers - MSVC 2017+, 2015, 2012, mingw g++ and clang. Some particularly backwater corporate establishments will insist on using even older MSVC sometimes. So a C++ interface is mostly a lost cause unless you provide shims for all those users. It’s not unthinkable in these days of continuous integration (CI) - you can easily build wrappers that consume your C API and expose it to the users via a C++ interface compatible with their preferred development system. But that doesn’t mean that you get to mess with their C++ objects directly from your C++ code. There’s still a C intermediary, and you still need to use the helpers in your adapter code. E.g. you cannot delete the user provided object directly - you can only do it by calling a helper in the adapter DLL - a helper that uses the same runtime and is ABI compatible with user’s C++ code.
And don’t forget that there are binary incompatible runtime libraries for any given compiler version - e.g. debug vs release and single vs multithreaded. So, for any MSVC version you have to provide four adapter DLLs that the plugin developers would link with. And then your code links to that adapter as well, to access user’s plugin. So you would be first parsing the binary headers in the plugin to determine what adapter DLL and runtime it’s using, and issue an error message if they don’t match (plugin devs are very likely to mess up). Then if there’s a match you load the plugin DLL. The dynamic linker will bring in the requisite adapter DLL. Then you’re ready to use the adapter DLL to access the plugin.
Having done this before, my advice is to have each adapter dll provide different C symbols to your host program, since invariably there will be multiple plugins each using a different adapter and this only complicates matters. Instead, you need to link to all the adapters via demand loading on Windows, and only access a particular adapter when you have parsed the plugin DLL to know what it’s using. Then when you call the adapter, the dynamic linker will resolve the demandload stubs for the real adapter functions. A similar approach can be used on non-Windows platforms, but requires writing helper code to provide the demand link feature - so it may be simplest to use dlopen explicitly on Unix. You’ll still need to parse the ELF headers of the plugin to figure out the C++ runtime it uses and the adapter library it expects, validate the combination; and only then load it. And then you’d dlopen the adapter to talk to the plugin. In no case you’d be directly calling any functions on the plugin itself - only the adapter can do that safely when you need to cross C++ runtime boundaries. There may be easier ways to do all that but my experience is that they only work 99% of the way so in the end there’s no cheap solution here - not unless someone wrote an open source project (or a product) to do all this. You will be expected to understand the dirty implementation details and deal with C++ runtime bugs anyway if you dabble in that. So if you’d rather avoid all that - don’t mess with C++ user visible APIs that require lodable libraries.
You can certainly do a header only C-to-C++ bridge for the user, but surprisingly enough some users don’t like header only solutions. Sigh. How do I know? I had to get rid of a header-only interface improves customer insistence… I wept and laughed like a maniac all at once that day.