I understand how to load an F# dll into a project. But I would like to be able to load specific parts of them dynamically and without a big performance hit from the dll being purely functional.
To unload an application domain
There is no way to unload an individual assembly without unloading all of the application domains that contain it. Use the Unload method from AppDomain to unload the application domains. For more information, see How to: Unload an Application Domain.
Taken from https://msdn.microsoft.com/en-us/library/ms173101.aspx
The idea will be to initialize a new application domain for the F# library, then drop it after you're done using it. I think you're going to have to make sure you compile the F# lib as a Strong-Named assembly and add it to the GAC to accomplish this, but I'm not positive.