Search code examples
c#comcom-interop

Calling function from ComImport class doesn't fail as expected


I'm trying to verify that the class I'm trying to use via COM works as expected. Unfortunately it seems to succeed on a call which should fail:

enum X509CertificateEnrollmentContext
{
    ContextUser = 0x1,
    ContextMachine = 0x2,
    ContextAdministratorForceMachine = 0x3
}

[ComImport(), Guid("884e2045-217d-11da-b2a4-000e7bbb2b09")]
class Cenroll { }

[Guid("728ab35d-217d-11da-b2a4-000e7bbb2b09")]
interface IX509CertificateRequestCmc2
{
    void InitializeFromTemplate(
        [In] X509CertificateEnrollmentContext Context,
        [In] IX509EnrollmentPolicyServer pPolicyServer,
        [In] IX509CertificateTemplate pTemplate);
}

static void Main(string[] args)
{
    var cr = new Cenroll();
    var cmc2 = (IX509CertificateRequestCmc2)cr;
    cmc2.InitializeFromTemplate(X509CertificateEnrollmentContext.ContextUser, null, null);
}

Casting from Cenroll to the interface works, which indicates that the guids are ok. (and it fails casting to other guids, so it's not random success)

But when I call InitializeFromTemplate, with both parameters set to null, it succeeds. The documentation says that the result should be an E_POINTER error:

Return code - Description
E_POINTER - The pPolicyServer and pTemplate parameters cannot be NULL.

So why don't I see an exception?


Solution

  • The problem is that you are redeclaring the interface, and the new definition is different from the original.

    Guids are OK, but underneath, QueryInterface implementation checks GUID, and returns the pointer to the implementation - this is the interface vtable and method addresses are calculated relative to this address (when a call to the method is compiled, offset of the method is added to this address to get the actuall address).

    In your implementation, InitializeFromTemplate is the first method and generated client code calls the method at the beginning of vtable.

    However, in the original interface, there are 56 other methods before InitializeFromTemplate because there is an inheritance chain:

    IX509CertificateRequest (25 methods)
    |
    +-> IX509CertificateRequestPkcs7 (8 methods)
        |
        +-> IX509CertificateRequestCmc (23 methods)
            |
            +-> IX509CertificateRequestCmc2
    

    Function addresses in the certenroll.dll adhere to this layout, so when you call InitializeFromTemplate as declared in your interface, you are calling the first method in chain which is actually IX509CertificateRequest::Initialize.

    As an experiment, if you add 56 dummy methods before InitializeFromTemplate in your IX509CertificateRequestCmc2 you will correctly receive an exception:

    [Guid("728ab35d-217d-11da-b2a4-000e7bbb2b09")]
    interface IX509CertificateRequestCmc
    {
        void fn1();
        void fn2();
        ...
        void fn56();
        void InitializeFromTemplate(...);
    }
    

    The call will throw: CertEnroll::CX509CertificateRequestCmc::InitializeFromTemplate: Invalid pointer 0x80004003 (-2147467261)

    Of course, the solution is not to add the dummy methods :) You should use the generated interop types instead of providing your own. As you are referencing the certenroll assembly, I don't understand why don't you simply use those generated interop classes. Here's the full example which behaves as expected:

    using CERTENROLLLib;
    
    namespace comcerttest
    {
        class Program
        {
            static void Main(string[] args)
            {
                // If you are embedding the interop types, note that you must
                // remove the `Class` suffix from generated type name in order
                // to instantiate it. See link at the bottom for explanation:
                var cr = new CX509CertificateRequestCmc();
                var cmc2 = (IX509CertificateRequestCmc2)cr;
                cmc2.InitializeFromTemplate(X509CertificateEnrollmentContext.ContextUser, null, null);
            }
        }
    }
    

    The issue with using class vs interface type is explained here: Using embedded interop types