Search code examples
.netwpfc++-cli3dsmax

3D Studio Max SDK .NET / C++ interoperability via NativePointer and Marshal


I'm working on a plugin for 3D Studio Max. The core of it is written in C++, and the user interface is written in WPF. Autodesk provides .NET wrappers of their C++ types, each of which implements an INativePointer interface providing a System::IntPtr NativePointer property.

Using C# and C++/CLI I had hoped to pass references to these objects back and forth from C++ to .NET. However, the address returned by NativePointer is close, but not quite right. For example:

// create instance of a CustAttribute, which max exposed to .NET as ICustAttribute
object instance = CoreInterface.CreateInstance(SClassID.CustAttribute, myCustomAttributeClassID );

// The C++ constructor of my custom attribute runs and the address of the
// created object is 0x45001000 for example.

// cast to ICustAttrib (successfully) - all methods on attrib work.
Autodesk.Max.ICustAttrib attrib = (Autodesk.Max.ICustAttrib)instance;

// get the native pointer
System.IntPtr ptr = attrib.NativePointer;

// Strangely, ptr is equal to 0x45001009

// call my C++/CLI function which casts ICustAttrib to CustAttrib and modifies object
Support.ModifyAttribute( ptr );

Crash! ptr is off by 9 bytes.

What's so strange is that my Support.ModifyAttribute( ) method, which casts the address to a CustAttrib, is crashing. The value of the address is 9 bytes greater than the actual value of the object created. This is consistent for ALL 3D Studio Max wrapper types I have tested. As a result my code crashes.

My question is this: How do I successfully pass a 3D Studio Max .NET wrapper object back to C++ for interoperability? the 3D Studio Max SDK does not indicate through samples or documentation how to convert a .NET instance of an object to a C++ instance, although this must work because it is being done all over their SDK. Has anyone else had some luck with this?

I notice that this question is related but unanswered.

I'm using 3D Studio Max 2016 and VS2013.


Solution

  • After reviewing the available managed assemblies provided with 3D Studio Max I found that the Autodesk.Max.Wrappers.DLL assembly containes a collection of classes called CustomMarshaler__typename, one for each type exposed via the wrappers. I was able to use them to satisfy my requirements of getting the native address:

    // create instance of a CustAttribute, which max exposed to .NET as ICustAttribute
    object instance = CoreInterface.CreateInstance(SClassID.CustAttribute, myCustomAttributeClassID );
    
    // The C++ constructor of my custom attribute runs and the address of the
    // created object is 0x45001000 for example.
    
    // cast to ICustAttrib (successfully) - all methods on attrib work.
    Autodesk.Max.ICustAttrib attrib = (Autodesk.Max.ICustAttrib)instance;
    
    // Get the marshaler for the type - not sure what the string is for
    ICustomMarshaler marshaler = Autodesk.Max.Wrappers.CustomMarshalerCustAttrib.GetInstance("what goes there?");
    
    
    // get the native pointer via the custom marshaler - it isn't clear if there is any type checking internally since MarshalManagedToNative takes on object
    System.IntPtr ptr = marshaler.MarshalManagedToNative(attrib);
    
    // Hurrah - ptr is equal to 0x45001000
    
    // call my C++/CLI function which casts intptr to native ::CustAttrib type and modifies it via native methods.
    Support.ModifyAttribute( ptr );
    
    // Huzzah - attribute is correctly modified in native C++
    

    That works fine, which is great. It works for all provided wrapper types:

    Autodesk.Max.Wrappers.CustomMarshalerINode.GetInstance("what goes there?");
    Autodesk.Max.Wrappers.CustomMarshalerModifier.GetInstance("what goes there?");
    Autodesk.Max.Wrappers.CustomMarshalerGUP.GetInstance("what goes there?");
    

    The only nagging concern is what the string in CustomMarshalerCustAttrib.GetInstance( String ) is for.

    Hopefully someone will find this useful!