Search code examples
c#pinvokemarshallingprojfs

Heap corruption shortly after receiving marshalled struct in C# app


I'm trying to wrap the new Windows 10 ProjFS features in C#. The first steps, as outlined on MSDN, work fine: I set up a directory as my virtualization root, and then register my projection provider including its callbacks as follows:

static void Main(string[] args)
{
    // Create and mark clean directory as virtualization root.
    const string directory = @"C:\test_projfs"; // Warning: Deleted to get clean directory.
    if (Directory.Exists(directory))
        Directory.Delete(directory);
    Directory.CreateDirectory(directory);

    Guid guid = Guid.NewGuid();
    Marshal.ThrowExceptionForHR(PrjMarkDirectoryAsPlaceholder(directory, null, IntPtr.Zero,
        ref guid));

    // Set up the callback table for the projection provider.
    PrjCallbacks callbackTable = new PrjCallbacks
    {
        StartDirectoryEnumerationCallback = StartDirectoryEnumerationCallback,
        EndDirectoryEnumerationCallback = EndDirectoryEnumerationCallback,
        GetDirectoryEnumerationCallback = GetDirectoryEnumerationCallback,
        GetPlaceholderInfoCallback = GetPlaceholderInfoCallback,
        GetFileDataCallback = GetFileDataCallback
    };
    // Start the projection provider.
    IntPtr instanceHandle = IntPtr.Zero;
    Marshal.ThrowExceptionForHR(PrjStartVirtualizing(directory, ref callbackTable,
        IntPtr.Zero, IntPtr.Zero, ref instanceHandle));

    // Keep a test console application running.
    Console.ReadLine();
}

// Managed callbacks, simply returning S_OK for now.
static int StartDirectoryEnumerationCallback(ref PrjCallbackData callbackData, ref Guid enumerationId)
{
    return 0;
}
static int EndDirectoryEnumerationCallback(ref PrjCallbackData callbackData, ref Guid enumerationId) => 0;
static int GetDirectoryEnumerationCallback(ref PrjCallbackData callbackData, ref Guid enumerationId, string searchExpression, IntPtr dirEntryBufferHandle) => 0;
static int GetPlaceholderInfoCallback(ref PrjCallbackData callbackData) => 0;
static int GetFileDataCallback(ref PrjCallbackData callbackData, ulong byteOffset, uint length) => 0;

The native declarations are as follows (sorry about the fact this code block can be scrolled - the basics are quite a lot to wrap already):

// Methods to mark directory as virtualization root, and start the projection provider.
[DllImport("ProjectedFSLib.dll", CharSet = CharSet.Unicode)]
static extern int PrjMarkDirectoryAsPlaceholder(string rootPathName,
    string targetPathName, IntPtr versionInfo, ref Guid virtualizationInstanceID);
[DllImport("ProjectedFSLib.dll", CharSet = CharSet.Unicode)]
static extern int PrjStartVirtualizing(string virtualizationRootPath,
    ref PrjCallbacks callbacks, IntPtr instanceContext, IntPtr options,
    ref IntPtr namespaceVirtualizationContext);

// Structure configuring the projection provider callbacks.
[StructLayout(LayoutKind.Sequential)]
struct PrjCallbacks
{
    public PrjStartDirectoryEnumerationCb StartDirectoryEnumerationCallback;
    public PrjEndDirectoryEnumerationCb EndDirectoryEnumerationCallback;
    public PrjGetDirectoryEnumerationCb GetDirectoryEnumerationCallback;
    public PrjGetPlaceholderInfoCb GetPlaceholderInfoCallback;
    public PrjGetFileDataCb GetFileDataCallback;
    public PrjQueryFileNameCb QueryFileNameCallback;
    public PrjNotificationCb NotificationCallback;
    public PrjCancelCommandCb CancelCommandCallback;
}

// Callback signatures.
delegate int PrjStartDirectoryEnumerationCb(ref PrjCallbackData callbackData, ref Guid enumerationId);
delegate int PrjEndDirectoryEnumerationCb(ref PrjCallbackData callbackData, ref Guid enumerationId);
delegate int PrjGetDirectoryEnumerationCb(ref PrjCallbackData callbackData, ref Guid enumerationId, string searchExpression, IntPtr dirEntryBufferHandle);
delegate int PrjGetPlaceholderInfoCb(ref PrjCallbackData callbackData);
delegate int PrjGetFileDataCb(ref PrjCallbackData callbackData, ulong byteOffset, uint length);
delegate int PrjQueryFileNameCb(ref PrjCallbackData callbackData);
delegate int PrjNotificationCb(ref PrjCallbackData callbackData, bool isDirectory, int notification, string destinationFileName, IntPtr operationParameters);
delegate int PrjCancelCommandCb(ref PrjCallbackData callbackData);

// Callback data passed to each of the callbacks above.
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
struct PrjCallbackData
{
    public uint Size;
    public uint Flags;
    public IntPtr NamespaceVirtualizationContext;
    public int CommandId;
    public Guid FileId;
    public Guid DataStreamId;
    public string FilePathName;
    public IntPtr VersionInfo;
    public uint TriggeringProcessId;
    public string TriggeringProcessImageFileName;
    public IntPtr InstanceContext;
}

However, I seem to have done something wrong when marshalling a structure passed from the OS to the callbacks, as it results in the managed app immediately stopping without a managed exception, and only logging a heap corruption error code in the Windows event viewer.

In detail, I receive the PRJ_START_DIRECTORY_ENUMERATION_CB callback (PrjStartDirectoryEnumerationCb in my code) to start enumerating my directories. It passes a pointer to a PRJ_CALLBACK_DATA struct along (PrjCallbackData in my code).

Now, while I apparently receive the struct nicely into my managed callback, with all values making sense down to the last member InstanceContext, the application crashes immediately upon trying to return the value 0 (S_OK).

static int StartDirectoryEnumerationCallback(ref PrjCallbackData callbackData, ref Guid enumerationId)
{
    return 0; // Crashes here or when stepping out of this method.
}

I tried to nail down where my error lies, but due to debugging immediately stopping without any exception (I'm not filtering out any), I didn't get far. I realized that when I change the callback to dumbly accept an IntPtr instead of a ref PrjCallbackData, the application doesn't crash.

static int StartDirectoryEnumerationCallback(IntPtr callbackData, ref Guid enumerationId)
{
    return 0; // No crash executing this with IntPtr passed in.
}

delegate int PrjStartDirectoryEnumerationCb(IntPtr callbackData, ref Guid enumerationId);

Logically, this doesn't get me far without the important info accessible by me.

Am I missing a step here? Is mapping a struct as simple like that not possible directly?

If of interest, the event viewer entry looks follows. I'm running the application with .NET Framework 4.6 in a new-style C# project file (which should explain the "dotnet.exe" executable name). If additional information is required, I'm happy to provide it.

Faulting application name: dotnet.exe, version: 2.1.26919.1, time stamp: 0x5ba1bb46
Faulting module name: ntdll.dll, version: 10.0.17763.1, time stamp: 0xa369e897
Exception code: 0xc0000374
Fault offset: 0x00000000000fb349
Faulting process id: 0xfa8
Faulting application start time: 0x01d46d623aee076d
Faulting application path: C:\Program Files\dotnet\dotnet.exe
Faulting module path: C:\WINDOWS\SYSTEM32\ntdll.dll
Report Id: adcfba5c-dfd4-428d-8eb5-81aceada1983
Faulting package full name: 
Faulting package-relative application ID: 

Note that if you want to try the sample code above, you have to install the Projected Filesystem feature in Windows 10 1809, and compile it as x64 (there are no native libraries for x86 / AnyCPU configurations).


Solution

  • As Simon commented, only having an IntPtr in the signature instead of the structure and then using Marshal.PtrToStructure<PrjCallbackData>(callbackData) to retrieve a copy of the structure passed to my callback works fine.

    Alternatively, Hans' solution with using IntPtr for the string fields in the structure (and keeping the structure in the signature) also works, but leaves me with no simple access to the string data.

    Luckily, I do not have to write anything back to this structure, otherwise I'd run into problems writing it back to the original, and not the copy of the structure, so Simon's solution is sufficient here.

    In case full code coverage is of interest, the repository is found here.