Search code examples
powershellpinvoke

sending WM_COPYDATA messages from powershell, trouble with pointers


I'm trying to send control messages from powershell to mpc-hc. Mpc's api makes use of WM_COPYDATA messages. Here's what I have so far, after looking here, here and here:

Add-Type @"
    using System;
    using System.Runtime.InteropServices;
    public class Messages {
        [DllImport("user32.dll", CharSet = CharSet.Auto)]
        public static extern IntPtr SendMessage(IntPtr hWnd, int Msg, int wParam, ref IntPtr lParam);
    }
    public struct COPYDATASTRUCT
    {
        public IntPtr dwData;    // Any value the sender chooses.  Perhaps its main window handle?
        public int cbData;       // The count of bytes in the message.
        public IntPtr lpData;    // The address of the message.
    }
"@

$WM_COPYDATA = 0x004A;
$CMD_OSDSHOWMESSAGE = 0xA0005000

$MpcWindow1 = Start-Process -PassThru -FilePath "C:\Program Files (x86)\K-Lite Codec Pack\MPC-HC64\mpc-hc64.exe" -ArgumentList "/new"
$MpcMessage = "hello"

$cds = New-Object COPYDATASTRUCT
$cds.dwData = $CMD_OSDSHOWMESSAGE
$cds.cbData = $MpcMessage.Length
$cds.lpData = $MpcMessage[0]     #without [0] throws an exception

[Messages]::SendMessage($MpcWindow1.MainWindowHandle, $WM_COPYDATA, (Get-Process powershell).MainWindowHandle, [ref]$cds[0])

Executing that gives me:

Exception calling "SendMessage" with "4" argument(s): "Cannot convert the "COPYDATASTRUCT" value of type
"COPYDATASTRUCT" to type "System.IntPtr"."
At C:\Users\Petersburg SDA\Videos\dev\wm_copydata.ps1:27 char:1
+ [Messages]::SendMessage($MpcWindow1.MainWindowHandle, $WM_COPYDATA, (Get-Process ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : PSInvalidCastException

I'm not sure the $MpcMessage[0] is correct, but using just the variable gets me this (in addition to the above)

Exception setting "lpData": "Cannot convert the "hello" value of type "System.String" to type "System.IntPtr"."
At C:\Users\Petersburg SDA\Videos\dev\wm_copydata.ps1:25 char:1
+ $cds.lpData = $MpcMessage     #throws an exception
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], SetValueInvocationException
    + FullyQualifiedErrorId : ExceptionWhenSetting

Changing that to [ref]$MpcMessage gets

Exception setting "lpData": "Cannot convert the "System.Management.Automation.PSReference`1[System.String]" value of
type "System.Management.Automation.PSReference`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089]]" to type "System.IntPtr"."
At C:\Users\Petersburg SDA\Videos\dev\wm_copydata.ps1:25 char:1
+ $cds.lpData = [ref]$MpcMessage     #throws an exception
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], SetValueInvocationException
    + FullyQualifiedErrorId : ExceptionWhenSetting

Admittedly I haven't worked with powershell much, but translating the code is proving to be unusually difficult.


Solution

  • You'll have to use the marshal class to marshal the struct to a pointer. Example :

        $StructPointerSize = [System.Runtime.InteropServices.Marshal]::SizeOf($CDS)
        $StructPointer = [System.Runtime.InteropServices.Marshal]::AllocHGlobal($StructPointerSize)
        [System.Runtime.InteropServices.Marshal]::StructureToPtr($CDS,$StructPointer,$true
    

    This will assign some memory, then it copies the struct to this block of memory. After this the pointer to your struct will be in the $StructPointer variable.

    Complete example which resumes a paused video (using reflection to create the types) :

    $Domain = [System.AppDomain]::CurrentDomain
    $AssemblyName = [System.Reflection.AssemblyName]::new('Messages')
    $Assembly = $Domain.DefineDynamicAssembly($AssemblyName,'Run')
    $ModuleBuilder = $Assembly.DefineDynamicModule('Messages')
    
    # Define the struct (source : https://msdn.microsoft.com/en-us/library/windows/desktop/ms649010(v=vs.85).aspx)
    $StructAttributes = 'AutoLayout, AnsiClass, Class, Public, SequentialLayout, BeforeFieldInit'
    $COPYDATASTRUCT = $ModuleBuilder.DefineType('COPYDATASTRUCT',$StructAttributes)
    $COPYDATASTRUCT.DefineField('dwData',[int],'Public')
    $COPYDATASTRUCT.DefineField('cbData',[Uint32],'Public')
    $COPYDATASTRUCT.DefineField('lpData',[System.IntPtr],'Public')
    $COPYDATASTRUCT.CreateType()
    
    # Define the class that will hold the PInvoke method
    $MessageClass = $ModuleBuilder.DefineType('Messages','Public')
    
    # Define the PInvoke Method
    $SendMessage = $MessageClass.DefinePInvokeMethod(
    'SendMessageW',
    'User32.dll',
    @('Public','Static','PinvokeImpl'),
    [System.Reflection.CallingConventions]::Standard,
    [System.IntPtr],
    @([System.IntPtr],[Uint32],[System.IntPtr],[System.IntPtr]),
    [System.Runtime.InteropServices.CallingConvention]::Winapi,
    [System.Runtime.InteropServices.CharSet]::Unicode
    )
    
    $SendMessage.DefineParameter(1,'In','hWnd')
    $SendMessage.DefineParameter(2,'In','Msg')
    $SendMessage.DefineParameter(3,'In','wParam')
    $SendMessage.DefineParameter(4,'In','lParam')
    
    
    $SendMessage.SetImplementationFlags($SendMessage.GetMethodImplementationFlags() -bor [System.Reflection.MethodImplAttributes]::PreserveSig)
    $MessageClass.CreateType()
    
    $WM_COPYDATA = 0x004A;
    $CMD_PLAY = 0xA0000004
    
    $CurrentHandle = [System.Diagnostics.Process]::GetCurrentProcess().MainWindowHandle
    $Arguments = '/Slave' + " " + $CurrentHandle.ToString()
    
    Start-Process -FilePath 'C:\Program Files\MPC-HC\mpc-hc64.exe' -ArgumentList $Arguments
    $CDS = [COPYDATASTRUCT]::new()
    $CDS.dwData = $CMD_PLAY
    $CDS.lpData = [System.IntPtr]::Zero
    $CDS.cbData = 0
    
    $StructPointerSize = [System.Runtime.InteropServices.Marshal]::SizeOf($CDS)
    $StructPointer = [System.Runtime.InteropServices.Marshal]::AllocHGlobal($StructPointerSize)
    [System.Runtime.InteropServices.Marshal]::StructureToPtr($CDS,$StructPointer,$true)
    $p = (Get-Process mpc-hc64).MainWindowHandle
    [messages]::SendMessageW($p,$WM_COPYDATA,0,$StructPointer)