Call Win32 CalculatePopupWindowPosition from C#

Question

I'm trying to call the WinAPI function CalculatePopupWindowPosition in C# using P/Invoke. From

http://msdn.microsoft.com/en-us/library/windows/desktop/dd565861(v=vs.85).aspx

I see that it's syntax is:

BOOL WINAPI CalculatePopupWindowPosition(  
  _In_      const POINT *anchorPoint,  
  _In_      const SIZE *windowSize,  
  _In_      UINT flags,  
  _In_opt_  RECT *excludeRect,  
  _Out_     RECT *popupWindowPosition  
);

I then tried to import it using the following code in C#

[DllImport("User32.dll", SetLastError = true, CallingConvention = CallingConvention.StdCall)]
public static extern bool CalculatePopupWindowPosition
(
    [In] ref POINT anchorPoint,
    [In] ref SIZE windowSize,
    [In] ref UInt32 flags,
    [In,Optional] ref RECT excludeRect,
    [Out] out SIZE popupWindowPosition
);

I also implemented the RECT, POINT and SIZE structures and initialized them. Finally I called the function like so.

CalculatePopupWindowPosition(ref nIconPos, ref windowSize, ref flags, ref nIconRect, out windowSize);

This doesn't seem to work though, windowSize contains nothing but zeros, which it shouldn't. Any ideas what I'm doing wrong here?

Answer 1

The flags parameter needs to be passed by value rather than by reference:

[DllImport("User32.dll", SetLastError = true)]
public static extern bool CalculatePopupWindowPosition
(
    ref POINT anchorPoint,
    ref SIZE windowSize,
    uint flags,
    ref RECT excludeRect,
    out RECT popupWindowPosition
);

Some general advice. When an API call fails, check the return value. In this case if the function returns false then call Marshal.GetLastWin32Error to find out the error status code.