Globally preventing mouse from entering screen area

Question

What I would like to do is to create an impenetrable edge on my desktop to protect a "zone". What I am looking for is kind of like mouse trapping, but I dont want to be strictly confined to a rect as this will be used in multi-monitor setups where the desktop area may not be a perfect rect and the area I want isolated may be somewhere in the middle.

I am using Gma.System.MouseKeyHook to hook the mouse cords, I know this isnt too hard to do in pinvoke but I went the library route.

So far what I have is:

    const uint MOUSEEVENTF_ABSOLUTE = 0x8000;
    const uint MOUSEEVENTF_LEFTDOWN = 0x0002;
    const uint MOUSEEVENTF_LEFTUP = 0x0004;
    const uint MOUSEEVENTF_MIDDLEDOWN = 0x0020;
    const uint MOUSEEVENTF_MIDDLEUP = 0x0040;
    const uint MOUSEEVENTF_MOVE = 0x0001;
    const uint MOUSEEVENTF_RIGHTDOWN = 0x0008;
    const uint MOUSEEVENTF_RIGHTUP = 0x0010;
    const uint MOUSEEVENTF_XDOWN = 0x0080;
    const uint MOUSEEVENTF_XUP = 0x0100;
    const uint MOUSEEVENTF_WHEEL = 0x0800;
    const uint MOUSEEVENTF_HWHEEL = 0x01000;

    [DllImport("User32.Dll")]
    public static extern long SetCursorPos(int x, int y);

    [DllImport("user32.dll")]
    static extern void mouse_event(uint dwFlags, int dx, int dy, uint dwData, int dwExtraInfo);

    private IKeyboardMouseEvents m_GlobalHook;

    public MainWindow()
    {
        InitializeComponent();

        m_GlobalHook = Hook.GlobalEvents();
        m_GlobalHook.MouseMove += M_GlobalHook_MouseMove;
    }

    private void M_GlobalHook_MouseMove(object sender, System.Windows.Forms.MouseEventArgs e)
    {
        if (e.X < 0)
        {
            //SetCursorPos(0, e.Y);
            //mouse_event((int)MOUSEEVENTF_MOVE | (int)MOUSEEVENTF_ABSOLUTE, 0, e.Y, 0, 0);
            System.Windows.Forms.Cursor.Position = new System.Drawing.Point(0, e.Y);
        }
    }

The issue I am facing is the actual mouse position and the cursor seem to be abstract or not tied together so I get a weird flicker back and forth between its position in the area I am trying to avoid and the edge. As you can see I have tried a couple different methods all behave the same.

Please note, for testing and simplification my test area is just anything with a X < 0, which is one of my side monitors.

Answer 1

The answer was to do a low level mouse hook, and instead of doing the CallNextHookEx when it was in the area I wanted to keep the mouse out of I returned (IntPtr)1 instead. This smoothly blocked the mouse from entering my no go area.

The code is messy as this was just thrown together a s a proof of concept, but someone can likely adapt it as needed.

public partial class MainWindow : Window
{
    public static IntPtr _hookID = IntPtr.Zero;
    private static MouseHook.LowLevelMouseProc _proc;

    public MainWindow()
    {
        InitializeComponent();

        _proc = new MouseHook.LowLevelMouseProc(HookCallback);
        _hookID = MouseHook.SetHook(_proc);
    }

    private static IntPtr HookCallback(int nCode, IntPtr wParam, IntPtr lParam)
    {
        if (nCode >= 0 &&  MouseHook.MouseMessages.WM_MOUSEMOVE == (MouseHook.MouseMessages)wParam)
        {
            MouseHook.MSLLHOOKSTRUCT hookStruct = (MouseHook.MSLLHOOKSTRUCT)Marshal.PtrToStructure(lParam, typeof(MouseHook.MSLLHOOKSTRUCT));
            if (hookStruct.pt.x < 0)
            {
                System.Windows.Forms.Cursor.Position = new System.Drawing.Point(0, hookStruct.pt.y);
                return (IntPtr)1;
            }
        }
        return MouseHook.CallNextHookEx(_hookID, nCode, wParam, lParam);
    }
}

class MouseHook
{
    private const int WH_MOUSE_LL = 14;

    public enum MouseMessages
    {
        WM_LBUTTONDOWN = 0x0201,
        WM_LBUTTONUP = 0x0202,
        WM_MOUSEMOVE = 0x0200,
        WM_MOUSEWHEEL = 0x020A,
        WM_RBUTTONDOWN = 0x0204,
        WM_RBUTTONUP = 0x0205
    }

    [StructLayout(LayoutKind.Sequential)]
    public struct POINT
    {
        public int x;
        public int y;
    }

    [StructLayout(LayoutKind.Sequential)]
    public struct MSLLHOOKSTRUCT
    {
        public POINT pt;
        public uint mouseData;
        public uint flags;
        public uint time;
        public IntPtr dwExtraInfo;
    }

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    public static extern IntPtr SetWindowsHookEx(int idHook, LowLevelMouseProc lpfn, IntPtr hMod, uint dwThreadId);

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    private static extern bool UnhookWindowsHookEx(IntPtr hhk);

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    public static extern IntPtr CallNextHookEx(IntPtr hhk, int nCode, IntPtr wParam, IntPtr lParam);

    [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr GetModuleHandle(string lpModuleName);

    [DllImport("kernel32.dll")]
    static extern IntPtr LoadLibrary(string lpFileName);

    public delegate IntPtr LowLevelMouseProc(int nCode, IntPtr wParam, IntPtr lParam);

    public static IntPtr SetHook(LowLevelMouseProc proc)
    {
        IntPtr hInstance = LoadLibrary("User32");
        return MouseHook.SetWindowsHookEx(WH_MOUSE_LL, proc, hInstance, 0);
    }
}