Search code examples
c#.netpositionleap-motionmouse-cursor

How to move the cursor or simulate clicks for other applications?


I am creating a C# Windows application using the Leap Motion Controller. I am developing on Windows 8 and Visual Studio 2010. I use SetCursorPos and mouse_event from user32.dll to move the cursor and simulate clicks.

I want the cursor to be moved when in any application. When I run/debug it from Visual Studio it only works when in the application itself or in Visual Studio. When in other applications the mouse does not move and the clicks don't work, but if I try to move the cursor with the real mouse it goes back to the place it was at. When run standalone it does not move in Visual Studio and the mouse can be moved with the real mouse in other applications.

I have this code to use SetCursorPos and mouse_event:

[DllImport("user32.dll")]
public static extern long SetCursorPos(int x, int y);

[DllImport("User32.Dll")]
public static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint cButtons, uint dwExtraInfo);

public static void MouseClick(uint x, uint y) {
    mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, x, y, 0, 0);
}

In my form class I have the mouse location stored in two fields. In a Timer's tick I set the cursor position and do necessary clicks like this:

if (!frame.Hands.IsEmpty) {
    Hand hand = frame.Hands.Leftmost;
    if (!hand.Fingers.IsEmpty) {
        // Get coordinates...
        SetCursorPos(mousex, mousey);
    }
}
foreach (Gesture gesture : gestures) {
    if (gesture.Type == Gesture.GestureType.TYPESCREENTAP) {
        MouseClick(mousex, mousey);
    }
}

The if statements are for the Leap device; I want to move the mouse and do other stuff only when there is a hand to get the coordinates.

Is it even possible to move the cursor or simulate mouse clicks for other applications? And if it is, how?


Solution

  • I managed to solve the problem myself.

    There were two problems. The first (and the main) one was that the Leap device was not sending frames to my application when it was in the background. That was fixed by adding this code to the listener's onConnect() method, as suggested here:

    controller.SetPolicyFlags(Controller.PolicyFlag.POLICYBACKGROUNDFRAMES);
    

    The second problem was that when run standalone the application didn't have access to send input to other applications. I followed instructions found here (scroll down, there's info about uiAccess) and here and signed my application and added this to the manifest:

    <requestedExecutionLevel level="asInvoker" uiAccess="true" />