Search code examples
c++cwindowswinapisendinput

Calculate normalized coordinates for SendInput() in a multi-monitor environment

I want to send mouse input programmatically to anyone of my connected displays.

I know I can use the SendInput function to do this, but my current approach doesn't work on a multi-display setup.

I am using the following approach:

  1. Get pixel coordinates of the point to click, relative to the current screen;
  2. Get pixel coordinates of the topleft pixel of the current screen, using GetMonitorInfoW function with the appropriate HMONITOR handle;
  3. Get the pixel coordinates relative to the whole virtual screen: to do this, we sum the coordinates of the current screen margin (topleft pixel) with the coordinates of the point to click;
  4. Transform the resulting pixel coordinates into absolute coordinates (0-65535), to be used by the SendInput function;
  5. Use the SendInput function with MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_VIRTUALDESK flags and the absolute coordinates relative to the entire virtual desktop.

However, with my current code, the mouse input gets wrongly positioned. I am confused on how to correctly calculate the absolute coordinates.

At the moment, I do retrieve the absolute coordinates using the formula discussed on another question. To get the width/height of the virtual screen, I use GetSystemMetrics with SM_CXVIRTUALSCREEN/SM_CYVIRTUALSCREEN parameter.

// ________________________________________________
//
// GetAbsoluteCoordinate
//
// PURPOSE: 
// Convert pixel coordinate to absolute coordinate (0-65535).
//
// RETURN VALUE:
// Absolute Coordinate
// ________________________________________________
//
UINT16 GetAbsoluteCoordinate(INT PixelCoordinate, INT ScreenResolution)
{
    UINT16 AbsoluteCoordinate = ( (65536 * PixelCoordinate) / ScreenResolution ) + 1;
    return AbsoluteCoordinate;
}


// ________________________________________________
//
// GetAbsoluteCoordinates
//
// PURPOSE: 
// Retrieve virtual screen absolute coordinates from pixel coordinates.
//
// PARAMETERS:
// x, y coordinates passed by reference. These will be changed by the function and used as return values.
//
// RETURN VALUE:
// None (see parameters)
// ________________________________________________
//
void GetAbsoluteCoordinates(INT32 &X, INT32 &Y)
{
    // Get multi-screen coordinates
    MONITORINFO MonitorInfo = { 0 };
    MonitorInfo.cbSize = sizeof(MonitorInfo);
    if (GetMonitorInfoW(hMonitor, &MonitorInfo))
    {
        // 1) Get pixel coordinates of topleft pixel of target screen, relative to the virtual desktop ( coordinates should be 0,0 on Main screen);
        // 2) Get pixel coordinates of mouse cursor, relative to the target screen;
        // 3) Sum topleft margin pixel coordinates with mouse cursor coordinates;
        X = MonitorInfo.rcMonitor.left + X;
        Y = MonitorInfo.rcMonitor.top + Y;

        // 4) Transform the resulting pixel coordinates into absolute coordinates.
        X = GetAbsoluteCoordinate(X, GetSystemMetrics(SM_CXVIRTUALSCREEN));
        Y = GetAbsoluteCoordinate(Y, GetSystemMetrics(SM_CYVIRTUALSCREEN));
    }
} 



// ________________________________________________
//
// SendMouseInput
//
// PURPOSE: 
// Send mouse input, supporting any of the connected displays
//
// PARAMETERS:
// X, Y are pixel coordinates, relative to the current screen.
// ________________________________________________
//
 void SendMouseInput(INT X, INT Y) 
  {
     INPUT Input;
     GetAbsoluteCoordinates(X, Y);
     memset(&Input, 0, sizeof(INPUT)); 
     Input.type = INPUT_MOUSE;
     Input.mi.dx = X;
     Input.mi.dy = Y;
     Input.mi.dwFlags = MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_VIRTUALDESK | MOUSEEVENTF_MOVE | MOUSEEVENTF_LEFTDOWN;
     SendInput(1, &Input, sizeof(Input));
  }
Solution
    1. Get the pixel coordinates relative to the whole virtual screen: to do this, we sum the coordinates of the current screen margin (topleft pixel) with the coordinates of the point to click;

    It looks like you are using the coordinates(X,Y) relative to the monitor. How did you get the coordinates?

    If so, then the sample is basically no problem. If your mouse coordinates are obtained by something similar to GetCursorPos, then you do not need to calculate the coordinates relative to the virtual screen (it is).

    The sample almost work for me, I've changed some code as below:

    #include <windows.h>
#include <iostream>
using namespace std;
// ________________________________________________
//
// GetAbsoluteCoordinate
//
// PURPOSE: 
// Convert pixel coordinate to absolute coordinate (0-65535).
//
// RETURN VALUE:
// Absolute Coordinate
// ________________________________________________
//
INT GetAbsoluteCoordinate(INT PixelCoordinate, INT ScreenResolution)
{
    INT AbsoluteCoordinate = MulDiv(PixelCoordinate, 65535, ScreenResolution);
    return AbsoluteCoordinate;
}

void GetAbsoluteCoordinates(HMONITOR hMonitor, INT32& X, INT32& Y)
{
    // Get multi-screen coordinates
    MONITORINFO MonitorInfo = { 0 };
    MonitorInfo.cbSize = sizeof(MonitorInfo);
    if (GetMonitorInfoW(hMonitor, &MonitorInfo))
    {
        // 1) Get pixel coordinates of topleft pixel of target screen, relative to the virtual desktop ( coordinates should be 0,0 on Main screen);
        // 2) Get pixel coordinates of mouse cursor, relative to the target screen;
        // 3) Sum topleft margin pixel coordinates with mouse cursor coordinates;
        X = MonitorInfo.rcMonitor.left + X;
        Y = MonitorInfo.rcMonitor.top + Y;

        // 4) Transform the resulting pixel coordinates into absolute coordinates.
        X = GetAbsoluteCoordinate(X, GetSystemMetrics(SM_CXVIRTUALSCREEN));
        Y = GetAbsoluteCoordinate(Y, GetSystemMetrics(SM_CYVIRTUALSCREEN));
    }
}

void SendMouseInput(HMONITOR hMonitor, INT X, INT Y)
{
    INPUT Input[2];
    GetAbsoluteCoordinates(hMonitor, X, Y);
    memset(Input, 0, sizeof(INPUT));
    Input[0].type = Input[1].type = INPUT_MOUSE;
    Input[0].mi.dx = Input[1].mi.dx = X;
    Input[0].mi.dy = Input[1].mi.dy = Y;
    Input[0].mi.dwFlags = MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_MOVE | MOUSEEVENTF_VIRTUALDESK;
    Input[1].mi.dwFlags = MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_LEFTUP | MOUSEEVENTF_MOVE | MOUSEEVENTF_VIRTUALDESK;

    SendInput(2, Input, sizeof(INPUT));
}
BOOL CALLBACK Monitorenumproc(
    HMONITOR Arg1,
    HDC Arg2,
    LPRECT Arg3,
    LPARAM Arg4)
{
    SendMouseInput(Arg1, 725, 85);
    return TRUE;
}
int main(void)
{
    EnumDisplayMonitors(NULL,NULL, Monitorenumproc,0);
    return 0;
}

    Result:

    I have 2 monitors with the same display resolution(1920 x 1080) to test like: enter image description here

    The sample will click at the same place on each monitor.

    Further reading: The Virtual Screen