please do not block me. I found similar question in this link: How to move the mouse pointer visually in selenium python but there was no answer. So I decided to ask again.
How may I VISUALLY move the mouse cursor to a specific position via Selenium in VBA Excel?
This is the code I tried. I want to move the cursor to 5,10. But the mouse cursor doesn't move visually.
Sub VisuallyCursorMove()
dim Driver as Selenium.ChromeDriver
Set Driver = New Selenium.ChromeDriver
Driver.Start
Driver.Get "http://www.yahoo.com"
Driver.Actions.MoveByOffset(5, 10).Perform
End Sub
If you are going to ask me that what does it matter to see visually the cursor, I should answer that it is because I am using Magnifier. I should move the mouse to any place to see that in the magnifier window. So I want every mouse actions in my Selenium project, able to see in magnifier window.
I have done something similar in the past, so I'll post some code you can try. I haven't tested it now, so there might be some bugs/issues with it. Also, a bunch of the declared API functions are not used in this snippet, I had no time to remove them now.
If this Sub doesn't perfectly fit your needs it should be easy to adapt, you could also move the mouse to a position defined by WebDriverRECT.right - offsetX and WebDriverRECT.bottom -offsetY...
Also, you can experiment with oWebDriver.Window.Position.x, oWebDriver.Window.Position.y, oWebDriver.Window.Size.Width and oWebDriver.Window.Size.Height, I exclusively used Windows API functions for consistency in the scaling.
Option Explicit
#If VBA7 Then
Private Declare PtrSafe Sub Sleep Lib "kernel32" (ByVal dwMilliSeconds As Long)
Private Declare PtrSafe Function GetCursorPos Lib "user32" (lpPoint As PointAPI) As Long
Private Declare PtrSafe Function SetCursorPos Lib "user32" (ByVal x As Long, ByVal y As Long) As Long
Private Declare PtrSafe Sub mouse_event Lib "user32" (ByVal dwFlags As Long, ByVal dx As Long, ByVal dy As Long, ByVal cButtons As Long, ByVal dwExtraInfo As Long)
Private Declare PtrSafe Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As LongPtr
Private Declare PtrSafe Function GetWindowRect Lib "user32" (ByVal hWnd As LongPtr, lpRect As RECT) As Long
Private Declare PtrSafe Function GetDesktopWindow Lib "user32" () As LongPtr
Private Declare PtrSafe Function GetWindow Lib "user32" (ByVal hWnd As LongPtr, ByVal wCmd As Long) As LongPtr
Private Declare PtrSafe Function GetWindowTextLength Lib "user32" Alias "GetWindowTextLengthA" (ByVal hWnd As LongPtr) As Long
Private Declare PtrSafe Function GetWindowText Lib "user32" Alias "GetWindowTextA" (ByVal hWnd As LongPtr, ByVal lpString As String, ByVal cch As Long) As Long
Private Declare PtrSafe Function GetClassName Lib "user32" Alias "GetClassNameA" (ByVal hWnd As LongPtr, ByVal lpStr As String, ByVal nMaxCount As Long) As Long
Private Declare PtrSafe Function GetForegroundWindow Lib "user32" () As LongPtr
Private Declare PtrSafe Function SetForegroundWindow Lib "user32" (ByVal hWnd As LongPtr) As Boolean
Private Declare PtrSafe Function SendMessage Lib "user32" Alias "SendMessageA" (ByVal hWnd As LongPtr, ByVal wMsg As Long, ByVal wParam As LongPtr, lParam As Any) As LongPtr
Private Declare PtrSafe Function SendMessageTimeout Lib "user32" Alias "SendMessageTimeoutA" (ByVal hWnd As LongPtr, ByVal msg As Long, ByVal wParam As LongPtr, ByVal lParam As Any, ByVal fuFlags As Long, ByVal uTimeout As Long, lpdwResult As Long) As Long
Private Declare PtrSafe Function SendMessageByString Lib "user32.dll" Alias "SendMessageA" (ByVal hWnd As LongPtr, ByVal wMsg As Long, ByVal wParam As Long, ByVal lParam As String) As Long
Private Declare PtrSafe Function EnumChildWindows Lib "user32" (ByVal hwndParent As LongPtr, ByVal lpEnumFunc As LongPtr, ByVal lParam As LongPtr) As Boolean
#If Win64 Then
Private Declare PtrSafe Function GetWindowLongPtr Lib "user32" Alias "GetWindowLongPtrA" (ByVal hWnd As LongPtr, ByVal nIndex As Long) As LongPtr
#Else
Private Declare PtrSafe Function GetWindowLongPtr Lib "user32" Alias "GetWindowLongA" (ByVal hWnd As Long, ByVal nIndex As Long) As Long
#End If
#Else
Private Declare Sub Sleep Lib "kernel32" Alias "sleep" (ByVal lngMilliSeconds As Long)
Private Declare Function GetCursorPos Lib "user32" (lpPoint As PointAPI) As Long
Private Declare Function SetCursorPos Lib "user32" (ByVal x As Long, ByVal y As Long) As Long
Private Declare Sub mouse_event Lib "user32" (ByVal dwFlags As Long, ByVal dx As Long, ByVal dy As Long, ByVal cButtons As Long, ByVal dwExtraInfo As Long)
Private Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As Long
Private Declare Function GetWindowRect Lib "user32" (ByVal hWnd As Long, lpRect As RECT) As Long
Private Declare Function GetDesktopWindow Lib "user32" () As Long
Private Declare Function GetWindow Lib "user32" (ByVal hWnd As Long, ByVal wCmd As Long) As Long
Private Declare Function GetWindowTextLength Lib "user32" Alias "GetWindowTextLengthA" (ByVal hWnd As Long) As Long
Private Declare Function GetWindowText Lib "user32" Alias "GetWindowTextA" (ByVal hWnd As Long, ByVal lpString As String, ByVal cch As Long) As Long
Private Declare Function GetClassName Lib "user32" Alias "GetClassNameA" (ByVal hWnd As Long, ByVal lpStr As String, ByVal nMaxCount As Long) As Long
Private Declare Function GetForegroundWindow Lib "user32" () As Long
Private Declare Function SetForegroundWindow Lib "user32" (ByVal hWnd As Long) As Boolean
Private Declare Function SendMessage Lib "user32" Alias "SendMessageA" (ByVal hWnd As Long, ByVal wMsg As Long, ByVal wParam As Long, lParam As Any) As Long
Private Declare Function SendMessageTimeout Lib "user32" Alias "SendMessageTimeoutA" (ByVal hWnd As Long, ByVal msg As Long, ByVal wParam As Long, ByVal lParam As Any, ByVal fuFlags As Long, ByVal uTimeout As Long, lpdwResult As Long) As Long
Private Declare Function SendMessageByString Lib "user32.dll" Alias "SendMessageA" (ByVal hWnd As Long, ByVal wMsg As Long, ByVal wParam As Long, ByVal lParam As String) As Long
Private Declare Function EnumChildWindows Lib "user32" (ByVal hwndParent As Long, ByVal lpEnumFunc As Long, ByVal lParam As Long) As Boolean
Private Declare Function GetWindowLongPtr Lib "user32" Alias "GetWindowLongA" (ByVal hWnd As Long, ByVal nIndex As Long) As Long
#End If
Private Type PointAPI
x As Long
y As Long
End Type
Private Type RECT
left As Long
top As Long
right As Long
bottom As Long
End Type
Private Sub MoveMouseToWebDriverPos(ByVal oWebDriver As Object, _
ByVal posX As Long, _
ByVal posY As Long)
#If VBA7 Then
Dim WebDriverHwnd As LongPtr
Dim dlgHwnd As LongPtr
#Else
Dim WebDriverHwnd As Long
Dim dlgHwnd As Long
#End If
WebDriverHwnd = FindTopLevelWindow(oWebDriver.Title, "Chrome_WidgetWin_1")
'Get Edge window position and dimensions:
'Note: It is necessary to use the Windows API Functions here,
'as the Selenium internal ones:
'oWebDriver.Window.Position.x, oWebDriver.Window.Position.y,
'oWebDriver.Window.Size.Width and oWebDriver.Window.Size.Height
'seem to use different scaling and return different (wrong) values
Dim WebDriverRECT As RECT: GetWindowRect WebDriverHwnd, WebDriverRECT
'If you want to rturn cursor to original position afterwards, cache pos here
'Dim CachedPos As PointAPI: GetCursorPos CachedPos
SetCursorPos WebDriverRECT.left + posX, WebDriverRECT.top + posY
'If you want to click:
' Const MOUSEEVENTF_LEFTDOWN = &H2
' Const MOUSEEVENTF_LEFTUP = &H4
' Const MOUSEEVENTF_RIGHTDOWN As Long = &H8
' Const MOUSEEVENTF_RIGHTUP As Long = &H10
'mouse_event MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0
'mouse_event MOUSEEVENTF_LEFTUP, 0, 0, 0, 0
'If you want to rturn cursor to original position after some action
'SetCursorPos CachedPos.X, CachedPos.y
End Sub
#If VBA7 Then
Private Function FindTopLevelWindow(partOfTitleStr As String, _
Optional className As String = "§$%³") As LongPtr
#Else
Private Function FindTopLevelWindow(partOfTitleStr As String, _
Optional className As String = "§$%³") As Long
#End If
Const GW_CHILD = 5
Const GW_HWNDNEXT = 2
Dim tLen As Long
Dim i As Long: i = 0
FindTopLevelWindow = GetWindow(GetDesktopWindow(), GW_CHILD)
Do While FindTopLevelWindow <> 0
tLen = GetWindowTextLength(FindTopLevelWindow)
Dim wname As String: wname = Space(tLen)
GetWindowText FindTopLevelWindow, wname, tLen
If InStr(1, wname, partOfTitleStr) > 0 Then
If className <> "§$%³" Then
Dim tClassName As String: tClassName = Space(256)
GetClassName FindTopLevelWindow, tClassName, Len(tClassName)
'Null termination has to be removed:
If className = left(Trim(tClassName), Len(Trim(tClassName)) - 1) Then
Exit Function
End If
Else
Exit Function
End If
End If
FindTopLevelWindow = GetWindow(FindTopLevelWindow, GW_HWNDNEXT)
i = i + 1
If i Mod 10 = 0 Then DoEvents
Loop
End Function
Edit:
As requested in your comment, to use this code, just call the Sub I provided in your code:
Sub VisuallyCursorMove()
dim Driver as Selenium.ChromeDriver
Set Driver = New Selenium.ChromeDriver
Driver.Start
Driver.Get "http://www.yahoo.com"
'The following moves the mouse cursor to the position 57 pixels from the top
'and 57 pixels from the left of your WebDriver window, so probably somewhere
'around the top left corner of the window.
MoveMouseToWebDriverPos(Driver, 57, 57)
End Sub