I have written an extensive on-screen keyboard (OSK) in Java. This is intended for people with disabilities and needs to be able to do everything that a standard keyboard can do. One thing I've noticed that doesn't work is context menus - in (most) Windows programs. I want my OSK user to be able to click the 'Context Menu' key (normally beside right-Ctrl on standard keyboard) and use the arrow keys to navigate that menu. The context menu appears fine, but when I click back onto my Frame (to press an arrow key), the context menu disappears.
Anybody got any ideas as to how this might be achieved (probably a fairly monstrous 'hack' required I would think)? I'm pretty sure the context is disappearing because a mouse-click is occurring elsewhere on the screen (hard to get around that when you need to actually click something!).
Below is a test app. I can use it to navigate down through a context menu in NetBeans successfully, but not in any other application I've tried (e.g. Windows Explorer).
import java.awt.Robot;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.KeyEvent;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.WindowConstants;
public class RobotContextMenuTest extends JFrame implements ActionListener
{
Robot R;
JPanel JP;
JButton JBmenu, JBdown;
public RobotContextMenuTest()
{
try
{
R = new Robot();
}
catch(Exception e)
{
e.printStackTrace();
}
JP = new JPanel();
getContentPane().add(JP);
JBmenu = new JButton("Menu");
JBmenu.addActionListener(this);
JP.add(JBmenu);
JBdown = new JButton("Down");
JBdown.addActionListener(this);
JP.add(JBdown);
setFocusableWindowState(false);
setAlwaysOnTop(true);
setSize(200,80);
setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
setVisible(true);
}
public static void main(String args[])
{
new RobotContextMenuTest();
}
public void actionPerformed(ActionEvent e)
{
if(e.getActionCommand().equals("Menu"))
{
R.keyPress(KeyEvent.VK_CONTEXT_MENU);
R.keyRelease(KeyEvent.VK_CONTEXT_MENU);
}
else if(e.getActionCommand().equals("Down"))
{
R.keyPress(KeyEvent.VK_DOWN);
R.keyRelease(KeyEvent.VK_DOWN);
}
}
}
Finally, here's what I've tried:
Sorry for the long-winded question but it's a fairly complicated problem! Incidentally, the standard Windows 7 OSK can't do this either but WiViK can. Thanks.
Finally cracked this with a lot of help from a friend (thanks Chris Gregan!). After many failed attempts, the answer was relatively simple. I used JNA and Windows API to create a hook for low-level mouse events. It's all described with 2 sample classes given here in an answer by 'prunge'. Note: I had to include the jars for jna version 3.4.0 and jna platform version 3.4.0 (newer versions of these jars resulted in errors!).
Using the above linked MouseHook, you can prevent a low-level event from propagating (reaching other applications) by returning -1 from the LowLevelMouseProc callback.
Here is a snippet I added to the callback method in MouseHook to intercept/suppress left mouse button presses (thereby preventing context menus from disappearing!):
if(nCode >= 0 && wParam.intValue() == WinUserX.WM_LBUTTONDOWN)
return new LRESULT(-1);
Thanks again to Alex Barker for taking the time to look at this and point me in the direction of JNA and Windows API.