Search code examples
c#.netautomationui-automation

AutomationElement shows up using Inspect.exe but does show not up when using UIAutomationCore.dll or System.Windows.Automation

TL;DR: What am I doing wrong that is causing the workspace pane to show up in Inspect Objects but not show up in my custom code?

I am trying to write some UI automation to a 3rd party program. I am using Inspect.exe that came with the Windows SDK, and I have tried both System.Windows.Automation and direct COM Calls (using the wrapper library from UIA Verify).

Process[] processes = Process.GetProcessesByName("Redacted Client");
if (processes.Length == 0) throw new Exception("Could not find \"Redacted Client\" process");

PropertyCondition parentFileCond = new PropertyCondition(AutomationElement.ProcessIdProperty, processes[0].Id);
PropertyCondition workspaceCond = new PropertyCondition(AutomationElement.NameProperty, "Workspace", PropertyConditionFlags.IgnoreCase);
PropertyCondition documentCond = new PropertyCondition(AutomationElement.NameProperty, "Untitled3", PropertyConditionFlags.IgnoreCase);

var parentElement = AutomationElement.RootElement.FindFirst(TreeScope.Children, parentFileCond);
var workspaceElement = parentElement.FindFirst(TreeScope.Children, workspaceCond); //Also does not work with TreeScope.Descendants
var documentElement = workspaceElement.FindFirst(TreeScope.Children, documentCond);

When I try the above code, parentElement does have the correct reference to the main program window, but workspaceElement is null.

A temporary workaround:

If I change my documentElement code to:

var documentElement = parentElement.FindFirst(TreeScope.Descendants, documentCond);

I will get the correct element returned. I can use this as a workaround as the document window is the one I really wanted anyway, but I would like to know why the Workspace pane would not show up so I can improve my skills in case I run into this in the future with a situation I cannot work around.

UPDATE: I tried MrGomez's suggestions

PropertyCondition parentFileCond = new PropertyCondition(AutomationElement.ProcessIdProperty, 5872);
PropertyCondition panelCond = new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane);

var parentElement = AutomationElement.RootElement.FindFirst(TreeScope.Children, parentFileCond);
var panels = parentElement.FindAll(TreeScope.Children, panelCond);

enter image description here

I get 3 results, unfortunately, I have 4 panels, and the one that did not show up was the panel named Workspace.

I also tried to use a TreeWalker

PropertyCondition parentFileCond = new PropertyCondition(AutomationElement.ProcessIdProperty, 5872);
PropertyCondition workspaceCond= new PropertyCondition(AutomationElement.NameProperty, "Workspace");

var walker = new TreeWalker(workspaceCond);
var parentElement = AutomationElement.RootElement.FindFirst(TreeScope.Children, parentFileCond);
var workspaceElement = walker.Normalize(parentElement);

but that also returns null for workspaceElement

Finally, in desperation, I tried the current value of "NativeWindowHandle" from Inspect and started the walking from the root node.

PropertyCondition workspaceCond = new PropertyCondition(AutomationElement.NativeWindowHandleProperty, 0x110906);
var walker = new TreeWalker(workspaceCond);
var workspaceElement = walker.Normalize(AutomationElement.RootElement);

Workspace element is STILL null.

Result Found

I finally did get Workspace to show up, but I had to perform 

PropertyCondition workspaceCond = new PropertyCondition(AutomationElement.NativeWindowHandleProperty, 0x110906);
var test = AutomationElement.RootElement.FindFirst(TreeScope.Subtree, workspaceCond);

and it took quite a while to run.

Old Screen Captures

Here is screenshots from Inspect.exe showing the tree view.

enter image description here

Here are the properties of the main window of the program.

How found:  Selected from tree...
RuntimeId:  "[42.2557552]"
BoundingRectangle:  {l:75 t:1 r:1311 b:1003}
ProcessId:  8160
ControlType:    UIA_WindowControlTypeId (0xC370)
LocalizedControlType:   "window"
Name:   "Redacted"
AccessKey:  ""
HasKeyboardFocus:   false
IsKeyboardFocusable:    true
IsEnabled:  true
ClassName:  "C:\Program Files (x86)\RedactedProgramFiles7\RedactedClientFolder"
HelpText:   ""
IsPassword: false
NativeWindowHandle: 0x270670
IsOffscreen:    false
FrameworkId:    "Win32"
ProviderDescription:    "[pid:4000,hwnd:0x270670 Main:Nested [pid:8160,hwnd:0x270670 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
Window.CanMaximize: true
Window.CanMinimize: true
Window.WindowVisualState:   Normal (0)
Window.WindowInteractionState:  ReadyForUserInteraction (2)
Window.IsModal: false
Window.IsTopmost:   false
Transform.CanMove:  true
Transform.CanResize:    true
Transform.CanRotate:    false
LegacyIAccessible.ChildId:  0
LegacyIAccessible.DefaultAction:    ""
LegacyIAccessible.Description:  ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "Redacted"
LegacyIAccessible.Role: client (0xA)
LegacyIAccessible.State:    focusable (0x100000)
LegacyIAccessible.Value:    ""
IsDockPatternAvailable: false
IsExpandCollapsePatternAvailable:   false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable:   false
IsLegacyIAccessiblePatternAvailable:    true
IsMultipleViewPatternAvailable: false
IsRangeValuePatternAvailable:   false
IsScrollPatternAvailable:   false
IsScrollItemPatternAvailable:   false
IsSelectionItemPatternAvailable:    false
IsSelectionPatternAvailable:    false
IsTablePatternAvailable:    false
IsTableItemPatternAvailable:    false
IsTextPatternAvailable: false
IsTogglePatternAvailable:   false
IsTransformPatternAvailable:    true
IsValuePatternAvailable:    false
IsWindowPatternAvailable:   true
IsItemContainerPatternAvailable:    false
IsVirtualizedItemPatternAvailable:  false
IsSynchronizedInputPatternAvailable:    false
FirstChild: "Workspace" pane
LastChild:  "Application" menu bar
Next:   "Inspect  (HWND: 0x01700F06)" window
Previous:   "Sandbox Console (Debugging) - Microsoft Visual Studio (Administrator)" window
Other Props:    Object has no additional properties
Children:   "Workspace" pane
    (null) title bar
    "Application" menu bar
Ancestors:  "Desktop" pane
    [ No Parent ]

Here are the properties of the problem "Workspace" pane.

How found:  Selected from tree...
RuntimeId:  "[42.34146524]"
BoundingRectangle:  {l:83 t:51 r:1303 b:995}
ProcessId:  8160
ControlType:    UIA_PaneControlTypeId (0xC371)
LocalizedControlType:   "pane"
Name:   "Workspace"
AccessKey:  ""
HasKeyboardFocus:   false
IsKeyboardFocusable:    true
IsEnabled:  true
ClassName:  "MDIClient"
HelpText:   ""
IsPassword: false
NativeWindowHandle: 0x20908DC
IsOffscreen:    false
FrameworkId:    "Win32"
ProviderDescription:    "[pid:4000,hwnd:0x20908DC Main:Nested [pid:8160,hwnd:0x20908DC Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
LegacyIAccessible.ChildId:  0
LegacyIAccessible.DefaultAction:    ""
LegacyIAccessible.Description:  ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "Workspace"
LegacyIAccessible.Role: client (0xA)
LegacyIAccessible.State:    focusable (0x100000)
LegacyIAccessible.Value:    ""
IsDockPatternAvailable: false
IsExpandCollapsePatternAvailable:   false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable:   false
IsLegacyIAccessiblePatternAvailable:    true
IsMultipleViewPatternAvailable: false
IsRangeValuePatternAvailable:   false
IsScrollPatternAvailable:   false
IsScrollItemPatternAvailable:   false
IsSelectionItemPatternAvailable:    false
IsSelectionPatternAvailable:    false
IsTablePatternAvailable:    false
IsTableItemPatternAvailable:    false
IsTextPatternAvailable: false
IsTogglePatternAvailable:   false
IsTransformPatternAvailable:    false
IsValuePatternAvailable:    false
IsWindowPatternAvailable:   false
IsItemContainerPatternAvailable:    false
IsVirtualizedItemPatternAvailable:  false
IsSynchronizedInputPatternAvailable:    false
FirstChild: "Untitled3" window
LastChild:  "Letters (32638 of 32638):" window
Next:   (null) title bar
Previous:   [null]
Other Props:    Object has no additional properties
Children:   "Untitled3" window
    "Letters (32638 of 32638):" window
Ancestors:  "Redacted" window
    "Desktop" pane
    [ No Parent ]

Here are the properties of the "Working" document window.

How found:  Selected from tree...
RuntimeId:  "[42.9505096]"
BoundingRectangle:  {l:85 t:53 r:651 b:491}
ProcessId:  8160
ControlType:    UIA_WindowControlTypeId (0xC370)
LocalizedControlType:   "window"
Name:   "Untitled3"
AccessKey:  ""
HasKeyboardFocus:   false
IsKeyboardFocusable:    true
IsEnabled:  true
AutomationId:   "10"
ClassName:  "ProToolsSubMDIWndClass"
HelpText:   ""
IsPassword: false
NativeWindowHandle: 0x910948
IsOffscreen:    false
FrameworkId:    "Win32"
ProviderDescription:    "[pid:4000,hwnd:0x910948 Main:Nested [pid:8160,hwnd:0x910948 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
Window.CanMaximize: true
Window.CanMinimize: true
Window.WindowVisualState:   Normal (0)
Window.WindowInteractionState:  ReadyForUserInteraction (2)
Window.IsModal: false
Window.IsTopmost:   false
Transform.CanMove:  true
Transform.CanResize:    true
Transform.CanRotate:    false
LegacyIAccessible.ChildId:  0
LegacyIAccessible.DefaultAction:    ""
LegacyIAccessible.Description:  ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "Untitled3"
LegacyIAccessible.Role: client (0xA)
LegacyIAccessible.State:    focusable (0x100000)
LegacyIAccessible.Value:    ""
IsDockPatternAvailable: false
IsExpandCollapsePatternAvailable:   false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable:   false
IsLegacyIAccessiblePatternAvailable:    true
IsMultipleViewPatternAvailable: false
IsRangeValuePatternAvailable:   false
IsScrollPatternAvailable:   false
IsScrollItemPatternAvailable:   false
IsSelectionItemPatternAvailable:    false
IsSelectionPatternAvailable:    false
IsTablePatternAvailable:    false
IsTableItemPatternAvailable:    false
IsTextPatternAvailable: false
IsTogglePatternAvailable:   false
IsTransformPatternAvailable:    true
IsValuePatternAvailable:    false
IsWindowPatternAvailable:   true
IsItemContainerPatternAvailable:    false
IsVirtualizedItemPatternAvailable:  false
IsSynchronizedInputPatternAvailable:    false
FirstChild: "" thumb
LastChild:  (null) title bar
Next:   "Letters (32638 of 32638):" window
Previous:   [null]
Other Props:    Object has no additional properties
Children:   "" thumb
    (null) title bar
Ancestors:  "Workspace" pane
    "Redacted" window
    "Desktop" pane
    [ No Parent ]
Solution

  • Very nice question. Based upon the problem you've documented, it's clear that your conditional:

    PropertyCondition workspaceCond = new PropertyCondition(
 AutomationElement.NameProperty, "Workspace", PropertyConditionFlags.IgnoreCase);

    ... fails evaluation. Why?

    The answer is how your conditional is evaluated. Looking at your element tree, we notice this property for Workspace:

    IsWindowPatternAvailable:   false

    And for the main window and Untitled3:

    IsWindowPatternAvailable:   true

    From MSDN:

    UIA_IsWindowPatternAvailablePropertyId 30044

    Identifies the IsWindowPatternAvailable property, which indicates whether the Window control pattern is available for the automation element. If TRUE, a client can retrieve an IUIAutomationWindowPattern interface from the element.

    We find a repro in this thread, which implies the same failure pattern as the one you are currently experiencing. We also note the lack of Window properties present for this element because IUIAutomationWindowPattern is inaccessible.

    A workaround is available from the aforelinked thread. Instead of PropertyCondition, one might use:

    public class ConditionMatcher : IMatchConditions
{
    public bool Matches(AutomationElement element, Condition condition)
    {
        return new TreeWalker(condition).Normalize(element) != null;
    }
}

    Or, alternately, one might use the workaround you've given, provided your tree structure is guaranteed to be shallow (and thus, appropriate to the name of this site, will not trigger a stack overflow).

    Admittedly, this wasn't the most obvious issue. In the perfect world, MSDN should have better documentation on this topic.