macoscocoamenuparent-childaccessibility-api
How to get an array of AXMenuItems from AXMenu?
For my code I am attempting to get an array of AXMenuItems from an AXMenu (AXUIElementRef). The menu logs successfully, and here is my code:
NSArray *anArray = [NSRunningApplication runningApplicationsWithBundleIdentifier:@"com.apple.dock"];
AXUIElementRef anAXDockApp = AXUIElementCreateApplication([[anArray objectAtIndex:0] processIdentifier]);
CFTypeRef aChildren;
AXUIElementCopyAttributeValue(anAXDockApp, kAXChildrenAttribute, &aChildren);
SafeCFRelease(anAXDockApp);
CFTypeRef aMenu = CFArrayGetValueAtIndex(aChildren, 0);
NSLog(@"aMenu: %@", aMenu);
// Get menu items
CFTypeRef aMenuChildren;
AXUIElementCopyAttributeValue(aMenu, kAXVisibleChildrenAttribute, &aMenuChildren);
for (NSInteger i = 0; i < CFArrayGetCount(aMenuChildren); i++) {
AXUIElementRef aMenuItem = [self copyAXUIElementFrom:aMenu role:kAXMenuItemRole atIndex:i];
NSLog(@"aMenuItem: %@", aMenuItem); // logs (null)
CFTypeRef aTitle;
AXUIElementCopyAttributeValue(aMenuItem, kAXTitleAttribute, &aTitle);
if ([(__bridge NSString *)aTitle isEqualToString:@"New Window"] || [(__bridge NSString *)aTitle isEqualToString:@"New Finder Window"]) /* Crashes here (i can see why)*/{
AXUIElementPerformAction(aMenuItem, kAXPressAction);
[NSThread sleepForTimeInterval:1];
break;
}
}
What is the correct way to get the list of AXMenuItems?
Screenshot of the Accessibility Inspector:
Solution
I have figured out an answer, using @Willeke answer of using AXUIElementCopyElementAtPosition() to get the menu. Since there were multiple dock orientations and hiding, I had to create enums in the .h file as it would be easier to read than 0, 1, or 2.
// .h
typedef enum {
kDockPositionBottom,
kDockPositionLeft,
kDockPositionRight,
kDockPositionUnknown
} DockPosition;
typedef enum {
kDockAutohideOn,
kDockAutohideOff
} DockAutoHideState;
Then, I added the methods to get these states in the .m
// .m
- (DockPosition)dockPosition
{
NSRect screenRect = [[NSScreen mainScreen] frame];
NSRect visibleRect = [[NSScreen mainScreen] visibleFrame];
// Dont need to remove menubar height
visibleRect.origin.y = 0;
if (visibleRect.origin.x > screenRect.origin.x) {
return kDockPositionLeft;
} else if (visibleRect.size.width < screenRect.size.width) {
return kDockPositionRight;
} else if (visibleRect.size.height < screenRect.size.height) {
return kDockPositionBottom;
}
return kDockPositionUnknown;
}
- (DockAutoHideState)dockHidden
{
NSString *plistPath = [NSHomeDirectory() stringByAppendingPathComponent:@"Library/Preferences/com.apple.dock.plist"];
NSDictionary *dockDict = [NSDictionary dictionaryWithContentsOfFile:plistPath];
CFBooleanRef autohide = CFDictionaryGetValue((__bridge CFDictionaryRef)dockDict, @"autohide");
if (CFBooleanGetValue(autohide) == true) {
return kDockAutohideOn;
}
return kDockAutohideOff;
}
For the dock position unknown, I added in case it was not able to calculate it from the screen positions.
Then, I used a method to get the dock item from the menubar:
- (AXUIElementRef)getDockItemWithName:(NSString *)name
{
NSArray *anArray = [NSRunningApplication runningApplicationsWithBundleIdentifier:@"com.apple.dock"];
AXUIElementRef anAXDockApp = AXUIElementCreateApplication([[anArray objectAtIndex:0] processIdentifier]);
AXUIElementRef aList = [self copyAXUIElementFrom:anAXDockApp role:kAXListRole atIndex:0];
CFTypeRef aChildren;
AXUIElementCopyAttributeValue(aList, kAXChildrenAttribute, &aChildren);
NSInteger itemIndex = -1;
for (NSInteger i = 0; i < CFArrayGetCount(aChildren); i++) {
AXUIElementRef anElement = CFArrayGetValueAtIndex(aChildren, i);
CFTypeRef aResult;
AXUIElementCopyAttributeValue(anElement, kAXTitleAttribute, &aResult);
if ([(__bridge NSString *)aResult isEqualToString:name]) {
itemIndex = i;
}
}
SafeCFRelease(aChildren);
if (itemIndex == -1) return nil;
// We have index now do something with it
AXUIElementRef aReturnItem = [self copyAXUIElementFrom:aList role:kAXDockItemRole atIndex:itemIndex];
SafeCFRelease(aList);
return aReturnItem;
}
This SafeCFRelease() method is a very simple method that checks if the passed value is not nil, then releases (had some issues earlier).
void SafeCFRelease( CFTypeRef cf )
{
if (cf) CFRelease(cf);
}
And this method [copyAXUIElementFrom: role: atIndex:] is a method from @Willeke answer from another one of my questions:
- (AXUIElementRef)copyAXUIElementFrom:(AXUIElementRef)theContainer role:(CFStringRef)theRole atIndex:(NSInteger)theIndex {
AXUIElementRef aResultElement = NULL;
CFTypeRef aChildren;
AXError anAXError = AXUIElementCopyAttributeValue(theContainer, kAXChildrenAttribute, &aChildren);
if (anAXError == kAXErrorSuccess) {
NSUInteger anIndex = -1;
for (id anElement in (__bridge NSArray *)aChildren) {
if (theRole) {
CFTypeRef aRole;
anAXError = AXUIElementCopyAttributeValue((__bridge AXUIElementRef)anElement, kAXRoleAttribute, &aRole);
if (anAXError == kAXErrorSuccess) {
if (CFStringCompare(aRole, theRole, 0) == kCFCompareEqualTo)
anIndex++;
SafeCFRelease(aRole);
}
}
else
anIndex++;
if (anIndex == theIndex) {
aResultElement = (AXUIElementRef)CFRetain((__bridge CFTypeRef)(anElement));
break;
}
}
SafeCFRelease(aChildren);
}
return aResultElement;
}
Taking all this code, I put it into one of my methods:
// Check if in dock (otherwise cant do it)
if ([self isAppOfNameInDock:[appDict objectForKey:@"AppName"]]) {
// Get dock item
AXUIElementRef aDockItem = [self getDockItemWithName:[appDict objectForKey:@"AppName"]];
AXUIElementPerformAction(aDockItem, kAXShowMenuAction);
[NSThread sleepForTimeInterval:0.5];
CGRect aRect;
CFTypeRef aPosition;
AXUIElementCopyAttributeValue(aDockItem, kAXPositionAttribute, &aPosition);
AXValueGetValue(aPosition, kAXValueCGPointType, &aRect.origin);
SafeCFRelease(aPosition);
CFTypeRef aSize;
AXUIElementCopyAttributeValue(aDockItem, kAXSizeAttribute, &aSize);
AXValueGetValue(aSize, kAXValueCGSizeType, &aRect.size);
SafeCFRelease(aSize);
SafeCFRelease(aDockItem);
CGPoint aMenuPoint;
if ([self dockHidden] == kDockAutohideOff) {
switch ([self dockPosition]) {
case kDockPositionRight:
aMenuPoint = CGPointMake(aRect.origin.x - 18, aRect.origin.y + (aRect.size.height / 2));
break;
case kDockPositionLeft:
aMenuPoint = CGPointMake(aRect.origin.x + aRect.size.width + 18, aRect.origin.y + (aRect.size.height / 2));
break;
case kDockPositionBottom:
aMenuPoint = CGPointMake(aRect.origin.x + (aRect.size.width / 2), aRect.origin.y - 18);
break;
case kDockPositionUnknown:
aMenuPoint = CGPointMake(0, 0);
break;
}
} else {
NSRect screenFrame = [[NSScreen mainScreen] frame];
switch ([self dockPosition]) {
case kDockPositionRight:
aMenuPoint = CGPointMake(screenFrame.size.width - 18, aRect.origin.y + (aRect.size.height / 2));
break;
case kDockPositionLeft:
aMenuPoint = CGPointMake(screenFrame.origin.x + 18, aRect.origin.y + (aRect.size.height / 2));
break;
case kDockPositionBottom:
aMenuPoint = CGPointMake(aRect.origin.x + (aRect.size.width / 2), screenFrame.size.height - 18);
break;
case kDockPositionUnknown:
aMenuPoint = CGPointMake(0, 0);
break;
}
}
if ((aMenuPoint.x != 0) && (aMenuPoint.y != 0)) {
AXUIElementRef _systemWideElement = AXUIElementCreateSystemWide();
AXUIElementRef aMenu;
AXUIElementCopyElementAtPosition(_systemWideElement, aMenuPoint.x, aMenuPoint.y, &aMenu);
SafeCFRelease(_systemWideElement);
// Get menu items
CFTypeRef aMenuChildren;
AXUIElementCopyAttributeValue(aMenu, kAXVisibleChildrenAttribute, &aMenuChildren);
NSRunningApplication *app = [[NSRunningApplication runningApplicationsWithBundleIdentifier:[appDict objectForKey:@"BundleID"]] objectAtIndex:0];
for (NSInteger i = 0; i < CFArrayGetCount(aMenuChildren); i++) {
AXUIElementRef aMenuItem = [self copyAXUIElementFrom:aMenu role:kAXMenuItemRole atIndex:i];
CFTypeRef aTitle;
AXUIElementCopyAttributeValue(aMenuItem, kAXTitleAttribute, &aTitle);
// Supports chrome, safari, and finder
if ([(__bridge NSString *)aTitle isEqualToString:@"New Window"] || [(__bridge NSString *)aTitle isEqualToString:@"New Finder Window"]) {
AXUIElementPerformAction(aMenuItem, kAXPressAction);
NSInteger numberOfWindows = [self numberOfWindowsOpenFromApplicationWithPID:[app processIdentifier]];
// Wait until open
while ([self numberOfWindowsOpenFromApplicationWithPID:[app processIdentifier]] <= numberOfWindows) {
}
break;
}
}
SafeCFRelease(aMenu);
SafeCFRelease(aMenuChildren);
}
}
This is pretty complicated, but it works. I probably can't explain it, but I have stress tested this code and it works quite well.
- Combine rows if the dates exist in another row
- Distinguish between two maximum values in a column with Power Query
- Power Query Conditional JOIN - JOIN with WHERE clause
- Calculate combinations for a range of text values in excel
- How to update a column's values in Power BI M Language
- What's the difference between DAX and Power Query (or M)?
- Sorting issue when consolidating monthly data into a year to date file. Using query, sorting by day, time, and letter
- Power Query `List.Generate` runs too slow
- Custom Colum based on specific dates in powerquery
- How do I properly use table.group in a PowerQuery query to dynamically summarize different rows and columns?
- Excel Power Query syntax issue transform multiple columns of records
- How to capitalize only first letter of a sentence
- Power Query Import For Power BI with group by on SelectRows
- Get data from config table cell when loading data source with power query
- Translate Excel Formula into Power Query
- Counting matching pairs of letters in lists of bigrams in Power Query M
- Bigrams of a string in Power Query M
- Power Query code to refer to another query (and how buffering works)
- M code for the equivalent of of Left (string, len(string)-11)
- How to add a missing column to the Transform Sample File
- How to add or insert ' (single quotes) for every string in a list in which strings are separated by commas in Power Query
- Power BI M code Split column from Lowercase to Uppercase won't include diacritics (accents)
- M Code(Power Query) to Remove Empty columns that runs fast
- How to find the value with date closest to another date
- Error when build AOSP 14 - internal error: undefined variable CTS_TEST_SUITES_DEFAULT
- Call a table using a dynamic identifier
- Problem getting the value from a cell using M in excel
- How to rename columns based on the data within them in Power Query M?
- Power Query return a field name based on parameter
- Power Query: after combining 2 tables got something strange