无法获取某AXGroup的AXChildren：如何访问其可见子元素？

我在模拟器中尝试用以下Objective-C代码获取一个AXGroup的所有子元素：

+ (NSArray<NSValue *> *)childrenOfUIElement:(AXUIElementRef)element {
    CFArrayRef children = nil;

    children = (__bridge CFArrayRef)([UIElementUtilities valueOfAttribute:NSAccessibilityChildrenAttribute ofUIElement:element]);

    NSArray *childrenValues = (__bridge NSArray<NSValue *> *)(children);

    if ([childrenValues count] > 0) {
        return childrenValues;
    } else {
        children = (__bridge CFArrayRef)([UIElementUtilities valueOfAttributes:NSAccessibilityChildrenAttribute ofUIElement:element]);
        return (__bridge NSArray<NSValue *> *)(children);
    }
}

+ (id)valueOfAttribute:(NSString *)attribute ofUIElement:(AXUIElementRef)element
{
    CFTypeRef result = nil;
    NSArray *attributeNames = [UIElementUtilities attributeNamesOfUIElement:element];
    
    if (attributeNames) {
        if ( [attributeNames indexOfObject:(NSString *)attribute] != NSNotFound
                &&
            AXUIElementCopyAttributeValue(element, (CFStringRef)attribute, &result) == kAXErrorSuccess
        ) {
        }
    }
    return (__bridge id)(result);
}

+ (id)valueOfAttributes:(NSString *)attribute ofUIElement:(AXUIElementRef)element
{
    CFArrayRef result = nil;
    NSArray *attributeNames = [UIElementUtilities attributeNamesOfUIElement:element];

    if (attributeNames) {
        if ( [attributeNames indexOfObject:(NSString *)attribute] != NSNotFound
            &&
            AXUIElementCopyAttributeValues(element, (CFStringRef)attribute, 0, 999, &result) == kAXErrorSuccess
            ) {
        }
    }
    return (__bridge id)(result);
}

该组在截图中以黄色边框标注，实际包含至少三个AXButton，但打印该AXGroup的信息显示其AXChildren为空数组，且在Accessibility Inspector中无法查看其层级结构：

description: <AXApplication: “Simulator”>
 <AXWindow: “iPhone SE (3rd generation) – iOS 18.0”>
  <AXGroup>
   <AXGroup>

Attributes:
   AXFocused:  “0”
   AXChildrenInNavigationOrder:  “<array of size 0>”
   AXFrame:  “x=977 y=121.51 w=387 h=98.4468”
   AXUserInputLabels:  “<array of size 0>”
   AXLanguage:  “(null)”
   AXPosition:  “x=977 y=121.51”
   AXSelected:  “0”
   AXLinkedUIElements:  “(null)”
   AXActivationPoint:  “x=1170.5 y=170.733”
   AXEnabled:  “1”
   AXCustomContent:  “(null)”
   AXHelp:  “(null)”
   AXChildren:  “<array of size 0>”
   AXRole:  “AXGroup”
   AXParent:  “<AXGroup>”
   AXTopLevelUIElement:  “<AXWindow: “iPhone SE (3rd generation) – iOS 18.0”>”
   AXCustomRotors:  “(null)”
   AXDescription:  “(null)”
   AXSubrole:  “(null)”
   AXCustomActions:  “(null)”
   AXValue:  “(null)”
   AXRoleDescription:  “Nav bar”
   AXSize:  “w=387 h=98.4468”
   AXIdentifier:  “Scheduled”
   AXWindow:  “<AXWindow: “iPhone SE (3rd generation) – iOS 18.0”>”

Actions:
   AXScrollToVisible - AXScrollToVisible
   AXShowMenu - show menu

编辑补充：在Accessibility Inspector中仅能看到该元素，无法查看层级。

解决方法 #

1. 尝试获取NSAccessibilityContainedElementsAttribute属性
   部分容器元素不会通过AXChildren暴露子元素，但会通过AXContainedElements提供。修改childrenOfUIElement方法，加入该属性的判断：

   + (NSArray<NSValue *> *)childrenOfUIElement:(AXUIElementRef)element {
       CFArrayRef children = nil;
   
       // 先尝试获取AXChildren
       children = (__bridge CFArrayRef)([UIElementUtilities valueOfAttribute:NSAccessibilityChildrenAttribute ofUIElement:element]);
       NSArray *childrenValues = (__bridge NSArray<NSValue *> *)(children);
       if ([childrenValues count] > 0) {
           return childrenValues;
       }
   
       // 尝试获取AXContainedElements
       CFArrayRef containedElements = (__bridge CFArrayRef)([UIElementUtilities valueOfAttribute:NSAccessibilityContainedElementsAttribute ofUIElement:element]);
       if (containedElements && CFArrayGetCount(containedElements) > 0) {
           return (__bridge NSArray<NSValue *> *)(containedElements);
       }
   
       // 最后尝试批量获取AXChildren
       children = (__bridge CFArrayRef)([UIElementUtilities valueOfAttributes:NSAccessibilityChildrenAttribute ofUIElement:element]);
       return (__bridge NSArray<NSValue *> *)(children);
   }
   

2. 遍历窗口所有元素，通过Frame筛选
   如果目标AXGroup的子元素未被正确关联，直接遍历整个窗口的元素，筛选出Frame落在目标组范围内的元素：

   + (NSArray<NSValue *> *)findElementsInGroupFrame:(CGRect)groupFrame fromElement:(AXUIElementRef)element {
       NSMutableArray *result = [NSMutableArray array];
       CFArrayRef children = (__bridge CFArrayRef)([UIElementUtilities valueOfAttribute:NSAccessibilityChildrenAttribute ofUIElement:element]);
       if (!children) return result;
       
       NSArray *childValues = (__bridge NSArray<NSValue *> *)(children);
       for (NSValue *childValue in childValues) {
           AXUIElementRef child = childValue.pointerValue;
           // 获取子元素Frame
           CGRect childFrame = CGRectZero;
           CFTypeRef frameRef = nil;
           if (AXUIElementCopyAttributeValue(child, kAXFrameAttribute, &frameRef) == kAXErrorSuccess) {
               childFrame = CGRectMakeWithDictionaryRepresentation((__bridge CFDictionaryRef)frameRef);
               CFRelease(frameRef);
           }
           // 判断是否在目标组范围内
           if (CGRectIntersectsRect(childFrame, groupFrame)) {
               [result addObject:childValue];
               // 递归检查子元素
               [result addObjectsFromArray:[self findElementsInGroupFrame:groupFrame fromElement:child]];
           }
       }
       return result;
   }
   

   使用时，先获取目标AXGroup的Frame，再传入窗口元素调用该方法。

3. 针对导航栏角色的特殊处理
   该元素的AXRoleDescription是"Nav bar"，可能是UINavigationBar的包装元素，尝试直接获取导航栏专用的AXBarButtons属性：

   + (NSArray<NSValue *> *)barButtonsOfNavBar:(AXUIElementRef)navBarElement {
       CFArrayRef barButtons = nil;
       if (AXUIElementCopyAttributeValue(navBarElement, kAXBarButtonsAttribute, &barButtons) == kAXErrorSuccess) {
           return (__bridge NSArray<NSValue *> *)(barButtons);
       }
       return @[];
   }
   

4. 适配iOS 18的API调整
   iOS 18对辅助功能API有部分调整，尝试调用AXUIElementCopyAttributeValues时将数量参数设为更大值（比如1000），确保不会漏掉元素；同时可以尝试遍历AXLayoutArea或AXLayoutItem类型的元素，这类元素可能包含你需要的按钮。

内容的提问来源于stack exchange，提问作者dasdom