Skip to content

perf: batch missing AX attributes during XML serialization - #1284

Closed
KazuCocoa wants to merge 1 commit into
masterfrom
codex/perf-batch-snapshot-attributes
Closed

KazuCocoa wants to merge 1 commit into
masterfrom
codex/perf-batch-snapshot-attributes

Conversation

@KazuCocoa

@KazuCocoa KazuCocoa commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

XML serialization can fetch multiple missing accessibility attributes through separate synchronous AX requests for the same snapshot. Prefetch the requested missing attributes in one request when at least two are needed, and cache the returned values on that snapshot.

Keep single-attribute reads lazy, preserve visibility inference, and leave missing or failed values available to existing fallback getters. Reuse prefetched custom-action payloads. Add tests showing one fetch for multiple attributes, reuse of cached false values, and fallback after an empty result.

Validation: all 178 native unit tests passed on an iOS 26.5 simulator using a local integration branch containing this review series. The regression tests are included in this PR. Native changes were built with Xcode 27.1. The live simulator experiment below measures both endpoint latency and AX request count. Physical-device profiling on representative applications remains outstanding.

Live WDA experiment (2026-09-28, Pacific time)

This experiment found an end-to-end benefit on the tested fixtures. With default settings, median /source latency fell by 12.80% on the menu, 26.10% with 256 labels, and 11.49% with 1,024 labels. A separate diagnostic run confirmed that per-element AX attribute calls were halved. These are simulator results for the tested screens, not a guarantee for every application.

Compared baseline 261c08d53917b138eaf2f7d8a744d1151fe3bc8d against this PR alone at 97572f6210916cbad17f0f59e15d11f350a419cc. No other optimization PRs, including #1276 or #1283, were included.

Method and results

  • Release WDA builds, Xcode 27.1 (27A9269), iOS 26.5 (23F77), same simulator and identical IntegrationApp binary. Fixed status bar at 9:41, Wi-Fi, battery 100%.
  • Actual XCTest WDA runners serving GET /session/{sessionId}/source?format=xml. HTTP timing includes the complete response body. JSON parsing, XML checks and artifact writes occur after timing. App launch and session creation are excluded.
  • Runner launch order: before → after → after → before. Fresh app session for each case. Each block/case has two warmups and ten measured requests: 20 measured requests per version per case, 200 measured requests total, plus 40 warmups. No measured samples or outliers were discarded.
  • Session capabilities: bundleId=com.facebook.IntegrationApp, forceAppLaunch=true, shouldTerminateApp=true, shouldWaitForQuiescence=true. Wide fixtures use arguments=["--source-benchmark-nodes", "256"], "1024", or "64".
  • First three cases use default source options. The fourth enables only enforceCustomSnapshots; the fifth enables includeNativeAccessibilityElementInPageSource and includeCustomActionsInPageSource. Hittable, native frame and min/max output remain disabled. Settings were explicitly reset for every case and verified identical across versions.
  • Fixtures: normal IntegrationApp menu and static, visible UIKit UILabel grids with unique node_i identifiers and Node i & value labels. The grids are synthetic stress fixtures. They do not have custom actions or slider min/max values.
Case Before median / p95 (ms) After median / p95 (ms) Median reduction
IntegrationApp menu (26 XML nodes) 64.17 / 70.41 55.96 / 60.19 12.80%
256 labels (261 XML nodes) 483.78 / 549.70 357.51 / 374.81 26.10%
1,024 labels (1,029 XML nodes) 2911.85 / 3186.38 2577.38 / 2784.18 11.49%
256 labels, enforceCustomSnapshots=true 2053.96 / 2339.92 1871.77 / 2111.77 8.87%
64 labels, native accessibility + custom actions enabled 145.59 / 152.86 117.78 / 130.73 19.10%

p95 uses linear interpolation of 20 measured samples. Per-launch medians (ms) show the effect of execution order:

Case Before 1 After 1 After 2 Before 2
IntegrationApp menu (26 XML nodes) 63.42 56.17 54.45 65.29
256 labels (261 XML nodes) 477.99 356.79 358.71 486.30
1,024 labels (1,029 XML nodes) 2947.65 2616.64 2575.67 2892.17
256 labels, enforceCustomSnapshots=true 2063.67 1902.65 1871.42 1933.88
64 labels, native accessibility + custom actions enabled 145.59 119.81 113.84 143.92

The custom-snapshot case varies more between launches; its observed 8.87% reduction should be interpreted cautiously. The default 256-label case consistently moved from approximately 478–486 ms to 357–359 ms in both launches.

XML compatibility

All 240 responses succeeded and matched byte-for-byte within each case after normalizing only the application processId. Exactly one such attribute was required per response. No ordering, geometry, index, visibility, accessibility, custom-action content, whitespace or other attributes were normalized. All wide responses contained the expected number of fixture labels. This validates the tested static fixtures and settings; it does not establish universal regression freedom.

Why it helps: actual AX request counts

After finishing all timing runs, both versions were rebuilt with temporary counters around FBXCAXClientProxy attributesForElement:attributes:error:. These instrumented builds were excluded from the performance table. Each case ran three /source requests; request counts were identical in all three. All 30 diagnostic XML responses also matched the corresponding uninstrumented output after process-ID normalization.

Case Attribute calls before → after Attribute names requested before → after Batch calls after
IntegrationApp menu (26 XML nodes) 52 → 26 52 → 52 26
256 labels (261 XML nodes) 522 → 261 522 → 522 261
1,024 labels (1,029 XML nodes) 2058 → 1029 2058 → 2058 1029
256 labels, enforceCustomSnapshots=true 522 → 261 522 → 522 261
64 labels, native accessibility + custom actions enabled 207 → 138 207 → 276 69

For the default fixtures, separate visibility and accessibility requests become a single two-attribute request per XML node. This is actual batching in the live WDA path.

The optional-attribute fixture has no custom actions. It still makes one batch and one single-attribute call per node, so calls fall from three to two rather than three to one. Logs report unavailable custom actions; the count pattern is consistent with the existing lazy custom-action fallback retrying an unavailable prefetched value. Total attribute names requested rise from 207 to 276. This is a limit of this case, not evidence that every requested attribute is fetched exactly once.

The diagnostic also times the initial snapshot call and individual AX attribute calls. For 256 labels, the median of the two post-warmup diagnostic requests was:

  • Initial snapshot: 61.81 → 59.10 ms.
  • AX attribute calls accumulated: 392.63 → 278.67 ms.
  • XML traversal including attribute getters: 410.97 → 295.19 ms.

AX time is contained within traversal time; these values must not be added together. The traversal measurement is not pure XML-string formatting. With only two post-warmup diagnostic samples, these phase timings illustrate the mechanism and are not a separate statistical performance result. They point to attribute retrieval, rather than the initial snapshot alone, as the dominant measured cost in this fixture.

Scope and reproduction

This is one simulator/SDK and a small sample of static UIs. Physical devices, changing hierarchies, deeply nested containers, text-input special cases, real custom-action payloads, min/max values and XPath projection require separate coverage. Peak memory and CPU time were not measured. The diagnostics count calls at the WDA AX proxy, not low-level transport messages.

Build both revisions with xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -configuration Release -sdk iphonesimulator -destination 'platform=iOS Simulator,id=SIMULATOR_UUID' -derivedDataPath VARIANT_PATH build-for-testing CODE_SIGNING_ALLOWED=NO USE_PORT=8213 USE_IP=127.0.0.1. Launch each generated .xctestrun with test-without-building, -only-testing:WebDriverAgentRunner/UITestingUITests/testRunner, and -parallel-testing-enabled NO. Install the same fixture app once, then create fresh sessions, apply the settings above, and repeat the requests in the balanced order. The fixture is the same as the reproducible patch documented in #1283.

The fixture patch, complete HTTP driver, raw responses, settings, measurements, binary hashes, summary script and temporary diagnostic patches are retained in the local experiment archive. No diagnostic code was committed or pushed.

@KazuCocoa
KazuCocoa marked this pull request as ready for review September 29, 2026 06:13
indexPath:(nullable NSString *)indexPath
includedAttributes:(nullable NSSet<Class> *)includedAttributes
{
FBXCElementSnapshotWrapper *wrapped = [FBXCElementSnapshotWrapper ensureWrapped:element];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this method is too big and needs refactoring

return [self fb_attributeValues:@[attribute] error:error][attribute];
}

- (NSDictionary *)fb_attributeValues:(NSArray<NSString *> *)attributes error:(NSError **)error

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was observing some apps where retrieving of a particular AX attribute takes too long, so that it may ruin the whole batch.

I think this change may be dangerous and may negatively affect some existing tests

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, ok. Then, I'll close this

@KazuCocoa KazuCocoa closed this Sep 30, 2026
@KazuCocoa
KazuCocoa deleted the codex/perf-batch-snapshot-attributes branch September 30, 2026 03:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants