Skip to content

perf: stream page-source XML with exception-safe resource cleanup - #1276

Merged
KazuCocoa merged 4 commits into
masterfrom
codex/perf-stream-xml-with-cleanup
Sep 30, 2026
Merged

KazuCocoa merged 4 commits into
masterfrom
codex/perf-stream-xml-with-cleanup

Conversation

@KazuCocoa

@KazuCocoa KazuCocoa commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Page-source generation builds a complete libxml2 document and then traverses it again to produce a string. Snapshot or attribute exceptions also bypass manual cleanup and leak native objects.

Write page-source XML directly to an xmlBuffer using a memory writer, preserving indentation and escaping. Keep document construction for XPath evaluation. Use @try/@finally around both paths to release writers, buffers, documents, and XPath results on success, early return, or exception.

This combines review finding 3 (libxml2 resource leaks) with the XML streaming optimization. Tests compare serialized output against the document-based writer and inject an attribute exception to check document cleanup.

Performance experiment

Compared the two libxml2 serialization strategies used before and after this change in a standalone C microbenchmark:

  • Before: xmlNewTextWriterDoc → build a document → xmlDocDumpFormatMemory → free the writer/document/output.
  • After: xmlNewTextWriterMemory → write directly into an xmlBuffer → copy the output → free the writer/buffer/output.
  • Fixture: one application root with 100, 1,000, 5,000, or 10,000 button children, each with 12 attributes. All attribute values are the synthetic string sample & value, exercising XML escaping. Both paths produce the same UTF-8 declaration and indentation.
  • Check output length and byte-for-byte equality before timing; all four sizes matched.
  • Three warmup iterations per implementation and size, then 30 measured iterations per implementation. Alternate which implementation runs first on every iteration. Timing uses CLOCK_MONOTONIC and includes allocation, serialization, output copying/dumping, and cleanup.
  • Environment: macOS 26.6.2, arm64, Apple Clang 21.0.0, -O2, system libxml2 reporting version 20913 (2.9.13). Results below are from one local run.
Child elements Output bytes Document + dump, median Memory writer, median Speedup Time reduction
100 35,198 0.781 ms 0.146 ms 5.35× 81.3%
1,000 351,098 4.294 ms 0.794 ms 5.41× 81.5%
5,000 1,755,098 21.134 ms 3.907 ms 5.41× 81.5%
10,000 3,510,098 42.530 ms 7.637 ms 5.57× 82.0%

For 5,000 children, the isolated serialization work decreased from 21.134 ms to 3.907 ms: 5.41× faster / 81.5% less time, with identical output.

Why this improves performance, and what the measurement covers

The document-based path allocates a DOM representation of the elements and attributes, walks it again to serialize the XML, and then frees that tree. The memory writer emits the XML during the initial traversal, avoiding the intermediate DOM and its second traversal. The benchmark includes a final output copy for the memory writer so it does not gain an artificial advantage from returning a borrowed buffer.

This measures the libxml2 serialization strategies, not the full Objective-C WDA endpoint. It excludes XCTest/accessibility snapshot acquisition, AX attribute retrieval, idle waits, NSString conversion, and HTTP transport. It does not establish a 5.41× end-to-end /source speedup, and it does not measure peak memory. Device-level profiling is still needed to quantify the overall benefit on real applications. XPath evaluation continues to use a DOM; this benchmark does not claim faster XPath queries. The exception cleanup is a separate correctness improvement.

Reproduce the benchmark

Save the following as /tmp/xml-bench.c:

#include <libxml/xmlwriter.h>
#include <libxml/tree.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
static double now(void) { struct timespec t; clock_gettime(CLOCK_MONOTONIC, &t); return t.tv_sec + t.tv_nsec / 1e9; }
static xmlChar *render(int stream, int count, int *len) {
  xmlDocPtr doc = NULL;
  xmlBufferPtr buffer = stream ? xmlBufferCreate() : NULL;
  xmlTextWriterPtr w = stream ? xmlNewTextWriterMemory(buffer, 0) : xmlNewTextWriterDoc(&doc, 0);
  if (stream) { xmlTextWriterSetIndent(w, 1); xmlTextWriterSetIndentString(w, BAD_CAST "  "); }
  xmlTextWriterStartDocument(w, NULL, "UTF-8", NULL);
  xmlTextWriterStartElement(w, BAD_CAST "XCUIElementTypeApplication");
  const char *names[] = {"type","value","name","label","enabled","visible","accessible","x","y","width","height","index"};
  for (int n=0;n<count;n++) {
    xmlTextWriterStartElement(w, BAD_CAST "XCUIElementTypeButton");
    for (int a=0;a<12;a++) xmlTextWriterWriteAttribute(w, BAD_CAST names[a], BAD_CAST "sample & value");
    xmlTextWriterEndElement(w);
  }
  xmlTextWriterEndElement(w); xmlTextWriterEndDocument(w);
  xmlChar *result = NULL;
  if (stream) { *len = xmlBufferLength(buffer); result = xmlStrndup(xmlBufferContent(buffer), *len); }
  else xmlDocDumpFormatMemory(doc, &result, len, 1);
  xmlFreeTextWriter(w);
  if (doc) xmlFreeDoc(doc);
  if (buffer) xmlBufferFree(buffer);
  return result;
}
static int compare(const void *a, const void *b) {
  double x = *(const double *)a, y = *(const double *)b;
  return (x > y) - (x < y);
}
int main(void) {
  const int sizes[] = {100, 1000, 5000, 10000};
  const int runs = 30;
  printf("libxml=%s runs=%d warmups=3 per mode\n", xmlParserVersion, runs);
  for (int size = 0; size < 4; size++) {
    int n = sizes[size], a, b;
    xmlChar *dom = render(0, n, &a), *stream = render(1, n, &b);
    int equal = a == b && memcmp(dom, stream, a) == 0;
    printf("children=%d attributesPerChild=12 equalBytes=%d bytes=%d\n", n, equal, a);
    xmlFree(dom); xmlFree(stream);
    if (!equal) return 1;
    for (int i = 0; i < 3; i++) {
      for (int mode = 0; mode < 2; mode++) {
        xmlChar *s = render(mode, n, &a); xmlFree(s);
      }
    }
    double times[2][30], sums[2] = {0, 0};
    for (int i = 0; i < runs; i++) {
      for (int j = 0; j < 2; j++) {
        int mode = (i + j) % 2; // Alternate which implementation runs first.
        double start = now();
        xmlChar *s = render(mode, n, &a); xmlFree(s);
        times[mode][i] = (now() - start) * 1000;
        sums[mode] += times[mode][i];
      }
    }
    double medians[2];
    for (int mode = 0; mode < 2; mode++) {
      qsort(times[mode], runs, sizeof(double), compare);
      medians[mode] = (times[mode][14] + times[mode][15]) / 2;
      printf("%s medianMs=%.3f meanMs=%.3f\n", mode ? "memory-writer" : "doc-writer+dump", medians[mode], sums[mode] / runs);
    }
    printf("medianSpeedup=%.2fx medianReduction=%.1f%%\n", medians[0] / medians[1], (1 - medians[1] / medians[0]) * 100);
  }
  xmlCleanupParser();
}

Compile and run on macOS with Xcode command-line tools:

xcrun clang -O2 \
  -I "$(xcrun --sdk macosx --show-sdk-path)/usr/include/libxml2" \
  /tmp/xml-bench.c -lxml2 -o /tmp/xml-bench
/tmp/xml-bench

The program prints output equality, median and mean times, and the median speedup for each fixture size. Absolute timings depend on the machine and system load.

Regression validation

All 178 native unit tests passed on an iOS 26.5 simulator using a local integration branch containing this review series, built with Xcode 27.1. The regression tests are included in this PR. The output-comparison fixture includes XML special characters, newlines, tabs, and non-ASCII text; the exception-injection test checks document cleanup.

Comment thread WebDriverAgentLib/Utilities/FBXPath.m Outdated
Comment thread WebDriverAgentLib/Utilities/FBXPath.m
@KazuCocoa
KazuCocoa marked this pull request as ready for review September 29, 2026 06:03
@KazuCocoa
KazuCocoa merged commit 4e8639b into master Sep 30, 2026
56 of 60 checks passed
@KazuCocoa
KazuCocoa deleted the codex/perf-stream-xml-with-cleanup branch September 30, 2026 07:17
github-actions Bot pushed a commit that referenced this pull request Sep 30, 2026
## [16.13.5](v16.13.4...v16.13.5) (2026-09-30)

### Performance Improvements

* stream page-source XML with exception-safe resource cleanup ([#1276](#1276)) ([4e8639b](4e8639b))
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 16.13.5 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants