Add ax_tree_walk: readable roles + addressable node paths for a11y dumps

WHATS_NEW.md

@@ -1,5 +1,11 @@
 # What's New — AutoControl
 
+## What's new (2026-06-24) — Readable, Addressable Accessibility Tree (role names + node paths)
+
+Turn a raw `ControlType_50000` tree dump into readable roles with a stable path per node. Full reference: [`docs/source/Eng/doc/new_features/v183_features_doc.rst`](docs/source/Eng/doc/new_features/v183_features_doc.rst).
+
+- **`control_type_name` / `humanize_role` / `humanize_tree` / `assign_node_paths` / `find_by_path`** (`AC_walk_tree`, `AC_humanize_role`): `dump_accessibility_tree` emits the platform's raw role (on Windows the bare UIA ControlType id, e.g. `ControlType_50000` for a button) and carries no stable per-node identity once serialised. This adds the pure post-processing it lacks: translate ControlType ids to friendly names, deep-copy a tree with every role humanised, stamp each node with a stable positional `path` (`"0.2.1"` — a pure stand-in for RuntimeId), and resolve a node back by path. `AC_walk_tree` is the readable counterpart to `AC_a11y_dump`. Pure-stdlib over `AXTreeNode`; unknown / non-UIA roles pass through unchanged. No `PySide6`.
+
 ## What's new (2026-06-24) — Native Text Reading via the UIA TextPattern (document / selection / visible)
 
 Read the text in multiline editors and document controls where ValuePattern returns nothing. Full reference: [`docs/source/Eng/doc/new_features/v182_features_doc.rst`](docs/source/Eng/doc/new_features/v182_features_doc.rst).

docs/source/Eng/doc/new_features/v183_features_doc.rst

@@ -0,0 +1,47 @@
+Readable, Addressable Accessibility Tree (role names + node paths)
+==================================================================
+
+``dump_accessibility_tree`` emits nodes with the platform's *raw* role — on
+Windows that is the bare UI Automation ControlType id, e.g. ``"ControlType_50000"``
+for a button. That is unreadable, and a serialised dump carries no stable
+per-node identity (UIA RuntimeId needs the live element, which the dump has
+thrown away). ``ax_tree_walk`` adds the pure, platform-agnostic post-processing
+the dump lacks, composable on top of any ``dump_accessibility_tree`` output:
+
+* :func:`control_type_name` / :func:`humanize_role` — translate a ControlType id
+  (or ``"ControlType_NNNNN"`` / ``"NNNNN"`` string) to a friendly name,
+* :func:`humanize_tree` — a deep copy of the tree with every role humanised,
+* :func:`assign_node_paths` — a deep copy stamping each node with a stable
+  positional ``path`` (``"0.2.1"``) — a pure stand-in for RuntimeId identity,
+* :func:`find_by_path` — resolve a node back from its path.
+
+Pure-stdlib over ``AXTreeNode`` values; no device or backend access. Imports no
+``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import (dump_accessibility_tree, humanize_tree,
+                                 assign_node_paths, find_by_path, humanize_role)
+
+    humanize_role("ControlType_50000")          # "Button"
+    humanize_role(50004)                         # "Edit"
+
+    tree = assign_node_paths(humanize_tree(dump_accessibility_tree()))
+    # every node now has a readable role and tree["attributes"]["path"]
+    node = find_by_path(tree, "0.0.1")           # re-resolve a node by its path
+
+Unknown ids and non-UIA roles (``"AXApplication"``) pass through unchanged, so
+nothing is lost. The path is stable for a given tree shape, giving scripts /
+agents a deterministic handle to a node across a dump → act round-trip.
+
+Executor commands
+-----------------
+
+``AC_walk_tree`` (``app_name`` / ``max_results``) returns the humanised,
+path-stamped tree as a nested dict — the readable counterpart to
+``AC_a11y_dump``. ``AC_humanize_role`` (``role``) returns ``{"role": ...}``.
+Both are exposed as read-only ``ac_*`` MCP tools and as Script Builder commands
+under **Native UI**.

docs/source/Zh/doc/new_features/v183_features_doc.rst

@@ -0,0 +1,43 @@
+可讀且可定址的無障礙樹(角色名稱 + 節點路徑)
+=============================================
+
+``dump_accessibility_tree`` 輸出的節點帶有平台的*原始*角色——在 Windows 上就是裸的
+UI Automation ControlType id,例如按鈕是 ``"ControlType_50000"``。這既難以閱讀,且序列化後的
+dump 不帶任何穩定的逐節點身分(UIA RuntimeId 需要存活的元素,而 dump 已將其丟棄)。
+``ax_tree_walk`` 補上 dump 所缺、純粹且跨平台的後處理,可疊加在任何
+``dump_accessibility_tree`` 輸出之上:
+
+* :func:`control_type_name` / :func:`humanize_role` ——把 ControlType id(或
+  ``"ControlType_NNNNN"`` / ``"NNNNN"`` 字串)轉成友善名稱,
+* :func:`humanize_tree` ——回傳一份每個角色都已人性化的樹深拷貝,
+* :func:`assign_node_paths` ——回傳一份深拷貝,為每個節點蓋上穩定的位置 ``path``
+  (``"0.2.1"``)——作為 RuntimeId 身分的純粹替代,
+* :func:`find_by_path` ——由 path 反解回節點。
+
+純標準庫,針對 ``AXTreeNode`` 值運算;不存取裝置或後端。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import (dump_accessibility_tree, humanize_tree,
+                                 assign_node_paths, find_by_path, humanize_role)
+
+    humanize_role("ControlType_50000")          # "Button"
+    humanize_role(50004)                         # "Edit"
+
+    tree = assign_node_paths(humanize_tree(dump_accessibility_tree()))
+    # 每個節點現在都有可讀角色與 tree["attributes"]["path"]
+    node = find_by_path(tree, "0.0.1")           # 由 path 重新解析節點
+
+未知 id 與非 UIA 角色(``"AXApplication"``)原樣通過,故不會遺失任何資訊。path 對於給定的
+樹形狀是穩定的,讓腳本 / agent 在 dump → 操作的往返中對某節點有確定性的把手。
+
+執行器指令
+----------
+
+``AC_walk_tree``(``app_name`` / ``max_results``)以巢狀 dict 回傳已人性化、已蓋上 path 的樹
+——即 ``AC_a11y_dump`` 的可讀對應版本。``AC_humanize_role``(``role``)回傳
+``{"role": ...}``。兩者皆以唯讀 ``ac_*`` MCP 工具及 Script Builder 指令(位於 **Native UI**
+分類下)形式提供。

je_auto_control/__init__.py

@@ -57,6 +57,11 @@
 from je_auto_control.utils.ax_text import (
     get_control_text, get_selected_text, get_visible_text,
 )
+# Readable / addressable a11y-tree post-processing (role names + node paths)
+from je_auto_control.utils.ax_tree_walk import (
+    assign_node_paths, control_type_name, find_by_path, humanize_role,
+    humanize_tree,
+)
 # VLM element locator (headless)
 from je_auto_control.utils.vision import (
     VLMNotAvailableError, click_by_description, locate_by_description,
@@ -1622,6 +1627,8 @@ def start_autocontrol_gui(*args, **kwargs):
     "select_control_item", "control_range", "set_control_range",
     "scroll_control_into_view",
     "get_control_text", "get_selected_text", "get_visible_text",
+    "control_type_name", "humanize_role", "humanize_tree",
+    "assign_node_paths", "find_by_path",
     # VLM locator
     "VLMNotAvailableError", "locate_by_description", "click_by_description",
     "verify_description",

je_auto_control/gui/script_builder/command_schema.py

@@ -1538,6 +1538,18 @@ def _add_native_control_specs(specs: List[CommandSpec]) -> None:
         fields=fields,
         description="Read only the on-screen text via TextPattern.GetVisibleRanges.",
     ))
+    specs.append(CommandSpec(
+        "AC_walk_tree", "Native UI", "Walk Accessibility Tree",
+        fields=(FieldSpec("app_name", FieldType.STRING, optional=True),
+                FieldSpec("max_results", FieldType.INT, optional=True,
+                          default=500)),
+        description="Dump the a11y tree with friendly roles + a path per node.",
+    ))
+    specs.append(CommandSpec(
+        "AC_humanize_role", "Native UI", "Humanize UIA Role",
+        fields=(FieldSpec("role", FieldType.STRING),),
+        description="Translate a raw UIA role (ControlType_50000) to a name.",
+    ))
 
 
 def _add_misc_specs(specs: List[CommandSpec]) -> None:

je_auto_control/utils/ax_tree_walk/__init__.py

@@ -0,0 +1,10 @@
+"""Readable, addressable accessibility-tree post-processing (role names + node paths)."""
+from je_auto_control.utils.ax_tree_walk.ax_tree_walk import (
+    assign_node_paths, control_type_name, find_by_path, humanize_role,
+    humanize_tree,
+)
+
+__all__ = [
+    "control_type_name", "humanize_role", "humanize_tree",
+    "assign_node_paths", "find_by_path",
+]

je_auto_control/utils/ax_tree_walk/ax_tree_walk.py

@@ -0,0 +1,108 @@
+"""Make an accessibility-tree dump readable and addressable.
+
+``dump_accessibility_tree`` emits nodes with the platform's *raw* role —
+on Windows that is the bare UI Automation ControlType id, e.g.
+``"ControlType_50000"`` for a button. That is unreadable, and the dump
+carries no stable per-node identity (UIA RuntimeId needs the live element,
+which a serialised dump has thrown away). ``ax_tree_walk`` adds the pure,
+platform-agnostic post-processing the dump lacks:
+
+* :func:`control_type_name` / :func:`humanize_role` — translate a UIA
+  ControlType id (or ``"ControlType_NNNNN"`` string) to a friendly name,
+* :func:`humanize_tree` — a deep copy of the tree with every role humanised,
+* :func:`assign_node_paths` — a deep copy stamping each node with a stable
+  positional ``path`` (``"0.2.1"``) — a pure stand-in for RuntimeId identity,
+* :func:`find_by_path` — resolve a node back from its path.
+
+Pure-stdlib over :class:`AXTreeNode` values; no device or backend access, no
+``PySide6``. Compose it on top of any ``dump_accessibility_tree`` output.
+"""
+from typing import Optional, Union
+
+from je_auto_control.utils.accessibility.tree import AXTreeNode
+
+# UIA ControlType ids → friendly names (UIAutomationClient ControlTypeId range).
+_CONTROL_TYPE_NAMES = {
+    50000: "Button", 50001: "Calendar", 50002: "CheckBox", 50003: "ComboBox",
+    50004: "Edit", 50005: "Hyperlink", 50006: "Image", 50007: "ListItem",
+    50008: "List", 50009: "Menu", 50010: "MenuBar", 50011: "MenuItem",
+    50012: "ProgressBar", 50013: "RadioButton", 50014: "ScrollBar",
+    50015: "Slider", 50016: "Spinner", 50017: "StatusBar", 50018: "Tab",
+    50019: "TabItem", 50020: "Text", 50021: "ToolBar", 50022: "ToolTip",
+    50023: "Tree", 50024: "TreeItem", 50025: "Custom", 50026: "Group",
+    50027: "Thumb", 50028: "DataGrid", 50029: "DataItem", 50030: "Document",
+    50031: "SplitButton", 50032: "Window", 50033: "Pane", 50034: "Header",
+    50035: "HeaderItem", 50036: "Table", 50037: "TitleBar", 50038: "Separator",
+    50039: "SemanticZoom", 50040: "AppBar",
+}
+_ROLE_PREFIX = "ControlType_"
+
+
+def control_type_name(control_type: int) -> str:
+    """Return the friendly name for a UIA ControlType id (e.g. ``50000`` → ``Button``).
+
+    Unknown ids fall back to ``"ControlType_<id>"`` so nothing is lost.
+    """
+    cid = int(control_type)
+    return _CONTROL_TYPE_NAMES.get(cid, f"{_ROLE_PREFIX}{cid}")
+
+
+def humanize_role(role: Union[str, int]) -> str:
+    """Map a raw UIA role to a friendly name.
+
+    Accepts an int id (``50000``), a ``"ControlType_50000"`` string, or a bare
+    ``"50000"`` string. Any role that is not a recognised ControlType — already
+    friendly (``"Button"``) or a non-UIA role (``"AXApplication"``) — is returned
+    unchanged.
+    """
+    if isinstance(role, int):
+        return control_type_name(role)
+    text = str(role)
+    digits = text[len(_ROLE_PREFIX):] if text.startswith(_ROLE_PREFIX) else text
+    if digits.isdigit():
+        return control_type_name(int(digits))
+    return text
+
+
+def humanize_tree(node: AXTreeNode) -> AXTreeNode:
+    """Return a deep copy of ``node`` with every role run through :func:`humanize_role`."""
+    return AXTreeNode(
+        name=node.name, role=humanize_role(node.role), bounds=node.bounds,
+        app_name=node.app_name, process_id=node.process_id,
+        attributes=dict(node.attributes),
+        children=[humanize_tree(child) for child in node.children],
+    )
+
+
+def assign_node_paths(node: AXTreeNode, prefix: str = "0") -> AXTreeNode:
+    """Return a deep copy stamping each node with a stable positional ``path``.
+
+    The root is ``"0"``; its third child is ``"0.2"``, and so on. The path is a
+    pure stand-in for a RuntimeId: stable for a given tree shape and re-resolvable
+    with :func:`find_by_path`. Stored under ``attributes["path"]``.
+    """
+    attributes = dict(node.attributes)
+    attributes["path"] = prefix
+    children = [assign_node_paths(child, f"{prefix}.{index}")
+                for index, child in enumerate(node.children)]
+    return AXTreeNode(
+        name=node.name, role=node.role, bounds=node.bounds,
+        app_name=node.app_name, process_id=node.process_id,
+        attributes=attributes, children=children,
+    )
+
+
+def find_by_path(root: AXTreeNode, path: str) -> Optional[AXTreeNode]:
+    """Resolve the node addressed by ``path`` (e.g. ``"0.2.1"``); ``None`` if absent."""
+    parts = str(path).split(".")
+    if not parts or parts[0] != "0":
+        return None
+    node = root
+    for part in parts[1:]:
+        if not part.isdigit():
+            return None
+        index = int(part)
+        if index >= len(node.children):
+            return None
+        node = node.children[index]
+    return node

je_auto_control/utils/executor/action_executor.py

@@ -182,6 +182,23 @@ def _a11y_dump(app_name: Optional[str] = None,
     ).to_dict()
 
 
+def _walk_tree(app_name: Optional[str] = None,
+               max_results: int = 500) -> Dict[str, Any]:
+    """Executor adapter: dump the a11y tree with friendly roles + node paths."""
+    from je_auto_control.utils.accessibility import dump_accessibility_tree
+    from je_auto_control.utils.ax_tree_walk import (
+        assign_node_paths, humanize_tree)
+    root = dump_accessibility_tree(app_name=app_name,
+                                   max_results=int(max_results))
+    return assign_node_paths(humanize_tree(root)).to_dict()
+
+
+def _humanize_role(role: str) -> Dict[str, Any]:
+    """Executor adapter: translate a raw UIA role to a friendly name."""
+    from je_auto_control.utils.ax_tree_walk import humanize_role
+    return {"role": humanize_role(role)}
+
+
 def _a11y_record_start(app_name: Optional[str] = None,
                         poll_interval_s: float = 0.25,
                         min_movement_px: int = 8) -> Dict[str, Any]:
@@ -6108,6 +6125,8 @@ def __init__(self):
             "AC_a11y_find": _a11y_find_as_dict,
             "AC_a11y_click": click_accessibility_element,
             "AC_a11y_dump": _a11y_dump,
+            "AC_walk_tree": _walk_tree,
+            "AC_humanize_role": _humanize_role,
             "AC_control_get_value": _control_get_value,
             "AC_control_set_value": _control_set_value,
             "AC_control_invoke": _control_invoke,

je_auto_control/utils/mcp_server/tools/_factories.py

@@ -1200,6 +1200,29 @@ def a11y_tree_tools() -> List[MCPTool]:
             handler=h.a11y_dump,
             annotations=READ_ONLY,
         ),
+        MCPTool(
+            name="ac_walk_tree",
+            description=("Dump the accessibility tree like ac_a11y_dump but with "
+                         "friendly role names (UIA 'ControlType_50000' → "
+                         "'Button') and a stable positional 'path' per node "
+                         "(addressable via the path attribute)."),
+            input_schema=schema({
+                "app_name": {"type": "string"},
+                "max_results": {"type": "integer"},
+            }),
+            handler=h.walk_tree,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_humanize_role",
+            description=("Translate a raw UIA role ('ControlType_50000' / "
+                         "'50000') to a friendly name: {role}. Unknown / "
+                         "already-friendly roles pass through unchanged."),
+            input_schema=schema({"role": {"type": "string"}},
+                                required=["role"]),
+            handler=h.humanize_role,
+            annotations=READ_ONLY,
+        ),
         MCPTool(
             name="ac_a11y_record_start",
             description=("Start the polling accessibility recorder. "

je_auto_control/utils/mcp_server/tools/_handlers.py

@@ -2887,6 +2887,16 @@ def a11y_dump(app_name: Optional[str] = None,
     ).to_dict()
 
 
+def walk_tree(app_name=None, max_results: int = 500):
+    from je_auto_control.utils.executor.action_executor import _walk_tree
+    return _walk_tree(app_name, max_results)
+
+
+def humanize_role(role):
+    from je_auto_control.utils.executor.action_executor import _humanize_role
+    return _humanize_role(role)
+
+
 def a11y_record_start(app_name: Optional[str] = None,
                       poll_interval_s: float = 0.25,
                       min_movement_px: int = 8) -> Dict[str, Any]: