diff --git a/README/WHATS_NEW_zh-CN.md b/README/WHATS_NEW_zh-CN.md index b925557e..cf759bb0 100644 --- a/README/WHATS_NEW_zh-CN.md +++ b/README/WHATS_NEW_zh-CN.md @@ -1,5 +1,11 @@ # 本次更新 — AutoControl +## 本次更新 (2026-06-23) — 窗口铺排 / 版面几何规划器 + +计算应用程序窗口该放在哪里——半边、网格、层叠。完整参考:[`docs/source/Zh/doc/new_features/v133_features_doc.rst`](../docs/source/Zh/doc/new_features/v133_features_doc.rst)。 + +- **`tile_rect` / `grid_rects` / `cascade_rects`**(`AC_tile_rect`、`AC_grid_rects`、`AC_cascade_rects`):`save/restore_window_layout` 重播*精确*的已存位置、`snap_window` 移动*一个*窗口——没有任何功能能*计算*出全新的多窗口版面。此纯几何规划器在给定屏幕工作区下,返回半边、四分之一、三分之一、R×C 网格与错位层叠的目标矩形,让脚本能以确定性方式排列窗口。返回 `WindowRect`(`.as_tuple()` / `.to_dict()`);`gap` 内缩铺排间距;跨平台且完全无头可测;可与任何窗口移动后端组合。 + ## 本次更新 (2026-06-23) — 以边缘 / 轮廓定位 UI 元素(免模板) 在从未见过的画面上找出可点击的方框。完整参考:[`docs/source/Zh/doc/new_features/v132_features_doc.rst`](../docs/source/Zh/doc/new_features/v132_features_doc.rst)。 diff --git a/README/WHATS_NEW_zh-TW.md b/README/WHATS_NEW_zh-TW.md index 18f2c014..b2ebf916 100644 --- a/README/WHATS_NEW_zh-TW.md +++ b/README/WHATS_NEW_zh-TW.md @@ -1,5 +1,11 @@ # 本次更新 — AutoControl +## 本次更新 (2026-06-23) — 視窗鋪排 / 版面幾何規劃器 + +計算應用程式視窗該放在哪裡——半邊、網格、層疊。完整參考:[`docs/source/Zh/doc/new_features/v133_features_doc.rst`](../docs/source/Zh/doc/new_features/v133_features_doc.rst)。 + +- **`tile_rect` / `grid_rects` / `cascade_rects`**(`AC_tile_rect`、`AC_grid_rects`、`AC_cascade_rects`):`save/restore_window_layout` 重播*精確*的已存位置、`snap_window` 移動*一個*視窗——沒有任何功能能*計算*出全新的多視窗版面。此純幾何規劃器在給定螢幕工作區下,回傳半邊、四分之一、三分之一、R×C 網格與錯位層疊的目標矩形,讓腳本能以決定性方式排列視窗。回傳 `WindowRect`(`.as_tuple()` / `.to_dict()`);`gap` 內縮鋪排間距;跨平台且完全無頭可測;可與任何視窗移動後端組合。 + ## 本次更新 (2026-06-23) — 以邊緣 / 輪廓定位 UI 元素(免模板) 在從未見過的畫面上找出可點擊的方框。完整參考:[`docs/source/Zh/doc/new_features/v132_features_doc.rst`](../docs/source/Zh/doc/new_features/v132_features_doc.rst)。 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index c2b54731..2a903c00 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -1,5 +1,11 @@ # What's New — AutoControl +## What's new (2026-06-23) — Window Tiling / Layout Geometry Planner + +Compute where to place application windows — halves, grids, cascades. Full reference: [`docs/source/Eng/doc/new_features/v133_features_doc.rst`](docs/source/Eng/doc/new_features/v133_features_doc.rst). + +- **`tile_rect` / `grid_rects` / `cascade_rects`** (`AC_tile_rect`, `AC_grid_rects`, `AC_cascade_rects`): `save/restore_window_layout` replay *exact* saved positions and `snap_window` moves *one* window — nothing *computes* a fresh multi-window layout. This pure-geometry planner returns the target rectangles for halves, quadrants, thirds, an R×C grid and a staggered cascade given a screen work area, so a script can lay out windows deterministically. Returns `WindowRect` (`.as_tuple()` / `.to_dict()`); `gap` insets tiles; cross-platform and fully headless-testable; composes with any window-move backend. + ## What's new (2026-06-23) — Locate UI Elements by Edge / Contour (No Template) Find the clickable boxes on a screen you have never seen. Full reference: [`docs/source/Eng/doc/new_features/v132_features_doc.rst`](docs/source/Eng/doc/new_features/v132_features_doc.rst). diff --git a/docs/source/Eng/doc/new_features/v133_features_doc.rst b/docs/source/Eng/doc/new_features/v133_features_doc.rst new file mode 100644 index 00000000..2fd53624 --- /dev/null +++ b/docs/source/Eng/doc/new_features/v133_features_doc.rst @@ -0,0 +1,47 @@ +Window Tiling / Layout Geometry Planner +======================================= + +``save_window_layout`` / ``restore_window_layout`` capture and replay the *exact* +positions a user already arranged, and ``snap_window`` moves *one* window to a half +or quarter. Nothing *computes* a fresh multi-window layout. ``tile_rect``, +``grid_rects`` and ``cascade_rects`` are a pure-geometry planner: given a screen +work area they return the target rectangles for the common tiling layouts — halves, +quadrants, thirds, an R×C grid, a staggered cascade — so a script can lay out +application windows deterministically. + +The planner is cross-platform and has no device dependency, so it is fully +unit-testable; the rectangles it returns compose with any window-move backend. +Imports no ``PySide6``. + +Headless API +------------ + +.. code-block:: python + + from je_auto_control import tile_rect, grid_rects, cascade_rects + + left = tile_rect((0, 0, 1920, 1080), "left_third", gap=8) + print(left.as_tuple()) # (8, 8, 624, 1064) + + for cell in grid_rects((0, 0, 1920, 1080), rows=2, cols=3): + window_move("Editor", *cell.as_tuple()) # 6-up grid + + plan = cascade_rects((0, 0, 1920, 1080), count=4, offset=40) + +``tile_rect`` returns a ``WindowRect`` (``x, y, width, height`` with ``.as_tuple()`` +and ``.to_dict()``) for a named ``slot`` — see :func:`available_slots` +(``left``, ``top_right``, ``center``, ``left_third`` …); ``gap`` insets all sides +for a margin between tiles. ``grid_rects`` returns one rectangle per cell of an +``rows`` × ``cols`` grid, row-major. ``cascade_rects`` returns ``count`` staggered, +overlapping rectangles clamped to the screen (``size`` defaults to 60% of the work +area). Unknown slots / non-positive grid dimensions raise ``ValueError``. + +Executor commands +----------------- + +``AC_tile_rect`` (``slot`` / ``screen`` / ``gap`` → ``{rect}``), ``AC_grid_rects`` +(``rows`` / ``cols`` / ``screen`` / ``gap`` → ``{count, rects}``) and +``AC_cascade_rects`` (``count`` / ``screen`` / ``offset`` / ``size`` → +``{count, rects}``). ``screen`` defaults to the live primary screen work area. They +are exposed as the MCP tools ``ac_tile_rect`` / ``ac_grid_rects`` / +``ac_cascade_rects`` and as Script Builder commands under **Window**. diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst index 7206eab2..f5a50b18 100644 --- a/docs/source/Eng/eng_index.rst +++ b/docs/source/Eng/eng_index.rst @@ -155,6 +155,7 @@ Comprehensive guides for all AutoControl features. doc/new_features/v130_features_doc doc/new_features/v131_features_doc doc/new_features/v132_features_doc + doc/new_features/v133_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/docs/source/Zh/doc/new_features/v133_features_doc.rst b/docs/source/Zh/doc/new_features/v133_features_doc.rst new file mode 100644 index 00000000..ff3953aa --- /dev/null +++ b/docs/source/Zh/doc/new_features/v133_features_doc.rst @@ -0,0 +1,37 @@ +視窗鋪排 / 版面幾何規劃器 +========================== + +``save_window_layout`` / ``restore_window_layout``擷取並重播使用者已經排好的*精確*位置,``snap_window`` 把*一個* +視窗移到一半或四分之一。沒有任何功能能*計算*出全新的多視窗版面。``tile_rect``、``grid_rects`` 與 ``cascade_rects`` +是純幾何規劃器:給定螢幕工作區,回傳常見鋪排版面的目標矩形——左右半、四分之一、三分之一、R×C 網格、錯位層疊—— +讓腳本能以決定性方式排列應用程式視窗。 + +此規劃器跨平台且無裝置相依,因此完全可單元測試;它回傳的矩形可與任何視窗移動後端組合。不匯入 ``PySide6``。 + +無頭 API +-------- + +.. code-block:: python + + from je_auto_control import tile_rect, grid_rects, cascade_rects + + left = tile_rect((0, 0, 1920, 1080), "left_third", gap=8) + print(left.as_tuple()) # (8, 8, 624, 1064) + + for cell in grid_rects((0, 0, 1920, 1080), rows=2, cols=3): + window_move("Editor", *cell.as_tuple()) # 6 格網格 + + plan = cascade_rects((0, 0, 1920, 1080), count=4, offset=40) + +``tile_rect`` 為具名 ``slot`` 回傳 ``WindowRect``(``x, y, width, height``,含 ``.as_tuple()`` 與 ``.to_dict()``) +——見 :func:`available_slots`(``left``、``top_right``、``center``、``left_third`` …);``gap`` 內縮各邊作為鋪排間距。 +``grid_rects`` 為 ``rows`` × ``cols`` 網格的每格(列優先)回傳一個矩形。``cascade_rects`` 回傳 ``count`` 個錯位、 +重疊且被夾在螢幕內的矩形(``size`` 預設為工作區的 60%)。未知 slot / 非正網格維度會丟出 ``ValueError``。 + +執行器命令 +---------- + +``AC_tile_rect``(``slot`` / ``screen`` / ``gap`` → ``{rect}``)、``AC_grid_rects``(``rows`` / ``cols`` / ``screen`` +/ ``gap`` → ``{count, rects}``)與 ``AC_cascade_rects``(``count`` / ``screen`` / ``offset`` / ``size`` → +``{count, rects}``)。``screen`` 預設為實際主螢幕工作區。它們以 MCP 工具 ``ac_tile_rect`` / ``ac_grid_rects`` / +``ac_cascade_rects`` 以及 Script Builder 中 **Window** 分類下的命令提供。 diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst index ab847e27..84c6f336 100644 --- a/docs/source/Zh/zh_index.rst +++ b/docs/source/Zh/zh_index.rst @@ -155,6 +155,7 @@ AutoControl 所有功能的完整使用指南。 doc/new_features/v130_features_doc doc/new_features/v131_features_doc doc/new_features/v132_features_doc + doc/new_features/v133_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py index cabefeaa..ae699b11 100644 --- a/je_auto_control/__init__.py +++ b/je_auto_control/__init__.py @@ -294,6 +294,10 @@ from je_auto_control.utils.shape_locator import ( find_rectangles, find_shapes, ) +# Window tiling/layout geometry planner (halves, quadrants, grids, cascade) +from je_auto_control.utils.window_layout import ( + WindowRect, available_slots, cascade_rects, grid_rects, tile_rect, +) # CI workflow annotations (GitHub Actions) from je_auto_control.utils.ci_annotations import ( emit_annotations, format_annotation, @@ -1097,6 +1101,11 @@ def start_autocontrol_gui(*args, **kwargs): "FeatureMatch", "find_shapes", "find_rectangles", + "WindowRect", + "available_slots", + "tile_rect", + "grid_rects", + "cascade_rects", "emit_annotations", "format_annotation", "ClipboardHistory", "default_clipboard_history", "analyze_heal_log", "heal_stats", "scan_secrets", diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py index b2472eeb..6db53ab9 100644 --- a/je_auto_control/gui/script_builder/command_schema.py +++ b/je_auto_control/gui/script_builder/command_schema.py @@ -552,6 +552,43 @@ def _add_window_specs(specs: List[CommandSpec]) -> None: fields=(FieldSpec("layout", FieldType.FILE_PATH),), description="Move windows back to a saved layout file.", )) + specs.append(CommandSpec( + "AC_tile_rect", "Window", "Tile Rect (compute)", + fields=( + FieldSpec("slot", FieldType.ENUM, + choices=("full", "left", "right", "top", "bottom", + "top_left", "top_right", "bottom_left", + "bottom_right", "center", "left_third", + "center_third", "right_third"), default="left"), + FieldSpec("screen", FieldType.STRING, optional=True, + placeholder="[x, y, width, height]"), + FieldSpec("gap", FieldType.INT, optional=True, default=0), + ), + description="Compute the rectangle for a tiling slot of the screen.", + )) + specs.append(CommandSpec( + "AC_grid_rects", "Window", "Grid Rects (compute)", + fields=( + FieldSpec("rows", FieldType.INT, default=2), + FieldSpec("cols", FieldType.INT, default=2), + FieldSpec("screen", FieldType.STRING, optional=True, + placeholder="[x, y, width, height]"), + FieldSpec("gap", FieldType.INT, optional=True, default=0), + ), + description="Compute the cell rectangles of an R×C screen grid.", + )) + specs.append(CommandSpec( + "AC_cascade_rects", "Window", "Cascade Rects (compute)", + fields=( + FieldSpec("count", FieldType.INT, default=3), + FieldSpec("screen", FieldType.STRING, optional=True, + placeholder="[x, y, width, height]"), + FieldSpec("offset", FieldType.INT, optional=True, default=30), + FieldSpec("size", FieldType.STRING, optional=True, + placeholder="[width, height]"), + ), + description="Compute staggered, overlapping window rectangles.", + )) def _add_flow_specs(specs: List[CommandSpec]) -> None: diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index e541111d..c63be708 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -3345,6 +3345,45 @@ def _find_rectangles(region: Any = None, min_area: Any = 400, max_area: Any = No return {"count": len(rects), "rectangles": rects} +def _resolve_screen(screen: Any) -> list: + """Parse a JSON screen rect, or default to the live primary screen work area.""" + import json + if isinstance(screen, str): + screen = json.loads(screen) if screen.strip() else None + if screen: + return list(screen) + from je_auto_control.wrapper.auto_control_screen import screen_size + width, height = screen_size() + return [0, 0, int(width), int(height)] + + +def _tile_rect(slot: str, screen: Any = None, gap: Any = 0) -> Dict[str, Any]: + """Adapter: rectangle for a named tiling slot of the screen work area.""" + from je_auto_control.utils.window_layout import tile_rect + rect = tile_rect(_resolve_screen(screen), str(slot), gap=int(gap)) + return {"rect": rect.to_dict()} + + +def _grid_rects(rows: Any, cols: Any, screen: Any = None, + gap: Any = 0) -> Dict[str, Any]: + """Adapter: one rectangle per cell of an rows x cols grid over the screen.""" + from je_auto_control.utils.window_layout import grid_rects + rects = grid_rects(_resolve_screen(screen), int(rows), int(cols), gap=int(gap)) + return {"count": len(rects), "rects": [rect.to_dict() for rect in rects]} + + +def _cascade_rects(count: Any, screen: Any = None, offset: Any = 30, + size: Any = None) -> Dict[str, Any]: + """Adapter: count staggered, overlapping window rectangles (a cascade).""" + import json + from je_auto_control.utils.window_layout import cascade_rects + if isinstance(size, str): + size = json.loads(size) if size.strip() else None + rects = cascade_rects(_resolve_screen(screen), int(count), offset=int(offset), + size=tuple(size) if size else None) + return {"count": len(rects), "rects": [rect.to_dict() for rect in rects]} + + def _with_modifiers(modifiers: Any, actions: Any) -> Dict[str, Any]: """Adapter: run nested actions while modifier keys are held down.""" import json @@ -5071,6 +5110,9 @@ def __init__(self): "AC_feature_match": _feature_match, "AC_find_shapes": _find_shapes, "AC_find_rectangles": _find_rectangles, + "AC_tile_rect": _tile_rect, + "AC_grid_rects": _grid_rects, + "AC_cascade_rects": _cascade_rects, "AC_find_color_region": _find_color_region, "AC_detect_drift": _detect_drift, "AC_categorical_drift": _categorical_drift, diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index d935e2ad..8f97e62e 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -2709,6 +2709,57 @@ def shape_locator_tools() -> List[MCPTool]: ] +def window_layout_tools() -> List[MCPTool]: + return [ + MCPTool( + name="ac_tile_rect", + description=("Compute the rectangle for a named tiling 'slot' of the " + "screen work area: full/left/right/top/bottom/top_left/" + "top_right/bottom_left/bottom_right/center/left_third/" + "center_third/right_third. 'screen' [x,y,w,h] defaults to " + "the live primary screen; 'gap' insets all sides. Returns " + "{rect:{x,y,width,height}} — feed to a window-move."), + input_schema=schema({ + "slot": {"type": "string"}, + "screen": {"type": "array", "items": {"type": "integer"}}, + "gap": {"type": "integer"}}, + required=["slot"]), + handler=h.tile_rect, + annotations=READ_ONLY, + ), + MCPTool( + name="ac_grid_rects", + description=("Split the screen work area into an 'rows' x 'cols' grid; " + "return {count, rects:[{x,y,width,height}]} row-major. " + "'screen' [x,y,w,h] defaults to the live screen; 'gap' " + "insets each cell."), + input_schema=schema({ + "rows": {"type": "integer"}, + "cols": {"type": "integer"}, + "screen": {"type": "array", "items": {"type": "integer"}}, + "gap": {"type": "integer"}}, + required=["rows", "cols"]), + handler=h.grid_rects, + annotations=READ_ONLY, + ), + MCPTool( + name="ac_cascade_rects", + description=("Compute 'count' staggered, overlapping window rectangles " + "(a cascade), each 'offset' px down-right of the previous, " + "clamped to the screen. 'size' [w,h] defaults to 60% of " + "the work area. Returns {count, rects}."), + input_schema=schema({ + "count": {"type": "integer"}, + "screen": {"type": "array", "items": {"type": "integer"}}, + "offset": {"type": "integer"}, + "size": {"type": "array", "items": {"type": "integer"}}}, + required=["count"]), + handler=h.cascade_rects, + annotations=READ_ONLY, + ), + ] + + def ssim_tools() -> List[MCPTool]: return [ MCPTool( @@ -6211,7 +6262,7 @@ def media_assert_tools() -> List[MCPTool]: key_hold_tools, mouse_relative_tools, text_unicode_tools, modifier_state_tools, grid_locator_tools, visual_match_tools, color_region_tools, ssim_tools, feature_match_tools, shape_locator_tools, - plugin_sdk_tools, governance_tools, + window_layout_tools, plugin_sdk_tools, governance_tools, credential_lease_tools, egress_tools, approval_testing_tools, trajectory_eval_tools, compliance_tools, agent_trace_tools, video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools, diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index 875b3d5b..cc6e1748 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -2130,6 +2130,21 @@ def find_rectangles(region=None, min_area=400, max_area=None, aspect_range=None, return _find_rectangles(region, min_area, max_area, aspect_range, epsilon) +def tile_rect(slot, screen=None, gap=0): + from je_auto_control.utils.executor.action_executor import _tile_rect + return _tile_rect(slot, screen, gap) + + +def grid_rects(rows, cols, screen=None, gap=0): + from je_auto_control.utils.executor.action_executor import _grid_rects + return _grid_rects(rows, cols, screen, gap) + + +def cascade_rects(count, screen=None, offset=30, size=None): + from je_auto_control.utils.executor.action_executor import _cascade_rects + return _cascade_rects(count, screen, offset, size) + + def detect_drift(reference, current, threshold=0.25, bins=10): from je_auto_control.utils.executor.action_executor import _detect_drift return _detect_drift(reference, current, threshold, bins) diff --git a/je_auto_control/utils/window_layout/__init__.py b/je_auto_control/utils/window_layout/__init__.py new file mode 100644 index 00000000..68a71b22 --- /dev/null +++ b/je_auto_control/utils/window_layout/__init__.py @@ -0,0 +1,7 @@ +"""Window tiling/layout geometry planner (halves, quadrants, grids, cascade).""" +from je_auto_control.utils.window_layout.window_layout import ( + WindowRect, available_slots, cascade_rects, grid_rects, tile_rect, +) + +__all__ = ["WindowRect", "available_slots", "cascade_rects", "grid_rects", + "tile_rect"] diff --git a/je_auto_control/utils/window_layout/window_layout.py b/je_auto_control/utils/window_layout/window_layout.py new file mode 100644 index 00000000..bd538977 --- /dev/null +++ b/je_auto_control/utils/window_layout/window_layout.py @@ -0,0 +1,127 @@ +"""Window tiling/layout geometry planner — halves, quadrants, grids, cascade. + +``save_window_layout`` / ``restore_window_layout`` capture and replay the *exact* +positions a user already arranged; nothing *computes* new ones. This is a pure- +geometry planner: given a screen work area it returns the target rectangles for the +common tiling layouts (left/right/top/bottom halves, quadrants, thirds, an R×C grid, +a staggered cascade) so a script can arrange application windows deterministically. + +The result is plain geometry that composes with any window-move backend; the planner +itself is cross-platform and has no device dependency, so it is fully unit-testable. +Imports no ``PySide6``. +""" +from dataclasses import asdict, dataclass +from typing import Dict, List, Optional, Sequence, Tuple + +Screen = Sequence[int] +Size = Optional[Sequence[int]] + +# slot name -> (x_fraction, y_fraction, width_fraction, height_fraction) +_THIRD = 1.0 / 3.0 +_SLOTS: Dict[str, Tuple[float, float, float, float]] = { + "full": (0.0, 0.0, 1.0, 1.0), + "left": (0.0, 0.0, 0.5, 1.0), + "right": (0.5, 0.0, 0.5, 1.0), + "top": (0.0, 0.0, 1.0, 0.5), + "bottom": (0.0, 0.5, 1.0, 0.5), + "top_left": (0.0, 0.0, 0.5, 0.5), + "top_right": (0.5, 0.0, 0.5, 0.5), + "bottom_left": (0.0, 0.5, 0.5, 0.5), + "bottom_right": (0.5, 0.5, 0.5, 0.5), + "center": (0.25, 0.25, 0.5, 0.5), + "left_third": (0.0, 0.0, _THIRD, 1.0), + "center_third": (_THIRD, 0.0, _THIRD, 1.0), + "right_third": (2.0 * _THIRD, 0.0, _THIRD, 1.0), +} + + +@dataclass(frozen=True) +class WindowRect: + """A target window geometry: top-left ``(x, y)`` and ``width`` / ``height``.""" + + x: int + y: int + width: int + height: int + + def as_tuple(self) -> Tuple[int, int, int, int]: + """Return ``(x, y, width, height)`` — ready for a window-move call.""" + return (self.x, self.y, self.width, self.height) + + def to_dict(self) -> Dict[str, int]: + """Return the rectangle as a plain dict.""" + return asdict(self) + + +def available_slots() -> List[str]: + """Return the names of the supported single-window slots.""" + return list(_SLOTS) + + +def _apply_gap(x: int, y: int, width: int, height: int, + gap: int) -> Tuple[int, int, int, int]: + """Inset a rectangle by ``gap`` on every side (min size 1).""" + return (x + gap, y + gap, max(1, width - 2 * gap), max(1, height - 2 * gap)) + + +def tile_rect(screen: Screen, slot: str, *, gap: int = 0) -> WindowRect: + """Return the rectangle for a named ``slot`` of the ``screen`` work area. + + ``screen`` is ``(x, y, width, height)``. ``slot`` is one of + :func:`available_slots` (``left``, ``top_right``, ``center``, ``left_third`` …). + ``gap`` insets the result on all sides for a margin between tiled windows. + """ + if slot not in _SLOTS: + raise ValueError(f"unknown slot: {slot!r}") + screen_x, screen_y, screen_w, screen_h = (int(value) for value in screen[:4]) + fx, fy, fw, fh = _SLOTS[slot] + rect = _apply_gap(screen_x + round(fx * screen_w), + screen_y + round(fy * screen_h), + round(fw * screen_w), round(fh * screen_h), int(gap)) + return WindowRect(*rect) + + +def grid_rects(screen: Screen, rows: int, cols: int, *, + gap: int = 0) -> List[WindowRect]: + """Return one rectangle per cell of an ``rows`` × ``cols`` grid, row-major. + + Splits the ``screen`` work area into equal cells; ``gap`` insets each cell for + uniform spacing. Raises ``ValueError`` if ``rows`` or ``cols`` is below 1. + """ + rows, cols = int(rows), int(cols) + if rows < 1 or cols < 1: + raise ValueError("rows and cols must be >= 1") + screen_x, screen_y, screen_w, screen_h = (int(value) for value in screen[:4]) + cell_w, cell_h = screen_w / cols, screen_h / rows + rects: List[WindowRect] = [] + for row in range(rows): + for col in range(cols): + rect = _apply_gap(screen_x + round(col * cell_w), + screen_y + round(row * cell_h), + round(cell_w), round(cell_h), int(gap)) + rects.append(WindowRect(*rect)) + return rects + + +def cascade_rects(screen: Screen, count: int, *, offset: int = 30, + size: Size = None) -> List[WindowRect]: + """Return ``count`` staggered, overlapping rectangles (a window cascade). + + Each window is ``offset`` pixels down-right of the previous one, clamped to stay + within the ``screen`` work area; ``size`` defaults to 60% of the work area. + """ + count = int(count) + if count < 1: + raise ValueError("count must be >= 1") + screen_x, screen_y, screen_w, screen_h = (int(value) for value in screen[:4]) + if size is not None: + win_w, win_h = int(size[0]), int(size[1]) + else: + win_w, win_h = round(screen_w * 0.6), round(screen_h * 0.6) + max_x, max_y = screen_x + screen_w - win_w, screen_y + screen_h - win_h + rects: List[WindowRect] = [] + for index in range(count): + pos_x = min(screen_x + index * int(offset), max_x) + pos_y = min(screen_y + index * int(offset), max_y) + rects.append(WindowRect(int(pos_x), int(pos_y), int(win_w), int(win_h))) + return rects diff --git a/test/unit_test/headless/test_remote_desktop_chat_and_multicursor.py b/test/unit_test/headless/test_remote_desktop_chat_and_multicursor.py index b5aace94..717de3cc 100644 --- a/test/unit_test/headless/test_remote_desktop_chat_and_multicursor.py +++ b/test/unit_test/headless/test_remote_desktop_chat_and_multicursor.py @@ -22,6 +22,19 @@ def jpeg_bytes(): return _make_jpeg() +def _await_clients(host, expected: int = 1, timeout: float = 5.0) -> None: + """Wait until the host has authenticated ``expected`` viewers. + + ``viewer.connect()`` returns once the *client* side is up; the host's accept + thread may not have authenticated the viewer yet, so a broadcast issued + immediately can reach zero clients (reliably so on a slow CI container). This + closes that race using the host's own authenticated-client count. + """ + deadline = time.monotonic() + timeout + while host.connected_clients < expected and time.monotonic() < deadline: + time.sleep(0.02) + + # --- Phase 5.2: chat ---------------------------------------------------- def test_host_broadcasts_chat_to_viewer(jpeg_bytes): @@ -44,6 +57,7 @@ def on_chat(sender: str, text: str) -> None: ) viewer.connect(timeout=5.0) try: + _await_clients(host) sent = host.broadcast_chat("hello viewer") assert sent == 1 deadline = time.monotonic() + 2.0 @@ -75,6 +89,7 @@ def host_on_chat(sender: str, text: str) -> None: viewer = RemoteDesktopViewer(host="127.0.0.1", port=host.port, token="t") viewer.connect(timeout=5.0) try: + _await_clients(host) viewer.send_chat("ping from viewer", sender="alice") deadline = time.monotonic() + 2.0 while not received and time.monotonic() < deadline: @@ -124,6 +139,7 @@ def test_viewer_cursor_payload_routes_to_separate_callback(jpeg_bytes): ) viewer.connect(timeout=5.0) try: + _await_clients(host) # Simulate MultiViewerHost relaying another operator's cursor. host.broadcast_viewer_cursor("alice", 200, 300) deadline = time.monotonic() + 2.0 diff --git a/test/unit_test/headless/test_window_layout_batch.py b/test/unit_test/headless/test_window_layout_batch.py new file mode 100644 index 00000000..2207d675 --- /dev/null +++ b/test/unit_test/headless/test_window_layout_batch.py @@ -0,0 +1,104 @@ +"""Headless tests for the window tiling/layout geometry planner. No Qt.""" +import je_auto_control as ac +from je_auto_control.utils.window_layout import ( + WindowRect, available_slots, cascade_rects, grid_rects, tile_rect, +) + +_SCREEN = (0, 0, 1920, 1080) + + +def test_tile_halves_and_quadrants(): + assert tile_rect(_SCREEN, "left").as_tuple() == (0, 0, 960, 1080) + assert tile_rect(_SCREEN, "right").as_tuple() == (960, 0, 960, 1080) + assert tile_rect(_SCREEN, "top").as_tuple() == (0, 0, 1920, 540) + assert tile_rect(_SCREEN, "bottom_right").as_tuple() == (960, 540, 960, 540) + assert tile_rect(_SCREEN, "full").as_tuple() == (0, 0, 1920, 1080) + + +def test_thirds_split_the_width(): + left = tile_rect(_SCREEN, "left_third") + center = tile_rect(_SCREEN, "center_third") + right = tile_rect(_SCREEN, "right_third") + assert left.x == 0 and center.x == 640 and right.x == 1280 + assert left.width == center.width == right.width == 640 + + +def test_screen_offset_is_honoured(): + rect = tile_rect((100, 50, 800, 600), "left") + assert rect.as_tuple() == (100, 50, 400, 600) + + +def test_gap_insets_all_sides(): + rect = tile_rect(_SCREEN, "left", gap=10) + assert rect.as_tuple() == (10, 10, 940, 1060) + + +def test_unknown_slot_raises(): + try: + tile_rect(_SCREEN, "diagonal") + except ValueError: + pass + else: + raise AssertionError("expected ValueError for unknown slot") + + +def test_grid_rects_row_major_and_tiled(): + rects = grid_rects(_SCREEN, 2, 2) + assert [r.as_tuple() for r in rects] == [ + (0, 0, 960, 540), (960, 0, 960, 540), + (0, 540, 960, 540), (960, 540, 960, 540)] + + +def test_grid_rejects_zero_dimensions(): + for rows, cols in ((0, 2), (2, 0)): + try: + grid_rects(_SCREEN, rows, cols) + except ValueError: + continue + raise AssertionError("expected ValueError for non-positive grid") + + +def test_cascade_staggers_and_clamps(): + rects = cascade_rects(_SCREEN, 3, offset=40, size=(800, 600)) + assert rects[0].as_tuple() == (0, 0, 800, 600) + assert rects[1].as_tuple() == (40, 40, 800, 600) + assert all(r.x + r.width <= 1920 and r.y + r.height <= 1080 for r in rects) + + +def test_cascade_default_size_is_60_percent(): + rect = cascade_rects(_SCREEN, 1)[0] + assert rect.width == 1152 and rect.height == 648 # 0.6 * 1920 / 1080 + + +def test_available_slots_lists_known_names(): + slots = available_slots() + assert {"left", "right", "center", "left_third"} <= set(slots) + + +# --- wiring --------------------------------------------------------------- + +def test_wiring(): + known = set(ac.executor.known_commands()) + assert {"AC_tile_rect", "AC_grid_rects", "AC_cascade_rects"} <= known + from je_auto_control.utils.mcp_server.tools import build_default_tool_registry + names = {t.name for t in build_default_tool_registry()} + assert {"ac_tile_rect", "ac_grid_rects", "ac_cascade_rects"} <= names + from je_auto_control.gui.script_builder.command_schema import _build_specs + specs = {s.command for s in _build_specs()} + assert {"AC_tile_rect", "AC_grid_rects", "AC_cascade_rects"} <= specs + + +def test_executor_adapters_return_plans(): + from je_auto_control.utils.executor.action_executor import ( + _cascade_rects, _grid_rects, _tile_rect) + assert _tile_rect("left", screen=[0, 0, 100, 80])["rect"] == { + "x": 0, "y": 0, "width": 50, "height": 80} + assert _grid_rects(2, 2, screen=[0, 0, 100, 80])["count"] == 4 + assert _cascade_rects(3, screen=[0, 0, 100, 80])["count"] == 3 + + +def test_facade_exports(): + for attr in ("tile_rect", "grid_rects", "cascade_rects", "available_slots", + "WindowRect"): + assert hasattr(ac, attr) and attr in ac.__all__ + assert isinstance(tile_rect(_SCREEN, "left"), WindowRect)