Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README/WHATS_NEW_zh-CN.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# 本次更新 — AutoControl

## 本次更新 (2026-06-23) — 排列多个窗口(网格 / 层叠)

一次调用排好一整组窗口。完整参考:[`docs/source/Zh/doc/new_features/v134_features_doc.rst`](../docs/source/Zh/doc/new_features/v134_features_doc.rst)。

- **`arrange_grid` / `arrange_cascade`**(`AC_arrange_grid`、`AC_arrange_cascade`):`snap_window` 移动*一个*窗口、版面规划器只*计算*矩形——这两个把循环补完,接受一组窗口标题并实际把每个匹配的窗口移入网格(自动近正方形,或明确 `rows`/`cols` + `gap`)或对角线层叠。以版面规划器为基础并沿用 `snap_window` 的可注入 `mover`/`screen_size` 接缝,因此完全无头可测;返回移动的窗口数。

## 本次更新 (2026-06-23) — 窗口铺排 / 版面几何规划器

计算应用程序窗口该放在哪里——半边、网格、层叠。完整参考:[`docs/source/Zh/doc/new_features/v133_features_doc.rst`](../docs/source/Zh/doc/new_features/v133_features_doc.rst)。
Expand Down
6 changes: 6 additions & 0 deletions README/WHATS_NEW_zh-TW.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# 本次更新 — AutoControl

## 本次更新 (2026-06-23) — 排列多個視窗(網格 / 層疊)

一次呼叫排好一整組視窗。完整參考:[`docs/source/Zh/doc/new_features/v134_features_doc.rst`](../docs/source/Zh/doc/new_features/v134_features_doc.rst)。

- **`arrange_grid` / `arrange_cascade`**(`AC_arrange_grid`、`AC_arrange_cascade`):`snap_window` 移動*一個*視窗、版面規劃器只*計算*矩形——這兩個把迴圈補完,接受一組視窗標題並實際把每個符合的視窗移入網格(自動近正方形,或明確 `rows`/`cols` + `gap`)或對角線層疊。以版面規劃器為基礎並沿用 `snap_window` 的可注入 `mover`/`screen_size` 接縫,因此完全無頭可測;回傳移動的視窗數。

## 本次更新 (2026-06-23) — 視窗鋪排 / 版面幾何規劃器

計算應用程式視窗該放在哪裡——半邊、網格、層疊。完整參考:[`docs/source/Zh/doc/new_features/v133_features_doc.rst`](../docs/source/Zh/doc/new_features/v133_features_doc.rst)。
Expand Down
6 changes: 6 additions & 0 deletions WHATS_NEW.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# What's New — AutoControl

## What's new (2026-06-23) — Arrange Multiple Windows (Grid / Cascade)

Lay out a whole set of windows in one call. Full reference: [`docs/source/Eng/doc/new_features/v134_features_doc.rst`](docs/source/Eng/doc/new_features/v134_features_doc.rst).

- **`arrange_grid` / `arrange_cascade`** (`AC_arrange_grid`, `AC_arrange_cascade`): `snap_window` moves *one* window and the layout planner only *computes* rectangles — these close the loop, taking a list of window titles and actually moving every match into a grid (auto near-square shape, or explicit `rows`/`cols` + `gap`) or a diagonal cascade. Build on the layout planner and reuse `snap_window`'s injectable `mover`/`screen_size` seams, so they are fully headless-testable; return the count moved.

## What's new (2026-06-23) — Window Tiling / Layout Geometry Planner

Compute where to place application windows — halves, grids, cascades. Full reference: [`docs/source/Eng/doc/new_features/v133_features_doc.rst`](docs/source/Eng/doc/new_features/v133_features_doc.rst).
Expand Down
44 changes: 44 additions & 0 deletions docs/source/Eng/doc/new_features/v134_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
Arrange Multiple Windows (Grid / Cascade)
=========================================

``snap_window`` moves *one* window to a half or quarter, and the
:doc:`v133_features_doc` planner *computes* rectangles but does not move anything.
``arrange_grid`` and ``arrange_cascade`` close the loop: given a list of window
titles they compute a layout and actually move every matching window — tile a set
of app windows into a grid, or fan them out in a diagonal cascade, in one call.

They build on the layout planner for the geometry and reuse the same injectable
``mover`` / ``screen_size`` seams as ``snap_window``, so the arrangement logic is
fully unit-testable without real windows. The default mover is Win32 today (other
platforms are a no-op until their backend lands). Imports no ``PySide6``.

Headless API
------------

.. code-block:: python

from je_auto_control import arrange_grid, arrange_cascade

# Tile three editors into an auto-shaped grid (here 2x2, first 3 cells).
arrange_grid(["Editor", "Browser", "Terminal"])

# Or an explicit 1x3 row with an 8px gutter.
arrange_grid(["Left", "Mid", "Right"], rows=1, cols=3, gap=8)

# Fan windows out diagonally.
arrange_cascade(["Doc 1", "Doc 2", "Doc 3"], offset=40)

``arrange_grid`` tiles the ``titles`` into an ``rows`` × ``cols`` grid (defaulting
to a near-square auto-shape for the window count) with an optional ``gap``;
``arrange_cascade`` staggers each window ``offset`` pixels down-right of the
previous, sized to 60% of the work area. Both return the number of windows
actually moved and leave any windows beyond the grid capacity untouched.

Executor commands
-----------------

``AC_arrange_grid`` (``titles`` JSON array + ``rows`` / ``cols`` / ``gap``) and
``AC_arrange_cascade`` (``titles`` + ``offset``), each returning
``{moved, count}``. They are exposed as the MCP tools ``ac_arrange_grid`` /
``ac_arrange_cascade`` (side-effecting) and as Script Builder commands under
**Window**.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ Comprehensive guides for all AutoControl features.
doc/new_features/v131_features_doc
doc/new_features/v132_features_doc
doc/new_features/v133_features_doc
doc/new_features/v134_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
36 changes: 36 additions & 0 deletions docs/source/Zh/doc/new_features/v134_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
排列多個視窗(網格 / 層疊)
============================

``snap_window`` 移動*一個*視窗到一半或四分之一,而 :doc:`v133_features_doc` 規劃器只*計算*矩形、並不移動任何東西。
``arrange_grid`` 與 ``arrange_cascade`` 把這個迴圈補完:給定一組視窗標題,計算版面並實際移動每個符合的視窗——
把一組應用程式視窗鋪成網格,或以對角線層疊散開,一次呼叫完成。

它們以版面規劃器取得幾何,並沿用與 ``snap_window`` 相同的可注入 ``mover`` / ``screen_size`` 接縫,因此排列邏輯
完全可在無真實視窗下做單元測試。預設 mover 目前為 Win32(其他平台在其後端完成前為 no-op)。不匯入 ``PySide6``。

無頭 API
--------

.. code-block:: python

from je_auto_control import arrange_grid, arrange_cascade

# 把三個編輯器鋪成自動形狀的網格(此處 2x2,使用前 3 格)。
arrange_grid(["Editor", "Browser", "Terminal"])

# 或明確的 1x3 列,含 8px 間距。
arrange_grid(["Left", "Mid", "Right"], rows=1, cols=3, gap=8)

# 將視窗以對角線散開。
arrange_cascade(["Doc 1", "Doc 2", "Doc 3"], offset=40)

``arrange_grid`` 把 ``titles`` 鋪成 ``rows`` × ``cols`` 網格(預設為依視窗數量的近正方自動形狀),可加 ``gap``;
``arrange_cascade`` 讓每個視窗在前一個的右下方錯位 ``offset`` 像素,尺寸為工作區的 60%。兩者都回傳實際移動的
視窗數,並對超出網格容量的視窗保持不動。

執行器命令
----------

``AC_arrange_grid``(``titles`` JSON 陣列 + ``rows`` / ``cols`` / ``gap``)與 ``AC_arrange_cascade``
(``titles`` + ``offset``),各回傳 ``{moved, count}``。它們以 MCP 工具 ``ac_arrange_grid`` / ``ac_arrange_cascade``
(有副作用)以及 Script Builder 中 **Window** 分類下的命令提供。
1 change: 1 addition & 0 deletions docs/source/Zh/zh_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ AutoControl 所有功能的完整使用指南。
doc/new_features/v131_features_doc
doc/new_features/v132_features_doc
doc/new_features/v133_features_doc
doc/new_features/v134_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
5 changes: 3 additions & 2 deletions je_auto_control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@
from je_auto_control.utils.color_stats import ColorStats, region_color_stats
# Per-window capture, window-layout save / restore, snap/tile.
from je_auto_control.utils.window_capture import (
capture_window, get_window_geometry, restore_window_layout,
save_window_layout, snap_window,
arrange_cascade, arrange_grid, capture_window, get_window_geometry,
restore_window_layout, save_window_layout, snap_window,
)
# Scroll until a target image / text is visible.
from je_auto_control.utils.scroll_find import scroll_until_visible
Expand Down Expand Up @@ -1365,6 +1365,7 @@ def start_autocontrol_gui(*args, **kwargs):
# Per-window capture + window-layout save / restore + snap
"capture_window", "get_window_geometry",
"save_window_layout", "restore_window_layout", "snap_window",
"arrange_grid", "arrange_cascade",
# Scroll-to-find
"scroll_until_visible",
# Recoverable deletion (recycle bin)
Expand Down
20 changes: 20 additions & 0 deletions je_auto_control/gui/script_builder/command_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -481,6 +481,26 @@ def _add_window_specs(specs: List[CommandSpec]) -> None:
),
description="Move a window to a screen half / quarter / maximize.",
))
specs.append(CommandSpec(
"AC_arrange_grid", "Window", "Arrange Windows in Grid",
fields=(
FieldSpec("titles", FieldType.STRING,
placeholder='["Editor", "Browser", "Terminal"]'),
FieldSpec("rows", FieldType.INT, optional=True),
FieldSpec("cols", FieldType.INT, optional=True),
FieldSpec("gap", FieldType.INT, optional=True, default=0),
),
description="Tile a list of windows into a grid (auto-shape if unset).",
))
specs.append(CommandSpec(
"AC_arrange_cascade", "Window", "Arrange Windows in Cascade",
fields=(
FieldSpec("titles", FieldType.STRING,
placeholder='["Editor", "Browser", "Terminal"]'),
FieldSpec("offset", FieldType.INT, optional=True, default=30),
),
description="Cascade a list of windows diagonally.",
))
specs.append(CommandSpec(
"AC_wait_window_closed", "Window", "Wait for Window to Close",
fields=(
Expand Down
27 changes: 27 additions & 0 deletions je_auto_control/utils/executor/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -2178,6 +2178,31 @@ def _snap_window(title: str, position: str = "left") -> Dict[str, Any]:
return {"moved": snap_window(title, position)}


def _arrange_grid(titles: Any, rows: Any = None, cols: Any = None,
gap: Any = 0) -> Dict[str, Any]:
"""Executor adapter: tile a list of window titles into a grid."""
import json
from je_auto_control.utils.window_capture import arrange_grid
if isinstance(titles, str):
titles = json.loads(titles)
moved = arrange_grid(list(titles),
rows=int(rows) if rows is not None else None,
cols=int(cols) if cols is not None else None,
gap=int(gap))
return {"moved": moved, "count": len(list(titles))}


def _arrange_cascade(titles: Any, offset: Any = 30) -> Dict[str, Any]:
"""Executor adapter: cascade a list of window titles diagonally."""
import json
from je_auto_control.utils.window_capture import arrange_cascade
if isinstance(titles, str):
titles = json.loads(titles)
titles = list(titles)
return {"moved": arrange_cascade(titles, offset=int(offset)),
"count": len(titles)}


def _save_window_layout(path: Optional[str] = None) -> Dict[str, Any]:
"""Executor adapter: snapshot every window's geometry (optionally to file)."""
from je_auto_control.utils.window_capture import save_window_layout
Expand Down Expand Up @@ -5480,6 +5505,8 @@ def __init__(self):
"AC_save_window_layout": _save_window_layout,
"AC_restore_window_layout": _restore_window_layout,
"AC_snap_window": _snap_window,
"AC_arrange_grid": _arrange_grid,
"AC_arrange_cascade": _arrange_cascade,
}

def known_commands(self) -> set:
Expand Down
33 changes: 32 additions & 1 deletion je_auto_control/utils/mcp_server/tools/_factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -2760,6 +2760,37 @@ def window_layout_tools() -> List[MCPTool]:
]


def window_arrange_tools() -> List[MCPTool]:
return [
MCPTool(
name="ac_arrange_grid",
description=("Tile the given window 'titles' into a grid and MOVE "
"them. 'rows'/'cols' default to a near-square auto-shape; "
"'gap' spaces cells. Returns {moved, count}."),
input_schema=schema({
"titles": {"type": "array", "items": {"type": "string"}},
"rows": {"type": "integer"},
"cols": {"type": "integer"},
"gap": {"type": "integer"}},
required=["titles"]),
handler=h.arrange_grid,
annotations=SIDE_EFFECT_ONLY,
),
MCPTool(
name="ac_arrange_cascade",
description=("Cascade the given window 'titles' diagonally and MOVE "
"them, each 'offset' px down-right of the previous. "
"Returns {moved, count}."),
input_schema=schema({
"titles": {"type": "array", "items": {"type": "string"}},
"offset": {"type": "integer"}},
required=["titles"]),
handler=h.arrange_cascade,
annotations=SIDE_EFFECT_ONLY,
),
]


def ssim_tools() -> List[MCPTool]:
return [
MCPTool(
Expand Down Expand Up @@ -6262,7 +6293,7 @@ def media_assert_tools() -> List[MCPTool]:
key_hold_tools, mouse_relative_tools, text_unicode_tools,
modifier_state_tools, grid_locator_tools, visual_match_tools,
color_region_tools, ssim_tools, feature_match_tools, shape_locator_tools,
window_layout_tools, plugin_sdk_tools, governance_tools,
window_layout_tools, window_arrange_tools, plugin_sdk_tools, governance_tools,
credential_lease_tools, egress_tools, approval_testing_tools,
trajectory_eval_tools, compliance_tools, agent_trace_tools,
video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools,
Expand Down
10 changes: 10 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2145,6 +2145,16 @@ def cascade_rects(count, screen=None, offset=30, size=None):
return _cascade_rects(count, screen, offset, size)


def arrange_grid(titles, rows=None, cols=None, gap=0):
from je_auto_control.utils.executor.action_executor import _arrange_grid
return _arrange_grid(titles, rows, cols, gap)


def arrange_cascade(titles, offset=30):
from je_auto_control.utils.executor.action_executor import _arrange_cascade
return _arrange_cascade(titles, offset)


def detect_drift(reference, current, threshold=0.25, bins=10):
from je_auto_control.utils.executor.action_executor import _detect_drift
return _detect_drift(reference, current, threshold, bins)
Expand Down
8 changes: 5 additions & 3 deletions je_auto_control/utils/window_capture/__init__.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
"""Per-window capture, window-layout save / restore, and snap/tile."""
"""Per-window capture, window-layout save / restore, snap/tile, and arrange."""
from je_auto_control.utils.window_capture.window_capture import (
capture_window, get_window_geometry, restore_window_layout,
save_window_layout, snap_window,
arrange_cascade, arrange_grid, capture_window, get_window_geometry,
restore_window_layout, save_window_layout, snap_window,
)

__all__ = [
"arrange_cascade",
"arrange_grid",
"capture_window",
"get_window_geometry",
"restore_window_layout",
Expand Down
62 changes: 62 additions & 0 deletions je_auto_control/utils/window_capture/window_capture.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
logic is fully unit-testable without real windows. GUI-free.
"""
import json
import math
import sys
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Tuple, Union
Expand Down Expand Up @@ -168,3 +169,64 @@ def snap_window(title: str, position: str = "left", *,
width, height = (screen_size or _default_screen_size)()
x, y, w, h = _snap_rect(position, int(width), int(height))
return (mover or _default_mover)(title, x, y, w, h)


def _move_into(titles: List[str], rects, move: WindowMover) -> int:
"""Move each title to the matching rectangle; return the number moved."""
moved = 0
for title, rect in zip(titles, rects):
if move(title, rect.x, rect.y, rect.width, rect.height):
moved += 1
return moved


def _grid_shape(count: int, rows: Optional[int],
cols: Optional[int]) -> Tuple[int, int]:
"""Resolve the (rows, cols) grid shape, auto-sizing to near-square when unset."""
if rows and cols:
return int(rows), int(cols)
if cols:
return math.ceil(count / int(cols)), int(cols)
if rows:
return int(rows), math.ceil(count / int(rows))
side = math.ceil(math.sqrt(count))
return math.ceil(count / side), side


def arrange_grid(titles: List[str], *, rows: Optional[int] = None,
cols: Optional[int] = None, gap: int = 0,
mover: Optional[WindowMover] = None,
screen_size: Optional[SizeProvider] = None) -> int:
"""Tile the given window ``titles`` into a grid; return the count moved.

``rows`` / ``cols`` default to a near-square auto-shape for the number of
windows; ``gap`` spaces the cells. The mover and size provider are injectable
for tests. Windows beyond the grid capacity are left untouched.
"""
from je_auto_control.utils.window_layout import grid_rects
titles = list(titles)
if not titles:
return 0
width, height = (screen_size or _default_screen_size)()
grid_rows, grid_cols = _grid_shape(len(titles), rows, cols)
rects = grid_rects((0, 0, int(width), int(height)), grid_rows, grid_cols,
gap=int(gap))
return _move_into(titles, rects, mover or _default_mover)


def arrange_cascade(titles: List[str], *, offset: int = 30,
mover: Optional[WindowMover] = None,
screen_size: Optional[SizeProvider] = None) -> int:
"""Cascade the given window ``titles`` diagonally; return the count moved.

Each window is ``offset`` pixels down-right of the previous, sized to 60% of
the work area and clamped on-screen. The mover and size provider are injectable.
"""
from je_auto_control.utils.window_layout import cascade_rects
titles = list(titles)
if not titles:
return 0
width, height = (screen_size or _default_screen_size)()
rects = cascade_rects((0, 0, int(width), int(height)), len(titles),
offset=int(offset))
return _move_into(titles, rects, mover or _default_mover)
Loading
Loading