Merge pull request #446 from Integration-Automation/dev

Release: perception lane (CVD, mark layout, contrast, theme-invariant match) v214-v217

Author: JE-Chen

WHATS_NEW.md

@@ -2,6 +2,30 @@
 
 ## What's new (2026-06-26)
 
+### Theme-Invariant Matching (Light Template, Dark Mode)
+
+Find a button captured in light mode even after the app switches to dark mode. Full reference: [`docs/source/Eng/doc/new_features/v217_features_doc.rst`](docs/source/Eng/doc/new_features/v217_features_doc.rst).
+
+- **`normalize_theme` / `match_theme`** (`AC_match_theme`): `match_template` correlates raw pixel intensities, so a light-mode template scores terribly against the same control in dark mode — the polarity is inverted. The fix is to compare *structure*. `normalize_theme` maps an image to a polarity-invariant single channel (`sobel`/`laplacian` gradient magnitude — identical for an image and its colour inverse — or `zscore`); `match_theme` normalizes both the template and the screen, then locates the template via `visual_match.match_template`, finding it across a light/dark flip that defeats raw matching. cv2/numpy are imported lazily so the module stays importable everywhere. Fourth feature of the ROUND-15 perception lane. No `PySide6`.
+
+### Sample a Region's Text Contrast (WCAG)
+
+Grade the legibility of on-screen text when you only have a region, not the two colours. Full reference: [`docs/source/Eng/doc/new_features/v216_features_doc.rst`](docs/source/Eng/doc/new_features/v216_features_doc.rst).
+
+- **`grade_contrast` / `dominant_pair` / `region_contrast`** (`AC_grade_contrast`, `AC_dominant_pair`, `AC_region_contrast`): `a11y_audit.contrast_ratio` grades a foreground/background pair you already know — but a button or label on screen is a *patch of pixels*, not two known colours. `dominant_pair` splits sampled pixels at the mean luminance into the dominant foreground (minority, the text) and background (majority); `grade_contrast` grades a pair against the WCAG 2.x AA/AAA thresholds (normal + large text); `region_contrast` samples a screen region (through an injectable `sampler`) and grades it. The grading and split are pure and reuse `a11y_audit.contrast_ratio`, fully testable without a screen. Third feature of the ROUND-15 perception lane. No `PySide6`.
+
+### Set-of-Marks Label Layout (No Overlap, Readable Colour)
+
+Number every element without the labels piling up or vanishing into the background. Full reference: [`docs/source/Eng/doc/new_features/v215_features_doc.rst`](docs/source/Eng/doc/new_features/v215_features_doc.rst).
+
+- **`place_labels` / `label_color`** (`AC_place_labels`, `AC_label_color`): Set-of-Marks draws each numbered label at a fixed offset, so on dense UIs the numbers pile on top of each other and a dark label on a dark element vanishes. `place_labels` is greedy non-overlap placement — for each mark it tries a ring of candidate positions around its box (above/below/inside, left/right aligned) and takes the first that stays in bounds and clears every already-placed label; `label_color` picks black or white by whichever has the better WCAG contrast against the element background (reusing `a11y_audit.contrast_ratio`). Pure standard library, deterministic, fully testable without rendering. Second feature of the ROUND-15 perception lane. No `PySide6`.
+
+### Colour-Vision-Deficiency Simulation + Collision Check
+
+Check whether your red/green status colours are distinguishable to colour-blind users. Full reference: [`docs/source/Eng/doc/new_features/v214_features_doc.rst`](docs/source/Eng/doc/new_features/v214_features_doc.rst).
+
+- **`simulate_cvd` / `colors_collide` / `color_distance`** (`AC_simulate_cvd`, `AC_colors_collide`): status UIs lean on colour (green "ok" vs red "error"), but for the ~8% of men with a colour-vision deficiency those can be indistinguishable — and nothing in the framework could check it. `simulate_cvd` maps an RGB colour through a dichromat simulation matrix (protanopia/deuteranopia/tritanopia) at a given `severity`; `colors_collide` simulates two colours and reports whether they become confusable (a perceptual `redmean` distance below `threshold`); `color_distance` is the underlying metric. Pure standard library — no numpy/OpenCV, operating on plain RGB tuples, fully testable. First feature of the ROUND-15 perception lane. No `PySide6`.
+
 ### Wait Until the App Is Idle
 
 Hold off the next click until the busy/wait cursor settles — don't act mid-churn. Full reference: [`docs/source/Eng/doc/new_features/v213_features_doc.rst`](docs/source/Eng/doc/new_features/v213_features_doc.rst).

docs/source/Eng/doc/new_features/v214_features_doc.rst

@@ -0,0 +1,48 @@
+Colour-Vision-Deficiency Simulation + Collision Check
+=====================================================
+
+Status UIs lean on colour — a green "ok" vs a red "error" dot, a colour-coded
+chart legend. For the ~8% of men with a colour-vision deficiency (CVD) those can
+be indistinguishable, and nothing in the framework could check it.
+``cvd_simulate`` adds the two primitives an accessibility / design check needs.
+
+* :func:`simulate_cvd` — map an ``(r, g, b)`` colour through a dichromat
+  simulation matrix (``protanopia`` / ``deuteranopia`` / ``tritanopia``) at a
+  given ``severity`` (0 = unaffected, 1 = full dichromacy).
+* :func:`colors_collide` — simulate two colours under a CVD type and report
+  whether they become too similar to tell apart (a perceptual ``redmean``
+  distance below ``threshold``).
+* :func:`color_distance` — the underlying ``redmean`` colour-difference metric.
+
+Pure standard library — no numpy / OpenCV — operating on plain RGB tuples, so it
+is fully testable. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import simulate_cvd, colors_collide
+
+    # How does the "error red" look to a deuteranope?
+    simulate_cvd((220, 40, 40), "deuteranopia")        # -> (r, g, b)
+
+    # Are my ok-green and error-red distinguishable for them?
+    report = colors_collide((60, 200, 60), (220, 60, 60), kind="deuteranopia")
+    report["collide"]    # True if the two are confusable
+    report["distance"]   # the perceptual distance after simulation
+
+``simulate_cvd`` accepts friendly aliases (``protan`` / ``deutan`` / ``tritan``,
+or ``red`` / ``green`` / ``blue``). ``severity`` interpolates between the
+original colour and the full dichromat simulation, for the milder anomalous
+trichromacies. ``colors_collide`` returns ``{collide, distance, kind, severity,
+simulated_left, simulated_right}``.
+
+Executor commands
+-----------------
+
+``AC_simulate_cvd`` (``rgb`` ``[r, g, b]`` + ``kind`` / ``severity`` →
+``{rgb}``) and ``AC_colors_collide`` (``left`` / ``right`` ``[r, g, b]`` +
+``kind`` / ``severity`` / ``threshold`` → the report). RGB inputs accept a JSON
+list. They are the matching read-only ``ac_*`` MCP tools and Script Builder
+commands under **Image**.

docs/source/Eng/doc/new_features/v215_features_doc.rst

@@ -0,0 +1,43 @@
+Set-of-Marks Label Layout (No Overlap, Readable Colour)
+=======================================================
+
+Set-of-Marks overlays a numbered label on every element so a vision model can
+say "click 7". ``set_of_marks`` draws each label at a fixed offset, so on dense
+UIs the numbers pile on top of each other (unreadable) and a dark label on a
+dark element vanishes. ``marks_layout`` fixes both with pure geometry.
+
+* :func:`place_labels` — greedy non-overlap placement: for each mark, try a ring
+  of candidate positions around its box (above, below, inside; left/right
+  aligned) and take the first that stays in bounds and clears every
+  already-placed label.
+* :func:`label_color` — pick the label text colour (black or white) with the
+  better WCAG contrast against the element's background.
+
+Pure standard library; reuses :func:`a11y_audit.contrast_ratio`. Fully testable
+without rendering. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import mark_elements, place_labels, label_color
+
+    marks = mark_elements(elements)            # [{id, bbox, ...}]
+    layout = place_labels(marks, bounds=(1920, 1080))
+    # [{'id': 1, 'label': [x, y, 22, 16], 'anchor': [bx, by]}, ...]
+
+    label_color((30, 30, 30))     # {'rgb': [255, 255, 255], 'contrast': ...}
+
+Feed the ``label`` boxes from :func:`place_labels` to your renderer instead of a
+naive fixed offset, and pick each number's colour with :func:`label_color` so it
+stays legible on its background. ``place_labels`` is deterministic and ordered by
+the input marks, so the same screen always numbers the same way.
+
+Executor commands
+-----------------
+
+``AC_place_labels`` (``marks`` JSON list + ``label_width`` / ``label_height`` /
+``bounds`` ``[w, h]`` → ``{labels}``) and ``AC_label_color`` (``background``
+``[r, g, b]`` → ``{rgb, contrast}``). They are the matching read-only ``ac_*``
+MCP tools and Script Builder commands under **Image**.

docs/source/Eng/doc/new_features/v216_features_doc.rst

@@ -0,0 +1,50 @@
+Sample a Region's Text Contrast (WCAG)
+======================================
+
+:func:`a11y_audit.contrast_ratio` grades a foreground / background pair you
+already know. But when you only have a *region* of the screen — a button, a
+label — you don't know those two colours; you have a patch of pixels.
+``contrast_map`` closes that gap: split a sampled region into its dominant
+foreground (the minority — usually the text) and background (the majority)
+colours, then grade their WCAG contrast.
+
+* :func:`grade_contrast` — pure: a foreground / background pair to
+  ``{ratio, aa, aaa, aa_large, aaa_large}`` against the WCAG 2.x thresholds.
+* :func:`dominant_pair` — pure: split a list of sampled RGB pixels into the
+  dominant ``{foreground, background}`` colours by luminance.
+* :func:`region_contrast` — sample a screen region and grade it, through an
+  injectable ``sampler`` (the real screen grab by default).
+
+The grading and split are pure and reuse :func:`a11y_audit.contrast_ratio`, so
+they are fully testable without a screen. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import grade_contrast, dominant_pair, region_contrast
+
+    # If you already know the colours:
+    grade_contrast((90, 90, 90), (255, 255, 255))
+    # {'ratio': 3.9, 'aa': False, 'aaa': False, 'aa_large': True, ...}
+
+    # If you only have a region of the screen, sample and grade it:
+    report = region_contrast(region=[x, y, w, h])
+    if not report["aa"]:
+        print("low-contrast text", report["foreground"], report["background"])
+
+``dominant_pair`` partitions the sampled pixels at the mean luminance and treats
+the larger group as the background and the smaller as the text — a uniform patch
+yields the same colour for both (no contrast). ``region_contrast`` accepts an
+injectable ``sampler`` (``region -> list of RGB pixels``) so the logic is tested
+without a real screen.
+
+Executor commands
+-----------------
+
+``AC_grade_contrast`` (``foreground`` / ``background`` ``[r, g, b]`` → the
+grade), ``AC_dominant_pair`` (``pixels`` JSON list of ``[r, g, b]`` →
+``{foreground, background}``) and ``AC_region_contrast`` (``region``
+``[x, y, w, h]`` → the grade + colours + ``samples``). They are the matching
+read-only ``ac_*`` MCP tools and Script Builder commands under **Image**.

docs/source/Eng/doc/new_features/v217_features_doc.rst

@@ -0,0 +1,49 @@
+Theme-Invariant Matching (Light Template, Dark Mode)
+====================================================
+
+``match_template`` correlates raw pixel intensities, so a template captured in
+light mode scores terribly against the same control in dark mode — the polarity
+is inverted. The fix is to compare *structure* (edges, gradients), which is the
+same regardless of which way the colours run. ``theme_normalize`` turns an image
+into a polarity-invariant representation before matching.
+
+* :func:`normalize_theme` — map an image to a normalised single-channel image.
+  ``sobel`` (default) and ``laplacian`` use gradient magnitude, which is
+  identical for an image and its colour-inverse; ``zscore`` standardises
+  intensity.
+* :func:`match_theme` — :func:`normalize_theme` both the template and the
+  haystack (the screen by default), then locate the template — finding it across
+  a light/dark theme flip that defeats raw matching.
+
+``cv2`` / ``numpy`` are imported lazily, so importing the module never requires
+them, and the locating logic reuses :func:`visual_match.match_template`. Imports
+no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import match_theme, normalize_theme
+
+    # A button template grabbed in light mode, found in the dark-mode app:
+    hit = match_theme("save_button_light.png", method="sobel", min_score=0.4)
+    if hit and hit["score"] >= 0.5:
+        click(hit["x"] + hit["width"] // 2, hit["y"] + hit["height"] // 2)
+
+    # The transform itself (e.g. to feed your own matcher):
+    edges = normalize_theme("template.png", method="sobel")
+
+Because gradient magnitude is identical for an image and its inverse,
+``normalize_theme(img, "sobel")`` equals ``normalize_theme(255 - img, "sobel")``
+— that invariance is exactly what lets one template match both themes. Use
+``min_score`` lower than for raw matching (structure correlation runs cooler).
+
+Executor commands
+-----------------
+
+``AC_match_theme`` (``template`` + ``region`` ``[x, y, w, h]`` / ``method`` /
+``min_score`` → ``{found, x, y, width, height, score}``) locates a template
+across a theme flip. It is the matching read-only ``ac_match_theme`` MCP tool and
+a Script Builder command under **Image**. :func:`normalize_theme` (which returns
+an image array) is the Python-API surface.

docs/source/Zh/doc/new_features/v214_features_doc.rst

@@ -0,0 +1,42 @@
+色覺辨認障礙模擬 + 碰撞檢查
+==========================
+
+狀態 UI 仰賴顏色——綠色「正常」對紅色「錯誤」的圓點、以顏色編碼的圖表圖例。對約 8% 有色覺辨認障礙
+(CVD)的男性而言,這些可能難以分辨,而框架原本無從檢查。``cvd_simulate`` 補上無障礙 / 設計檢查
+所需的兩個原語。
+
+* :func:`simulate_cvd` ——把 ``(r, g, b)`` 顏色透過二色覺模擬矩陣(``protanopia`` /
+  ``deuteranopia`` / ``tritanopia``)在給定 ``severity``(0 = 不受影響,1 = 完全二色覺)下映射。
+* :func:`colors_collide` ——在某 CVD 類型下模擬兩個顏色,並回報它們是否變得太相似而難以區分
+  (模擬後的感知 ``redmean`` 距離低於 ``threshold``)。
+* :func:`color_distance` ——底層的 ``redmean`` 色差度量。
+
+純標準函式庫——不需 numpy / OpenCV——以單純的 RGB tuple 運作,故能完整測試。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import simulate_cvd, colors_collide
+
+    # 「錯誤紅」在綠色弱者眼中看起來如何?
+    simulate_cvd((220, 40, 40), "deuteranopia")        # -> (r, g, b)
+
+    # 我的正常綠與錯誤紅對他們是否可區分?
+    report = colors_collide((60, 200, 60), (220, 60, 60), kind="deuteranopia")
+    report["collide"]    # 若兩者易混淆則為 True
+    report["distance"]   # 模擬後的感知距離
+
+``simulate_cvd`` 接受友善別名(``protan`` / ``deutan`` / ``tritan``,或
+``red`` / ``green`` / ``blue``)。``severity`` 在原色與完全二色覺模擬之間插值,
+用於較輕微的異常三色覺。``colors_collide`` 回傳 ``{collide, distance, kind, severity,
+simulated_left, simulated_right}``。
+
+執行器指令
+----------
+
+``AC_simulate_cvd``(``rgb`` ``[r, g, b]`` 加上 ``kind`` / ``severity`` →
+``{rgb}``)與 ``AC_colors_collide``(``left`` / ``right`` ``[r, g, b]`` 加上
+``kind`` / ``severity`` / ``threshold`` → 報告)。RGB 輸入接受 JSON 清單。皆以對應的唯讀
+``ac_*`` MCP 工具及 Script Builder 指令(位於 **Image** 分類下)形式提供。

docs/source/Zh/doc/new_features/v215_features_doc.rst

@@ -0,0 +1,37 @@
+Set-of-Marks 標籤佈局(不重疊、可讀顏色)
+=========================================
+
+Set-of-Marks 在每個元素上疊一個編號標籤,讓視覺模型能說「點 7」。``set_of_marks`` 以固定偏移繪製
+每個標籤,故在密集 UI 上數字會互相疊壓(難以辨讀),而深色標籤在深色元素上會消失。``marks_layout``
+以純幾何修正兩者。
+
+* :func:`place_labels` ——貪婪式不重疊放置:對每個 mark,在其方框周圍嘗試一圈候選位置
+  (上、下、內;左/右對齊),取第一個仍在邊界內且不與任何已放置標籤重疊者。
+* :func:`label_color` ——挑選標籤文字顏色(黑或白),取對元素背景 WCAG 對比較佳者。
+
+純標準函式庫;重用 :func:`a11y_audit.contrast_ratio`。無需繪製即可完整測試。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import mark_elements, place_labels, label_color
+
+    marks = mark_elements(elements)            # [{id, bbox, ...}]
+    layout = place_labels(marks, bounds=(1920, 1080))
+    # [{'id': 1, 'label': [x, y, 22, 16], 'anchor': [bx, by]}, ...]
+
+    label_color((30, 30, 30))     # {'rgb': [255, 255, 255], 'contrast': ...}
+
+把 :func:`place_labels` 產生的 ``label`` 方框餵給你的繪製器(取代固定偏移),並用 :func:`label_color`
+挑選每個編號的顏色,使其在背景上維持可讀。``place_labels`` 是確定性的且依輸入 marks 排序,
+故同一畫面總是以相同方式編號。
+
+執行器指令
+----------
+
+``AC_place_labels``(``marks`` JSON 清單加上 ``label_width`` / ``label_height`` /
+``bounds`` ``[w, h]`` → ``{labels}``)與 ``AC_label_color``(``background``
+``[r, g, b]`` → ``{rgb, contrast}``)。皆以對應的唯讀 ``ac_*`` MCP 工具及 Script Builder 指令
+(位於 **Image** 分類下)形式提供。

docs/source/Zh/doc/new_features/v216_features_doc.rst

@@ -0,0 +1,42 @@
+取樣區域的文字對比(WCAG)
+==========================
+
+:func:`a11y_audit.contrast_ratio` 對你已知的前景 / 背景配對評分。但當你只有螢幕上的一個*區域*——
+一個按鈕、一個標籤——你並不知道那兩個顏色;你有的是一片像素。``contrast_map`` 補上這道缺口:
+把取樣區域拆成其主要前景(少數——通常是文字)與背景(多數)顏色,再評其 WCAG 對比。
+
+* :func:`grade_contrast` ——純函式:把前景 / 背景配對對 WCAG 2.x 門檻轉為
+  ``{ratio, aa, aaa, aa_large, aaa_large}``。
+* :func:`dominant_pair` ——純函式:依亮度把一串取樣 RGB 像素拆成主要的 ``{foreground, background}``。
+* :func:`region_contrast` ——取樣螢幕區域並評分,透過可注入的 ``sampler``(預設為真實螢幕擷取)。
+
+評分與拆分皆為純函式並重用 :func:`a11y_audit.contrast_ratio`,故能在沒有螢幕的情況下完整測試。
+不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import grade_contrast, dominant_pair, region_contrast
+
+    # 若你已知顏色:
+    grade_contrast((90, 90, 90), (255, 255, 255))
+    # {'ratio': 3.9, 'aa': False, 'aaa': False, 'aa_large': True, ...}
+
+    # 若你只有螢幕的一個區域,取樣並評分:
+    report = region_contrast(region=[x, y, w, h])
+    if not report["aa"]:
+        print("低對比文字", report["foreground"], report["background"])
+
+``dominant_pair`` 以平均亮度切分取樣像素,把較大的一群視為背景、較小的視為文字——
+均勻一片會讓兩者得到相同顏色(無對比)。``region_contrast`` 接受可注入的 ``sampler``
+(``region -> RGB 像素清單``),故邏輯能在沒有真實螢幕的情況下測試。
+
+執行器指令
+----------
+
+``AC_grade_contrast``(``foreground`` / ``background`` ``[r, g, b]`` → 評分)、
+``AC_dominant_pair``(``pixels`` JSON 清單 ``[r, g, b]`` → ``{foreground, background}``)與
+``AC_region_contrast``(``region`` ``[x, y, w, h]`` → 評分 + 顏色 + ``samples``)。皆以對應的
+唯讀 ``ac_*`` MCP 工具及 Script Builder 指令(位於 **Image** 分類下)形式提供。