Add contrast_map: grade on-screen text contrast by sampling its colours

WHATS_NEW.md

@@ -2,6 +2,12 @@
 
 ## What's new (2026-06-26)
 
+### Sample a Region's Text Contrast (WCAG)
+
+Grade the legibility of on-screen text when you only have a region, not the two colours. Full reference: [`docs/source/Eng/doc/new_features/v216_features_doc.rst`](docs/source/Eng/doc/new_features/v216_features_doc.rst).
+
+- **`grade_contrast` / `dominant_pair` / `region_contrast`** (`AC_grade_contrast`, `AC_dominant_pair`, `AC_region_contrast`): `a11y_audit.contrast_ratio` grades a foreground/background pair you already know — but a button or label on screen is a *patch of pixels*, not two known colours. `dominant_pair` splits sampled pixels at the mean luminance into the dominant foreground (minority, the text) and background (majority); `grade_contrast` grades a pair against the WCAG 2.x AA/AAA thresholds (normal + large text); `region_contrast` samples a screen region (through an injectable `sampler`) and grades it. The grading and split are pure and reuse `a11y_audit.contrast_ratio`, fully testable without a screen. Third feature of the ROUND-15 perception lane. No `PySide6`.
+
 ### Set-of-Marks Label Layout (No Overlap, Readable Colour)
 
 Number every element without the labels piling up or vanishing into the background. Full reference: [`docs/source/Eng/doc/new_features/v215_features_doc.rst`](docs/source/Eng/doc/new_features/v215_features_doc.rst).

docs/source/Eng/doc/new_features/v216_features_doc.rst

@@ -0,0 +1,50 @@
+Sample a Region's Text Contrast (WCAG)
+======================================
+
+:func:`a11y_audit.contrast_ratio` grades a foreground / background pair you
+already know. But when you only have a *region* of the screen — a button, a
+label — you don't know those two colours; you have a patch of pixels.
+``contrast_map`` closes that gap: split a sampled region into its dominant
+foreground (the minority — usually the text) and background (the majority)
+colours, then grade their WCAG contrast.
+
+* :func:`grade_contrast` — pure: a foreground / background pair to
+  ``{ratio, aa, aaa, aa_large, aaa_large}`` against the WCAG 2.x thresholds.
+* :func:`dominant_pair` — pure: split a list of sampled RGB pixels into the
+  dominant ``{foreground, background}`` colours by luminance.
+* :func:`region_contrast` — sample a screen region and grade it, through an
+  injectable ``sampler`` (the real screen grab by default).
+
+The grading and split are pure and reuse :func:`a11y_audit.contrast_ratio`, so
+they are fully testable without a screen. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import grade_contrast, dominant_pair, region_contrast
+
+    # If you already know the colours:
+    grade_contrast((90, 90, 90), (255, 255, 255))
+    # {'ratio': 3.9, 'aa': False, 'aaa': False, 'aa_large': True, ...}
+
+    # If you only have a region of the screen, sample and grade it:
+    report = region_contrast(region=[x, y, w, h])
+    if not report["aa"]:
+        print("low-contrast text", report["foreground"], report["background"])
+
+``dominant_pair`` partitions the sampled pixels at the mean luminance and treats
+the larger group as the background and the smaller as the text — a uniform patch
+yields the same colour for both (no contrast). ``region_contrast`` accepts an
+injectable ``sampler`` (``region -> list of RGB pixels``) so the logic is tested
+without a real screen.
+
+Executor commands
+-----------------
+
+``AC_grade_contrast`` (``foreground`` / ``background`` ``[r, g, b]`` → the
+grade), ``AC_dominant_pair`` (``pixels`` JSON list of ``[r, g, b]`` →
+``{foreground, background}``) and ``AC_region_contrast`` (``region``
+``[x, y, w, h]`` → the grade + colours + ``samples``). They are the matching
+read-only ``ac_*`` MCP tools and Script Builder commands under **Image**.

docs/source/Zh/doc/new_features/v216_features_doc.rst

@@ -0,0 +1,42 @@
+取樣區域的文字對比(WCAG)
+==========================
+
+:func:`a11y_audit.contrast_ratio` 對你已知的前景 / 背景配對評分。但當你只有螢幕上的一個*區域*——
+一個按鈕、一個標籤——你並不知道那兩個顏色;你有的是一片像素。``contrast_map`` 補上這道缺口:
+把取樣區域拆成其主要前景(少數——通常是文字)與背景(多數)顏色,再評其 WCAG 對比。
+
+* :func:`grade_contrast` ——純函式:把前景 / 背景配對對 WCAG 2.x 門檻轉為
+  ``{ratio, aa, aaa, aa_large, aaa_large}``。
+* :func:`dominant_pair` ——純函式:依亮度把一串取樣 RGB 像素拆成主要的 ``{foreground, background}``。
+* :func:`region_contrast` ——取樣螢幕區域並評分,透過可注入的 ``sampler``(預設為真實螢幕擷取)。
+
+評分與拆分皆為純函式並重用 :func:`a11y_audit.contrast_ratio`,故能在沒有螢幕的情況下完整測試。
+不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import grade_contrast, dominant_pair, region_contrast
+
+    # 若你已知顏色:
+    grade_contrast((90, 90, 90), (255, 255, 255))
+    # {'ratio': 3.9, 'aa': False, 'aaa': False, 'aa_large': True, ...}
+
+    # 若你只有螢幕的一個區域,取樣並評分:
+    report = region_contrast(region=[x, y, w, h])
+    if not report["aa"]:
+        print("低對比文字", report["foreground"], report["background"])
+
+``dominant_pair`` 以平均亮度切分取樣像素,把較大的一群視為背景、較小的視為文字——
+均勻一片會讓兩者得到相同顏色(無對比)。``region_contrast`` 接受可注入的 ``sampler``
+(``region -> RGB 像素清單``),故邏輯能在沒有真實螢幕的情況下測試。
+
+執行器指令
+----------
+
+``AC_grade_contrast``(``foreground`` / ``background`` ``[r, g, b]`` → 評分)、
+``AC_dominant_pair``(``pixels`` JSON 清單 ``[r, g, b]`` → ``{foreground, background}``)與
+``AC_region_contrast``(``region`` ``[x, y, w, h]`` → 評分 + 顏色 + ``samples``)。皆以對應的
+唯讀 ``ac_*`` MCP 工具及 Script Builder 指令(位於 **Image** 分類下)形式提供。

je_auto_control/__init__.py

@@ -137,6 +137,10 @@
 )
 # Lay out Set-of-Marks labels without overlap + readable colour
 from je_auto_control.utils.marks_layout import label_color, place_labels
+# Grade on-screen text legibility by sampling its actual colours (WCAG)
+from je_auto_control.utils.contrast_map import (
+    dominant_pair, grade_contrast, region_contrast,
+)
 # Rich clipboard formats — RTF + CSV/TSV codecs and Windows get / set
 from je_auto_control.utils.clipboard_rich_formats import (
     build_rtf, csv_to_rows, get_clipboard_csv, get_clipboard_rtf, rows_to_csv,
@@ -1763,6 +1767,7 @@ def start_autocontrol_gui(*args, **kwargs):
     "wait_until_app_idle", "idle_point",
     "simulate_cvd", "colors_collide", "color_distance",
     "place_labels", "label_color",
+    "grade_contrast", "dominant_pair", "region_contrast",
     "build_rtf", "rtf_to_text", "rows_to_csv", "csv_to_rows",
     "set_clipboard_rtf", "get_clipboard_rtf",
     "set_clipboard_csv", "get_clipboard_csv",

je_auto_control/gui/script_builder/command_schema.py

@@ -4525,7 +4525,7 @@
     specs.append(CommandSpec(
         "AC_simulate_cvd", "Image", "Simulate Colour-Vision Deficiency",
         fields=(
             FieldSpec("rgb", FieldType.STRING, placeholder="[r, g, b]"),
             FieldSpec("kind", FieldType.STRING, optional=True,
                       default="deuteranopia",
                       placeholder="protanopia / deuteranopia / tritanopia"),
@@ -4567,6 +4567,30 @@
         ),
         description="Higher-contrast label colour (black/white) for a background.",
     ))
+    specs.append(CommandSpec(
+        "AC_grade_contrast", "Image", "Grade Contrast (WCAG)",
+        fields=(
+            FieldSpec("foreground", FieldType.STRING, placeholder="[r, g, b]"),
+            FieldSpec("background", FieldType.STRING, placeholder="[r, g, b]"),
+        ),
+        description="Grade a foreground/background colour pair against WCAG.",
+    ))
+    specs.append(CommandSpec(
+        "AC_dominant_pair", "Image", "Dominant FG/BG Colours",
+        fields=(
+            FieldSpec("pixels", FieldType.STRING,
+                      placeholder="JSON list of [r, g, b] pixels"),
+        ),
+        description="Split sampled pixels into dominant foreground/background.",
+    ))
+    specs.append(CommandSpec(
+        "AC_region_contrast", "Image", "Region Text Contrast",
+        fields=(
+            FieldSpec("region", FieldType.STRING, optional=True,
+                      placeholder="[x, y, w, h]"),
+        ),
+        description="Sample a screen region and grade its text contrast.",
+    ))
     specs.append(CommandSpec(
         "AC_normalize_ext", "Shell", "Normalize Extension",
         fields=(

je_auto_control/utils/contrast_map/__init__.py

@@ -0,0 +1,6 @@
+"""Grade the legibility of on-screen text by sampling its actual colours."""
+from je_auto_control.utils.contrast_map.contrast_map import (
+    dominant_pair, grade_contrast, region_contrast,
+)
+
+__all__ = ["grade_contrast", "dominant_pair", "region_contrast"]

je_auto_control/utils/contrast_map/contrast_map.py

@@ -0,0 +1,114 @@
+"""Grade the legibility of on-screen text by sampling its actual colours.
+
+:func:`a11y_audit.contrast_ratio` grades a foreground / background pair you
+already know. But when you only have a *region* of the screen — a button, a
+label — you don't know those two colours; you have a patch of pixels.
+``contrast_map`` closes that gap: split a sampled region into its dominant
+foreground (the minority — usually the text) and background (the majority)
+colours, then grade their WCAG contrast.
+
+* :func:`grade_contrast` — pure: a foreground / background pair to
+  ``{ratio, aa, aaa, aa_large, aaa_large}`` against the WCAG 2.x thresholds.
+* :func:`dominant_pair` — pure: split a list of sampled RGB pixels into the
+  dominant ``{foreground, background}`` colours by luminance.
+* :func:`region_contrast` — sample a screen region and grade it, through an
+  injectable ``sampler`` (the real screen grab by default).
+
+The grading and split are pure and reuse :func:`a11y_audit.contrast_ratio`, so
+they are fully testable without a screen. Imports no ``PySide6``.
+"""
+from typing import Any, Callable, Dict, List, Optional, Sequence
+
+from je_auto_control.utils.a11y_audit import contrast_ratio
+from je_auto_control.utils.a11y_audit.audit import relative_luminance
+
+# WCAG 2.x contrast thresholds.
+_AA_NORMAL = 4.5
+_AAA_NORMAL = 7.0
+_AA_LARGE = 3.0
+_AAA_LARGE = 4.5
+
+RGB = List[int]
+PixelSampler = Callable[[Optional[Sequence[int]]], List[Sequence[int]]]
+
+
+def grade_contrast(foreground: Sequence[float],
+                   background: Sequence[float]) -> Dict[str, Any]:
+    """Grade a foreground / background colour pair against WCAG (pure).
+
+    Returns ``{ratio, aa, aaa, aa_large, aaa_large}`` — the contrast ratio and
+    whether it meets AA / AAA for normal and large text.
+    """
+    ratio = contrast_ratio(foreground, background)
+    return {
+        "ratio": round(ratio, 3),
+        "aa": ratio >= _AA_NORMAL,
+        "aaa": ratio >= _AAA_NORMAL,
+        "aa_large": ratio >= _AA_LARGE,
+        "aaa_large": ratio >= _AAA_LARGE,
+    }
+
+
+def _mean_color(pixels: Sequence[Sequence[int]]) -> RGB:
+    """Average a non-empty list of RGB pixels to one colour (pure)."""
+    count = len(pixels)
+    totals = [0, 0, 0]
+    for pixel in pixels:
+        totals[0] += int(pixel[0])
+        totals[1] += int(pixel[1])
+        totals[2] += int(pixel[2])
+    return [totals[0] // count, totals[1] // count, totals[2] // count]
+
+
+def _partition_by_luminance(rgbs: Sequence[Sequence[int]]) -> tuple:
+    """Split RGB pixels into (darker, lighter) groups at the mean luminance."""
+    lums = [relative_luminance(rgb) for rgb in rgbs]
+    average = sum(lums) / len(lums)
+    low = [rgb for rgb, lum in zip(rgbs, lums) if lum < average]
+    high = [rgb for rgb, lum in zip(rgbs, lums) if lum >= average]
+    return low, high
+
+
+def dominant_pair(pixels: Sequence[Sequence[int]]) -> Dict[str, RGB]:
+    """Split sampled pixels into dominant foreground / background colours (pure).
+
+    Pixels are partitioned at the mean luminance; the larger group is the
+    background and the smaller the foreground (text). A uniform region yields
+    the same colour for both (no contrast). Returns ``{foreground, background}``.
+    """
+    rgbs = [(int(p[0]), int(p[1]), int(p[2])) for p in pixels]
+    if not rgbs:
+        return {"foreground": [0, 0, 0], "background": [0, 0, 0]}
+    low, high = _partition_by_luminance(rgbs)
+    if not low or not high:
+        mean = _mean_color(rgbs)
+        return {"foreground": mean, "background": mean}
+    background, foreground = (high, low) if len(high) >= len(low) else (low,
+                                                                        high)
+    return {"foreground": _mean_color(foreground),
+            "background": _mean_color(background)}
+
+
+def _default_sampler(region: Optional[Sequence[int]]) -> List[RGB]:
+    """Grab a screen region and return a capped list of its RGB pixels."""
+    from je_auto_control.utils.color_region.color_region import _grab_rgb
+    array = _grab_rgb(region)
+    flat = array.reshape(-1, array.shape[-1])
+    step = max(1, len(flat) // 4096)
+    return [[int(px[0]), int(px[1]), int(px[2])] for px in flat[::step]]
+
+
+def region_contrast(*, sampler: Optional[PixelSampler] = None,
+                    region: Optional[Sequence[int]] = None) -> Dict[str, Any]:
+    """Sample a screen ``region`` and grade its text contrast.
+
+    Pass ``sampler`` (``region -> list of RGB pixels``) to supply pixels in
+    tests; the default grabs the real screen. Returns the
+    :func:`grade_contrast` result plus ``{foreground, background, samples}``.
+    """
+    sample = sampler if sampler is not None else _default_sampler
+    pixels = list(sample(region))
+    pair = dominant_pair(pixels)
+    grade = grade_contrast(pair["foreground"], pair["background"])
+    return {**grade, "foreground": pair["foreground"],
+            "background": pair["background"], "samples": len(pixels)}

je_auto_control/utils/executor/action_executor.py

@@ -2880,6 +2880,26 @@ def _label_color(background: Any) -> Dict[str, Any]:
     return label_color(_coerce_rgb(background))
 
 
+def _grade_contrast(foreground: Any, background: Any) -> Dict[str, Any]:
+    """Adapter: grade a foreground/background colour pair vs WCAG (pure)."""
+    from je_auto_control.utils.contrast_map import grade_contrast
+    return grade_contrast(_coerce_rgb(foreground), _coerce_rgb(background))
+
+
+def _dominant_pair(pixels: Any) -> Dict[str, Any]:
+    """Adapter: split sampled RGB pixels into fg/bg dominant colours (pure)."""
+    from je_auto_control.utils.contrast_map import dominant_pair
+    items = [_coerce_rgb(pixel) for pixel in _coerce_list(pixels)] \
+        if pixels else []
+    return dominant_pair(items)
+
+
+def _region_contrast(region: Any = None) -> Dict[str, Any]:
+    """Adapter: sample a screen region and grade its text contrast (device)."""
+    from je_auto_control.utils.contrast_map import region_contrast
+    return region_contrast(region=_coerce_region(region))
+
+
 def _normalize_ext(target: str) -> Dict[str, Any]:
     """Adapter: the lowercased extension of a path / bare ext (pure)."""
     from je_auto_control.utils.file_assoc import normalize_ext
@@ -6915,6 +6935,9 @@ def __init__(self):
             "AC_colors_collide": _colors_collide,
             "AC_place_labels": _place_labels,
             "AC_label_color": _label_color,
+            "AC_grade_contrast": _grade_contrast,
+            "AC_dominant_pair": _dominant_pair,
+            "AC_region_contrast": _region_contrast,
             "AC_normalize_ext": _normalize_ext,
             "AC_file_association": _file_association,
             "AC_get_control_text": _get_control_text,

je_auto_control/utils/mcp_server/tools/_factories.py

@@ -4057,6 +4057,40 @@ def img_histogram_tools() -> List[MCPTool]:
             handler=h.label_color,
             annotations=READ_ONLY,
         ),
+        MCPTool(
+            name="ac_grade_contrast",
+            description=("Grade a 'foreground'/'background' [r,g,b] colour pair "
+                         "against WCAG. Returns {ratio, aa, aaa, aa_large, "
+                         "aaa_large}. Pure."),
+            input_schema=schema({"foreground": {"type": "array",
+                                               "items": {"type": "integer"}},
+                                 "background": {"type": "array",
+                                               "items": {"type": "integer"}}},
+                                required=["foreground", "background"]),
+            handler=h.grade_contrast,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_dominant_pair",
+            description=("Split sampled RGB 'pixels' (list of [r,g,b]) into the "
+                         "dominant {foreground, background} colours by "
+                         "luminance. Pure."),
+            input_schema=schema({"pixels": {"type": "array",
+                                           "items": {"type": "array"}}},
+                                required=["pixels"]),
+            handler=h.dominant_pair,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_region_contrast",
+            description=("Sample a screen 'region' [x,y,w,h] and grade its text "
+                         "contrast: {ratio, aa, aaa, aa_large, aaa_large, "
+                         "foreground, background, samples}."),
+            input_schema=schema({"region": {"type": "array",
+                                           "items": {"type": "integer"}}}),
+            handler=h.region_contrast,
+            annotations=READ_ONLY,
+        ),
     ]
 
 

je_auto_control/utils/mcp_server/tools/_handlers.py

@@ -754,6 +754,21 @@ def label_color(background):
     return _label_color(background)
 
 
+def grade_contrast(foreground, background):
+    from je_auto_control.utils.executor.action_executor import _grade_contrast
+    return _grade_contrast(foreground, background)
+
+
+def dominant_pair(pixels):
+    from je_auto_control.utils.executor.action_executor import _dominant_pair
+    return _dominant_pair(pixels)
+
+
+def region_contrast(region=None):
+    from je_auto_control.utils.executor.action_executor import _region_contrast
+    return _region_contrast(region)
+
+
 def normalize_ext(target):
     from je_auto_control.utils.executor.action_executor import _normalize_ext
     return _normalize_ext(target)