Skip to content

Gate the built site in a browser, and its scripts in the markup - #11

Merged
nicodes merged 2 commits into
mainfrom
issue-35-browser-quality
Aug 21, 2026
Merged

Gate the built site in a browser, and its scripts in the markup#11
nicodes merged 2 commits into
mainfrom
issue-35-browser-quality

Conversation

@nicodes

@nicodes nicodes commented Aug 21, 2026

Copy link
Copy Markdown
Member

Fixes aviorstudio/termcade-be#35

Stacked on #10 (aviorstudio/termcade-be#29); this diff is only the second layer.

What

Script gatetools/check_scripts.py walks every HTML file in dist/ and rejects any <script src=...> and any inline <script> whose type is not application/ld+json. The old gate only searched for *.js files, which an inline script sails straight past. The JSON-LD structured-data block is data, not code, and stays allowed. The existing find dist -name '*.js' check is kept — it catches a script file nothing references, which the markup scan cannot see.

Browser gatetools/browser-check.mjs serves dist/ over a throwaway local HTTP server and loads the page in headless Chrome (preinstalled on ubuntu-latest, so no browser download; playwright-core is the only new dependency, with node pinned in .mise.toml because playwright-core does not run on bun). It fails on:

  • console errors, uncaught exceptions, failed requests, HTTP ≥ 400 responses — tolerance zero
  • missing landmarks: not exactly one <h1>, no <header>/<main>/<footer>, missing html[lang], <title>, or meta description
  • links without link text, inline SVGs neither aria-hidden nor labelled
  • horizontal overflow: page-level scrollbar and any top-level section extending past the viewport (the section check is the binding one — .page-shell's overflow-x: hidden masks the scrollbar for in-shell content)
  • broken links: every internal href must return 200 from the built site; every external href (the GitHub repo/releases/docs links and wazero.io) must answer 200 within the timeout

Documented parameters (in the script header, the CI step comment, and the README): viewports 1280×800 desktop and 390×844 mobile (touch + isMobile), external link timeout 10000ms, one attempt, redirects followed, accessibility threshold = the landmark/alternative-text assertions above (no raster images exist, so no alt rule binds).

Proof the gates fail

Inline script through the real build (required by the issue): added <script is:inline>alert("deliberate")</script> to src/pages/index.astro, rebuilt:

dist ships executable JavaScript — this site is meant to ship none:
  dist/index.html:97: inline executable script <script type=None>
exit=1

Restored, rebuilt, green. The external-src and type="module" rejection branches were also exercised against synthetic files and fail with their own messages.

Every browser assertion proven reachable by sabotaging a copy of dist (never the committed sources):

Sabotage Observed failure
inline script logging + throwing console error: saboteur, uncaught exception: Error: boom
removed <h1>, unwrapped <footer> expected exactly one <h1>, found 0, no <footer> landmark
hero forced to 3000px wide 1 section(s) extend past the viewport
same + overflow-x:visible on the shell page has a horizontal scrollbar (proves the scrollbar backstop is reachable)
broken internal + external href, emptied link text link /nope: HTTP 404, link https://wazero.io/definitely-not-a-page: HTTP 404, 1 link(s) with no link text
stylesheet on a closed port request failed: ... (net::ERR_UNSAFE_PORT)
stripped aria-hidden from an SVG 1 svg(s) neither hidden nor labelled

One proposed assertion failed this audit and was removed: aria-label on the frames' plain <div> wrappers is not exposed by assistive technology (no role), and the frames contain text so the "no text" fallback never fired — dead code. The frames are text by design; the honest alternative-text rules are the SVG and link-text assertions. This is recorded in the script header.

Verification

  • python3 tools/check_scripts.py dist — pass, 1 page, ld+json allowed.
  • node tools/browser-check.mjs dist — pass at both viewports; all 8 page links (2 internal /, 6 external) returned 200.
  • Visual spot-check of the built page at 1280×800 and 390×844 (screenshots): layout intact, no overflow, pixel frames render as before.
  • bun run build clean; bun.lock updated via bun add -d playwright-core (frozen-lockfile safe).

@vercel

vercel Bot commented Aug 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
termcade-web Ready Ready Preview Aug 21, 2026 6:31am

@nicodes

nicodes commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Review 1

P2 — Accessibility gate accepts empty or unresolved accessible names

tools/browser-check.mjs:92-95, 110

The SVG check only tests for the presence of aria-label or aria-labelledby, not whether either produces a non-empty accessible name. A future unhidden SVG with aria-label="" or aria-labelledby="missing-id" passes this gate while being unnamed to assistive technology. The link check similarly uses raw textContent, which can count text hidden from the accessibility tree. Validate computed accessible names rather than attribute/text presence.

Residual risks

Reviewed only the pinned diff f051a6624ec27edcd174dc096f977de4a2b2ada2...0313f2ede981e89158da47b4f428ab3550244219, fetched the changed files at the pinned head, and inspected the CI workflow and terminal head CI result. The Chrome check passed in CI, but external-link availability remains inherently network-dependent; this review did not execute the gate locally.

Verdict: NEEDS ATTENTION

@nicodes

nicodes commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Review 1 reply

Fixed in 3705611. Link and SVG/image checks now consume Chrome’s computed accessibility tree through Accessibility.getFullAXTree at both viewports rather than matching raw attributes or text. Empty aria-label without fallback, unresolved aria-labelledby, and display-none-only text each fail as links with empty computed accessible names; an unhidden unlabeled SVG fails as an image with an empty computed name. Decorative hidden SVGs do not enter the tree, while an empty label with genuinely visible text correctly falls back to that content.

All prior sabotage classes were re-run and remain reachable; the emptied-text case now fails through the computed-name assertion. Browser checks still use 1280x800 and 390x844 viewports with a 10000ms link timeout. Head CI and Vercel are green.

@nicodes

nicodes commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Review 2

No findings.

Review 1 is correctly dispositioned: tools/browser-check.mjs:90-118,204 reads computed CDP accessibility names for live link/image nodes at both configured viewports.

Reviewed exact range 4bfa4e4...3705611, CI workflow, PR checks, and reply. CI/Vercel succeeded. Residual risk: external-link checks remain network-dependent and Chromium-only.

Verdict: APPROVE

@nicodes

nicodes commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Review 2 reply

No findings require changes. The network dependency and Chromium scope are accepted and documented with exact parameters: one attempt per critical external link, 10000ms timeout, redirects followed, and Chrome at 1280x800 plus 390x844. This is a Chromium smoke/accessibility gate rather than a cross-browser conformance claim. Head CI/Vercel are green.

Disposition: Review 2 is approving.

Base automatically changed from issue-29-capture-freshness to main August 21, 2026 06:31
@nicodes
nicodes force-pushed the issue-35-browser-quality branch from 3705611 to 344384a Compare August 21, 2026 06:31
The no-JavaScript gate only looked for *.js files, so an inline <script>
shipped past it. check_scripts.py walks the built HTML and rejects any
<script src> and any inline <script> whose type is not application/ld+json
-- the one script this page ships, and data rather than code.

The site also had no browser verification at all. browser-check.mjs serves
dist/ locally and loads it in headless Chrome at 1280x800 and 390x844,
failing on console errors, uncaught exceptions, failed or >=400 requests,
missing landmarks (one h1, header/main/footer, lang, title, description),
links or SVGs without text alternatives, horizontal overflow (page-level,
and any top-level section past the viewport edge), and broken internal or
external links (200 expected, 10s timeout, redirects followed). Every
assertion was proven reachable by sabotaging a build copy; one that was not
-- aria-labels on the frames' role-less div wrappers -- was removed rather
than kept as theatre.

Proven to fail end to end: an is:inline script in the page source builds to
'dist/index.html:97: inline executable script <script type=None>', exit 1.

Fixes aviorstudio/termcade-be#35
Review 1, finding 3. The link and SVG name checks matched raw attributes
and text, which waves through an empty aria-label, an aria-labelledby
pointing at nothing, and text hidden with display:none. The check now reads
each node's computed accessible name from Chrome's accessibility tree (CDP
Accessibility.getFullAXTree) at both viewports and fails on any link or
unhidden svg/image whose computed name is empty, naming the offending
element.

Proven reachable against a sabotaged build copy: empty aria-label,
unresolved aria-labelledby, and display:none-only link text each fail with
'link with an empty computed accessible name: <a ...>'; an unhidden
unlabelled svg fails with 'unhidden svg or image with an empty computed
accessible name: <svg ...>'. The previously emptied-link-text sabotage is
now caught by the same computed-name assertion with a better message.
@nicodes
nicodes force-pushed the issue-35-browser-quality branch from 344384a to 92fdd3c Compare August 21, 2026 06:31
@nicodes
nicodes merged commit f66f587 into main Aug 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant