Skip to content

fix(edge-apps): use Playwright Clock API for idempotent screenshots#712

Merged
nicomiguelino merged 2 commits intomasterfrom
fix/idempotent-screenshots
Feb 27, 2026
Merged

fix(edge-apps): use Playwright Clock API for idempotent screenshots#712
nicomiguelino merged 2 commits intomasterfrom
fix/idempotent-screenshots

Conversation

@nicomiguelino
Copy link
Contributor

@nicomiguelino nicomiguelino commented Feb 26, 2026

Summary

Idempotent screenshots means that running bun run screenshots multiple times always produces pixel-identical output, regardless of when the tests are executed. Without a frozen clock, apps that render the current date or time produce different screenshots on every run.

  • Add FIXED_SCREENSHOT_DATE constant and setupClockMock() utility to edge-apps-library using Playwright's Clock API (setFixedTime)
  • Apply setupClockMock() in all 8 screenshot specs (clock, simple-timer, weather, menu-board, qr-code, grafana, cap-alerting, bun-create template) before page.goto() to freeze time prior to any JS execution
  • Regenerate screenshots for clock, simple-timer, and weather with the frozen clock

- Add FIXED_SCREENSHOT_DATE constant and setupClockMock() utility to edge-apps-library
- Apply setupClockMock() in all screenshot specs (clock, simple-timer, weather, menu-board, qr-code, grafana, cap-alerting, and bun-create template)
- Regenerate screenshots for clock, simple-timer, and weather with frozen clock

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions
Copy link

github-actions bot commented Feb 26, 2026

PR Reviewer Guide 🔍

(Review updated until commit eebae78)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Compatibility

Verify the Playwright Clock API usage is compatible with the repo’s Playwright version and configuration. Some versions/flows require installing/enabling the clock before calling setFixedTime, and failures here could make the screenshot suite flaky or fail at runtime across environments.

/**
 * Freeze the browser clock to a fixed point in time before page load.
 * Must be called before `page.goto()` to ensure the clock is set before
 * any JavaScript runs. This makes screenshots idempotent regardless of
 * when the test is executed.
 *
 * @param page - Playwright page object
 * @param date - The date to freeze the clock at (defaults to FIXED_SCREENSHOT_DATE)
 */
export async function setupClockMock(
  page: {
    clock: { setFixedTime(time: Date | number | string): Promise<void> }
  },
  date: Date | number | string = FIXED_SCREENSHOT_DATE,
): Promise<void> {
  await page.clock.setFixedTime(date)
}
Determinism

Freezing time may still produce non-idempotent screenshots if rendering depends on timezone/locale (e.g., Date formatting). Consider enforcing a consistent timezone/locale in browser.newContext for screenshot runs to avoid environment-dependent output.

for (const { width, height } of RESOLUTIONS) {
  test(`screenshot ${width}x${height}`, async ({ browser }) => {
    const screenshotsDir = getScreenshotsDir()

    const context = await browser.newContext({ viewport: { width, height } })
    const page = await context.newPage()

    // Setup mocks
    await setupClockMock(page)
    await setupScreenlyJsMock(page, screenlyJsContent)
    await setupOpenWeatherMocks(page, {
Mutability

FIXED_SCREENSHOT_DATE is a shared Date instance, which is mutable. To prevent accidental mutation from impacting other tests, consider storing an immutable timestamp (number/string) or returning a fresh Date instance per call.

/**
 * Fixed date used for screenshot tests to ensure idempotent output.
 * Matches the timestamp used in mock weather API responses (2025-02-19T21:20:00Z).
 */
export const FIXED_SCREENSHOT_DATE = new Date('2025-02-19T21:20:00Z')

@github-actions
Copy link

Persistent review updated to latest commit eebae78

@github-actions
Copy link

github-actions bot commented Feb 26, 2026

PR Code Suggestions ✨

Latest suggestions up to eebae78
Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Install clock before freezing time

Playwright’s Clock API typically requires installing the clock before calling
setFixedTime; otherwise setFixedTime can be a no-op or throw depending on Playwright
version/config. Install the clock first (when available) to ensure the time-freeze
reliably applies before navigation.

edge-apps/edge-apps-library/src/test/screenshots.ts [194-201]

 export async function setupClockMock(
   page: {
-    clock: { setFixedTime(time: Date | number | string): Promise<void> }
+    clock: {
+      install?: (options?: { time?: Date | number | string }) => Promise<void>
+      setFixedTime(time: Date | number | string): Promise<void>
+    }
   },
   date: Date | number | string = FIXED_SCREENSHOT_DATE,
 ): Promise<void> {
+  if (page.clock.install) {
+    await page.clock.install({ time: date })
+  }
   await page.clock.setFixedTime(date)
 }
Suggestion importance[1-10]: 7

__

Why: Calling page.clock.setFixedTime() without first installing the clock can fail or be ineffective depending on the Playwright version/fixture setup, so adding an install() step improves reliability of setupClockMock. The proposed change is compatible (guards on install) and directly supports the PR goal of deterministic screenshots.

Medium

Previous suggestions

Suggestions up to commit eebae78
CategorySuggestion                                                                                                                                    Impact
Possible issue
Install and validate the clock

Ensure the Playwright clock is installed before calling setFixedTime, otherwise this
can fail at runtime on pages where the clock hasn’t been initialized. Also normalize
string inputs to a Date/timestamp and fail fast on invalid values to avoid silently
setting an invalid time.

edge-apps/edge-apps-library/src/test/screenshots.ts [194-201]

 export async function setupClockMock(
   page: {
-    clock: { setFixedTime(time: Date | number | string): Promise<void> }
+    clock: {
+      install(options?: { time?: Date | number }): Promise<void>
+      setFixedTime(time: Date | number): Promise<void>
+    }
   },
   date: Date | number | string = FIXED_SCREENSHOT_DATE,
 ): Promise<void> {
-  await page.clock.setFixedTime(date)
+  const fixedTime =
+    typeof date === 'string'
+      ? (() => {
+          const parsed = Date.parse(date)
+          if (Number.isNaN(parsed)) {
+            throw new Error(`Invalid date string provided to setupClockMock: ${date}`)
+          }
+          return parsed
+        })()
+      : date
+  await page.clock.install({ time: fixedTime })
+  await page.clock.setFixedTime(fixedTime)
 }
Suggestion importance[1-10]: 8

__

Why: Calling page.clock.setFixedTime(...) without first calling page.clock.install(...) can fail at runtime in Playwright, so this is a meaningful correctness/stability improvement for the new screenshot tests. The added validation/normalization is reasonable, though install({ time }) plus setFixedTime(...) may be redundant depending on Playwright behavior.

Medium
Use an immutable fixed time

Avoid exporting a mutable Date instance, since any accidental mutation can leak
across tests and make runs flaky. Prefer exporting an immutable timestamp/ISO string
and constructing a Date (or using the number) at the call site.

edge-apps/edge-apps-library/src/test/screenshots.ts [20]

-export const FIXED_SCREENSHOT_DATE = new Date('2025-02-19T21:20:00Z')
+export const FIXED_SCREENSHOT_DATE = Date.parse('2025-02-19T21:20:00Z')
Suggestion importance[1-10]: 5

__

Why: Exporting a mutable Date object (FIXED_SCREENSHOT_DATE) can allow accidental mutation by consumers and lead to flaky tests, so using an immutable timestamp reduces risk. This is a smaller design/robustness improvement rather than a direct bug fix.

Low

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements idempotent screenshot generation by freezing the browser clock to a fixed date using Playwright's Clock API. The change ensures that running screenshot tests multiple times produces pixel-identical output regardless of execution time, which is critical for apps that render current date/time information.

Changes:

  • Added FIXED_SCREENSHOT_DATE constant and setupClockMock() utility function to the edge-apps-library test module
  • Applied clock freezing in all 8 screenshot test specs before page navigation
  • Regenerated screenshots for clock, simple-timer, and weather apps with the frozen clock

Reviewed changes

Copilot reviewed 9 out of 39 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
edge-apps/edge-apps-library/src/test/screenshots.ts Added FIXED_SCREENSHOT_DATE constant and setupClockMock() utility function
edge-apps/clock/e2e/screenshots.spec.ts Added setupClockMock() call before page.goto()
edge-apps/simple-timer/e2e/screenshots.spec.ts Added setupClockMock() call before page.goto()
edge-apps/weather/e2e/screenshots.spec.ts Added setupClockMock() call before page.goto()
edge-apps/menu-board/e2e/screenshots.spec.ts Added setupClockMock() call before page.goto()
edge-apps/qr-code/e2e/screenshots.spec.ts Added setupClockMock() call before page.goto()
edge-apps/grafana/e2e/screenshots.spec.ts Added setupClockMock() call before page.goto()
edge-apps/cap-alerting/e2e/screenshots.spec.ts Added setupClockMock() call before page.goto()
edge-apps/.bun-create/edge-app-template/e2e/screenshots.spec.ts Added setupClockMock() call before page.goto()
edge-apps/clock/screenshots/*.webp Regenerated screenshot files with frozen clock
edge-apps/simple-timer/screenshots/*.webp Regenerated screenshot files with frozen clock
edge-apps/weather/screenshots/*.webp Regenerated screenshot files with frozen clock

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@nicomiguelino nicomiguelino merged commit 5fb2ae2 into master Feb 27, 2026
38 of 40 checks passed
@nicomiguelino nicomiguelino deleted the fix/idempotent-screenshots branch February 27, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants