Skip to content

feat(har): include WebSocket in .har#41015

Open
dcrousso wants to merge 1 commit into
microsoft:mainfrom
dcrousso:drousso/feat-har-websocket
Open

feat(har): include WebSocket in .har#41015
dcrousso wants to merge 1 commit into
microsoft:mainfrom
dcrousso:drousso/feat-har-websocket

Conversation

@dcrousso
Copy link
Copy Markdown

add an entry for each WebSocket when generating a .har

also include data for each sent/received frame in a custom property _webSocketMessages (for more info see https://developer.chrome.com/blog/new-in-devtools-76#websocket)

both Chrome and WebKit already capture the wallTime for the initial request and a timestamp for each subsequent message (i.e. diff the timestamp relative to the initial timestamp and add the wallTime in order to determine the current wallTime)

unfortunately Firefox does not have this so fall back to Date.now() / 1000

fixes #30315

@dcrousso
Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Microsoft"

Comment thread packages/playwright-core/src/server/har/harTracer.ts
this._requestTimestamp = timestamp;
}

private _normalizeTimestamp(timestamp: number | undefined): number {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to _toWalltime?

}

setTimestampBaseline(wallTime: number, timestamp: number) {
this._requestWallTime = wallTime;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In createHarEntry we only use startedDateTime and _monotonicTime both of which are computed on the playwright end at the moment the entry is created. Maybe it's good enough to have the same for the websocket frames?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i figured since both Chromium and WebKit already have these values we might as well use them

but yeah if you feel strongly that we shouldn't then i absolutely can remove

Comment thread tests/library/har.spec.ts Outdated
const wsEntry = log.entries.find(e => e.request.url === wsUrl)!;
expect(wsEntry).toBeTruthy();
expect(wsEntry.response.status).toBe(101);
expect((wsEntry as any)._resourceType).toBe('websocket');
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cast the entry once to the har type to avoid repetitive as any

Copy link
Copy Markdown
Member

@yury-s yury-s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of content: 'attach' we need to store the content as external resources, let's also add a test for that.

return;

const pageEntry = this._createPageEntryIfNeeded(page);
const harEntry = createHarEntry(pageEntry?.id, 'GET', url, page.mainFrame().guid, this._options);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll need to populate headers too

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i realized that after i uploaded this PR 😅

working on that right now

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch from 91ad605 to 9f3d97c Compare May 27, 2026 21:14
@dcrousso dcrousso requested a review from yury-s May 27, 2026 21:15
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch from 9f3d97c to b423303 Compare May 27, 2026 22:24
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch from b423303 to 92e554b Compare May 27, 2026 23:21
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch 3 times, most recently from fc6a59c to bd42bf8 Compare May 28, 2026 22:18
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch from bd42bf8 to 4d9e532 Compare May 28, 2026 22:25
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

frameId: this._frameId,
wsid: webSocketSerialID + '',
opcode: frame.opCode,
data: frame.opCode !== 1 ? btoa(frame.payload) : frame.payload,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should go into playwright-browsers

wsid: string;
opcode: number;
data: string;
timestamp: number;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is regenerated during firefox rolls, no manual changes please

}

private _buildSessionListeners(session: WVSession): RegisteredListener[] {
const setCookieSeparator = process.platform === 'darwin' ? ',' : 'playwright-set-cookie-separator';
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this will work in stock webkit as the constant is a part of our patch: https://wksearch.azurewebsites.net/?path=%2Fhome%2Fjoe%2Fwebkit%2FSource%2FWebCore%2Fplatform%2Fnetwork%2FHTTPHeaderMap.cpp&line=242 We can probably address it later, but let's reuse it in WVInterceptableRequest, simular to WKPage.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooohhh good point this is using system WebKit

Comment thread packages/playwright-core/src/server/webkit/wkPage.ts Outdated
Comment thread packages/playwright-core/src/server/frames.ts Outdated
}
(request as any)[this._entrySymbol] = harEntry;
// In Firefox, WebSockets have additional events once opened.
// In Chromium and WebKit, WebSockets have an entirely different lifecycle and won't reach this.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should unify this logic across browsers, either add a synthetic opened event or merge opened logic into onWebSocketResponse or something else. We should be able to hide this specifics from the clients. Or maybe I don't understand why this comment is here despite the code being the same for all browsers.

};

constructor(context: contexts.BrowserContext, frame: frames.Frame | null, serviceWorker: pages.Worker | null, redirectedFrom: Request | null, documentId: string | undefined,
constructor(context: contexts.BrowserContext, frame: frames.Frame | null, serviceWorker: pages.Worker | null, redirectedFrom: Request | null, requestId: string, documentId: string | undefined,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since requestId is a required parameter, let's place it before all optional ones.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically none of these are actually optional as they explicitly state the allowed types instead of using ? (e.g. documentId: string | undefined vs documentId?: string)

regardless, i wasnt sure about the ordering here since there are other required properties (e.g. url, resourceType, method, etc.) that come after the optional ones

i figured id put it near the other ID parameter but i can move it earlier if you think that would read better

Comment thread packages/playwright-core/src/server/network.ts Outdated
@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch from 4d9e532 to b52e342 Compare May 29, 2026 01:36
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch from b52e342 to 1c3d64f Compare May 29, 2026 20:16
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch from 1c3d64f to 64d3c80 Compare May 29, 2026 21:27
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


const pageEntry = this._createPageEntryIfNeeded(page);
const harEntry = createHarEntry(pageEntry?.id, 'GET', url, page.mainFrame().guid, this._options);
harEntry._resourceType = 'websocket';
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we populate it for other types from Request.resourceType and maybe place to harEntry.request._resourceType (to match playwright api) or is it not worth it?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i can do that in a followup

}),
eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Close, () => {
if (this._started)
this._delegate.onEntryFinished(harEntry);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to remove listeners on this event? my concern is the case when the page creates/closes many web sockets, all these listeners will stay in _eventListeners map till har stops, buy perhaps it's fine.

Comment thread packages/trace/src/har.ts

export type WebSocketMessage = {
type: 'send' | 'receive';
time: number;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp? there is already time field in har that means duration which might be confusing.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree that it's confusing, but this matches how Chrome does it <https://developer.chrome.com/blog/new-in-devtools-76#websocket>

Comment thread tests/library/har.spec.ts Outdated
});
});

it.describe('WebSocket', () => {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: consider putting it in a separate file

add an entry for each `WebSocket` when generating a `.har`

also include data for each sent/received frame in a custom property `_webSocketMessages` (for more info see <https://developer.chrome.com/blog/new-in-devtools-76#websocket>)

both Chrome and WebKit already capture the `wallTime` for the initial request and a `timestamp` for each subsequent message (i.e. diff the `timestamp` relative to the initial `timestamp` and add the `wallTime` in order to determine the current `wallTime`)

unfortunately Firefox does not have this so fall back to `Date.now() / 1000`
@dcrousso dcrousso force-pushed the drousso/feat-har-websocket branch from 64d3c80 to ea79b41 Compare May 30, 2026 17:29
@github-actions
Copy link
Copy Markdown
Contributor

Test results for "MCP"

7233 passed, 1113 skipped


Merge workflow run.

@github-actions
Copy link
Copy Markdown
Contributor

Test results for "tests 1"

1 failed
❌ [firefox-library] › library/har-websocket.spec.ts:96 › should include websocket messages @firefox-ubuntu-22.04-node20

1 flaky ⚠️ [chromium-library] › library/beforeunload.spec.ts:130 › should support dismissing the dialog multiple times `@chromium-ubuntu-22.04-node24`

44068 passed, 864 skipped


Merge workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: include websocket traffic in har file

2 participants