Skip to content

[ExecuTorch][WebGPU] Add optimized conv2d op#20849

Open
JCNTH wants to merge 1 commit into
gh/JCNTH/24/basefrom
gh/JCNTH/24/head
Open

[ExecuTorch][WebGPU] Add optimized conv2d op#20849
JCNTH wants to merge 1 commit into
gh/JCNTH/24/basefrom
gh/JCNTH/24/head

Conversation

@JCNTH

@JCNTH JCNTH commented Jul 10, 2026

Copy link
Copy Markdown

Stack from ghstack (oldest at bottom):

Adds a general fp32 2D convolution kernel — direct and transposed — to the WebGPU backend, enabling the patch-embed / conv-stem and spatial-downsample layers of vision encoders (Florence-2 DaViT, SigLIP) to run GPU-accelerated in the browser.

Problematen.convolution.default had no WebGPU kernel, so any model with a convolutional stem or downsample block (the DaViT patch-embed and its strided downsample convs) could not lower into the delegate and broke the graph. Both the direct and the transposed forms serialize to the same aten.convolution.default op (distinguished only by the transposed arg), so a single handler has to cover both.

Solution

  • Before: no conv kernel — a convolution in the exported graph fell out of the delegate.
  • After: one conv2d_impl handler (registered once for aten.convolution.default) routes to three WGSL kernels by shape: conv2d_vec4.wgsl (vec4 fast path over input channels), conv2d.wgsl (scalar general path), and conv_transpose2d.wgsl (gather-form transposed conv). The transposed arg folds the transpose path into the same registration (a second WEBGPU_REGISTER_OP(aten.convolution.default) would be silently dropped), and among the non-transposed kernels the host picks vec4 vs scalar from the input-channels-per-group count.

Implementation

  • Direct conv is one GPU thread per (b, oc, oh, ow) output element, looping over input-channel-per-group then (kh, kw), with continue guards skipping out-of-bounds padded taps; conv2d.wgsl accumulates scalar input[...] * weight[...].
  • The vec4 path is selected only when icpg % 4 == 0 (use_vec4 in the handler); because NCHW's channel dim has stride IH*IW (not memory-contiguous), conv2d_vec4.wgsl gathers four strided scalar loads into a vec4<f32> and uses dot(in4, w4) — a register-packing vec4, not a coalesced load — cutting the input-channel loop trip count 4x. A real RGB stem (icpg=3) correctly falls to the scalar path.
  • conv_transpose2d.wgsl implements the scatter-inversion as a gather: for each tap (kh, kw) an input row contributes only when (oh + pH - kh*dH) is divisible by sH and in range; weight is read in the un-flipped torch transposed layout [IC, OC/groups, KH, KW].
  • All shape/stride/pad/dilation/groups constants ride in an 80-byte ConvParams uniform (16-byte-aligned); the handler parses stride/padding/dilation int-lists via utils::parse_hw (broadcasting a single value to both spatial dims).
  • Dispatch uses the shared adaptive-grid helper utils::compute_dispatch_grid (workgroup clamped to the device max, default 256) with a 2D spill past the 65535 per-dimension ceiling; the stride_x override constant lets the shader decode i = gid.y*stride_x + gid.x. Pipeline creation, the uniform upload, the grid override constants, and the optional-bias binding go through the shared utils::make_compute_pipeline, utils::make_uniform, utils::make_grid_constants, and utils::make_optional_binding helpers (a 4-byte dummy storage satisfies the bias binding when bias is None, gated in WGSL by has_bias).
  • Mirrors Vulkan backends/vulkan/runtime/graph/ops/impl/Convolution.cpp (its Conv2dMethod specializations Depthwise / Pointwise / SlidingWindow / Transposed collapse here into the vec4 / scalar / transpose routing).

Constraints — fp32 only (bails if nbytes != numel * sizeof(float)); 4D input/weight/output in NCHW; groups must divide both IC and OC and weight.dims[1] must equal IC/groups; output element count must fit uint32 for the (up to 2D) dispatch; non-zero output_padding is rejected on the non-transposed path (and output_padding >= stride on the transposed path); the fused et_vk.conv_with_clamp variant is not handled and would error clearly at load.

Co-authored-with: Claude Code.

Differential Revision: D110836669

[ghstack-poisoned]
@pytorch-bot

pytorch-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20849

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 1c5c7b2 with merge base aceeb40 (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants