You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adds a general fp32 2D convolution kernel — direct and transposed — to the WebGPU backend, enabling the patch-embed / conv-stem and spatial-downsample layers of vision encoders (Florence-2 DaViT, SigLIP) to run GPU-accelerated in the browser.
Problem — aten.convolution.default had no WebGPU kernel, so any model with a convolutional stem or downsample block (the DaViT patch-embed and its strided downsample convs) could not lower into the delegate and broke the graph. Both the direct and the transposed forms serialize to the same aten.convolution.default op (distinguished only by the transposed arg), so a single handler has to cover both.
Solution
Before: no conv kernel — a convolution in the exported graph fell out of the delegate.
After: one conv2d_impl handler (registered once for aten.convolution.default) routes to three WGSL kernels by shape: conv2d_vec4.wgsl (vec4 fast path over input channels), conv2d.wgsl (scalar general path), and conv_transpose2d.wgsl (gather-form transposed conv). The transposed arg folds the transpose path into the same registration (a second WEBGPU_REGISTER_OP(aten.convolution.default) would be silently dropped), and among the non-transposed kernels the host picks vec4 vs scalar from the input-channels-per-group count.
Implementation
Direct conv is one GPU thread per (b, oc, oh, ow) output element, looping over input-channel-per-group then (kh, kw), with continue guards skipping out-of-bounds padded taps; conv2d.wgsl accumulates scalar input[...] * weight[...].
The vec4 path is selected only when icpg % 4 == 0 (use_vec4 in the handler); because NCHW's channel dim has stride IH*IW (not memory-contiguous), conv2d_vec4.wgsl gathers four strided scalar loads into a vec4<f32> and uses dot(in4, w4) — a register-packing vec4, not a coalesced load — cutting the input-channel loop trip count 4x. A real RGB stem (icpg=3) correctly falls to the scalar path.
conv_transpose2d.wgsl implements the scatter-inversion as a gather: for each tap (kh, kw) an input row contributes only when (oh + pH - kh*dH) is divisible by sH and in range; weight is read in the un-flipped torch transposed layout [IC, OC/groups, KH, KW].
All shape/stride/pad/dilation/groups constants ride in an 80-byte ConvParams uniform (16-byte-aligned); the handler parses stride/padding/dilation int-lists via utils::parse_hw (broadcasting a single value to both spatial dims).
Dispatch uses the shared adaptive-grid helper utils::compute_dispatch_grid (workgroup clamped to the device max, default 256) with a 2D spill past the 65535 per-dimension ceiling; the stride_x override constant lets the shader decode i = gid.y*stride_x + gid.x. Pipeline creation, the uniform upload, the grid override constants, and the optional-bias binding go through the shared utils::make_compute_pipeline, utils::make_uniform, utils::make_grid_constants, and utils::make_optional_binding helpers (a 4-byte dummy storage satisfies the bias binding when bias is None, gated in WGSL by has_bias).
Mirrors Vulkan backends/vulkan/runtime/graph/ops/impl/Convolution.cpp (its Conv2dMethod specializations Depthwise / Pointwise / SlidingWindow / Transposed collapse here into the vec4 / scalar / transpose routing).
Constraints — fp32 only (bails if nbytes != numel * sizeof(float)); 4D input/weight/output in NCHW; groups must divide both IC and OC and weight.dims[1] must equal IC/groups; output element count must fit uint32 for the (up to 2D) dispatch; non-zero output_padding is rejected on the non-transposed path (and output_padding >= stride on the transposed path); the fused et_vk.conv_with_clamp variant is not handled and would error clearly at load.
If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.
To add a label, you can comment to pytorchbot, for example @pytorchbot label "release notes: none"
meta-claBot
added
the
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Adds a general fp32 2D convolution kernel — direct and transposed — to the WebGPU backend, enabling the patch-embed / conv-stem and spatial-downsample layers of vision encoders (Florence-2 DaViT, SigLIP) to run GPU-accelerated in the browser.
Problem —
aten.convolution.defaulthad no WebGPU kernel, so any model with a convolutional stem or downsample block (the DaViT patch-embed and its strided downsample convs) could not lower into the delegate and broke the graph. Both the direct and the transposed forms serialize to the sameaten.convolution.defaultop (distinguished only by thetransposedarg), so a single handler has to cover both.Solution
conv2d_implhandler (registered once foraten.convolution.default) routes to three WGSL kernels by shape:conv2d_vec4.wgsl(vec4 fast path over input channels),conv2d.wgsl(scalar general path), andconv_transpose2d.wgsl(gather-form transposed conv). Thetransposedarg folds the transpose path into the same registration (a secondWEBGPU_REGISTER_OP(aten.convolution.default)would be silently dropped), and among the non-transposed kernels the host picks vec4 vs scalar from the input-channels-per-group count.Implementation
(b, oc, oh, ow)output element, looping over input-channel-per-group then(kh, kw), withcontinueguards skipping out-of-bounds padded taps;conv2d.wgslaccumulates scalarinput[...] * weight[...].icpg % 4 == 0(use_vec4in the handler); because NCHW's channel dim has strideIH*IW(not memory-contiguous),conv2d_vec4.wgslgathers four strided scalar loads into avec4<f32>and usesdot(in4, w4)— a register-packing vec4, not a coalesced load — cutting the input-channel loop trip count 4x. A real RGB stem (icpg=3) correctly falls to the scalar path.conv_transpose2d.wgslimplements the scatter-inversion as a gather: for each tap(kh, kw)an input row contributes only when(oh + pH - kh*dH)is divisible bysHand in range; weight is read in the un-flipped torch transposed layout[IC, OC/groups, KH, KW].ConvParamsuniform (16-byte-aligned); the handler parsesstride/padding/dilationint-lists viautils::parse_hw(broadcasting a single value to both spatial dims).utils::compute_dispatch_grid(workgroup clamped to the device max, default 256) with a 2D spill past the 65535 per-dimension ceiling; thestride_xoverride constant lets the shader decodei = gid.y*stride_x + gid.x. Pipeline creation, the uniform upload, the grid override constants, and the optional-bias binding go through the sharedutils::make_compute_pipeline,utils::make_uniform,utils::make_grid_constants, andutils::make_optional_bindinghelpers (a 4-byte dummy storage satisfies the bias binding when bias isNone, gated in WGSL byhas_bias).backends/vulkan/runtime/graph/ops/impl/Convolution.cpp(itsConv2dMethodspecializationsDepthwise/Pointwise/SlidingWindow/Transposedcollapse here into the vec4 / scalar / transpose routing).Constraints — fp32 only (bails if
nbytes != numel * sizeof(float)); 4D input/weight/output in NCHW;groupsmust divide bothICandOCandweight.dims[1]must equalIC/groups; output element count must fituint32for the (up to 2D) dispatch; non-zerooutput_paddingis rejected on the non-transposed path (andoutput_padding >= strideon the transposed path); the fusedet_vk.conv_with_clampvariant is not handled and would error clearly at load.Co-authored-with: Claude Code.
Differential Revision: D110836669