Skip to content

perf(canvas): native hosts and GPU snapshots for drawable views - #255

Merged
triniwiz merged 1 commit into
masterfrom
feat/html-in-canvas-gpu
Nov 18, 2026
Merged

triniwiz merged 1 commit into
masterfrom
feat/html-in-canvas-gpu

Conversation

@triniwiz

@triniwiz triniwiz commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Stacked on #254.

The drawable children of a content="drawable" canvas are now held by a native host view, so per-frame work, touch routing and capture stay native, and snapshots are drawn from the GPU.

Change

Android: NSCDrawableHost (Kotlin, in the AAR)

  • Lays the children out with NativeScript's own CommonLayoutParams measure/layout (looked up at runtime: widgets ships with the app) and never draws them.
  • Marks a view dirty from onDescendantInvalidated; below API 26, idle views are polled.
  • Routes touches from a geometry table that JS sets only when geometry changes.
  • API 29+: records each changed view into a RenderNode and renders it with HardwareRenderer into an ImageReader, waiting on the image's fence on API 33+. Below 29 it draws into a Bitmap.
  • 2D draws the HardwareBuffer through the EGLImage path video already uses; WebGPU imports it as a native texture.

iOS: NSCDrawableHost (Swift, in CanvasNative)

  • Hit testing and the layer transforms that keep UIKit's touch coordinates in step with where a view is drawn, re-synced with layout every frame.
  • Renders into an IOSurface that 2D (canvas_native_ios_context_draw_external_metal_texture) and WebGPU read as a Metal texture.
  • A layer-tree signature taken every frame catches geometry, property, contents and animation changes without rendering. A quarter-size probe render, checked less often while nothing changes, catches the rest before any full-size render.

JS walks the NativeScript tree for drawable views and makes one call to the host per frame. drawElementImage no longer uploads the snapshot's bytes on every draw, and the geometry it sets on each draw only reaches the host when it changed. WebGL still reads the snapshot back on the CPU.

Verified

  • iOS 26.4 simulator: htmlincanvas 24/24 (including a new test that the snapshot draws through Metal), 2d 182/182, canvassource 18/18, offscreen 64/64, lifecycle 17/17, webgl 96/96.
  • Android emulator (Pixel 9 Pro, API 37), arm64 AAR from this branch: htmlincanvas 24/24 on three runs (including a new test that the snapshot is a HardwareBuffer drawn through EGL), 2d 182/182, webgl 96/96. lifecycle fails the same present-timing tests as a master build with the same AAR.
  • canvas-html demo on the emulator: taps and typing reach the rotated form through the native touch routing, at 60 fps.
  • iOS simulator, demo form at 840x917 px, idle: the paint loop averages about 0.3 ms a frame against 1.1 ms in feat(canvas): draw NativeScript views into a canvas (HTML-in-Canvas) #254, and its largest frame is 4–7 ms against about 9 ms every 10 frames.

@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8585026d-49da-4bca-9fd6-082694cc20dd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@triniwiz
triniwiz force-pushed the feat/html-in-canvas-gpu branch 3 times, most recently from 1b9e6d5 to 51887cb Compare November 18, 2026 14:05
@triniwiz
triniwiz force-pushed the feat/html-in-canvas-gpu branch from 51887cb to 9d438bc Compare November 18, 2026 15:05
Base automatically changed from feat/html-in-canvas to master November 18, 2026 15:30
The drawable children of a content="drawable" canvas are now held by a
native host view, so the per-frame work, touch routing and capture no
longer cross into JS:

- Android: NSCDrawableHost (Kotlin) lays the children out with
  NativeScript's own CommonLayoutParams helpers, tracks invalidations
  from onDescendantInvalidated, and routes touches from a geometry table
  JS sets only when geometry changes. On API 29+ it records each changed
  view into a RenderNode and renders it with HardwareRenderer into an
  ImageReader, waiting on the image's fence on API 33+. 2D draws the
  HardwareBuffer through the existing EGLImage path video uses, and
  WebGPU imports it as a native texture.
- iOS: NSCDrawableHost (Swift) does the hit testing and keeps the layer
  transforms in sync with layout. It renders into an IOSurface that 2D
  and WebGPU read as a Metal texture. A layer-tree signature taken every
  frame catches most changes without rendering; a quarter-size probe,
  checked less often while nothing changes, catches the rest.

JS walks the NativeScript tree for drawable views and makes one call to
the host per frame. drawElementImage no longer uploads the snapshot's
bytes on every draw. WebGL still reads the snapshot back on the CPU.
@triniwiz
triniwiz force-pushed the feat/html-in-canvas-gpu branch from 9d438bc to 57b9484 Compare November 18, 2026 15:31
@triniwiz
triniwiz merged commit 5656bea into master Nov 18, 2026
25 of 28 checks passed
@triniwiz
triniwiz deleted the feat/html-in-canvas-gpu branch November 18, 2026 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant