Skip to content

feat(offload): stream images and retained geometry on 3DS and PSP - #377

Draft
doodlewind wants to merge 20 commits into
mainfrom
feat/offload-images
Draft

feat(offload): stream images and retained geometry on 3DS and PSP#377
doodlewind wants to merge 20 commits into
mainfrom
feat/offload-images

Conversation

@doodlewind

@doodlewind doodlewind commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Behavior

Tiled PocketJS apps can hand image decoding and vector preparation to a desktop while retaining local camera movement and GPU rendering. Binary pixels and geometry stay outside guest JavaScript. The same resource collections support 3DS over paired LAN and PSP over PSPLINK USB; Pocket Map demonstrates both with OSM vectors and a local Hyrule raster atlas.

Resource and rendering contracts

  • createOffloadImageCollection / createOffloadMeshCollection own upload, cost admission, raw-response release and resident disposal. ResourceImage / ResourceMesh borrow native handles and expose component fallbacks. Merged view demand, retry, invalidation and eviction apply to both.
  • Demand accessors are sampled every frame, including nonreactive planners. Unchanged scalar snapshots now skip union reconstruction and cache reconciliation; mutations to a reused demand array, owner cleanup and clear still take effect.
  • Image and mesh replies share eight bounded native staging slots. Cancelled sent work retains transport credit until a response or session change. Maximum-size transfers, mismatched replies, materialization failures and throwing consumers release staging.
  • PMH1 contains at most 4,096 quantized vertices and 2,048 indexed colored triangles (36,880 wire bytes). Native validation produces generation-tagged handles. Opaque mesh Views emit ten-word handle/affine/clip commands. 3DS PICA and PSP GE transform retained GPU vertices without rebuilding triangles each frame; other backends and translucent Views keep bounded CPU-clipped TRI output.
  • The 3DS vertex pool is capped at 8 MiB; PSP GE allocations are capped at 4 MiB, charged in allocator size classes. Both include retired buffers until the preceding GPU frame completes. Ui.take_texture() invalidates a texture handle while transferring storage to a backend for the same delayed-release rule.

PSP USB execution

connectOffloadUsbProvider() mounts an app-specific directory below the root served by usbhostfs_pc. It runs the same isolated capability worker module as the LAN provider.

A lower-priority native thread performs all host0 file operations. The UI only exchanges preallocated slots: at most one submit, completion and binary upload per frame. The worker allocates no QuickJS, UI or Rust heap objects. Each slot has a 4 KiB request, 64-byte header and 128 KiB response buffer, with release/acquire ownership transitions. Mac epoch, PSP boot nonce, sequence and checksum fence incomplete or obsolete replies. Physical USB access replaces LAN pairing for this adapter.

Eight capability calls bound Mac work. A 12-second execution timeout terminates the subprocess and only releases credit after exit. The UI expires stale contact after three seconds even while the worker is blocked in a USB call, retaining materialized content for cached navigation. Pocket YouTube provided the host0 reference; maps need independent tile identities and completion ownership instead of replacing whole video frames.

PSP offload builds also calibrate a stable analog center in a bounded startup window and support SELECT recentering. This fixes the test PSP's untouched raw (162,133) nub without changing ordinary PSP demo input. The old synchronous DevTools file probe is disabled for these builds.

Continuous input and app boundaries

inputDeltaSeconds() exposes bounded host sampling time as recorded frame data. Both 3DS and PSP supply the seventh frame argument; tape v4 preserves it, and older tapes retain nominal timing. A 66.666 ms cap bounds resume movement without changing virtual timers, core ticks or resource budgets.

Tile world bounds and source/display LOD stay explicit. createTileIntent preserves direction across gesture lifts and planTileWindow supports a bounded forward corridor. The application decides loading importance, prediction and provider policy. PBF parsing, styling, simplification, tessellation and label selection remain in Pocket Map's Mac worker; no map parser or app state enters the framework.

Validation

  • Final targeted resource/offload/clock/tile suites: 71 tests, 522 assertions, passing. USB integration covers record bounds, JSON/image replies, stale-slot reuse, malformed requests and executor crash/recovery. New path-filtered CI runs USB and native packet/analog tests.
  • Rust core: 133 tests, passing, including retained texture ownership and stale-handle reuse. Native packet validation: one test; analog calibration: three tests. Root TypeScript passes.
  • Native C socket tests cover maximum-size images/meshes with thread/process providers under sanitizers. Wasm core and production 3DS/PSP builds pass with pinned toolchains.
  • Real PSP scripted checkpoints cover search/keyboard, pan, zoom, bookmark create/delete, Hyrule and OSM. A suspended usbhostfs_pc blocked transport for over five seconds while the UI thread continued running; cached rendering and communication recovered. These are scripted hardware receipts, not physical-button acceptance.
  • Same settled PSP SF scene: approximately 33.5 → 18.1 → 14.3 ms CPU-side frame work after unchanged-demand and stationary-app optimizations; stable intervals reach 120 frames per two seconds. Loading and component mounting still cause longer frames. Debugger samples are not an uncontaminated frame-time percentile benchmark.
  • The 3DS timing binary was uploaded and read back byte-for-byte; settled intervals reach 120 frames per two seconds. Compiled held-stick replay changes old 320/160/160 pixels per second under 60/30/mixed timing to approximately 320 in all three cases. Physical sustained-input acceptance remains pending.

Limits and related app

The PSP adapter targets an app-specific embedded executable under PSPLINK; it does not bootstrap the USB driver from XMB, route multiple apps or add video playback. Offloading prevents host0 waits from blocking UI; it does not eliminate JavaScript, mounting, GC or GPU cost. There is no continuous 60 fps claim during loading, no device MVT parser, Media Engine work, or SDF text shader.

PSP ownership and transport documentation. Application implementation, hardware screenshots and installation evidence: pocket-map#1.

Keep this PR Draft and unmerged, as requested.

@doodlewind doodlewind changed the title feat(offload): stream bounded image resources for tiled viewports feat(offload): stream bounded images and prepared geometry for tiled views Sep 7, 2026
@doodlewind doodlewind changed the title feat(offload): stream bounded images and prepared geometry for tiled views feat(offload): stream images and retained geometry on 3DS and PSP Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant