D2M: Mon, Jun 15, 2026 (v6.15.26)

[github-actions]

Summary

This PR lands a large batch of features across iOS, macOS, and the Go runtime: on-device WebKit browsing (BrowseSkill), HTML story/infographic generation ported to macOS, Google Workspace integration (Drive, Gmail, Calendar), VM-backed cron agents, a music mini-player banner, iMessage-style swipe-to-reveal timestamps, Sparkle auto-update for LoopMac, and a background handoff mechanism that migrates an in-flight local inference turn to a Loop Runner when the app backgrounds. It also fixes several audio ducking/TTS cleanup regressions, Apple on-device fallback state leaks, Claude tool-call text bleeding into the streaming bubble, and playlist/album playback using library IDs.

Changes by area

Browse skill (iOS-only, new)

• BrowseSession: drives a non-persistent WKWebView offscreen; full JS bridge (click, type, scroll, wait_for, querySelectorAll, eval_js, finish); per-step screenshot + DOM snapshot persisted to workspace://browse/<id>/ as a replay bundle.
• BrowseGenerationService: coordinates sessions, streams per-step attachment updates to MessagingVC.
• BrowsePlayerVC: full-screen viewer with live-mirror mode (read-only, gesture-swallowing overlay) and scrubbable replay mode (slider + play/pause + per-frame DOM/screenshot/share actions).
• BrowseSkill: browse user-facing tool + internal browse_action tool for the nested model loop; registered in AgentHarness, SkillDispatcher, ToolRouter.

Stories / HTML infographic

• StorySkill, StoryGenerator, StoryGenerationService, StoryAttachment, StoryBundledTemplates, StoryPlayerVC/View ported to #if os(iOS) || os(macOS).
• StoryMacUI.swift: StoryBubbleView card + StoryPlayerWindowController WKWebView player for macOS.
• ConversationWindowController wired as StorySkillHost on macOS.
• story- prefix added to message-filter guards in AnthropicChat, OpenAIChat, and ConversationTitleService.

Google Workspace integration (new)

• GoogleWorkspaceClient: shared networking layer with typed errors including token_expired on 401.
• GoogleDriveSkill (list/get/read/create), GoogleGmailSkill (search/get/send), GoogleCalendarSkill (list/create events).
• KeyStore: google_workspace_access_token, optional refresh/client tokens.
• IntegrationsVC: Google Workspace row with connected status and Revoke/Remove.
• Registered in AgentHarness, SkillDispatcher, ToolRouter, VoiceLoopCoordinator.

VM cron agents (new)

• VMCronSkill, VMCronManager, VMCronPoller, VMCronTasksVC: schedule/list/delete recurring agent jobs that run on the SSH VM and push results back.
• VMAgentRuntime, RunnerTurnApplier, BackgroundTurnRunner, RunnerProvisioner: pipeline for running and applying runner turns, including deduplication.
• AppDelegate: handles runner_turn push type — reconciles reply into conversation, suppresses redundant banner, cold-start tap routing via PendingConversationOpen.
• Linux runner binaries (loop-runner-linux-amd64.gz, loop-runner-linux-arm64.gz) bundled as dataset assets.

Background handoff to Loop Runner

• LocalInferenceController: tracks the active URLSessionDataTask for each provider (Anthropic, OpenAI, Fireworks) so it can be hard-cancelled.
• MessagingVC.handoffInFlightTurnIfEligible(): on backgrounding, cancels local inference, marks turn abandoned, submits to runner via LoopRunnerPoller.submitHandoff; posts a local notification when the reply arrives; falls back to an error notice if handoff fails.
• discardIfHandedOff guard in every Cloud.connection.chat completion block.

Audio ducking / TTS cleanup

• iOS MessagingVC: deactivateAudioSession() (with .notifyOthersOnDeactivation) called from stopSpeaking(), audioPlayerDidFinishPlaying, decode error, speechSynthesizer(_:didFinish:), and DeepgramTTS callbacks.
• All TTS category options switched from []/.mixWithOthers to .duckOthers.
• visionOS VisionVoiceCoordinator: matching deactivation calls after TTS and recording teardown.
• macOS DeepgramTTS: mixer output tap removed in finishAfterDrain() and the error path.
• MusicController: duck music synchronously before earcon in switchToRecordingState(); resume token guards against orphaned auto-resumes; new play_music/set_music_mood calls mid-session re-pause via reduckIfVoiceSessionActive().

Music mini-player banner

• MusicMiniPlayerView: compact pill and expanded card states with play/pause, skip, progress bar, album art, and deep-link.
• TopBannerScrollView: horizontally scrollable container below the sub-agent status bar hosting the mini-player.
• Auto-dismisses on stop, auto-minimizes on voice recording, swipe-down to collapse.

STT engine badge

• VoiceLoopCoordinator gains STTEngine enum and activeSTTEngine property.
• MessageBox publishes engine on start and on Deepgram→SFSpeech fallback.
• iOS: monospaced pill beside transcribingLabel; macOS: appended to recorder bar placeholder.
• sttEngine persisted to the model column for user rows and restored on load; shown as a dictation byline under user bubbles.

iMessage-style swipe-to-reveal timestamps

• MessagingCell.timeLabel: pinned just past the right edge of contentView.
• setTimeRevealOffset(_:) translates contentView; layoutSubviews re-applies the transform after UITableViewCell resets the frame each layout pass.
• MessagingVC pan gesture (handleTimeRevealPan) drives all visible cells in lock-step with rubber-banding.
• AgentLargeView.setChromeHidden(_:animated:) added; chrome held back until the orb lands to fix "labels appearing before orb arrives" visual.

Sparkle auto-update (macOS)

• Sparkle 2.9.2 added as a Swift package dependency for LoopMac.
• release-mac.yml overhauled from commented-out stub to a fully active workflow: build → sign → notarize app → notarize DMG → package Sparkle zip (post-staple) → generate_appcast with EdDSA signature → publish per-build release (DMG + zip) → upsert appcast release with appcast.xml.

Apple on-device fallback fixes

• Both Apple Task{} fallback paths (primary send and tool-loop) now call ActiveRequestTracker.markIdle, clear streamingPartial, reset VoiceLoopCoordinator to .idle, and surface a user-facing error message with the error earcon.
• Previously empty catch blocks filled in for both paths.

Claude tool-call streaming fix

• AnthropicStreamReader: sawToolUse flag suppresses onDelta for text_delta events once any tool_use block starts; pre-tool reasoning text is still accumulated in contentBuffer for the disclosure but no longer leaks into the live streaming bubble.
• AnthropicStreamReaderTests added covering text-only, tool-call assembly, delta suppression, connectivity error, and usage parsing.

Playlist/album playback fix

• Library playlist IDs (p.), album IDs (l.), and song IDs (i.) now use MusicLibraryRequest instead of MusicCatalogResourceRequest.
• queue_mode: append now works for albums and playlists, not just songs.

APNs / push registration

• PushBridge: runtime-discovered seam (mirrors AppSignals) that passes the APNs token to a private sender when present.
• APNs environment resolved from embedded provisioning profile; token registered immediately after permission grant.

Geocoding skill (new)

• GeocodingSkill: converts an address or place name to lat/lon using CLGeocoder; registered in AgentHarness, SkillDispatcher, VoiceLoopCoordinator.

AgentMail skill (new)

• AgentMailClient + AgentMailSkill: send/receive email via the AgentMail API; keys wired through KeyStore and Info.plist.

Managed-mode / AppFlags

• AppFlags.isManaged reads LOOP_FLAG from Info.plist; when set, pins TTS provider to ElevenLabs Flash v2.5, shows "Managed" in the backend indicator (non-interactive), drops the provider picker from the speaker menu, and allows managed onboarding to speak its messages.
• Secrets_managed.xcconfig added to .gitignore; Config.xcconfig switches include to Secrets_managed.xcconfig.

SSH / runner improvements

• SSHConfigStore, SSHConnectionsVC, SSHSettingsVC: multiple SSH profiles, passphrase support, managed pre-seeding from Info.plist.
• LoopRunnerSSHClient, LoopRunnerPoller, LoopRunnerClient: handoff submission, backstop catch-up polling for missed pushes.
• RunnerEditVC, RunnerModels, RunnerProvisioner added.

Go runtime

• agent.go, handlers.go, storage.go, main.go, config.go: managed-secrets support, improved cron handling, new integration test coverage.

Notable risks

• Background handoff is destructive: local inference is hard-cancelled and the turn marked abandoned before the runner submit completes. If submitHandoff fails and the runner is unreachable, the user sees a notice but has permanently lost the local result for that turn.
• BrowseSession parks a WKWebView offscreen in the key window: if keyWindow() returns nil (e.g. during early launch or on multi-scene), the web view is never attached and WebKit may not render/snapshot correctly, silently producing empty frames.
• Secrets_managed.xcconfig replaces Secrets.xcconfig as the build config include: developers who have Secrets.xcconfig locally must rename or symlink; the old filename is no longer read by Config.xcconfig.
• Sparkle EdDSA key (SPARKLE_ED_PRIVATE_KEY) is a new required CI secret: the release workflow will fail on main pushes until it is added.
• sttEngine piggybacked on the model DB column for user rows: any existing user-role rows with a non-empty model value (e.g. from a future schema change) would be misread as a dictation byline.
• storyCardView, browseCardView constraints grow with each prepareForReuse call if NSLayoutConstraint.deactivate does not fully remove them — the lazy-view superview == nil guard prevents duplicate additions, but constraint deactivation on recycled cells for the story/browse paths should be verified under heavy scrolling.

---

Auto-generated by .github/workflows/auto-develop-pr.yml

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
loop-harness	Ready	Preview, Comment	Jun 15, 2026 4:54pm