Skip to content

Integrate Butterchurn visualizer and UI controls#14030

Merged
julianbaker merged 9 commits intomainfrom
visualizer-milkdrop
Apr 1, 2026
Merged

Integrate Butterchurn visualizer and UI controls#14030
julianbaker merged 9 commits intomainfrom
visualizer-milkdrop

Conversation

@julianbaker
Copy link
Copy Markdown
Member

Replace the old WebGL visualizer with a Butterchurn (Milkdrop) based engine and add visualizer UI/controls. Adds a butterchurn visualizer singleton (utils/visualizer/butterchurnVisualizer.ts), TypeScript typings for butterchurn presets, new presets list, WebGL2 capability check, and hotkeys for nav/randomize. Introduces a new options popup (auto-advance / auto-hide now playing), track-corner UI behavior tied to mouse activity, and CSS updates for the control chrome. Updates Redux slice/selectors to persist auto-cycle and auto-hide settings, wires analytics/open/close flows, and adjusts binding logic to use the audio player's AudioContext. Removes legacy GL visualizer files and switches package deps to butterchurn + presets in the web package.

Replace the old WebGL visualizer with a Butterchurn (Milkdrop) based engine and add visualizer UI/controls. Adds a butterchurn visualizer singleton (utils/visualizer/butterchurnVisualizer.ts), TypeScript typings for butterchurn presets, new presets list, WebGL2 capability check, and hotkeys for nav/randomize. Introduces a new options popup (auto-advance / auto-hide now playing), track-corner UI behavior tied to mouse activity, and CSS updates for the control chrome. Updates Redux slice/selectors to persist auto-cycle and auto-hide settings, wires analytics/open/close flows, and adjusts binding logic to use the audio player's AudioContext. Removes legacy GL visualizer files and switches package deps to butterchurn + presets in the web package.
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Mar 31, 2026

⚠️ No Changeset found

Latest commit: 5eff033

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Ensure the visualizer rebinds when the audio source or context changes and avoid connecting null/stale nodes. VisualizerProvider: add an audioBindKey (based on currentSrc and audioCtx) to the effect deps and skip setup when not visible. butterchurnVisualizer: guard against missing source, disconnect a previously connected node if it differs from the new one (catching errors), update connectedAudioNode, and always connect the new node. These changes prevent stale audio node hookups after track changes and handle browser quirks.
Stop loading the Extra/Extra2/MD1 preset packs and only use the main butterchurn-presets bundle to avoid lower-quality duplicates and reduce bundle/import complexity. Add a missing disconnectAudio type to the butterchurn declaration and remove explicit module declarations for the extra preset packs. Update presets loader to import only the main module and adjust Vite optimizeDeps to stop pre-bundling the removed extra packs.
@socket-security
Copy link
Copy Markdown

socket-security bot commented Mar 31, 2026

No dependency changes detected. Learn more about Socket for GitHub.

👍 No dependency changes detected in pull request

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 31, 2026

🌐 Web preview ready

Preview URL: https://audius-web-preview-pr-14030.audius.workers.dev

Unique preview for this PR (deployed from this branch).
Workflow run

Replace Left/Right arrow keycodes with comma (188) and period (190) in Visualizer.tsx so history navigation uses ',' for back and '.' for forward. This avoids conflicts with the global play/pause hotkey and clarifies key mappings (onHistoryBack/onHistoryForward).
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the web visualizer from a legacy custom WebGL implementation to a Butterchurn (Milkdrop) engine and adds new UI controls (options popup, auto-advance/auto-hide, nav + hotkeys) with Redux-backed settings.

Changes:

  • Replace legacy WebGL visualizer implementation with a Butterchurn-based singleton and lazy-loaded preset pack handling.
  • Add new visualizer UI chrome (options popup, prev/next, auto-advance + auto-hide) and mouse-activity-driven visibility behavior.
  • Update build/deps (Vite optimizeDeps + new butterchurn deps) and remove legacy visualizer/util/shader sources.

Reviewed changes

Copilot reviewed 24 out of 98 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
packages/web/vite.config.ts Pre-bundles butterchurn deps for faster dev startup.
packages/web/src/utils/visualizer/visualizer-1.js Removes legacy WebGL visualizer implementation.
packages/web/src/utils/visualizer/shaders/visualizer-1.vert Removes legacy vertex shader.
packages/web/src/utils/visualizer/shaders/visualizer-1.frag Removes legacy fragment shader.
packages/web/src/utils/visualizer/presets.ts Adds lazy-loaded preset loading + curated ordering + random selection utilities.
packages/web/src/utils/visualizer/GLAudioAnalyser.js Removes legacy GL audio analyser helper.
packages/web/src/utils/visualizer/gl-vignette-background/vert.glsl Removes vendored legacy vignette background shader source.
packages/web/src/utils/visualizer/gl-vignette-background/README.md Removes vendored legacy vignette background docs.
packages/web/src/utils/visualizer/gl-vignette-background/package.json Removes vendored legacy vignette background package metadata.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/without_basedir/node_modules/mymodule.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/without_basedir/main.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/quux/foo/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/incorrect_main/package.json Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/incorrect_main/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/foo.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/cup.coffee Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/biz/node_modules/tiv/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/biz/node_modules/grux/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/biz/node_modules/garply/package.json Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/biz/node_modules/garply/lib/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/baz/quux.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/baz/package.json Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver/bar/node_modules/foo/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/resolver_sync.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/node_path/y/ccc/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/node_path/y/bbb/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/node_path/x/ccc/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/node_path/x/aaa/index.js Removes vendored dependency test fixture.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/node_path.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/mock.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/mock_sync.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/filter.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/filter_sync.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/faulty_basedir.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/test/core.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/readme.markdown Removes vendored dependency docs.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/package.json Removes vendored dependency metadata.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/LICENSE Removes vendored dependency license.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/lib/sync.js Removes vendored dependency source.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/lib/node-modules-paths.js Removes vendored dependency source.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/lib/core.json Removes vendored dependency data.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/lib/core.js Removes vendored dependency source.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/lib/caller.js Removes vendored dependency source.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/lib/async.js Removes vendored dependency source.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/index.js Removes vendored dependency entrypoint.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/example/sync.js Removes vendored dependency example.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/example/async.js Removes vendored dependency example.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/resolve/.travis.yml Removes vendored dependency CI config.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/simple-adapter.js Removes vendored dependency source.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/README.md Removes vendored dependency docs.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/package.json Removes vendored dependency metadata.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/LICENSE Removes vendored dependency license.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/index.js Removes vendored dependency source.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/example.js Removes vendored dependency example.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/cli.js Removes vendored dependency CLI source.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/browser.js Removes vendored dependency browser stub.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/bin/glslify Removes vendored dependency bin wrapper.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/adapter.js Removes vendored dependency adapter.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/glslify/.npmignore Removes vendored dependency ignore file.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/test/runner.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/test/run.js Removes vendored dependency tests runner.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/test/reflect.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/test/compat.js Removes vendored dependency tests.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/README.md Removes vendored dependency docs.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/package.json Removes vendored dependency metadata.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/LICENSE.BSD Removes vendored dependency license.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/ChangeLog Removes vendored dependency changelog.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/bin/esvalidate.js Removes vendored dependency bin.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/esprima/bin/esparse.js Removes vendored dependency bin.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/.bin/glslify Removes vendored dependency bin link.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/.bin/esvalidate Removes vendored dependency bin link.
packages/web/src/utils/visualizer/gl-vignette-background/node_modules/.bin/esparse Removes vendored dependency bin link.
packages/web/src/utils/visualizer/gl-vignette-background/LICENSE.md Removes vendored legacy package license.
packages/web/src/utils/visualizer/gl-vignette-background/index.js Removes vendored legacy vignette background implementation.
packages/web/src/utils/visualizer/gl-vignette-background/frag.glsl Removes vendored legacy vignette fragment shader.
packages/web/src/utils/visualizer/gl-vignette-background/demo/index.js Removes vendored legacy demo code.
packages/web/src/utils/visualizer/gl-vignette-background/create-shader.js Removes vendored legacy helper.
packages/web/src/utils/visualizer/gl-vignette-background/.npmignore Removes vendored legacy ignore file.
packages/web/src/utils/visualizer/gl-line-3d.js Removes legacy line rendering helper.
packages/web/src/utils/visualizer/butterchurnVisualizer.ts Adds Butterchurn visualizer singleton + render loop + preset/history/autocycle logic.
packages/web/src/types/butterchurn-preset-packs.d.ts Adds TS typings for butterchurn + preset pack modules.
packages/web/src/pages/visualizer/VisualizerProvider.tsx Wires Butterchurn visualizer, adds options UI, mouse-driven controls, analytics flows.
packages/web/src/pages/visualizer/VisualizerProvider.module.css Updates visualizer chrome/layout CSS (controls pill, options popup, track corner fade).
packages/web/src/pages/visualizer/Visualizer.tsx Adds hotkeys for preset navigation/randomize and uses new visualizer.
packages/web/src/pages/visualizer/utils.ts Adds WebGL2 capability check helper.
packages/web/src/pages/visualizer/store/slice.ts Extends Redux slice with auto-cycle + auto-hide settings.
packages/web/src/pages/visualizer/store/selectors.ts Adds selectors for new settings with defaults.
packages/web/package.json Swaps legacy WebGL deps for butterchurn + butterchurn-presets.
package-lock.json Locks new butterchurn dependencies and updates lock metadata.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +309 to +324
const ButterchurnVisualizer = isSupported()
? {
show,
hide,
bind,
stop,
setDominantColors,
randomPreset,
historyBack,
historyForwardOrNext,
canHistoryBack,
startAutoCycle,
stopAutoCycle,
setOnHistoryChange
}
: null
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isSupported() is executed at module initialization to decide whether to export the visualizer object. In SSR / non-browser environments (the web package has src/ssr/), this can throw because window/document may be undefined (either directly or inside butterchurn's support check). Consider guarding with typeof window !== 'undefined' && typeof document !== 'undefined' (and only then calling isSupported()), or deferring the support check until show()/bind() so the module can be safely imported on the server.

Copilot uses AI. Check for mistakes.
Comment on lines 403 to 406
const dominantColors = null
const dominantColor = dominantColors
? dominantColors[0]
: { r: 0, g: 0, b: 0 }
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dominantColors is hard-coded to null, so dominantColor will always fall back to { r: 0, g: 0, b: 0 } and the conditional branch for real dominant colors is now unreachable. Either wire this back up to a real dominant-color source (as before) or remove the dead code and pass an explicit constant/default so the intent is clear.

Suggested change
const dominantColors = null
const dominantColor = dominantColors
? dominantColors[0]
: { r: 0, g: 0, b: 0 }
const dominantColor = { r: 0, g: 0, b: 0 }

Copilot uses AI. Check for mistakes.
Comment on lines +138 to +144
initPresets().then(() => {
if (presets && presetKeys.length > 0 && visualizer) {
const key = getRandomPresetKey(presets, currentPresetName)
const idx = presetKeys.indexOf(key)
loadPresetByIndex(idx >= 0 ? idx : 0, 0)
}
})
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initPresets().then(...) has no .catch(). If the dynamic import fails for any reason, this will create an unhandled promise rejection and leave the visualizer without a preset. Consider adding error handling (e.g., .catch(() => {/* fallback / disable auto-cycle */})) so failures don't surface as global unhandled errors.

Suggested change
initPresets().then(() => {
if (presets && presetKeys.length > 0 && visualizer) {
const key = getRandomPresetKey(presets, currentPresetName)
const idx = presetKeys.indexOf(key)
loadPresetByIndex(idx >= 0 ? idx : 0, 0)
}
})
initPresets()
.then(() => {
if (presets && presetKeys.length > 0 && visualizer) {
const key = getRandomPresetKey(presets, currentPresetName)
const idx = presetKeys.indexOf(key)
loadPresetByIndex(idx >= 0 ? idx : 0, 0)
}
})
.catch((error) => {
// Prevent unhandled promise rejections if preset initialization fails.
// Optionally log for diagnostics; avoid throwing to keep the visualizer usable.
// eslint-disable-next-line no-console
console.error('Failed to initialize butterchurn presets:', error)
})

Copilot uses AI. Check for mistakes.
Add a CSS class (.visualizerCursorHidden) that sets cursor: none and apply it to the visualizer root when top controls are hidden and the options menu is closed. Introduce chromeIdleHidden computed flag (true when !showControls && !optionsOpen) and include the class via cn in VisualizerProvider.tsx so the cursor is hidden during idle UI state while keeping it visible when the options menu is open.
Copy link
Copy Markdown
Member

@raymondjacobson raymondjacobson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple high level things:

  • We have code that does all the dominant color work, but it seems unused now for the visualizer. We should investigate and determine if it's impacting track load at all and if so remove it
  • If we can lazy load the butterchurn package that would be great. It's not huge, but we will pay for it on every page load. The old visualizer was lazy loaded here:
    const VisualizerProvider = lazy(() => import('./VisualizerProvider')) which is essentially disregarded in this PR
  • If you hit next track while on the visualizer screen, the app crashes

"body-scroll-lock": "4.0.0-beta.0",
"bs58": "5.0.0",
"canvas-loop": "1.0.7",
"butterchurn": "^2.6.7",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please pin these dependencies (remove the ^)

@@ -0,0 +1,326 @@
// Butterchurn (Milkdrop) visualizer engine singleton.
// Drop-in replacement for the old visualizer-1.js with the same public API shape.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment is kinda meaningless b/c visualizer-1.js was removed

Move visualizer route constants to a dedicated file and pin butterchurn deps in package.json. Lazily load butterchurn and its isSupported check via dynamic import (singleflight), making the visualizer library load-resilient and avoiding top-level imports; add robust handling for multiple AudioContexts and teardown to prevent stale nodes and WebGL/context loss. Update butterchurn types to reflect default export and isSupported module. UI/UX changes: add a visualizer pill overlay on the now-playing artwork (new CSS module + component changes), tweak VisualizerProvider styles, and implement a short "now playing" peek to briefly reveal track details after queue changes when auto-hide is enabled. Misc: refactor presets loading to be defensive, simplify render/start logic, and add various safety checks to prevent runtime errors.
Introduce guards and generation tracking to avoid connecting visualizer to audio nodes from a different AudioContext or stale async binds. Add bindGeneration and pass a generation token through loadButterchurnLib to cancel in-flight binds, a sourceBelongsToContext helper, and instanceof checks. Wrap visualizer.connectAudio in try/catch and add logic in bindAfterLibLoaded to teardown or recreate the visualizer when contexts change, and to ignore sources that don't belong to the target context. These changes prevent cross-context/disposed-node errors and race conditions when loading the butterchurn library.
Add support for showing full track title and artist in the visualizer overlay by introducing a fullTrackText prop on PlayingTrackInfo and applying a .fullTrackText CSS class (removes ellipsis, disables max-width, and lets text wrap). Update VisualizerProvider styles to set overflow: visible so wrapped text can extend outside the tile, and pass fullTrackText to the PlayingTrackInfo instance used in the visualizer.
@julianbaker julianbaker merged commit 0493991 into main Apr 1, 2026
15 checks passed
@julianbaker julianbaker deleted the visualizer-milkdrop branch April 1, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants