fix(registry/coder/modules/vscode-web): install extensions when reusing a cached copy - #1057
Conversation
…ng a cached copy Reusing a cached or pre-installed VS Code Web (use_cached=true) exited before extension installation, so both the extensions list and auto_install_extensions were skipped. Skip only the download now; extensions are installed and the server started in both the fresh and cached paths. Offline behavior is unchanged. Bump vscode-web to 1.6.2.
Module Scorecard Check
|
| Theme | Before | After |
|---|---|---|
| Presentation & Onboarding | 25 / 25 | 25 / 25 |
| Integration | 22 / 25 | 23 / 25 |
| Credential Hygiene | 20 / 20 | 20 / 20 |
| Restricted-Environment | 7 / 20 | 10 / 20 |
| Engineering Quality | 8 / 10 | 8 / 10 |
| Overall | 82 / 100 | 86 / 100 |
Full scorecard for this PR
| Presentation & Onboarding | IDE Integration | Credential Hygiene | Restricted-Environment Readiness | Engineering Quality | Overall |
|---|---|---|---|---|---|
| 25 / 25 | 23 / 25 | 20 / 20 | 10 / 20 | 8 / 10 | 86 / 100 |
Drilldown
Presentation & Onboarding — 25 / 25
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Configuration-mode examples | 12 | 12 | Multiple documented examples cover major modes: custom folder installation, extension installation, machine settings configuration, version pinning, workspace opening. Each has sensible defaults. |
| Coder-context framing | 8 | 8 | README clearly states "Automatically install Visual Studio Code Server in a workspace and create an app to access it via the dashboard." Names both Coder (workspace, dashboard, agent) and VS Code. Shows Coder fits as the deployment/access layer. |
| Visual preview | 5 | 5 | README embeds vscode-web.gif showing VS Code Web with GitHub Copilot and live-share in action. Image verified to exist at 5277.4 KB. |
IDE Integration — 23 / 25
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Dashboard entry point | 7 | 7 | coder_app resource with proper healthcheck, configurable subdomain, share levels, and open_in behavior. Documented in main.tf and tested in vscode-web.tftest.hcl. |
| Managed configuration | 6 | 6 | Documented support for managed Machine settings via settings variable. README example shows theme configuration. Script merges settings with existing file using jq or python3 fallback. Tested in main.test.ts. |
| Configurable folder or workdir | 6 | 6 | folder variable documented with example "Install VS Code Web to a custom folder". workspace variable documented for opening .code-workspace files. Precondition enforces mutual exclusivity. |
| Pre-installed extensions | 6 | 4 | extensions variable documented with example installing github.copilot, ms-python.python, ms-toolsai.jupyter. auto_install_extensions documented for reading .vscode/extensions.json and .code-workspace recommendations. However, auto-install requires jq (not pre-installed), and the JSONC stripping logic is complex/fragile. Partial credit for under-documented dependency and implementation complexity. |
Credential Hygiene — 20 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Secrets marked sensitive | 16 | 16 | No sensitive inputs in this module. VS Code Server uses license acceptance (boolean) and runs without authentication tokens (--without-connection-token). No API keys or secrets required. README examples contain no inline secrets. |
| Non-hardcoded auth path | 4 | 4 | Module runs VS Code Server without connection tokens, relying on Coder's built-in workspace authentication. No credential management needed beyond accepting the license. |
Restricted-Environment Readiness — 10 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Mirrorable artifact source | 10 | 0 | No variable overrides the download URL. The script hardcodes https://update.code.visualstudio.com/api/commits/... and https://vscode.download.prss.microsoft.com/.... commit_id pins a version but does not change the source URL. install_prefix changes the destination, not the source. |
| Bring-your-own binary | 5 | 5 | use_cached variable documented: "Uses cached copy of VS Code Web in the background, otherwise fetches it from internet." offline variable documented: "Just run VS Code Web in the background, don't fetch it from the internet." Both disable download when the binary is pre-baked. |
| Egress transparency | 3 | 1.5 | No dedicated README section for network requirements or air-gapped environments. Endpoints are scattered across examples and inferable from code (update.code.visualstudio.com, vscode.download.prss.microsoft.com, extension marketplace). Warning in settings example mentions jq/python3 requirement but doesn't enumerate network endpoints. Half credit for partial visibility. |
| Runs without sudo | 2 | 2 | Inspected run.sh: no sudo invocations. Script uses mkdir, curl, tar, base64, jq/python3 (optional), all as unprivileged user. Creates directories under user-controlled paths (INSTALL_PREFIX defaults to /tmp, settings in ~/.vscode-server). Full credit from code verification. |
Engineering Quality — 8 / 10
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Input quality | 6 | 6 | Variables have clear descriptions. Sensible defaults (port 13338, telemetry_level "error", install_prefix "/tmp/vscode-web"). Validation on accept_license (must be true), share (owner/authenticated/public), telemetry_level (off/crash/error/all), open_in (tab/slim-window), platform (linux/darwin/alpine/win32). Preconditions enforce offline/use_cached mutual exclusivity and workspace/folder mutual exclusivity. |
| Test coverage | 4 | 2 | TypeScript tests (main.test.ts) cover settings creation, merging with jq, merging with python3 fallback, preservation without merge tools, JSONC extension parsing, workspace recommendations, cached mode extension installation, and auto-install with cached mode. Terraform tests (vscode-web.tftest.hcl) cover open_in validation. However, no tests verify the actual VS Code Web download/install path, platform detection, or end-to-end IDE launch. Tests mock the binary and stub downloads. Half credit for good business logic coverage but missing end-to-end verification. |
Overall — 86 / 100
Scored against SCORECARD.md with claude-sonnet-4-5. Language-model scores are advisory.
This is a false positive I am going to address this seperately but other than that this looks good. |
The LLM scoring Visual preview only saw markdown image references like `` but never received proof that the referenced file exists (images are stored in a shared .images/ directory outside the module folder). This led to score variance — sometimes the LLM inferred the image exists, sometimes it marked the reference as "not included in module content". PR #1057 hit this: vscode-web dropped from 82 to 79 due to a 5→2 Visual preview regression, despite the GIF existing and rendering correctly. Fix: verifyReadmeImages() now resolves relative image paths and checks existence with existsSync(). The context sent to the LLM includes a verification section: === README IMAGE VERIFICATION === ✓ ../../.images/vscode-web.gif — exists (5277.4 KB) → https://example.com/img.png — external URL Or for broken references: ✗ ../.images/missing.png — NOT FOUND This makes scoring deterministic and also catches real broken references (e.g. jetbrains-gateway has a typo: ../.images/ instead of ../../.images/). Fixes false positive from: #1057
…1067) ## Problem PR #1057 (a bug fix for `vscode-web`) triggered a scorecard regression from 82 → 79 due to Visual preview dropping from 5 → 2. The LLM noted: > README references an image (``) but the actual file is not included in the provided module content, only the reference exists. The image exists at `registry/coder/.images/vscode-web.gif` and renders correctly on the registry website. This is the documented pattern — module screenshots live in a shared `.images/` directory outside the module folder. ## Root Cause `gatherModuleContext()` only collects files inside the module directory. The LLM sees markdown image references like `` but receives no evidence that the referenced file exists. Without verification, it sometimes gives full credit (inferring the image exists) and sometimes gives partial credit (noting the file isn't in the provided content). ## Fix Add `verifyReadmeImages()` which: 1. Parses the README for markdown image references 2. Resolves relative paths against the module directory 3. Checks existence with `existsSync()` 4. Appends verification results to the context sent to the LLM The LLM now sees deterministic proof: ``` === README IMAGE VERIFICATION === ✓ ../../.images/vscode-web.gif — exists (5277.4 KB) ``` Or for broken references: ``` === README IMAGE VERIFICATION === ✗ ../.images/missing.png — NOT FOUND ``` ## Testing | Module | Before | After | |--------|--------|-------| | vscode-web | Visual preview: 2/5 (false positive) | Visual preview: 5/5 ✓ | | jetbrains-gateway | Broken ref undetected | Visual preview: 0/5 + "NOT FOUND" ✓ | The fix also caught a real bug: `jetbrains-gateway` uses `../.images/` (one level) instead of `../../.images/` (two levels).
|
Tested this on a live deployment the fix works. One small thing in 1. The "Found a copy" message now prints on the default path, right before it downloads anyway Hoisting the if [ -f "$VSCODE_WEB" ]; then
echo "🥳 Found a copy of VS Code Web"
if [ "${OFFLINE}" = true ]; thenBefore this PR the echo lived inside Anyone with a persistent Which reads like the cache was used when it wasn't. Purely cosmetic, but moving the echo into the two branches that act on it would be accurate. |
…opy is reused The "Found a copy of VS Code Web" message was hoisted to the top of the `if [ -f "$VSCODE_WEB" ]` block, so it printed on the default path right before re-downloading. Log it only in the offline and use_cached branches that actually reuse the copy, and add a regression test.
Description
Separate "skip the download" from "skip extensions" so a cached or pre-installed VS Code Web still installs extensions:
use_cached): reuse the existing copy, skip only the download, then install extensions and start the server like a fresh install.Type of Change
Module Information
Path:
registry/coder/modules/vscode-webNew version:
v1.6.2Breaking change: [ ] Yes [x] No
Testing & Validation
bun test)bun fmt)Related Issues
REG-63
🤖 Generated by Coder Agents on behalf of @35C4n0r.