Skip to content

fix(registry/coder/modules/vscode-web): install extensions when reusing a cached copy - #1057

Merged
35C4n0r merged 7 commits into
mainfrom
35C4n0r/reg-63-vscode-web-cached-extensions
Aug 23, 2026
Merged

fix(registry/coder/modules/vscode-web): install extensions when reusing a cached copy#1057
35C4n0r merged 7 commits into
mainfrom
35C4n0r/reg-63-vscode-web-cached-extensions

Conversation

@35C4n0r

@35C4n0r 35C4n0r commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Description

Separate "skip the download" from "skip extensions" so a cached or pre-installed VS Code Web still installs extensions:

  • Offline: unchanged. Runs the pre-baked copy and exits, or fails when no copy is present.
  • Cached (use_cached): reuse the existing copy, skip only the download, then install extensions and start the server like a fresh install.
  • Default: unchanged (always downloads).

Type of Change

  • New module
  • New template
  • Bug fix
  • Feature/enhancement
  • Documentation
  • Other

Module Information

Path: registry/coder/modules/vscode-web

New version: v1.6.2

Breaking change: [ ] Yes [x] No

Testing & Validation

  • Tests pass (bun test)
  • Code formatted (bun fmt)
  • Changes tested locally

Related Issues

REG-63


🤖 Generated by Coder Agents on behalf of @35C4n0r.

…ng a cached copy

Reusing a cached or pre-installed VS Code Web (use_cached=true) exited before
extension installation, so both the extensions list and auto_install_extensions
were skipped. Skip only the download now; extensions are installed and the
server started in both the fresh and cached paths. Offline behavior is
unchanged. Bump vscode-web to 1.6.2.
@linear-code

linear-code Bot commented Aug 18, 2026

Copy link
Copy Markdown

REG-63

@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Module Scorecard Check

coder/vscode-web: 82 → 86

Score improvement: 82 → 86 (+4).

Theme Before After
Presentation & Onboarding 25 / 25 25 / 25
Integration 22 / 25 23 / 25
Credential Hygiene 20 / 20 20 / 20
Restricted-Environment 7 / 20 10 / 20
Engineering Quality 8 / 10 8 / 10
Overall 82 / 100 86 / 100
Full scorecard for this PR
Presentation & Onboarding IDE Integration Credential Hygiene Restricted-Environment Readiness Engineering Quality Overall
25 / 25 23 / 25 20 / 20 10 / 20 8 / 10 86 / 100
Drilldown

Presentation & Onboarding — 25 / 25

Criterion Max Score Notes
Configuration-mode examples 12 12 Multiple documented examples cover major modes: custom folder installation, extension installation, machine settings configuration, version pinning, workspace opening. Each has sensible defaults.
Coder-context framing 8 8 README clearly states "Automatically install Visual Studio Code Server in a workspace and create an app to access it via the dashboard." Names both Coder (workspace, dashboard, agent) and VS Code. Shows Coder fits as the deployment/access layer.
Visual preview 5 5 README embeds vscode-web.gif showing VS Code Web with GitHub Copilot and live-share in action. Image verified to exist at 5277.4 KB.

IDE Integration — 23 / 25

Criterion Max Score Notes
Dashboard entry point 7 7 coder_app resource with proper healthcheck, configurable subdomain, share levels, and open_in behavior. Documented in main.tf and tested in vscode-web.tftest.hcl.
Managed configuration 6 6 Documented support for managed Machine settings via settings variable. README example shows theme configuration. Script merges settings with existing file using jq or python3 fallback. Tested in main.test.ts.
Configurable folder or workdir 6 6 folder variable documented with example "Install VS Code Web to a custom folder". workspace variable documented for opening .code-workspace files. Precondition enforces mutual exclusivity.
Pre-installed extensions 6 4 extensions variable documented with example installing github.copilot, ms-python.python, ms-toolsai.jupyter. auto_install_extensions documented for reading .vscode/extensions.json and .code-workspace recommendations. However, auto-install requires jq (not pre-installed), and the JSONC stripping logic is complex/fragile. Partial credit for under-documented dependency and implementation complexity.

Credential Hygiene — 20 / 20

Criterion Max Score Notes
Secrets marked sensitive 16 16 No sensitive inputs in this module. VS Code Server uses license acceptance (boolean) and runs without authentication tokens (--without-connection-token). No API keys or secrets required. README examples contain no inline secrets.
Non-hardcoded auth path 4 4 Module runs VS Code Server without connection tokens, relying on Coder's built-in workspace authentication. No credential management needed beyond accepting the license.

Restricted-Environment Readiness — 10 / 20

Criterion Max Score Notes
Mirrorable artifact source 10 0 No variable overrides the download URL. The script hardcodes https://update.code.visualstudio.com/api/commits/... and https://vscode.download.prss.microsoft.com/.... commit_id pins a version but does not change the source URL. install_prefix changes the destination, not the source.
Bring-your-own binary 5 5 use_cached variable documented: "Uses cached copy of VS Code Web in the background, otherwise fetches it from internet." offline variable documented: "Just run VS Code Web in the background, don't fetch it from the internet." Both disable download when the binary is pre-baked.
Egress transparency 3 1.5 No dedicated README section for network requirements or air-gapped environments. Endpoints are scattered across examples and inferable from code (update.code.visualstudio.com, vscode.download.prss.microsoft.com, extension marketplace). Warning in settings example mentions jq/python3 requirement but doesn't enumerate network endpoints. Half credit for partial visibility.
Runs without sudo 2 2 Inspected run.sh: no sudo invocations. Script uses mkdir, curl, tar, base64, jq/python3 (optional), all as unprivileged user. Creates directories under user-controlled paths (INSTALL_PREFIX defaults to /tmp, settings in ~/.vscode-server). Full credit from code verification.

Engineering Quality — 8 / 10

Criterion Max Score Notes
Input quality 6 6 Variables have clear descriptions. Sensible defaults (port 13338, telemetry_level "error", install_prefix "/tmp/vscode-web"). Validation on accept_license (must be true), share (owner/authenticated/public), telemetry_level (off/crash/error/all), open_in (tab/slim-window), platform (linux/darwin/alpine/win32). Preconditions enforce offline/use_cached mutual exclusivity and workspace/folder mutual exclusivity.
Test coverage 4 2 TypeScript tests (main.test.ts) cover settings creation, merging with jq, merging with python3 fallback, preservation without merge tools, JSONC extension parsing, workspace recommendations, cached mode extension installation, and auto-install with cached mode. Terraform tests (vscode-web.tftest.hcl) cover open_in validation. However, no tests verify the actual VS Code Web download/install path, platform detection, or end-to-end IDE launch. Tests mock the binary and stub downloads. Half credit for good business logic coverage but missing end-to-end verification.

Overall — 86 / 100


Scored against SCORECARD.md with claude-sonnet-4-5. Language-model scores are advisory.

@35C4n0r
35C4n0r marked this pull request as ready for review August 19, 2026 11:23
@35C4n0r 35C4n0r self-assigned this Aug 19, 2026
@35C4n0r 35C4n0r added the version:patch Add to PRs requiring a patch version upgrade label Aug 19, 2026
@DevelopmentCats

Copy link
Copy Markdown
Collaborator

Visual preview | 5 | 2 | README includes a GIF (vscode-web.gif) showing VS Code Web with GitHub Copilot and live-share, but the path ../../.images/vscode-web.gif appears broken (relative path goes outside module directory). Partial credit for attempting visual preview with unclear accessibility.

This is a false positive I am going to address this seperately but other than that this looks good.

DevelopmentCats added a commit that referenced this pull request Aug 20, 2026
The LLM scoring Visual preview only saw markdown image references like
`![alt](../../.images/foo.gif)` but never received proof that the
referenced file exists (images are stored in a shared .images/ directory
outside the module folder).

This led to score variance — sometimes the LLM inferred the image exists,
sometimes it marked the reference as "not included in module content".
PR #1057 hit this: vscode-web dropped from 82 to 79 due to a 5→2 Visual
preview regression, despite the GIF existing and rendering correctly.

Fix: verifyReadmeImages() now resolves relative image paths and checks
existence with existsSync(). The context sent to the LLM includes a
verification section:

  === README IMAGE VERIFICATION ===
    ✓ ../../.images/vscode-web.gif — exists (5277.4 KB)
    → https://example.com/img.png — external URL

Or for broken references:

    ✗ ../.images/missing.png — NOT FOUND

This makes scoring deterministic and also catches real broken references
(e.g. jetbrains-gateway has a typo: ../.images/ instead of ../../.images/).

Fixes false positive from: #1057
DevelopmentCats added a commit that referenced this pull request Aug 20, 2026
…1067)

## Problem

PR #1057 (a bug fix for `vscode-web`) triggered a scorecard regression
from 82 → 79 due to Visual preview dropping from 5 → 2. The LLM noted:

> README references an image (`![VS Code
Web...](../../.images/vscode-web.gif)`) but the actual file is not
included in the provided module content, only the reference exists.

The image exists at `registry/coder/.images/vscode-web.gif` and renders
correctly on the registry website. This is the documented pattern —
module screenshots live in a shared `.images/` directory outside the
module folder.

## Root Cause

`gatherModuleContext()` only collects files inside the module directory.
The LLM sees markdown image references like
`![alt](../../.images/foo.gif)` but receives no evidence that the
referenced file exists. Without verification, it sometimes gives full
credit (inferring the image exists) and sometimes gives partial credit
(noting the file isn't in the provided content).

## Fix

Add `verifyReadmeImages()` which:
1. Parses the README for markdown image references
2. Resolves relative paths against the module directory  
3. Checks existence with `existsSync()`
4. Appends verification results to the context sent to the LLM

The LLM now sees deterministic proof:

```
=== README IMAGE VERIFICATION ===
  ✓ ../../.images/vscode-web.gif — exists (5277.4 KB)
```

Or for broken references:

```
=== README IMAGE VERIFICATION ===
  ✗ ../.images/missing.png — NOT FOUND
```

## Testing

| Module | Before | After |
|--------|--------|-------|
| vscode-web | Visual preview: 2/5 (false positive) | Visual preview:
5/5 ✓ |
| jetbrains-gateway | Broken ref undetected | Visual preview: 0/5 + "NOT
FOUND" ✓ |

The fix also caught a real bug: `jetbrains-gateway` uses `../.images/`
(one level) instead of `../../.images/` (two levels).
@DevelopmentCats

DevelopmentCats commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Tested this on a live deployment the fix works. use_cached now reuses the copy, skips only the download, and still installs both the extensions list and the auto_install_extensions recommendations. Confirmed by wiping extensions_dir between restarts and watching it repopulate. offline and the default download path are unchanged.

One small thing in run.sh:

1. The "Found a copy" message now prints on the default path, right before it downloads anyway

Hoisting the echo out of the mode check means it fires whenever a copy exists, including when neither offline nor use_cached is set:

if [ -f "$VSCODE_WEB" ]; then
  echo "🥳 Found a copy of VS Code Web"
  if [ "${OFFLINE}" = true ]; then

Before this PR the echo lived inside if [ "${OFFLINE}" = true ] || [ "${USE_CACHED}" = true ], so it only appeared when the copy was actually going to be used.

Anyone with a persistent install_prefix who hasn't opted into use_cached now gets this on every restart:

🥳 Found a copy of VS Code Web
Installing Microsoft Visual Studio Code Server!
VS Code Web commit id version 110a328ea54b42367b803ec53ee0bf52ef26b419.
VS Code Web has been installed.

Which reads like the cache was used when it wasn't. Purely cosmetic, but moving the echo into the two branches that act on it would be accurate.

…opy is reused

The "Found a copy of VS Code Web" message was hoisted to the top of the
`if [ -f "$VSCODE_WEB" ]` block, so it printed on the default path right before
re-downloading. Log it only in the offline and use_cached branches that actually
reuse the copy, and add a regression test.
@35C4n0r
35C4n0r enabled auto-merge (squash) August 23, 2026 16:18
@35C4n0r
35C4n0r merged commit ca87482 into main Aug 23, 2026
5 checks passed
@35C4n0r
35C4n0r deleted the 35C4n0r/reg-63-vscode-web-cached-extensions branch August 23, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

version:patch Add to PRs requiring a patch version upgrade

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants