Skip to content

[DEVIN: Konsti] Fix failing E2E tests: CLI error string and MCP prompt name#1439

Merged
N2D4 merged 2 commits into
devfrom
devin/1779211590-fix-ci-test-failures
May 19, 2026
Merged

[DEVIN: Konsti] Fix failing E2E tests: CLI error string and MCP prompt name#1439
N2D4 merged 2 commits into
devfrom
devin/1779211590-fix-ci-test-failures

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented May 19, 2026

Fixes three deterministic E2E test failures on the dev branch:

  1. CLI test (apps/e2e/tests/general/cli.test.ts:611): The test expected 'File not found' but the actual CliError message is 'Config file not found: ...'. Updated to match the real error message from packages/stack-cli/src/lib/config-file-path.ts.

  2. MCP prompt test (apps/e2e/tests/backend/endpoints/api/v1/internal/mcp.test.ts:114): The prompts/list test expected prompt name 'ask_stack_auth' but the MCP handler registers the prompt as 'skill' (apps/mcp/src/mcp-handler.ts:93-94). Updated to match.

  3. MCP resource test (mcp.test.ts:131-132): The resources/list test expected resource name 'stack-auth-mcp-setup' with URI 'stack-auth://mcp/setup', but the handler now registers the resource as 'skill' with URI 'https://skill.stack-auth.com' (apps/mcp/src/mcp-handler.ts:76-78). Updated to match.

These failures were consistent across all E2E test workflows (regular, local emulator, custom port prefix). Other CI failures in the setup-tests and DB-migration-compat workflows appear to be flaky/timeout-related and also occur on the base branch.

Link to Devin session: https://app.devin.ai/sessions/5ee786b351194d4fb24b150fd358d6f5
Requested by: @N2D4

- cli.test.ts: Update expected error from 'File not found' to 'Config file not found' to match actual CliError message
- mcp.test.ts: Update expected prompt name from 'ask_stack_auth' to 'skill' to match MCP handler implementation

Co-Authored-By: Konstantin Wohlwend <n2d4xc@gmail.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
stack-auth-hosted-components Ready Ready Preview, Comment May 19, 2026 7:37pm
stack-auth-mcp Ready Ready Preview, Comment May 19, 2026 7:37pm
stack-auth-skills Ready Ready Preview, Comment May 19, 2026 7:37pm
stack-backend Error Error May 19, 2026 7:37pm
stack-dashboard Ready Ready Preview, Comment May 19, 2026 7:37pm
stack-demo Ready Ready Preview, Comment May 19, 2026 7:37pm
stack-docs Ready Ready Preview, Comment May 19, 2026 7:37pm
stack-preview-backend Ready Ready Preview, Comment May 19, 2026 7:37pm
stack-preview-dashboard Ready Ready Preview, Comment May 19, 2026 7:37pm

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 19, 2026

Greptile Summary

This PR fixes two E2E test assertions that were mismatched against their implementations. Both corrections are verified against the source code they test.

  • CLI test (cli.test.ts:611): Fixes the expected error substring from "File not found" to "Config file not found", which matches the CliError message in packages/stack-cli/src/lib/config-file-path.ts line 14.
  • MCP test (mcp.test.ts:114): Fixes the expected prompt name from "ask_stack_auth" (the MCP tool name) to "skill" (the actual prompt registered in apps/mcp/src/mcp-handler.ts line 93–94).

Confidence Score: 5/5

Both test fixes are straightforward assertion corrections verified against the actual implementation — safe to merge.

Each changed assertion was confirmed against the source it tests: the CLI error string matches the exact CliError message in config-file-path.ts, and 'skill' matches the prompt name registered in mcp-handler.ts. No logic, no runtime paths, and no application code are modified.

No files require special attention.

Important Files Changed

Filename Overview
apps/e2e/tests/general/cli.test.ts Updated error string assertion from "File not found" to "Config file not found" to match the actual CliError message thrown in config-file-path.ts
apps/e2e/tests/backend/endpoints/api/v1/internal/mcp.test.ts Updated prompt name assertion from "ask_stack_auth" (the tool name) to "skill" (the actual registered prompt name in mcp-handler.ts)

Sequence Diagram

sequenceDiagram
    participant Test as E2E Test
    participant CLI as stack-cli (init --mode link-config)
    participant CFP as config-file-path.ts
    participant MCP as mcp-handler.ts

    Note over Test,CFP: CLI test fix
    Test->>CLI: run with --config-file /nonexistent/stack.config.ts
    CLI->>CFP: "resolveConfigFilePathOption(path, {mustExist: true})"
    CFP-->>CLI: throw CliError("Config file not found: ...")
    CLI-->>Test: stderr contains "Config file not found" ✓

    Note over Test,MCP: MCP test fix
    Test->>MCP: prompts/list request
    MCP-->>Test: "[{name: "skill"}]"
    Test->>Test: "assert name === "skill" ✓"
Loading

Reviews (1): Last reviewed commit: "Fix failing E2E tests: update CLI error ..." | Re-trigger Greptile

…ation

The resources/list assertion still expected the old 'stack-auth-mcp-setup' name and 'stack-auth://mcp/setup' URI, but the MCP handler now registers the resource as 'skill' with URI 'https://skill.stack-auth.com'.

Co-Authored-By: Konstantin Wohlwend <n2d4xc@gmail.com>
@N2D4 N2D4 merged commit deff6c3 into dev May 19, 2026
29 of 34 checks passed
@N2D4 N2D4 deleted the devin/1779211590-fix-ci-test-failures branch May 19, 2026 21:20
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

Devin is archived and cannot be woken up. Please unarchive Devin if you want to continue using it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant