Add public API commands by ramilamparo · Pull Request #58 · Hypersequent/qas-cli

ramilamparo · 2026-03-18T14:09:36Z

Summary

Adds a new qasphere api command that provides direct CLI access to the full QA Sphere public API. Users can now manage projects, test runs, test cases, results, and other resources programmatically without writing custom API integration code.

What's included

16 resource subcommands: projects, runs, test-cases, results, folders, milestones, tags, requirements, shared-steps, shared-preconditions, custom-fields, audit-logs, settings, test-plans, files, users
JSON argument flexibility: Complex args accept inline JSON or @filename references
Zod validation with path-based errors: Detailed error messages like [0].tcaseIds: not allowed for "live" runs
Lazy env loading: QAS_URL/QAS_TOKEN loaded only when the API is called, so CLI validation errors surface first
Path parameter validation: Regex-based validation prevents injection via crafted identifiers
Agent Skills: New Skill.md file allows installing skills via npx skills add Hypersequent/qas-cli

Folder structure

src/api/                          # API client layer
├── index.ts                      # createApi() — composes fetch chain and assembles all sub-modules
├── schemas.ts                    # Shared types, RequestValidationError, validateRequest(), common Zod fields
├── utils.ts                      # Fetch decorators (withBaseUrl, withApiKey, withJson, withDevAuth)
└── <resource>.ts                 # One per resource — exports create<Resource>Api(fetcher) factory,
                                  #   defines Zod request schemas for early validation before API calls

src/commands/api/                 # CLI command layer
├── main.ts                       # Registers all resource subcommands
├── utils.ts                      # Shared helpers: apiHandler, parseAndValidateJsonArg,
│                                 #   handleValidationError, buildArgumentMap, printJson, etc.
└── <resource>/
    ├── command.ts                # Yargs command definitions and handler logic
    └── help.ts                   # Help text and descriptions

src/tests/api/                    # API command tests
├── test-helper.ts                # Shared infrastructure: useMockServer(), runCli(), test fixture
│                                 #   with project lifecycle (mock or live), helper creators
├── utils.spec.ts                 # Unit tests for API command utilities
└── <resource>/
    └── <action>.spec.ts          # One spec per action — mocked tests (MSW), validation error
                                  #   tests, and live tests (tagged, run against real instance)

Other changes

New API modules: audit-logs, custom-fields, milestones, requirements, results, settings, shared-preconditions, shared-steps, tags, test-plans, users, file — plus expanded runs, tcases, projects, folders
CLAUDE.md updated with full architecture docs for the API command
README.md updated with usage documentation
SKILL.md added for AI coding agent support
CI workflow updated
npm audit fix applied to dependencies (high vulnerability issues were being reported)

Testing

55+ new test files under src/tests/api/ organized by resource, one spec per action (e.g., projects/list.spec.ts, runs/create.spec.ts)
Each spec typically contains: mocked tests with MSW handlers, validation error tests, and live tests tagged with { tags: ['live'] }
Shared test infrastructure in test-helper.ts with useMockServer(), runCli(), and a test fixture that provides project lifecycle management
Global setup (global-setup.ts) handles live API authentication
Missing subcommand help tests ensure incomplete commands show help text

🤖 Generated with Claude Code

gemini-code-assist · 2026-03-18T14:11:25Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new api command to the QAS CLI, providing direct access to the QA Sphere public API. This enhancement allows users to manage their QA Sphere projects, test runs, and results programmatically from the command line. The PR also includes significant documentation updates and comprehensive testing to ensure the new functionality is robust and easy to use.

Highlights

API Command: Introduces a new api command for direct access to the QA Sphere public API, enabling users to manage projects, test runs, and results programmatically.
Documentation: Extensive documentation updates in README.md and a new SKILL.md file to support AI coding agents, providing detailed command usage and examples.
Testing: Adds comprehensive tests for the new api command, including both mocked and live modes, ensuring reliability and correctness.

Ignored Files

Ignored by pattern: .github/workflows/** (1)
- .github/workflows/ci.yml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a comprehensive api command to the CLI, providing direct access to the QA Sphere public API. The changes are extensive and well-executed, including new commands for various API resources, detailed documentation in README.md and SKILL.md, and a robust testing suite with both mocked and live tests. The code is well-structured, modular, and follows best practices. My review includes one minor suggestion to enhance error reporting for better user experience when dealing with malformed JSON files.

gemini-code-assist · 2026-03-18T14:17:17Z

src/commands/api/utils.ts

+		try {
+			return JSON.parse(content)
+		} catch {
+			throw new Error(`Failed to parse JSON from file ${filePath} for ${optionName}`)
+		}


For better error reporting and consistency with the other JSON.parse error handling in this file, it would be beneficial to include the original error message when parsing from a file fails. This will provide more context to the user when debugging malformed JSON files.

try { return JSON.parse(content) } catch (e) { const errorMessage = e instanceof Error ? e.message : String(e) throw new Error( `Failed to parse JSON from file ${filePath} for ${optionName}: ${errorMessage}` ) }

- Add Zod schemas at API level for all request types with validateRequest() that throws RequestValidationError, caught by apiHandler for clean error output - Add handleValidationError() + buildArgumentMap() to map API field names to CLI argument names (e.g., title → --title) for user-friendly error messages - Move validation schemas from command-level to API-level as single source of truth; command handlers pass args directly to API functions - Add customFields, parameterValues, and filledTCaseTitleSuffixParams to test-case create/update commands with proper validation (parameterValues restricted to template type) - Fix jsonResponse null crash, expand ResultStatus with custom statuses, relax test-cases update check, extract shared superRefine for run schemas - Add tests for new fields (mocked + live) and batch-create validation errors

claude · 2026-03-24T16:43:56Z

Claude finished @ramilamparo's task in 6m 24s —— View job

Reviewing PR #58: Add public API commands

Gather PR context and diff
Launch specialized review subagents (code quality, security, performance, test coverage)
Aggregate findings, deduplicate, and post review
Clean up outdated bot comments

Reviewed the full diff (~424K chars, 100 files) with 4 specialized subagents (code quality, security, performance, test coverage). Posted a new review with 2 inline comments. Resolved 9 outdated bot threads, minimized 9 outdated bot review summaries, and deleted 2 progress comments.

7 issues fixed since last review:

ResultStatus type now includes custom1-4 statuses
test-cases update .check() rewritten (checks empty merged body instead of requiring --body/--title)
DRY body-merging extracted to mergeBodyWithOverrides helper
DRY superRefine extracted to shared validateQueryPlans function
projects create now uses API-layer handleValidationError
test-cases create/update use explicit field picking (no more ...restArgs body leak)
listRunTCases correctly unwraps response

Correction on "yargs metadata leak": After deeper analysis, the ...args spread in list/count handlers is NOT a runtime bug — all API functions use validateRequest() which strips unknown keys via Zod before they reach appendSearchParams. Resolved those threads.

2 remaining issues (inline comments posted):

milestoneIds/tags arrays missing .int().positive() — allows NaN through (root cause of comma-parsing issue)
Shared-preconditions response unwrapping still inconsistent with shared-steps

claude · 2026-03-25T08:58:47Z

Claude finished @ramilamparo's task in 6m 31s —— View job

Reviewing PR #58: Add public API commands

Gather PR context and diff
Launch specialized review subagents (code quality, security, performance, test coverage)
Aggregate findings, deduplicate, and post review
Clean up outdated bot comments

All 12 previously raised issues are now resolved — fixed in code or confirmed correct by the author. Posted review with 1 minor nit (explicit field picking vs ...args spread for readability). Resolved 2 remaining open threads, minimized 10 outdated bot review summaries, and deleted 1 progress comment.

claude

Review Summary (re-review after `e484fb2`)

All previously raised issues have been addressed:

milestoneIds/tags missing .int().positive() — Fixed in latest commits. All numeric array schemas now use .int().positive(), which also prevents NaN from .split(',').map(Number).
Shared-preconditions response unwrapping — Confirmed correct by author; thread resolved.
ResultStatus type — Now includes custom1-4 statuses.
test-cases update .check() — Rewritten to check for empty merged body.
DRY body-merging — Extracted to mergeBodyWithOverrides helper.
DRY superRefine — Extracted to shared validateQueryPlans function.
projects create validation — Now uses API-layer handleValidationError.
test-cases create/update body leak — Uses explicit field picking via mergeBodyWithOverrides.
listRunTCases response unwrapping — Now correctly unwraps.
Path parameter injection — Mitigated via validatePathParams regex ([a-zA-Z0-9_-]+).
Schemas inlined — results/schemas.ts and settings/schemas.ts removed, schemas moved into their respective command.ts files.

1 minor nit posted inline (explicit field picking vs ...args spread — not a bug, just readability).

No new security, performance, or correctness issues found. Well-structured PR with good separation of concerns.

src/commands/api/runs/command.ts

claude · 2026-03-25T09:07:07Z

Claude finished @ramilamparo's task in 8m 43s —— View job

Reviewing PR #58: Add public API commands

Gather PR context and diff
Launch specialized review subagents (code quality, security, performance, test coverage)
Aggregate findings, deduplicate, and post review
Clean up outdated bot comments

Reviewed the full diff (~424K chars, 102 files) with 4 specialized subagents. All 12 previously raised issues are resolved. Posted an approving review with no new blocking issues. Minimized 10 outdated bot review summaries.

claude

Review Summary

All 12 previously raised issues have been resolved. Reviewed the full diff (~424K chars, 102 files) with 4 specialized subagents (code quality, security, performance, test coverage). No new critical, high, or medium issues found.

Security: Clean — path parameter validation via validatePathParams regex, Zod schema enforcement at all API boundaries, no hardcoded credentials, proper auth token handling.

Performance: No issues in PR-scoped changes.

Code quality: Well-structured PR with consistent patterns (apiHandler + connectApi lazy-loading, per-resource command.ts/help.ts organization, mergeBodyWithOverrides DRY helper, shared validateQueryPlans). The author's approach of relying on Zod's validateRequest() to strip unknown keys from ...args spread is a valid design choice.

Test coverage observation (non-blocking): Validation error paths are lightly tested for several commands (settings update-statuses, folders bulk-create, test-plans create, results create, files upload). The runs create and test-cases create specs are good models for what coverage could look like. This is a quality improvement opportunity, not a blocker.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e484fb2a50

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

.github/workflows/ci.yml

chatgpt-codex-connector · 2026-03-25T09:21:56Z

src/api/tcases.ts

+	description: z.string(),
+	expected: z.string().optional(),
+	sharedStepId: z.number().int().positive().optional(),


Accept shared-step-only payloads in step schema

The stepSchema makes description mandatory, so requests like steps: [{"sharedStepId": 123}] are rejected before hitting the API. QA Sphere's Test Cases API docs define step fields as optional (sharedStepId?, description?, expected?), which means shared-step references without inline text are valid; this client-side requirement blocks that supported workflow.

Useful? React with 👍 / 👎.

ramilamparo · 2026-03-25T09:24:46Z

src/api/run.ts

Renamed the file to runs.ts to keep it uniform with the others.

ramilamparo · 2026-03-25T09:25:07Z

skills/SKILL.md

New skill can be added with npx skills add Hypersequent/qas-cli

ramilamparo · 2026-03-25T09:27:26Z

.github/workflows/ci.yml

+      - name: Run live API tests
+        env:
+          QAS_TEST_URL: ${{ secrets.QAS_TEST_URL }}
+          QAS_TEST_TOKEN: ${{ secrets.QAS_TEST_TOKEN }}
+          QAS_TEST_USERNAME: ${{ secrets.QAS_TEST_USERNAME }}
+          QAS_TEST_PASSWORD: ${{ secrets.QAS_TEST_PASSWORD }}
+          QAS_DEV_AUTH: ${{ secrets.QAS_DEV_AUTH }}
+        run: npm run test:live


Live tests are added that allows running automated tests with a real live qasphere instance. QAS_TEST_USERNAME and QAS_TEST_PASSWORD are required because we don't have a public API yet for deleting projects, which is required for clean-ups after the tests. So we use login credentials and call a non-public API instead.

ramilamparo · 2026-03-25T09:30:41Z

package.json

    "typescript": "^5.8.3",
    "typescript-eslint": "^8.31.1",
-    "vitest": "^3.1.2"
+    "vitest": "^4.1.0"


Updated vitest to allow specifying tags. Which is useful for live tests and excluding them on normal testing.

Add public API commands

67480e3

gemini-code-assist bot reviewed Mar 18, 2026

View reviewed changes