Added CLI e2e Pipeline by neha-mishraa · Pull Request #3312 · jfrog/jfrog-cli

neha-mishraa · 2026-01-20T09:26:18Z

All tests have passed. If this feature is not already covered by the tests, new tests have been added.
The pull request is targeting the master branch.
The code has been validated to compile successfully by running go vet ./....
The code has been formatted properly using go fmt ./....

agrasth

PR Review: Added CLI e2e Pipeline

Issues & Concerns

1. Hardcoded Repository Reference (Line 11)

path: neha-mishraa/jfrog-cli

This should reference jfrog/jfrog-cli not a personal fork. This is critical - the pipeline won't work on the main repository.

2. Hardcoded Master Key (Lines 110, 121)

master_key=0098a3db926121d657955ebb9461483c

Hardcoded cryptographic keys are a security risk. This should be:

Stored in a secure integration/secret
Passed via environment variable
Never committed to version control

3. Mixed Authentication Methods (Lines 333, 834)

Most tests use jfrog.adminToken=${OAUTH_TOKEN}
npm test (line 834) uses jfrog.user=admin --jfrog.password=password

Why the inconsistency? The commented line 833 shows it should use the token.

4. Commented Code (Lines 219-240)

Large block of commented-out project creation code. Either:

Remove if not needed
Uncomment if needed
Add a comment explaining why it's disabled

5. Missing docker_cli_tests Dependency

The docker_cli_tests step runs independently but isn't included in the teardown_env input dependencies. If Docker tests are the last to finish, teardown might happen too early.

6. Resource Validation

Lines 267-270 validate ARTIFACTORY_URL but several other steps don't validate the OAuth token resource before use. Consider adding consistent validation across all test steps.

7. Hard Timeouts

Line 470: 600-second timeout for Artifactory startup. This might be too short for slower environments or too long for fast ones. Consider making this configurable.

Suggestions

Extract Common Setup: Go installation is repeated in every step. Consider a shared setup step or a custom base image.
Version Variables: Extract all tool versions (Go 1.21.5, Java 11, Maven 3.8.8, etc.) to pipeline-level variables for easier maintenance.
Add Documentation: Consider adding a README or comments explaining:
- How to run the pipeline
- Required integrations
- Environment variables
- What each test step validates
Test Ordering: Some tests could potentially run in parallel (e.g., pip and pipenv, Go and Maven). Consider if the sequential execution is intentional.
Logging Levels: Line 557 sets JFROG_CLI_LOG_LEVEL: DEBUG only for Artifactory tests. Should this be consistent across all tests?

github-actions · 2026-02-11T04:49:20Z

🐸 JFrog Frogbot

agrasth · 2026-02-11T05:02:21Z

.jfrog-pipelines/pipelines.yml

+  - name: jfrog_cli
+    type: GitRepo
+    configuration:
+      path: neha-mishraa/jfrog-cli


Is it expected to use the forked one not the upstream one(jfrog/jfrog-cli) ?

will update this with new name of integration and and path once I have that created - so keeping it as is for now

agrasth · 2026-02-11T07:27:02Z

.jfrog-pipelines/pipelines.yml

+            ACCOUNT_TYPE: "enterprise_plus"
+            GROUP: "ARTIFACTORY"
+            EXPIRY: 2d
+            EXTRA_PARAMS: "conf_artifactory_unified_version=${RT_VERSION} master_key=0098a3db926121d657955ebb9461483c"


these hard-coded values, are safe to use like this?

default being passed while creation of a d2c not a sensitive value

agrasth · 2026-02-11T07:27:50Z

.jfrog-pipelines/pipelines.yml

+              # # Create pipegh project in target Artifactory
+              # echo "Creating pipegh project in ${ART_URL}..."
+
+              # RESPONSE=$(curl -s -w "\n%{http_code}" --connect-timeout 30 --max-time 60 -X POST "${ART_URL}/access/api/v1/projects" \
+              #   -H "Authorization: Bearer ${OAUTH_TOKEN}" \
+              #   -H "Content-Type: application/json" \
+              #   -d '{"display_name":"cliTests","description":"The tests for JFrog CLI","admin_privileges":{"manage_members":true,"manage_resources":true,"index_resources":true},"storage_quota_bytes":1073741824,"project_key":"pipegh"}')
+
+              # HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
+              # RESPONSE_BODY=$(echo "$RESPONSE" | sed '$d')
+
+              # echo "HTTP Status: $HTTP_CODE"
+
+              # if [[ "$HTTP_CODE" == "201" ]]; then
+              #   echo ">>> Project 'pipegh' created successfully"
+              # elif [[ "$HTTP_CODE" == "409" ]]; then
+              #   echo ">>> Project 'pipegh' already exists"
+              # else
+              #   echo ">>> Failed to create project. HTTP Status: $HTTP_CODE"
+              #   echo "Response: $RESPONSE_BODY"
+              #   echo "Warning: Continuing with setup anyway..."
+              # fi


If not needed can we remove it? any specific reason these are here?

will check this - might have left

reshmifrog · 2026-02-12T07:55:30Z

@neha-mishraa please add a proper description explaining what this PR does, what was the problem and what is the solution we are going ahead with to resolve the problem.

reshmifrog · 2026-02-12T07:53:36Z

.jfrog-pipelines/pipelines.yml

@@ -0,0 +1,1302 @@
+resources:
+  - name: jfrog_cli


please share the forked repo PR where this has been tested.

@reshmifrog please refer the document as to what this PR does - https://docs.google.com/document/d/12XEmCxxMpq4CgbgVELKcQkuGLmvj0ksLkGE6vYegqV4/edit?usp=sharing
also the forked repo is here neha-mishraa:master

reshmifrog · 2026-02-12T07:57:55Z

.jfrog-pipelines/pipelines.yml

+              description: "Artifactory URL to use for tests (required)"
+              allowCustom: true
+            JOIN_KEY:
+              default: "0098a3db926121d657955ebb9461483c"


why are we hard-coding this?

default being passed while creation of a d2c not a sensitive value

reshmifrog · 2026-02-12T08:10:53Z

.jfrog-pipelines/pipelines.yml

+              fi
+
+              # Install JFrog CLI
+              echo "Installing JFrog CLI, Version: 2.54.0"


why are we installing 2.54.0 version of cli , it should be latest right?

I took reference from other yamls - you can suggest which version explicitly do we need to check - AFAIK the cli functions are used from code base and not the version so...

reshmifrog · 2026-02-12T08:11:41Z

.jfrog-pipelines/pipelines.yml

+
+              # Install JFrog CLI Access Plugin
+              echo "Installing JFrog CLI Access Plugin, Version: 7.66.0"
+              jfrog plugin install access@v7.66.0


why are we installing the 7.66.0 version of plugin?

I took reference from other yamls - you can suggest which version explicitly do we need to check with it

reshmifrog · 2026-02-12T08:12:24Z

.jfrog-pipelines/pipelines.yml

+              jfrog plugin install access@v7.66.0
+
+              # Generate tokens
+              JOIN_KEY="${JOIN_KEY:-0098a3db926121d657955ebb9461483c}"


we should not hard-code these values, you can add these as secrets in jfrog-cli repo and use it instead of hard-coding.

default being passed while creation of a d2c not a sensitive value - it is for local infra will not be reachable from here

reshmifrog · 2026-02-12T08:14:05Z

.jfrog-pipelines/pipelines.yml

+                exit 1
+              }
+
+              # Fixed parsing - extract everything after JF_ACCESS_ADMIN_TOKEN=


let's store it as a secret and then use it and please check this everywhere.

not referenced anywhere and is a masked value in pipelines and os created on the fly

nehamjfrog and others added 2 commits January 20, 2026 14:50

Added CLI e2e Pipeline

f566f14

Merge branch 'master' into master

6982553

agrasth reviewed Feb 11, 2026

View reviewed changes

reshmifrog self-requested a review February 12, 2026 07:49

reshmifrog requested changes Feb 12, 2026

View reviewed changes

Conversation

neha-mishraa commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agrasth left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

PR Review: Added CLI e2e Pipeline

Issues & Concerns

1. Hardcoded Repository Reference (Line 11)

2. Hardcoded Master Key (Lines 110, 121)

3. Mixed Authentication Methods (Lines 333, 834)

4. Commented Code (Lines 219-240)

5. Missing docker_cli_tests Dependency

6. Resource Validation

7. Hard Timeouts

Suggestions

Uh oh!

github-actions bot commented Feb 11, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reshmifrog commented Feb 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

neha-mishraa commented Jan 20, 2026 •

edited

Loading

agrasth left a comment •

edited

Loading