Command-line runner

apiworkbench-cli runs a collection without opening the app. It sends the requests in order, runs their scripts, tests and captures like the collection runner, prints the results and exits with a code that tells a script or a CI system whether everything passed.

Where to find it

The command-line runner is installed with the desktop app. It is new in ApiWorkbench 1.4.0.

You installed	Run it as
The `.deb` package	`apiworkbench-cli` (it is on your `PATH` at `/usr/bin/apiworkbench-cli`)
The Linux tarball	`./apiworkbench-cli` in the extracted folder, next to the app
The Windows zip	`.\apiworkbench-cli` in the extracted folder, next to the app
The snap	`apiworkbench.cli`

The snap version can read files under your home folder and on removable media only. Keep collections and reports there.

See Install ApiWorkbench for the packages themselves. Check that the runner works:

bash

apiworkbench-cli --version

It prints the version, such as 1.4.0. The examples on this page use apiworkbench-cli; with the snap, type apiworkbench.cli instead.

Your first run

Get a file to run. In the app, export a collection or your workspace to a JSON file: see Export. A Postman collection file works too.
Open a terminal in the folder that holds the file.
Run it:
bash
```
apiworkbench-cli run shop.json
```
Read the output. Each request is listed with its status, size and time, and each test with a tick or a cross. A summary table and a final line, All checks passed. or The run failed., close the run.

Usage

text

apiworkbench-cli run <file> [options]
apiworkbench-cli --version
apiworkbench-cli --help

<file> is an ApiWorkbench workspace export or a Postman collection. Exactly one file is given. If it holds more than one collection, name the one to run with --collection.

Options

Choosing what to run

Option	What it does
`--collection <name>`	Which collection to run when the file has several, by name or id. Needed only for a workspace export or a Postman dump.
`--folder <path>`	Run only this folder and the folders inside it, such as `Orders` or `Orders/Refunds`. Repeat the option for more folders.

Variables

Option	What it does
`-e`, `--environment <file>`	The environment to use: a Postman environment file, an ApiWorkbench workspace file with one environment, or the name of an environment inside the run file.
`-g`, `--globals <file>`	A Postman globals file, or a file with one environment to use as globals.
`--env-var <key=value>`	Set an environment variable, over the file's value. Repeatable.
`--global-var <key=value>`	Set a global variable, over the file's value. Repeatable.
`-d`, `--iteration-data <file>`	A CSV or JSON data file: one iteration per row, each column a variable.

Run control

Option	What it does	Default
`-n`, `--iteration-count <count>`	How many times to run the collection. With more iterations than data rows, the rows are cycled.	One per data row, or 1 without data
`--delay-request <ms>`	Wait this long after each request.	`0`
`--timeout-request <ms>`	Give up on a request after this long.	The app's default timeouts
`--bail`	Stop the run at the first failed request.	Off
`--insecure`	Accept self-signed or invalid TLS certificates.	Off

Output

Option	What it does	Default
`-r`, `--reporters <list>`	Reporters to use, comma separated: `cli`, `json`, `junit`, `html`.	`cli`
`--reporter-json-export <file>`	Where the json reporter writes. Turns the json reporter on.	See below
`--reporter-junit-export <file>`	Where the junit reporter writes. Turns the junit reporter on.	See below
`--reporter-html-export <file>`	Where the html reporter writes. Turns the html reporter on.	See below
`--color`, `--no-color`	Use colours in the cli reporter. Colours appear only when writing to a terminal.	`--color`
`--verbose`	Show response headers and bodies in the cli reporter.	Off
`-h`, `--help`	Show the help text.

Colours are also switched off when the NO_COLOR environment variable is set.

Reporters

A reporter decides how results are presented. You can use several at once.

Reporter	Output	Use it for
`cli`	Text in the terminal	Reading a run as it happens
`json`	A JSON file with every request, test, log line and a summary	Your own scripts and dashboards
`junit`	A JUnit XML file	CI systems that show test results
`html`	A single HTML page	Sharing a run with people

A file reporter without an export path writes to a folder named apiworkbench-reports in the current directory, with a name like apiworkbench-run-20261031-142500.json (.xml for junit, .html for html). Missing folders in an export path are created.

Values of variables marked secret are masked in every report.

The cli reporter

For each request it prints the name, then the method, URL, status, size and time, then one line per test. Output from console.log in scripts is shown under the request. With --verbose it adds the response headers and the start of the body.

The run ends with a table of iterations, requests, tests and assertions, each with how many ran and how many failed, followed by the total duration, the data received and the average, minimum and maximum response time. Here tests counts pm.test calls in scripts, and assertions counts the no-code checks from the Tests tab.

Failures are listed again at the end, each with a type:

Type	Meaning
`RequestError`	No response came back, or the request timed out
`ScriptError`	A script threw an error
`AssertionError`	A no-code test failed
`TestError`	A `pm.test` failed
`StatusError`	The status was 4xx or 5xx and the request has no tests

The json reporter

The top level of the file has tool, version, collection, environment, startedAt, durationMs, passed, stoppedEarly, summary, executions, failures and warnings. Each entry of executions is one request with its request, response, assertions, tests, scriptError and logs.

Read the overall result with a tool such as jq:

bash

jq '.passed, .summary.requests' apiworkbench-reports/*.json

The junit reporter

Each request becomes a test suite, and each of its tests a test case. A failed test is a failure; a request or script error is an error; a request skipped by a script is skipped.

Exit codes

Code	Meaning
`0`	Every request and test passed
`1`	A request or test failed
`2`	Wrong arguments, a file that cannot be read, or a report that cannot be written

A request passes or fails by the same rules as in the app: all its tests pass, or with no tests, the status is below 400. See When a request passes or fails.

How a run behaves

Requests run in collection order. Only HTTP requests are run.
Scripts can change the order with pm.execution.setNextRequest, as in the app's runner.
Captures and script variable changes carry from one request to the next for the length of the run. Nothing is written back to your files.
Captured values go to the environment when you pass one with -e or --env-var, and to the collection's variables otherwise.
Globals are loaded from the file you pass with -g, and from --global-var.
Cookies are kept for the run only.
An environment name given to -e must match the name in the run file exactly.

Examples

Run a collection with an environment file:

bash

apiworkbench-cli run shop.postman_collection.json -e staging.postman_environment.json

Run one collection from a workspace export, using an environment stored in the same file:

bash

apiworkbench-cli run workspace.json --collection "Shop API" -e "Staging"

Run only one folder and stop at the first failure:

bash

apiworkbench-cli run workspace.json --collection "Shop API" --folder Orders --bail

With a three-row data file, go through the rows twice:

bash

apiworkbench-cli run shop.json -d users.csv -n 6

Pass a secret from the shell instead of storing it in a file:

bash

apiworkbench-cli run shop.json -e staging.json --env-var "apiKey=$API_KEY"

Write a JUnit file and an HTML page as well as the terminal output:

bash

apiworkbench-cli run shop.json -r cli,junit,html \
  --reporter-junit-export results/junit.xml \
  --reporter-html-export results/report.html

Slow down for a rate-limited API and allow ten seconds per request:

bash

apiworkbench-cli run shop.json --delay-request 250 --timeout-request 10000

Use the exit code in a shell script:

bash

if apiworkbench-cli run shop.json -e staging.json; then
  echo "API checks passed"
else
  echo "API checks failed"
fi

Run in CI

Commit the exported collection and environment to your repository, without secrets in them. Give secrets to the run with --env-var from your CI system's secret store. The snippets below install the .deb package from the 1.4.0 release on an x64 Ubuntu runner.

GitHub Actions

yaml

name: API tests

on: [push]

jobs:
  api-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install apiworkbench-cli
        run: |
          curl -fsSLO https://github.com/Dhiva-Labs/ApiWorkbench/releases/download/v1.4.0/apiworkbench_1.4.0_amd64.deb
          sudo apt-get update
          sudo apt-get install -y ./apiworkbench_1.4.0_amd64.deb

      - name: Run the collection
        env:
          API_KEY: ${{ secrets.API_KEY }}
        run: |
          apiworkbench-cli run tests/shop.json \
            -e tests/staging.json \
            --env-var "apiKey=$API_KEY" \
            -r cli,junit,html \
            --reporter-junit-export results/junit.xml \
            --reporter-html-export results/report.html

      - name: Keep the reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: api-test-reports
          path: results/

The job fails when the runner exits with 1 or 2. if: always() keeps the reports even then.

GitLab CI

yaml

api-tests:
  image: ubuntu:24.04
  before_script:
    - apt-get update
    - apt-get install -y curl ca-certificates
    - curl -fsSLO https://github.com/Dhiva-Labs/ApiWorkbench/releases/download/v1.4.0/apiworkbench_1.4.0_amd64.deb
    - apt-get install -y ./apiworkbench_1.4.0_amd64.deb
  script:
    - >
      apiworkbench-cli run tests/shop.json
      -e tests/staging.json
      --env-var "apiKey=$API_KEY"
      -r cli,junit
      --reporter-junit-export results/junit.xml
  artifacts:
    when: always
    reports:
      junit: results/junit.xml
    paths:
      - results/

Define API_KEY as a masked CI/CD variable in the project's settings. GitLab reads the JUnit file and shows the tests in the pipeline view.

Troubleshooting

"has N collections … Pick one with --collection"

The file is a workspace export with several collections. Add --collection "Name", using a name from the message.

"Cannot find the environment"

The value of -e is neither an existing file nor the exact name of an environment inside the run file. Check the path and the spelling.

"this collection has scripts, but this build … has no script engine"

The runner could not load its script engine, so scripts are skipped and only no-code tests run. Use the runner from an official package, kept in the folder it was installed to.

The run passes in the app and fails on the command line

The command line does not use the app's selected environment, globals or cookies. Pass the environment with -e, globals with -g, and any secret values with --env-var.

Next steps

CI automation: where API tests fit in a pipeline.
Data-driven runs: the data file formats.
Export: produce the files the runner reads.

Command-line runner ​

Where to find it ​

Your first run ​

Usage ​

Options ​

Choosing what to run ​

Variables ​

Run control ​

Output ​

Reporters ​

The cli reporter ​

The json reporter ​

The junit reporter ​

Exit codes ​

How a run behaves ​

Examples ​

Run in CI ​

GitHub Actions ​

GitLab CI ​

Troubleshooting ​

"has N collections … Pick one with --collection" ​

"Cannot find the environment" ​

"this collection has scripts, but this build … has no script engine" ​

The run passes in the app and fails on the command line ​

Next steps ​

Command-line runner

Where to find it

Your first run

Usage

Options

Choosing what to run

Variables

Run control

Output

Reporters

The cli reporter

The json reporter

The junit reporter

Exit codes

How a run behaves

Examples

Run in CI

GitHub Actions

GitLab CI

Troubleshooting

"has N collections … Pick one with --collection"

"Cannot find the environment"

"this collection has scripts, but this build … has no script engine"

The run passes in the app and fails on the command line

Next steps