# GitHub Action: run TestWatch from CI

Source: https://testwatch.dev/docs/github-action
Updated: 2026-10-17

testwatch-dev/action starts a run of your app’s tests, waits for the final verdict and fails the job when the verdict is one you chose. It is available on Starter and up.

## Set it up

1. In TestWatch, open **Settings → API keys** and create a key. Only owners and admins can. Keep both scopes ticked: `runs:write` starts runs, `runs:read` is needed to wait for the verdict. You can pin the key to one app. The key starts with `tw_` and is shown once.
2. Add it to your repository as a secret named `TESTWATCH_API_KEY`.
3. Add a workflow. The app’s **CI & deploys** page in TestWatch shows one with your app id filled in.

```
# .github/workflows/testwatch.yml
name: TestWatch
on:
  deployment_status:      # after each deploy, against the deployed URL
  workflow_dispatch:
jobs:
  e2e:
    if: github.event_name != 'deployment_status' || github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: testwatch-dev/action@v1
        with:
          api-key: ${{ secrets.TESTWATCH_API_KEY }}
          app: <your app id>
          # environment: staging
          # tags: smoke
          # fail-on: real_bug      # only confirmed bugs fail the job
          # junit: testwatch.xml
```

The action waits for the **final** verdict: when a test fails, TestWatch first checks whether it is a real bug or a page change, and the verdict is final once that is done. Without a final verdict before `timeout`, the job fails with the output `verdict=timeout`.

## Verdicts

| Verdict | Meaning |
| --- | --- |
| `pass` | Every test passed, flaky passes included. |
| `real_bug` | The app is broken. |
| `needs_review` | A test failed and needs a person, often because the app changed. |
| `infra` | The app or its login wasn’t usable, or the run was cancelled. |

When several tests fail, the worst verdict wins: `real_bug`, then `needs_review`, then `infra`.

## Inputs

| Input | Default | What it does |
| --- | --- | --- |
| `api-key` | Required | A TestWatch API key. Store it as a repository secret. |
| `app` | None | The app id. Not needed when the key is pinned to one app. |
| `environment` | The app’s default | Environment name or id to run against. |
| `environment-url` | The deployed URL on a deployment_status event | A deployed URL. Runs against the environment whose own URL, or one of whose preview patterns, matches it. |
| `url` | None | A preview URL to run against, with environment or the app’s default environment. It must match that environment’s preview patterns. |
| `tags` | All tests | Only tests with these tags, comma separated. |
| `tests` | All tests | Only these test ids, comma separated. |
| `account` | The environment’s | Test account id to sign in as. |
| `fail-on` | `real_bug,needs_review,infra` | Verdicts that fail the job. "none" never fails it. |
| `wait` | `true` | Wait for the final verdict, after failures have been checked. |
| `timeout` | `30m` | How long to wait for the verdict. |
| `junit` | None | Write a JUnit XML report to this path. |
| `api-url` | `https://api.testwatch.dev` | The API address. Leave it unset. |

Use `url` or `environment-url`, not both. `url` is the strict one: the run is refused when the URL doesn’t match the environment’s preview patterns. `environment` and `environment-url` can’t be combined either. On a `deployment_status` event without `environment`, `url` or `environment-url`, the action uses the deployed URL, and skips the run with a notice when the deployment didn’t succeed.

## Outputs

| Output | Value |
| --- | --- |
| `run-id` | The TestWatch run id. |
| `url` | Link to the run in TestWatch. |
| `verdict` | `pass`, `real_bug`, `needs_review`, `infra` or `timeout`. |

The job summary lists the failing tests with links, and each one is also an annotation.

## Recipe: Vercel previews

Vercel sends a `deployment_status` event for every deploy, previews included. Add your preview hosts as preview patterns on the environment first, for example `*-acme.vercel.app` (see [Preview URLs](https://testwatch.dev/docs/preview-urls-and-protection)). The deployed URL is then matched to the environment by those patterns.

```
on:
  deployment_status:
jobs:
  e2e:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: testwatch-dev/action@v1
        with:
          api-key: ${{ secrets.TESTWATCH_API_KEY }}
          app: <your app id>
```

## Recipe: Netlify deploy previews

Netlify doesn’t send `deployment_status` by default. Either add a TestWatch [deploy hook](https://testwatch.dev/docs/deploy-hooks-and-webhooks) to Netlify’s deploy notifications, or pass the preview URL yourself:

```
on:
  pull_request:
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      # ...wait for the Netlify deploy preview here...
      - uses: testwatch-dev/action@v1
        with:
          api-key: ${{ secrets.TESTWATCH_API_KEY }}
          app: <your app id>
          environment: staging
          url: https://deploy-preview-${{ github.event.number }}--acme.netlify.app
```

## When a preview run fails

A run against a preview URL stays out of the app’s main health numbers. When a test needs a fix on that branch, TestWatch proposes a PR fix and the job summary and annotations link to it. Accept it in TestWatch, then re-run the job.

## Source

The action is the [TestWatch CLI](https://testwatch.dev/docs/cli) wrapped for GitHub, so both behave the same way. Code: [github.com/testwatch-dev/action](https://github.com/testwatch-dev/action); listing: [TestWatch Playwright E2E on GitHub Marketplace](https://github.com/marketplace/actions/testwatch-playwright-e2e). On other CI systems, use the CLI or the [REST API](https://testwatch.dev/docs/api).