# Connect your AI coding agent (MCP)

Source: https://testwatch.dev/docs/ai-agents
Updated: 2026-10-27

The TestWatch MCP server lets a coding agent run your end-to-end tests, wait for the verdict, read why a test failed and generate new tests, so it can check its own changes against the running app. It is on Starter and up.

## Setup

1. An owner or admin turns on **AI coding agents (MCP)** under **Settings → API keys**. It is off by default.
2. Add the server to your AI app (below). The first time it connects, it opens TestWatch in your browser: sign in, check what it may do, and approve.

On the approval page you choose what the app may do: read results (always), start and cancel runs, and generate tests. You can limit it to one app. It connects your current workspace; to connect another one, switch to it in TestWatch first. Every connection is listed under **Settings → API keys → Connected AI apps**, where you can disconnect it. Changing your password or signing out everywhere disconnects your AI apps too; removing a member disconnects theirs.

### Claude Code

```
claude mcp add --transport http testwatch https://mcp.testwatch.dev/mcp
# then, in Claude Code: /mcp → testwatch → Authenticate
```

### Cursor

```
// .cursor/mcp.json — Cursor opens TestWatch to sign in
{
  "mcpServers": {
    "testwatch": { "url": "https://mcp.testwatch.dev/mcp" }
  }
}
```

### VS Code (Copilot)

```
// .vscode/mcp.json — VS Code opens TestWatch to sign in
{
  "servers": {
    "testwatch": { "type": "http", "url": "https://mcp.testwatch.dev/mcp" }
  }
}
```

### Codex

```
codex mcp add testwatch --url https://mcp.testwatch.dev/mcp
codex mcp login testwatch
```

### claude.ai, ChatGPT and other apps

Add a custom connector with the URL `https://mcp.testwatch.dev/mcp`; the app opens TestWatch to sign in.

### With an API key instead

For CI, headless agents or clients without sign-in, create an API key under **Settings → API keys** and send it as a Bearer token. “Start and cancel runs” lets the agent start runs, “Generate tests” lets it write new ones; a key with only “Read run results” can only read.

```
# TESTWATCH_API_KEY = a key from TestWatch: Settings, API keys.
# Single quotes: your shell must not expand the key. Claude Code keeps ${TESTWATCH_API_KEY}
# in .mcp.json and reads the variable when it connects, so the key is never written to a file.
claude mcp add --transport http --scope project testwatch https://mcp.testwatch.dev/mcp \
  --header 'Authorization: Bearer ${TESTWATCH_API_KEY}'

# Codex (~/.codex/config.toml)
[mcp_servers.testwatch]
url = "https://mcp.testwatch.dev/mcp"
bearer_token_env_var = "TESTWATCH_API_KEY"
```

For clients that only start local servers, use the bridge (it keeps the key in that client’s config file):

```
// For clients that only start local (stdio) servers, the mcp-remote bridge.
// The client fills ${TESTWATCH_AUTH} from "env"; this keeps the key in the client's config.
{
  "mcpServers": {
    "testwatch": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://mcp.testwatch.dev/mcp", "--header", "Authorization:${TESTWATCH_AUTH}"],
      "env": { "TESTWATCH_AUTH": "Bearer tw_..." }
    }
  }
}
```

## Tools

| Tool | What it does | Permission |
| --- | --- | --- |
| `testwatch_status` | Apps, latest results per environment, review backlog and plan usage. | `runs:read` |
| `testwatch_find_tests` | Search an app’s tests by text, tag or status. | `runs:read` |
| `testwatch_get_test` | A test’s goal, steps, status, recent results and (optionally) its code. | `runs:read` |
| `testwatch_run_tests` | Start a run (all, by tag or by id; optionally on a preview URL) and wait ≤45 s. | `runs:write` |
| `testwatch_get_run` | Wait ≤45 s for progress, then the verdict (provisional or final) and failures. | `runs:read` |
| `testwatch_get_result` | Why one test failed: category, failing step, assertion diff, logs, screenshot. | `runs:read` |
| `testwatch_cancel_run` | Cancel the tests of a run that haven’t started yet. | `runs:write` |
| `testwatch_review_inbox` | Real bugs, suggested fixes and blocked tests waiting for a person. | `runs:read` |
| `testwatch_generate_test` | Write a new Playwright test from a plain-English goal (asks you first; 3–15 min). | `tests:write` |

Deleting tests, apps or environments is not possible over MCP; that stays in the dashboard.

## The loop it is built for

```
You: "I changed the checkout. Check it on the preview before I merge."

Agent → testwatch_status(app)                  environments + preview URL patterns
Agent → testwatch_find_tests(app, "checkout")  the tests that cover it
Agent → testwatch_run_tests(app, tags: ["checkout"], target_url: "https://pr-42.shop.vercel.app")
Agent → testwatch_get_run(run)                 waits, until the verdict is final
Agent → testwatch_get_result(result)           why it failed: real bug, step, assertion diff
Agent    fixes the code, pushes, and re-runs only the failed tests
```

Runs need no AI on our side, so they count like any other run against your monthly runs. A run takes from about 30 seconds to a few minutes. `testwatch_run_tests` and `testwatch_get_run` wait up to 45 seconds and then return progress; the agent calls `testwatch_get_run` again until the verdict is **final**, after TestWatch has checked each failure. An identical call within two minutes (a client retrying after a dropped connection) maps to the same run; pass a new `idempotency_key` to force a fresh one.

## Generating tests

```
You: "Add an end-to-end test for the new gift-card checkout."

Agent → testwatch_find_tests(app, "gift card")     nothing covers it yet
Agent → testwatch_generate_test(app, goal: "Buy a product with a gift card and check the order total")
        TestWatch asks for a confirmation (it uses a generation)
Agent → testwatch_get_test(test, wait_seconds: 45) …until it is ready (3–15 min)
Agent → testwatch_run_tests(app, tests: [test])
```

A generation uses the workspace’s generation budget, so TestWatch asks first. Apps that support confirmation prompts show you one, and then only your answer counts; with others, the agent has to show you the goal and call again with `confirm: true`, and your AI app’s own approval prompt is the second check. Goals that contain a password, key or token are refused: tests sign in with the app’s test accounts. Each connection can start up to 10 generations an hour.

## Evidence is untrusted

Error text, console errors, failed requests and emails come from the site under test, and a page can put text there that tries to give your agent instructions. TestWatch strips invisible characters, masks secrets and wraps that text — and every test title, step and name — in marked `<untrusted-data-…>` blocks, and the server tells the agent to treat them as data. That lowers the risk; it can't remove it. Keep your agent asking before it runs commands, and only allow what an agent needs.

Screenshots are sent only when the agent asks for one, and never for apps with screenshots turned off. Evidence you send to an agent goes to the AI provider that agent uses, under your agreement with them.

## Limits

- Each connection can start up to 10 runs a minute and 10 generations an hour, and call the server up to 120 times a minute.
- Up to 3 waiting calls per connection at a time; more answer at once without waiting.
- Screenshots are never sent for an app whose screenshots are turned off, including ones taken before the switch.
- Plan limits apply as everywhere else; a plan limit comes back as a message the agent shows you.