Connect your AI coding agent (MCP)

Setup

1. An owner or admin turns on AI coding agents (MCP) under Settings → API keys. It is off by default.
2. Add the server to your AI app (below). The first time it connects, it opens TestWatch in your browser: sign in, check what it may do, and approve.

On the approval page you choose what the app may do: read results (always), start and cancel runs, and generate tests. You can limit it to one app. It connects your current workspace; to connect another one, switch to it in TestWatch first. Every connection is listed under Settings → API keys → Connected AI apps, where you can disconnect it. Changing your password or signing out everywhere disconnects your AI apps too; removing a member disconnects theirs.

Claude Code

claude mcp add --transport http testwatch https://mcp.testwatch.dev/mcp
# then, in Claude Code: /mcp → testwatch → Authenticate

Cursor

// .cursor/mcp.json — Cursor opens TestWatch to sign in
{
  "mcpServers": {
    "testwatch": { "url": "https://mcp.testwatch.dev/mcp" }
  }
}

VS Code (Copilot)

// .vscode/mcp.json — VS Code opens TestWatch to sign in
{
  "servers": {
    "testwatch": { "type": "http", "url": "https://mcp.testwatch.dev/mcp" }
  }
}

Codex

codex mcp add testwatch --url https://mcp.testwatch.dev/mcp
codex mcp login testwatch

claude.ai, ChatGPT and other apps

Add a custom connector with the URL https://mcp.testwatch.dev/mcp; the app opens TestWatch to sign in.

With an API key instead

For CI, headless agents or clients without sign-in, create an API key under Settings → API keys and send it as a Bearer token. “Start and cancel runs” lets the agent start runs, “Generate tests” lets it write new ones; a key with only “Read run results” can only read.

# TESTWATCH_API_KEY = a key from TestWatch: Settings, API keys.
# Single quotes: your shell must not expand the key. Claude Code keeps ${TESTWATCH_API_KEY}
# in .mcp.json and reads the variable when it connects, so the key is never written to a file.
claude mcp add --transport http --scope project testwatch https://mcp.testwatch.dev/mcp \
  --header 'Authorization: Bearer ${TESTWATCH_API_KEY}'

# Codex (~/.codex/config.toml)
[mcp_servers.testwatch]
url = "https://mcp.testwatch.dev/mcp"
bearer_token_env_var = "TESTWATCH_API_KEY"

For clients that only start local servers, use the bridge (it keeps the key in that client’s config file):

// For clients that only start local (stdio) servers, the mcp-remote bridge.
// The client fills ${TESTWATCH_AUTH} from "env"; this keeps the key in the client's config.
{
  "mcpServers": {
    "testwatch": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://mcp.testwatch.dev/mcp", "--header", "Authorization:${TESTWATCH_AUTH}"],
      "env": { "TESTWATCH_AUTH": "Bearer tw_..." }
    }
  }
}

Tools

Deleting tests, apps or environments is not possible over MCP; that stays in the dashboard.

The loop it is built for

You: "I changed the checkout. Check it on the preview before I merge."

Agent → testwatch_status(app)                  environments + preview URL patterns
Agent → testwatch_find_tests(app, "checkout")  the tests that cover it
Agent → testwatch_run_tests(app, tags: ["checkout"], target_url: "https://pr-42.shop.vercel.app")
Agent → testwatch_get_run(run)                 waits, until the verdict is final
Agent → testwatch_get_result(result)           why it failed: real bug, step, assertion diff
Agent    fixes the code, pushes, and re-runs only the failed tests

Runs need no AI on our side, so they count like any other run against your monthly runs. A run takes from about 30 seconds to a few minutes. testwatch_run_tests and testwatch_get_run wait up to 45 seconds and then return progress; the agent calls testwatch_get_run again until the verdict is final, after TestWatch has checked each failure. An identical call within two minutes (a client retrying after a dropped connection) maps to the same run; pass a new idempotency_key to force a fresh one.

Generating tests

You: "Add an end-to-end test for the new gift-card checkout."

Agent → testwatch_find_tests(app, "gift card")     nothing covers it yet
Agent → testwatch_generate_test(app, goal: "Buy a product with a gift card and check the order total")
        TestWatch asks for a confirmation (it uses a generation)
Agent → testwatch_get_test(test, wait_seconds: 45) …until it is ready (3–15 min)
Agent → testwatch_run_tests(app, tests: [test])

A generation uses the workspace’s generation budget, so TestWatch asks first. Apps that support confirmation prompts show you one, and then only your answer counts; with others, the agent has to show you the goal and call again with confirm: true, and your AI app’s own approval prompt is the second check. Goals that contain a password, key or token are refused: tests sign in with the app’s test accounts. Each connection can start up to 10 generations an hour.

Evidence is untrusted

Error text, console errors, failed requests and emails come from the site under test, and a page can put text there that tries to give your agent instructions. TestWatch strips invisible characters, masks secrets and wraps that text — and every test title, step and name — in marked <untrusted-data-…> blocks, and the server tells the agent to treat them as data. That lowers the risk; it can't remove it. Keep your agent asking before it runs commands, and only allow what an agent needs.

Screenshots are sent only when the agent asks for one, and never for apps with screenshots turned off. Evidence you send to an agent goes to the AI provider that agent uses, under your agreement with them.

Limits

• Each connection can start up to 10 runs a minute and 10 generations an hour, and call the server up to 120 times a minute.
• Up to 3 waiting calls per connection at a time; more answer at once without waiting.
• Screenshots are never sent for an app whose screenshots are turned off, including ones taken before the switch.
• Plan limits apply as everywhere else; a plan limit comes back as a message the agent shows you.