Test every preview deploy before it merges
A preview URL is the cheapest place to find a broken signup: before review, before main. TestWatch runs your existing tests against each preview from GitHub Actions or a deploy hook, on Starter and above.
The workflow
# .github/workflows/testwatch.yml
on:
deployment_status:
jobs:
e2e:
if: github.event.deployment_status.state == 'success'
runs-on: ubuntu-latest
steps:
- uses: testwatch-dev/action@v1
with:
api-key: ${{ secrets.TESTWATCH_API_KEY }}
app: <your app id>
From the Action’s README. Vercel sends a deployment_status event for every deploy, previews included, and the Action uses the deployed URL automatically.
How it works
On the environment in TestWatch, an admin adds preview patterns such as *-acme.vercel.app: one *, in the leftmost part of the host. A preview run uses that environment’s tests and test accounts.
Netlify doesn’t send deployment_status by default. Add a TestWatch deploy hook to Netlify’s deploy notifications, or pass the preview address in the Action’s url input. url is strict: the run is refused when the address doesn’t match a preview pattern. Deploy hooks are signed, either with a generic HMAC or with Netlify’s own signature. GitLab and Bitbucket have no integration; the CLI (--url) and the REST API work there.
When you pass environment-url, or the Action reads it from the deploy event, TestWatch first looks for an environment with that exact address, then for one whose patterns match. If several environments match, the app’s default one is used.
Anyone can create a host like evil-acme.vercel.app. So on shared hosts, logins and protection secrets go to a preview only after an admin marks those patterns as trusted. Until then, tests that need a login stop as “login withheld”; tests without a login still run.
Protected previews
Previews and staging behind protection can be tested on every plan. TestWatch supports HTTP Basic auth, a Vercel protection bypass token, a Cloudflare Access service token, or up to five custom headers. They are stored encrypted and sent only to the run’s exact origin. They aren’t sent to third-party hosts or on redirect hops.
What you see when it fails
The job waits for the final verdict, after TestWatch has checked each failure. By default it fails on a real bug, a test that needs review, or an app that wasn’t usable; fail-on narrows that. The job summary links each failing test, and each one is also an annotation.
Preview runs stay out of the app’s health numbers, the weekly digest and the “last pass” comparison. Slack and email alerts include them only when a rule says so.
When a PR changes the page on purpose, say “Register” becomes “Sign up”, TestWatch proposes a fix for that PR only. It may change the text an assertion expects, not what it checks. Click “Accept for this PR” and later preview runs on that branch use it, while main keeps the old test. When the change ships and the main test fails the same way, the accepted fix is tried against main and applied only after it passes twice, with an Undo.
What breaks it
- Preview runs, the Action, the API and deploy hooks need Starter or above.
- A gate that checks only a custom header, with no cookie, loses the header after a redirect.
- WebSocket connections don’t get the protection headers.
Related
- Checkout flows
- Email verification
- Clean test data
- Docs: GitHub Action
- Docs: Preview URLs and protected staging
Free: 5 tests, 500 runs a month, 2 apps and one daily schedule. Unlimited teammates, no card.