Amikoo Blog

How to Generate Playwright Tests on Every Pull Request

Written by The Amikoo Team | Sep 2, 2026, 7:54:31 PM

 

You can configure agents to watch your repository, read the diff on every pull request, find the coverage gaps the change introduced, and commit the missing Playwright tests to that PR before a human reviews it. In this live demo, MuukLabs CTO Renan Ugalde shipped two features — a login redesign and a new dashboard — without hand-writing a single test, then used a second agent to separate real defects from environment noise in the failures. 

Key takeaways

  • Tests can arrive with the PR, not after it. A Test Generator worker reads the code changes, identifies what's newly uncovered, and pushes the tests to the PR branch automatically.

  • The bottleneck was never writing scripts. It's the gap between shipping a feature and having coverage for it — a gap that widens every time the team is busy.
  • Volume of tests isn't the metric that matters. Renan's example: a 700-test regression suite with 200 failures. When nobody can triage 200 results, the team ships anyway and defects reach production.
  • Failure triage is its own job. In the demo, the agent classified a batch of failures as an environment error, not an application bug, named the branch, and reported that 7 test cases were affected.
  • Coverage work can start before the commit. A VS Code integration brings the same generation and gap analysis into the editor, ahead of any PR.

     

Why do tests always lag behind the code?

Because the work has always been sequential. You ship the feature, then someone writes the tests — in review if you're disciplined, and often not at all.

LLMs made writing the script fast, but they didn't remove the bottleneck. Someone still has to stop, decide what's uncovered, prompt for it, check the result, and commit it. That's still a person interrupting their work, which means it still gets deferred.

The shift-left approach Renan demonstrated removes the person from the loop entirely: you configure the trigger once, and the coverage work happens on every push whether anyone remembers it or not.

How does an agent write tests for a pull request automatically? 

By reading the diff, comparing it against existing coverage, and committing the missing tests to the PR branch.

Demo one: a login screen redesign. Renan ran the existing authentication suite on main first:  31 Playwright tests, all passing. Then he branched, and vibe-coded four new components into the login flow: sign up, remember me, magic link, and a cookie banner. The existing tests no longer covered any of it.

He pushed the branch and opened a PR containing only application code and no tests.

Amikoo picked up the event, analyzed what the change introduced, and pushed a new commit to the PR with the tests for the new components. Renan didn't write one.

Demo two was a bigger case: an entirely new executive dashboard replacing the old one, with forecasting and allocation components. The old dashboard tests failed against it, as expected. Same flow: open the PR, and the missing coverage arrives as a commit.

The point isn't that AI can write a Playwright test. It's that no human had to decide it was time to write one.

How do you tell a real defect from an environment failure fast? 

By making triage an agent's job instead of a person's.

This is the problem Renan spent the most time on, and it's the one that quietly kills trust in a suite:

You have 700 tests. 200 fail. Some are selectors, some are environment, some are real. Nobody has time to open 200 results — so the team ships anyway.

In the demo, the dashboard suite threw a batch of failures. Renan asked Amikoo to classify them. The agent pulled the run data, the execution video, and the failure context, and came back with a verdict: an environment error — not a test bug, not an application bug — traced to a URL pointing at the wrong branch, affecting 7 test cases, with the next step spelled out.

That's the difference between 200 red results and one actionable sentence. And it's what a Defect Logger worker uses to decide whether a failure is worth filing as a GitHub issue and pinging the team on Slack — so the notifications you get are the ones that mean something.

What are Workers, and which ones ran in this demo? 

Workers are background agents you configure once around a trigger (a PR opening, a test failing, a run finishing) that then act without anyone filing a ticket.

Three ran during the session:

  • Test Generator — watches for a PR, reads the diff, writes the missing Playwright tests, commits them to the branch.

  • Defect Logger — confirms a failure is a real defect before filing it as a GitHub issue and notifying Slack.
  • Auto-Fixer — takes the issue and repairs the affected test.

Renan also described the pattern most teams set up first: a scheduled run that emails a daily digest — everything's green, or here's what broke. A Worker can be built around any repeatable QA job you want off your team's plate.

Can I generate tests before I even commit?

Yes, that's what the VS Code integration is for.

The last segment previewed Amikoo embedded directly in VS Code, walked through on the same dashboard branch. From inside the editor, with access to the terminal and the working tree, find the coverage gaps for these changes and write the tests. It produced roughly 13 test cases in the project folder, ready to run locally, with the option to generate a coverage report as PDF or HTML.

The split is straightforward. The GitHub integration is for teams that want the safety net at the PR. The VS Code integration is for developers who don't want to wait that long.

Automate the Whole Process

Amikoo is a purpose-built AI QA agent system that analyzes your codebase, user flows, and test reports to find coverage gaps and generate tests in Playwright TypeScript — readable code committed to your own repo, that your team owns and runs wherever it already runs tests.

It works with GitHub and Azure DevOps repositories today, with Bitbucket coming. Migrating from Cypress or Selenium to Playwright? Amikoo can help with that migration.

New to this series? Start with our first webinar recap: Why LLM Test Automation Breaks at Scale.

Watch the full session above, and try Amikoo free at qa.amikoo.ai

FAQ

Can AI write the tests for a pull request automatically? 

 Yes. An agent can be configured to trigger when a PR opens, read the code changes in the diff, compare them against your existing test coverage, generate the missing tests, and commit them to the PR branch before human review — with no prompt from a developer. This is what Amikoo's Test Generator worker does: you configure the trigger once, and every subsequent PR arrives with its missing Playwright TypeScript tests already committed. 

How do you triage hundreds of failing tests without reviewing each one? 

A person can diagnose any single failure. The problem is doing it 200 times before a release. Amikoo consolidates the run data, execution video, and failure context for every run, then reports the failure type (application defect, test issue, or environment error) along with how many test cases are affected and the recommended next step. In this webinar's demo, Amikoo correctly identified a batch of failures as an environment error caused by a URL pointing at the wrong branch, affecting 7 test cases. Its Defect Logger worker uses that classification to decide whether a failure is worth filing as a GitHub issue. 

Where do AI-generated Playwright tests actually run?

 Wherever you want them to. Amikoo commits the tests to your repository as standard Playwright TypeScript, so you can run them through GitHub Actions or any CI/CD provider, locally on a developer machine, or on a dedicated test-execution server. You can also run them directly in the Amikoo app, which consolidates the results, execution videos, and failure context in one place. The code is readable and your team owns it either way, so running it in Amikoo is a convenience, not a dependency. 

Can I generate tests inside my IDE, before committing? 

Yes. Amikoo's VS Code integration analyzes your working changes in the editor, identifies coverage gaps, and writes the tests directly into your project folder so you can run them locally before pushing — instead of waiting for the PR. 

How is this different from asking Claude Code to write tests?

Test creation quality can be comparable if you follow best practices. Amikoo's agents are tuned for QA specifically, which makes generation faster, but that's not the key differentiator. The difference is that with Amikoo, you can automate the whole process: the PR trigger, coverage gap analysis, committed tests, failure classification, defect filing, repair, and reporting, connected as one system. A general-purpose coding agent produces an isolated artifact when you ask for it. Workers run the process when you don't.