6d • Builds & Questions
Stop Trusting Your Coding Agent When It Says Your App Works. Test It.
One of the biggest mistakes I see people making when building software with LLMs is assuming that because their agent says something works, it actually works.
We've all done it.
You ask Claude Code or Codex to build a feature. It writes the code, tells you everything is working, and gives you a nice summary of what it accomplished.
You open the application, click around a little, and move on to the next feature.
But did you actually test it?
Does the form save the correct information? What happens when someone enters invalid data? Can one user access another user's records? What happens when the network fails halfway through a request?
These are the kinds of questions a quality assurance (QA) tester asks.
And you don't need to know how to write code to start asking them.
There's an open-source tool called **Playwright** that can help you turn your coding agent into a much more useful testing assistant.
## What is Playwright?
Playwright is a browser automation framework developed by Microsoft.
It allows software to control a real browser, interact with websites, complete forms, click buttons, and verify what happens afterward.
With **Playwright MCP**, you can connect those browser capabilities directly to Claude Code, Codex, or another compatible agent.
MCP stands for Model Context Protocol. Think of it as a standard way of giving your LLM access to external tools.
Instead of just reading your application's source code, your agent can now open the application in a browser and interact with it.
It can:
- Navigate between pages and test links.
- Fill out forms and submit information.
- Test authentication and protected routes.
- Check whether buttons perform their intended actions.
- Inspect browser errors and failed network requests.
- Capture screenshots to help diagnose problems.
- Compare the application's actual behavior against your requirements.
Playwright MCP primarily uses a structured representation of the webpage, called an accessibility snapshot, to understand its controls and content.
This means it doesn't need to rely exclusively on screenshots to identify buttons, fields, and other interface elements.
But Playwright isn't automatically a complete QA department. You still need to define what correct behavior looks like, and some problems require tests outside the browser.
## Getting started in your terminal
If you already use Claude Code or Codex, getting started is fairly simple.
You'll need Node.js and an MCP-compatible coding agent.
For Claude Code, run:
```bash
claude mcp add playwright npx @playwright/mcp@latest
```
For Codex, run:
```bash
codex mcp add playwright npx "@playwright/mcp@latest"
```
These commands register the Playwright MCP server with your agent. Once connected, your agent can use its browser tools.
You can confirm the configuration with:
```bash
claude mcp list
```
Or, for Codex:
```bash
codex mcp list
```
Now open your development project, start your application locally, and launch your coding agent.
I recommend starting with a test environment rather than your live production application.
## Your first QA test
Let's say you've built a customer management application.
It has a login page, a dashboard, and a form for adding customers.
Instead of asking your agent whether the application works, give it a specific job.
**Try this prompt:**
> You are acting as my QA engineer.
>
> Use Playwright to test my locally running application.
>
> Test the login process, navigation, and customer creation form.
>
> Check what happens when fields are empty, contain invalid information, or are submitted correctly.
>
> Verify that submitted information actually appears in the application and persists after refreshing the page.
>
> Inspect browser errors and failed network requests.
>
> Do not modify any application code.
>
> Use only test accounts and synthetic customer data.
>
> Document each test, the expected result, the actual result, and whether it passed or failed.
>
> Include evidence for any failures. If a result cannot be verified, mark it as unverified rather than passed.
Now your agent has a defined responsibility.
It's not being asked to build something. It's being asked to find out whether the thing you already built behaves correctly.
## Start thinking like a QA engineer
You don't need to memorize testing frameworks or understand every line of your application's code.
You need to get better at asking what could go wrong.
Let's look at a simple example.
You've created a form that saves a customer's name, email address, and phone number.
A basic test would enter the information, click Save, and confirm that a success message appears.
That's a start, but it isn't enough.
A better testing process would ask:
**What happens if the email address is invalid?**
Does the application reject it, or does it save bad information?
**What happens if someone clicks Save twice?**
Does it create two customer records?
**What happens if the network connection drops?**
Does the application lose the information without warning?
**What happens if the user refreshes the page?**
Is the customer still there?
**What happens if another user signs in?**
Can they see information they aren't supposed to have access to?
These questions help you understand your application beyond whether its interface looks correct.
Some can be checked directly through Playwright. Others, especially authorization and data isolation, should also be verified through backend tests and database permissions.
## Don't confuse clicking a button with verifying the result
This is probably the most important part of the entire process.
Let's say your agent fills out a form and clicks Save.
The application displays:
"Customer successfully created."
Does that prove the customer was created?
No.
It proves that a success message appeared.
The application might have failed to write the record to the database. The message might display before the server responds. It might have saved the information under the wrong account.
A proper test should verify the resulting state.
For example:
1. Submit the customer form.
2. Confirm the expected record appears in the customer list.
3. Refresh the application.
4. Confirm the information persists.
5. Check the database or API response through an authorized test interface when appropriate.
6. Verify that an unrelated account cannot access the record.
You're checking what actually happened instead of what the application claims happened.
That is a habit worth developing early.
## Turn your QA process into a repeatable system
Once you find a bug, don't just fix it and forget about it.
Create a test that would catch the same problem if it happened again.
This is called regression testing.
For example, imagine your agent discovers that submitting a form twice creates duplicate records.
You fix the problem.
Two weeks later, another coding agent changes the form's submission logic.
Without a regression test, the same bug could return without anyone noticing.
Playwright also provides a separate testing framework called Playwright Test. You can use it to create automated tests that run every time you make changes.
I would eventually have my development workflow follow a process like this:
**Build:** Claude Code or Codex implements a feature.
**Review:** Another agent examines the changes and identifies possible failure conditions.
**Test:** Playwright exercises the affected user workflows.
**Verify:** Automated assertions and independent checks confirm the expected results.
**Report:** The agent documents what passed, what failed, and what couldn't be verified.
**Approve:** Only after the required checks pass do you consider the feature ready to merge or deploy.
For high-risk functionality, including payments, patient records, authentication, and permissions, browser tests are only one layer. You also need appropriate backend, security, and domain-specific validation.
## One thing I would change about how people learn to build with LLMs
A lot of tutorials focus on how quickly someone can build an application.
Build a dashboard in ten minutes.
Create a SaaS product with one prompt.
Launch an application without writing code.
But very few spend enough time teaching people how to determine whether what they built is reliable.
Your agent can be wrong. Your application can appear functional while producing incorrect results. Your database permissions can be misconfigured even when every button works.
You don't have to become an experienced software engineer to start understanding those risks.
Start by learning how to test your own work.
Install Playwright. Connect it to your agent. Give it a feature you already built and ask it to find everything that could go wrong.
Don't let it fix the problems immediately. Review what it found first.
Understand why the problem happened, what could have been affected, and how you'll know the fix actually worked.
We're spending way too much time following hype trains, trying every new model, and seeing how quickly we can generate software.
Not nearly enough time is being spent understanding what we're building.
**If you're going to use LLMs to build software, learning how to test that software should be part of the process from day one.**
0
0 comments
Stop Trusting Your Coding Agent When It Says Your App Works. Test It.
skool.com/cmdandconquer
CMD & Conquer helps designers, filmmakers, photographers, developers, and business owners build an office of agents around their work.
Powered by