Your tests pass. What are they actually protecting?

If AI writes a feature and then writes tests based on that feature, both can agree on the same mistake. The tests pass because they confirm the behaviour that was implemented, even when that behaviour differs from what the customer needed.

That is the concern behind a question I raised on LinkedIn: how much confidence should we take from a green test report?

For a founder, the useful next step is to ask which promises to customers those tests actually check.

Choose a real customer journey

Consider a fictional appointment service. The normal booking journey is easy to describe: choose a time, pay, receive confirmation.

Now ask what should happen if two people try to book the last appointment together. Or if a customer sees an error after paying and tries again. Or if the booking succeeds but the confirmation email fails.

These questions expose decisions the business needs to make. Should the appointment remain reserved while payment is pending? How does a customer know whether to retry? What should staff see when a message has failed?

Your developer can work out how to implement and test the answers. You need to help establish what the answers should be. OWASP's business logic guidance recommends testing business rules rather than simply reproducing implementation details, including situations involving repeated or simultaneous actions.

Make the expected result specific

“Check that refunds work” leaves a lot open. A better example describes the starting situation, the action and the expected result.

For example: a customer has one paid booking, staff approve a full refund, and the booking history should show the refund without deleting the original payment record. The actual rules will depend on your service, but the example gives everyone something concrete to discuss.

Ask how the team checks the final result. A success message on screen does not answer every question about the payment, the booking record or what the customer sees later.

This is also a good way to expose disagreements early. If the founder expects a refund to cancel the appointment and the developer expects those to be separate actions, the test discussion has already prevented a misunderstanding.

Ask where each check belongs

You do not need every scenario tested by opening a browser and clicking through the whole product. Some checks concern a calculation, others concern communication between systems, and others need to follow a customer's journey through the interface.

Martin Fowler's test pyramid describes the value of balancing these kinds of checks. For a founder, the practical question is whether the team can explain how its choices cover the risks in your product.

I would ask for a walkthrough of one important journey and the checks around it. Which failures would stop a release? Which depend on a manual check? Which would only become visible after a customer reported them?

Those answers are more informative than a test count on its own.

Use recurring bugs to find a missing check

When the same problem returns, ask the team to compare the latest failure with the earlier one. Was the previous fix incomplete? Did another route produce the same result? Did the intended rule change without the test changing?

A useful follow-up records the cause, the correction and how the team will detect that particular failure next time. It should also identify anything still untested. That gives you an honest basis for deciding whether the product is ready for another release.

JiNi helps investigate recurring bugs and fix broken features in existing products. If releases keep bringing back problems you thought were resolved, contact us about the affected workflow. Tell us what should happen, what customers are experiencing and what has already been tried, so we can discuss how to investigate it.