Not every website AB test is trying to win...

Sometimes in CRO / Experimentation, we test purely to not lose.

You would maybe be surprised how often we run experiments where the goal isn’t uplift.

It’s confirmation that a new element doesn’t tank conversions.

We might be:
- Rolling out accessibility improvements
- Rebranding a key page
- Simplifying layout to boost page speed

In these cases, we’re not chasing a shiny 12% lift . We just want statistical confidence that the change won’t hurt performance. 

This kind of experiment is called a non-inferiority test.

Not sexy. Not headline-grabbing. But bloody important.

Because it’s often the tiny drops in conversion rate (the 0.3% here, the 0.7% there) that go unnoticed.

Until one day your website’s bleeding out from a thousand little cuts and the fingers start pointing (or worse they attribute the economy or a competitor).

That’s why we say: "any change made to a website should be tested" unless it's an obvious fix.

Sometimes, holding the line is the win.