A/B Testing Basics for Small Websites
How small sites can run honest A/B tests: choosing what to test, writing a hypothesis, sample size reality, running time, reading results and alternatives.
An A/B test shows two versions of a page to different visitors and compares which gets more of a chosen action. It is the cleanest way to learn what works, because it removes the guesswork about whether a change helped or whether traffic just happened to be better that week. But testing is easy to do badly, particularly on small sites. Here is how to use it honestly, and when to use other methods instead.
What an A/B test is
You create a variant of a page or element, such as a different headline. A testing tool splits incoming visitors randomly: half see the original (the control) and half see the variant. After enough visitors, you compare conversion rates. Randomisation matters: comparing this month with last month is not a test, because seasons, campaigns and news all change behaviour.
The traffic problem
Statistical confidence needs enough conversions in each version. If your site gets a few hundred visitors a month and a handful of enquiries, a test may take months to detect anything but a huge difference. Early results often look dramatic and then fade as more data arrives. Before testing, estimate honestly: if you currently get 20 conversions a month, a change that improves them by ten percent produces two extra, which no test can distinguish from chance in a short time. Many sample size calculators online will tell you the numbers needed, based on your baseline rate and the smallest improvement you care about.
If you do not have the traffic, do not despair. Use the alternatives later in this article.
Start with a hypothesis
Do not test random ideas. Write a hypothesis in this form: "Because we saw X, we believe changing Y will cause Z, which we will measure by W." For example: "Because session recordings show visitors scrolling past the form, we believe moving the form above the fold will increase quote requests, measured by form submissions per visit." The reasoning behind a hypothesis keeps the test meaningful and helps you learn even when the variant loses.
What to test first
| Element | Example variants |
|---|---|
| Headline | Benefit-led versus feature-led wording |
| Call-to-action text | "Get my quote" versus "Contact us" |
| Form length | Four fields versus seven |
| Proof placement | Testimonial near the button versus lower down |
| Price presentation | Showing a starting price versus "contact us" |
| Page layout | Long page versus short page |
Test one meaningful change at a time. Tiny tweaks, like a slightly different button colour, rarely produce differences you can detect. Bigger, evidence-driven changes do.
Running the test properly
- Choose one primary metric before you start, and do not switch after seeing results.
- Decide the minimum run time and sample size in advance.
- Run for whole weeks, since weekday and weekend behaviour differ.
- Do not peek and stop the moment one version leads; repeated checking inflates false positives.
- Avoid running overlapping tests on the same page.
- Keep other changes, such as new ad campaigns, out of the test period where you can.
Check that both versions load equally fast. Some tools add a flicker that disadvantages the variant, which spoils the comparison.
Reading results
If the variant wins with a result your tool calls statistically significant, that suggests it is better under the conditions tested, not that the gain will be exactly the number shown. If there is no clear winner, you have learned that the change did not matter much, which is useful: you can pick the version you prefer for other reasons. If the variant loses, you learned something about your audience. Record each test, the hypothesis, dates and result, so you do not repeat experiments and so patterns emerge.
Alternatives when traffic is low
- Before-and-after comparisons, treated with caution and noted alongside other changes.
- User testing: ask five people to complete a task while talking aloud. Patterns appear quickly.
- Session recordings and heatmaps, where privacy rules and consent allow.
- Customer interviews and surveys asking what almost stopped them.
- Best-practice fixes: clear forms, fast loading and trust signals rarely need a test to justify.
Ethical and practical limits
Never test deceptive tactics, such as misleading prices or fake urgency. Be aware of privacy rules, because testing tools may set cookies that need consent. Use them in line with your cookie banner and privacy policy.
A small-site testing routine
If you do test, keep the process light. Keep a single document with a backlog of ideas, each with the reason behind it and how much effort it takes. Pick the idea with the strongest evidence and lowest cost, run it, record the result and move on. Over a year, even five or six well-run tests, combined with sensible fixes learned from user observation, can improve results more than dozens of random tweaks. The goal is learning, not winning every test.
Make speed a given
A slow page can swamp any headline improvement. Make sure the basics are in place before testing. Cloudflare CDN is available on Zebra Hosting sites; read the CDN page, and check WordPress hosting if your testing tool runs as a plugin. For the broader view of measurement, see the knowledgebase.
