How to design a validation experiment that tells the truth

Start with the riskiest assumption

Test the assumption that, if wrong, would sink the business. Not the one that's easiest to test.

Most owners are tempted to test something comfortable, like which logo people prefer. That feels productive, but it doesn't change the decision. Go back to your list of deciding assumptions (the go/no-go checklist shows how to build one) and pick the shakiest. Usually it's one of these:

• Will enough people in my area buy this at all?
• Will they pay my price, not the cheapest competitor's?
• Can I reach them at a cost per customer my margins allow?
• Will they buy again, or only once?

One experiment should answer one of these questions. Trying to answer all four in a single test usually answers none of them clearly.

Choose the lightest test that gives a real signal

Different tests give signals of different strength. Pick the lightest one that would actually change your decision.

Interviews done properly

Interviews are cheap but easy to get wrong. Ask about what people already do, not what they'd do.

Weak question: "Would you use a mobile dog-grooming service?" Strong questions: "When did you last get your dog groomed? Where? What did it cost? What was annoying about it?"

If ten people describe the same annoyance unprompted, you've found something. If they all say "it's fine, I go to the place down the road," that's information too.

Landing pages

A single page that describes the offer, shows the price, and asks for one action. Keep it plain. The test is the offer, not the design. Aim a small paid campaign at the area you'd actually serve, and measure how many visitors take the action and what each one cost.

Refundable deposits

The strongest signal short of opening. Even $20 separates genuine buyers from the merely curious. Be clear on the page that the deposit is refundable, what it secures, and when the service is expected to start.

Set the pass mark before the test goes live

Write down, before launch, the result that means proceed and the result that means stop.

A worked example. Your break-even model says you need 60 paying customers a month and can afford to spend up to $70 to win each one. From experience in comparable offers you expect about 40% of deposit-payers to become customers. So the test needs to show:

• Proceed: at least 25 deposits in three weeks, at a cost of $28 or less per deposit (40% of $70).
• Stop: fewer than 10 deposits, or a cost above $50 per deposit.
• In between: change one thing, such as the headline, price or area, and run a second three-week test.

Share the pass mark with a partner or adviser before the test starts. It's much harder to rationalise a weak result when someone else has seen what you wrote.

Run it honestly

An honest test is also a more accurate one, because people respond to what's really on offer.

• Say it's launching. Don't describe a service as available if it isn't. Use plain words: "We're launching in [Month]. Reserve a spot."
• Make deposits refundable, and refund promptly if you don't proceed.
• Don't make claims you can't back. Canada's Competition Bureau treats misleading representations seriously, including environmental claims; its guidance on environmental claims is worth reading if your offer has a green angle. If a claim isn't proven, leave it out.
• Show the real price. A test without the price tells you about curiosity, not demand.

Handle contact details properly

Any test that collects names, emails or phone numbers collects personal information, and Canadian rules apply.

• Consent for follow-up. The federal anti-spam legislation asks businesses to obtain consent before sending commercial electronic messages. Ask clearly on the page whether people want to hear from you when you launch, and keep a record.
• Privacy. The federal private-sector privacy law, PIPEDA, covers personal information collected in commercial activity in most provinces. Alberta, BC and Quebec have their own similar laws. Collect only what you need and say what you'll use it for.
• Ownership. Run the test from accounts in your business's name so the data stays with you.

This is general information, not legal advice. If your test involves contracts, deposits or regulated activities, check with a lawyer.

Read the result without fooling yourself

When the test ends, report four numbers and compare them with the pass mark.

1. Reach: how many people saw the offer.
2. Action: how many took the step.
3. Cost per action: total spend divided by actions.
4. Quality: how many of those who acted match your target buyer.

Then decide: proceed, stop, or adjust and retest. Avoid three common traps:

• Explaining away a miss. "The ads weren't quite right" may be true, but then the next step is a better test, not a launch.
• Counting the wrong people. Sign-ups from outside your service area, or from friends, don't count.
• Over-reading a small sample. Twelve deposits at a good cost is encouraging. It's not proof of a 500-customer business. Scale the test before you scale the business.

A two-week starter plan

If you want to try a first test yourself:

• Days 1–2: pick the riskiest assumption and write the pass mark.
• Days 3–5: build a one-page offer with the price and a single action.
• Days 6–13: run a small, tightly targeted ad campaign to your service area.
• Day 14: count reach, actions, cost per action and quality. Decide.

If the result is borderline or the stakes are high, we can design and run a more rigorous version; see pricing or call, text or email to talk it through.