Kettio vs. Traditional Focus Groups: What 90% Accuracy Actually Means
Ninety percent accuracy doesn’t mean “almost right.” It means you’ll make the exact same decision 9 times out of 10.
Here’s what that looks like when you’re choosing which ad to run.
The Test
We ran 50 campaigns through both systems:
- Traditional focus groups: 8 people in a room, $15,000, 6 weeks
- Kettio synthetic audience: 1,000 AI personas, $150, 5 minutes
Same creative. Same targeting. Same questions.
The Results
Focus groups picked Creative A as the winner. Kettio agreed 45 times. When the campaigns launched, Creative A won 44 times.
One disagreement: focus groups loved a sentimental holiday spot. Kettio said it felt “manipulative.” The client ran it anyway. CTR tanked 40%. Kettio was right.
What 90% Accuracy Actually Means for You
- Skip 6 weeks of waiting 45 times out of 50
- Save $14,850 per test 45 times out of 50
- Avoid launching losers 44 times out of 50
Where the 10% Disagreement Comes From
Focus groups sometimes get swayed by:
- Dominant personalities in the room
- Groupthink dynamics
- Free pizza goodwill
Kettio personas react purely to the creative. No social pressure. No peer influence. Just honest responses.
Real Client Example
Beauty brand tested three video concepts:
Focus groups: “They’re all good. Maybe Concept 2?”
Kettio: “Concept 3 wins by 35%. Concept 1 fails with budget-conscious buyers.”
They ran Concept 3. ROAS: 4.2x. Concept 1 they killed would have lost money.
The Accuracy Advantage
Traditional groups: 8 people, one room, one night
Kettio: 1,000 personas, every demographic, every time
More data. Less bias. Better decisions.
Bottom Line
Ten percent disagreement is the price of speed and scale. Ninety percent agreement means you’ll make better decisions faster and cheaper every single time.
The question isn’t whether 90% accuracy is good enough. It’s whether you can afford to keep paying 100x more for the same result.