The CFO doesn’t guess. Why should the CMO?
Countless B2B SaaS marketers have found themselves standing in a boardroom defending a bold creative direction with nothing but survey results. It’s a precarious place to be. As Kate O’Keeffe, CEO and founder of Heatseeker, pointed out on a recent episode of SaaS Half Full, a CFO would never walk into a meeting and say, “I talked to 30 people, and they think we’re solvent.”
Yet, marketers are constantly forced to justify millions in spend based on speculative data.
O’Keeffe says we can put an end to the false confidence we get from surveys by using behavioral signals.
The 30% problem: Why surveys sink campaigns
The biggest disconnect in modern marketing? Trusting that people actually do what they claim to do.
She explains:
“There’s only about a 20–30% correlation between what someone says in a survey and what they actually do.”
That leaves a massive margin of error, which often shows up later as underperforming campaigns, missed launches, and a lot of post-mortems where marketing ends up holding the bag.
Stop polling, start testing
Marketers can and should be moving away from opinion-based insights. To get the truth, you have to look at behavior.
There are two main ways to get behavioral signals today. One is first-party data, such as performance marketing, support tickets, and loyalty data. This is super valuable; however, this type of information has limits.
“First-party data can’t test ideas you’re not already selling or ideas you had in the shower this morning,” said O’Keeffe.
The other source: live market experiments.
Meet your new (synthetic) best friend
Heatseeker runs live market experiments by putting real ads into real feeds and measuring what people actually do (clicks, sign-ups, and intent) at scale.
Instead of debating survey responses, teams can look at how hundreds of thousands of people respond in-market. It’s a practical way to explore new ideas without overcommitting.
Those experiments feed into synthetic personas, and yes, despite the name, they’re grounded in real data.
Wait, what is a synthetic persona? It’s a private predictive model built entirely on real behavior. The personas are trained on:
- First-party data
- High-quality ethnographic interviews
- Verbatim customer language
- Live experiment results, including uplift metrics
These models act as a proxy for your ideal customer, and you can test your ideas on them. When a persona gives feedback, it shows where it got its answers.
Better data, fewer arguments
One of the most satisfying side effects of this new strategy is how it improves your internal dynamics.
“When you can walk into a boardroom with results from a live market experiment — results that show a bold idea that made people uncomfortable actually spoke to customers more deeply, more cleanly, more profoundly and gave us a 44% uplift over what we’ve been putting out for the last 12 months — everything changes,” said O’Keeffe.
With better data, conversations move faster. Debates are less personal. Decisions feel easier to defend, whether that’s with leadership, product or the board. And marketers get the freedom to push creative ideas.
Listen to the full episode with Kate O’Keeffe on SaaS Half Full, If you want to discuss how real insight can create bold PR and marketing strategy, chat with the team at PANBlast.
Want to be a guest on SaaS Half Full? Send us a note about your experience and potential topics.
Transcript
Generated by YouTube and cleaned up with ChatGPT Pro.
[0:00] Kate O’Keeffe:
One of the things that excites me most about great, quant-backed market insights is that they actually allow us to be more creative.
Because I work in AI and marketing, I hear a lot about “AI slop.” I hear about the race to the bottom, about volume over quality. But I really believe that when you can walk into a boardroom with results from a live market experiment—results that show a bold idea that made people uncomfortable actually spoke to customers more deeply, more cleanly, more profoundly—and gave us a 44% uplift over the boring crap we’ve been putting out for the last 12 months, everything changes.
That’s the joy of what we do: how deeply quant-backed these insights are, and how believable they are.
[0:56] Lindsey Groepper:
Hi, welcome back to SaaS Half Full, the only show serving B2B SaaS marketers. I’m Lindsey Groepper, EVP at PANBlast. I’ll be both your host and bartender today.
I had a great conversation with Kate O’Keeffe, CEO and founder of Heatseeker. Today we’re talking about rethinking market research strategies in the AI-driven era, including something called synthetic personas—where you’re learning from AI personas trained on real data.
So grab a drink and join me. Hi Kate, welcome to SaaS Half Full.
[1:32] Kate O’Keeffe:
Hi Lindsey. Wildly excited to be here.
[1:35] Lindsey Groepper:
I’m excited too—and also impressed. You’re usually based in the Bay Area, but you’re in Australia right now, and it’s about 6:00 a.m.
[1:48] Kate O’Keeffe:
Fresh as a daisy.
[1:50] Lindsey Groepper:
Not only is it early, but you’re sticking to the SaaS Half Full rules—you’re joining me for a drink. Your family is probably questioning your life choices, but you grabbed a glass of champagne. Cheers.
[2:10] Lindsey Groepper:
Today we’re talking about how marketers should rethink market research in the AI era. It’s been a couple of years since we’ve tackled this topic, and a lot has changed.
But before we dive in, Kate, tell us a bit about your background and what led you to found Heatseeker.
[3:02] Kate O’Keeffe:
I had a consumer brand in my 20s, spent my 30s in the Bay Area working at Cisco in innovation roles—building products, spinning startups out of a massive organization. After that, I joined BCG Digital Ventures, creating digital attacker brands for large companies.
So I’ve built brands from scratch many times. And what really hit me—especially in strategy consulting—is how much tooling finance, risk, and performance teams have. Marketing doesn’t have that same version of the truth.
A CFO never walks into a board meeting and says, “I asked 30 people and they think we’re solvent.” But marketers are often asked to justify decisions that way.
[5:06] Kate O’Keeffe:
We rely on surveys and interviews—people chasing gift cards, bots chasing gift cards—and we always have to wait. Nobody else in the business waits on the truth the way marketers do.
That was my ambition with Heatseeker: give marketers access to real, actionable truth, in real time. Not next month—right now, even while you’re arguing with a colleague in a meeting.
[5:52] Lindsey Groepper:
Who does Heatseeker work best for today?
[6:08] Kate O’Keeffe:
We work primarily with mid-to-large brands, usually brought in by the CMO because it’s a big shift in how decisions get made.
We serve large consumer brands—DoorDash, Uber—and we recently brought on L’Oréal. They evaluated about 50 AI startups globally and chose Heatseeker because of our approach and alignment with how they think about customers and technology.
[7:18] Lindsey Groepper:
Let’s talk about where you see the biggest disconnect between how companies think they should do research and what they could be doing today.
[7:52] Kate O’Keeffe:
The biggest unlock right now is shifting to purely behavioral signal. Surveys and interviews are speculative—“Would you buy this?”—and there’s only about a 20–30% correlation between what someone says in a survey and what they actually do.
That kind of signal sinks careers, products, and campaigns.
[8:45] Kate O’Keeffe:
There are two main ways to get behavioral signal today. One is first-party data—performance marketing, support tickets, loyalty data. That’s the holy grail.
But first-party data isn’t exploratory. It can’t test ideas you’re not already selling or ideas you had in the shower this morning.
[9:59] Kate O’Keeffe:
That’s where Heatseeker comes in. We combine first-party data with live market experiments.
Our AI tool runs real ads—usually on Meta—testing different value props, features, or offers. Real people see them. They click, sign up, or buy. It’s speculative, but still behavioral.
[11:22] Kate O’Keeffe:
We then make that data accessible through synthetic personas. If you have five customer personas, you can talk to them directly—ask them to draft briefs, pressure-test messaging, or explain what would make a campaign work.
And while they’re talking, they show their sources: verbatims, experiment results, uplift percentages.
[12:20] Lindsey Groepper:
Most people assume synthetic personas are just something AI made up. What makes them credible?
[12:42] Kate O’Keeffe:
Trust comes from provenance. It’s not enough to rewrite copy—you need to see where insights came from.
That’s the difference between Heatseeker and a generic LLM. We’re trained on your data, we show correlation to real-world experiments, and customers can tune that correlation over time by running more experiments.
[14:20] Lindsey Groepper:
So synthetic personas don’t replace human research?
[14:38] Kate O’Keeffe:
No. I’m a huge fan of high-quality ethnography. Humans talking about lived experience is still some of the best data there is.
What we avoid is massive incentive-driven surveys. That kind of speculative preference muddies the signal.
[16:06] Kate O’Keeffe:
AB testing isn’t new, but it’s usually used too late—just for creative. We use it earlier to test pain points and jobs-to-be-done.
The headline is the pain point. That’s how you learn what really matters.
[17:48] Lindsey Groepper:
Please tell me your personas still have alliterative names.
[17:53] Kate O’Keeffe:
They absolutely do. And depending on the data, they have sass too. If your customers are grumpy, you’ll hear about it.
[18:47] Lindsey Groepper:
Speed can derail quality. How do you manage bias and accuracy?
[19:40] Kate O’Keeffe:
Quality shows up in two ways. First, quant-backed insights let marketers be more creative. When 300,000 real people see an experiment, that’s not opinion—that’s signal.
Second, every experiment runs alongside a synthetic comparison. Customers can see correlation drift and correct it with small media spends. It’s not a black box.
[23:12] Lindsey Groepper:
Can you give a real-world example?
[23:46] Kate O’Keeffe:
Brand partnerships are a great example—two brands, totally new customer. We can test that audience in days, before launch, under stealth brands, and build personas that didn’t even exist before.
That’s how you end up marketing to someone like “Last-Minute Linda”—a busy sports mom who Instacarts everything to her hotel during tournaments.
[27:15] Lindsey Groepper:
What’s the biggest objection you face selling this to enterprise teams?
[27:52] Kate O’Keeffe:
Honestly? Most marketers say, “Thank God you’re here.”
The resistance usually comes from people playing with the personas for fun. Marketers get to work. Executives ask what the persona watched on TV in the ’90s.
[30:37] Kate O’Keeffe:
Marketers are stuck in a cycle: boards demand quant proof, surveys give false confidence, campaigns fail, and marketing takes the blame.
Behavioral signal breaks that cycle. At least then, you’re not owning mistakes that weren’t yours.
[32:12] Kate O’Keeffe:
When teams trust the customer signal, they argue less, move faster, and make fewer mistakes.
And when marketers are confident in performance, they swing for the fences. That’s what we’re here for.
[33:13] Lindsey Groepper:
We end every episode with a toast.
[33:26] Kate O’Keeffe:
I’m Irish, so: May you get to heaven a half hour before the devil knows you’re dead.
[33:45] Lindsey Groepper:
Cheers. Thanks again for joining us on SaaS Half Full. Until next time—bottoms up.