Table of Contents
The Problem Every Growth Marketer Has
Every growth marketer has made the same expensive mistake.
You build a product page, or a landing page, or an ad. It feels right. The copy is tight, the visuals look good, the offer makes sense to you. You put budget behind it. And then you find out, from your analytics, two weeks and several thousand rupees later, that it wasn’t converting.
The feedback loop is broken. By the time real data tells you something is wrong, you’ve already paid for the lesson.
So what do you do instead?
What Everyone’s Trying Now: Just Ask ChatGPT
The newer answer, the one that more marketers are trying, is to just ask an AI.
“ChatGPT, rate this landing page 1 to 10.”
The answer is almost always somewhere between 7 and 9. The AI is polite. It wants to be helpful. And a helpful AI does not tell you that your page is terrible. So you get a number that feels like validation, and you’re no better off than when you started.
It doesn’t actually simulate a real person’s hesitation, scepticism, or confusion. To understand why “rate this 1 to 10” doesn’t work, you have to understand what you’re actually asking for. When you ask an AI to rate your product page, you’re asking it to produce a single number that summarises how good the page is. The AI doesn’t have a stake in the outcome. It has no buying history, no budget anxiety, no particular concern about whether the ingredients are safe for their kid. It’s not a customer. It’s a language model doing its best to produce a plausible answer to your question.
The result is a number that tells you almost nothing useful:
- It doesn’t tell you who would buy and who wouldn’t
- It doesn’t tell you why they would or wouldn’t
- It doesn’t tell you which specific part of your page is the problem
- And it doesn’t give you a distribution, which is the most important thing missing
A real audience isn’t a 7 out of 10. A real audience is 30% of people who would buy immediately, 40% who are on the fence, and 30% who would never buy no matter what. Those three groups need completely different things from your page. A single score collapses all of that into noise.
A Different Question to Ask the AI
The fix starts with asking a fundamentally different question.
Instead of asking the AI to rate your page, you ask it to respond as a customer would.
Imagine you hired 35 actors. Each one is playing a specific customer, a different age, a different worry (price, safety, effectiveness). You show them your product page. You don’t ask them to rate it. You ask them to just talk: “What do you think about this?”
You build a synthetic persona, a specific, detailed description of a real buyer type. Their age. Their primary concern is when making this type of purchase. Their level of familiarity with your brand. Their mindset when they encounter your content.
Then you show them your page and ask them to just talk. Not rate. No score. Just respond the way a real person would after reading it.
The responses you get are rich and varied:
“I’d order this immediately; the ingredient list is exactly what I’ve been looking for.”
“I’m interested, but I don’t know this brand. I’d want to see some reviews before I commit.”
“Feels expensive for something I’ve never tried. Would need a trial size first.”
“The claims sound strong, but I’d want to see some clinical backing before I put this on my child.”
Now you have text responses that carry real consumer nuance. But you still have a problem: how do you convert 35 different text responses into something you can measure and compare?
Semantic Similarity Rating: Converting Text into Probability Distributions
This is where SSR, Semantic Similarity Rating, comes in.
SSR is an open-source methodology developed by PyMC Labs. The core idea is elegant: instead of asking the AI to assign a number to your content, you measure how similar a persona’s text response is to a set of reference sentences that represent different positions on a scale.
For purchase intent, you might have five reference sentences:
- “I would never buy this product under any circumstances.”
- “I am very unlikely to purchase this product.”
- “I might consider buying this product, but I’m not sure.”
- “I would likely buy this product in the near future.”
- “I would definitely purchase this product immediately.”
Each persona’s response gets compared to all five using sentence embeddings and cosine similarity. The result isn’t “this response equals 4 out of 5.” It’s a probability distribution: this response is 5% likely to represent position 1, 10% likely to represent position 2, 25% likely to represent position 3, 40% likely to represent position 4, and 20% likely to represent position 5.
That distribution is called a PMF, a Probability Mass Function. And when you aggregate the PMFs across all 35 personas, you get something much more useful than a single score: you get a picture of how your whole audience is distributed across the response spectrum.
Instead of counting thumbs up or down, you’re measuring how similar what each person said sounds to someone who would definitely buy versus someone who absolutely would not. That measurement becomes:
“32% would immediately consider buying. 45% are on the fence. 23% are sceptical.”
That shape, that distribution, is the real insight. Because now you know:
- Is there a vocal minority that loves it? (Retargeting gold)
- Is the middle too flat? (Awareness problem, not a product problem)
- Is a specific trust dimension tanking your score? (Fix that, not everything)
Why the Shape of the Distribution Matters
Here is the insight that changes how you read research results.
A single expected score of 3.1 out of 5 could mean very different things depending on the shape of the distribution behind it:
Scenario A: 20% of personas score 1, 20% score 2, 20% score 3, 20% score 4, 20% score 5. Perfectly flat. This means no segment of your audience has a strong reaction either way. You have an awareness problem, nobody feels strongly enough about your product to buy it or dismiss it.
Scenario B: 40% of personas score 1 or 2, and 40% score 4 or 5. Almost nothing in the middle. This is a bimodal distribution, two completely different audience segments with opposing reactions. A messaging problem won’t fix this. You need to know who the believers are and who the sceptics are, and serve them differently.
Scenario C: 60% score 2 or 3, with a long tail toward 4 and 5. Most people are uncertain, but there’s a meaningful minority who are convinced. This is actually a good signal for retargeting, the sceptics need more touchpoints, but the high conviction tail is ready to buy now.
None of this is visible in a single number. The PMF makes it visible.
And because you run the same synthetic audience through your competitor’s page too, you get a benchmark. Not “our page scores 3.2” but:
“Our page scores 3.2. Theirs scores 3.4, despite having 4x more reviews and a lower price. We’re actually punching above our weight.”
How the Pipeline Works
In practice, running a synthetic research panel involves four steps:
1. Build your persona panel
For a product page analysis, a typical setup uses 35 personas: 7 different age points for the end user, crossed with 5 different primary purchase concerns (safety, effectiveness, value, convenience, brand trust). Each persona gets a detailed system prompt that describes who they are, what they care about, and their level of familiarity with your brand.
Cold brand framing, telling every persona they have never heard of this brand before, is the most useful default. It simulates a customer encountering your product page from a cold traffic source like a Meta ad or Google Shopping. The hardest conversion scenario.
2. Generate responses
Each persona is shown your content and asked to respond naturally. Gemini 2.5 Flash works well here, cost efficient and well suited for persona simulation at scale. 35 personas means 35 API calls. The whole generation step takes a few minutes.
3. Run SSR
The ResponseRater class from the SSR library handles this. You pass in your reference sentences (one set per research dimension) and your generated responses. It computes sentence embeddings, runs cosine similarity against the reference set, normalises the result, and returns PMF distributions for each response.
The key optimisation: compute response similarities once, then derive PMFs for each dimension without re-embedding. This matters when you’re running four dimensions across 35 personas.
4. Aggregate and interpret
The library’s get_survey_response_pmf() function aggregates individual PMFs into a panel level distribution. The output is a five point probability distribution per dimension, plus an expected value score. The score is useful for quick comparison. The distribution shape is where the real insight lives.
A Real Example: boAt vs. Noise
To make this concrete, here is a benchmark run on two of India’s most popular TWS earbud brands: boAt Airdopes Alpha and Noise Aura Buds.
Both products sit in a similar price band: boAt at ₹1,399, Noise at ₹1,499. Both claim IPX5 water resistance and 50ms low latency gaming mode. On paper, they are close competitors targeting the same buyer.
The test setup:
- 35 synthetic personas: 7 age points (18 to 33) x 5 primary purchase concerns (audio quality, call clarity, battery life, value for money, brand reliability)
- Cold brand framing: every persona was told they had never purchased from either brand before
- Same 35 personas run against both product pages under identical conditions
- 4 research dimensions: Purchase Intent, Product Sentiment, Audio Quality Trust, Value for Money
- LLM: Gemini 2.5 Flash
Results:
| Dimension | boAt Airdopes Alpha | Noise Aura Buds | Gap |
| Purchase Intent | 3.75/5 | 3.53/5 | +0.22 boAt |
| Product Sentiment | 2.91/5 | 2.88/5 | +0.03 boAt |
| Audio Quality Trust | 2.62/5 | 2.66/5 | +0.04 Noise |
| Value for Money | 3.24/5 | 2.89/5 | +0.35 boAt |
What the scores say: boAt wins on the two dimensions that drive conversion, purchase intent and value for money. Product sentiment and audio quality trust are essentially tied.
But the scores are only half the story. Here is what the PMF distributions reveal.
Purchase Intent distributions:
boAt: 8% | 11% | 8% | 46% | 27%, 73% of personas land at P4 or P5. Strong right skew, clear positive signal.
Noise: 16% | 11% | 4% | 44% | 26%, similar to P4+P5 mass (70%), but 16% at P1 (the “would never buy” position) versus boAt’s 8%. Noise has twice as many hard rejectors in the cold audience.
Value for Money, the most revealing distribution:
boAt: 21% | 9% | 25% | 14% | 31%, a striking 31% of personas score boAt at P5 (outstanding value). Bimodal, with believers concentrated at the top.
Noise: 17% | 13% | 45% | 12% | 12%, almost half the personas land at P3 (price seems about right). Much flatter. Only 12% at P5.
Both products sit within ₹100 of each other in actual price. Noise’s discount is actually larger, ₹3,000 off (67%) versus boAt’s ₹2,091 off (60%). So the value for money gap cannot be explained by pricing signals alone.
The more likely driver: feature perception at the price point. boAt lists a 13mm driver versus Noise’s 11mm. Bigger numbers read as better hardware to a cold audience, regardless of whether driver size alone determines audio quality. More importantly, boAt’s ASAP Charge, “10 minutes of charging equals 120 minutes of playback,” is a concrete, branded, verifiable claim that reads as a premium feature inside a budget product. Noise offers no equivalent. When a buyer is trying to justify ₹1,399 to themselves, a specific and surprising feature claim does more work than a larger headline battery number.
Audio Quality Trust, a category insight, not a brand insight:
boAt: 29% | 25% | 24% | 0% | 22%
Noise: 29% | 21% | 27% | 1% | 22%
These distributions are nearly identical, which is the point. Cold audiences reading spec sheets cannot verify audio quality claims. The 29% who score P1 (“seriously doubt the claims”) will not be converted by better copy. They need audio samples, verified reviews, or third party endorsements. This is a category problem that neither brand is solving on their product page, and a conventional survey would have averaged this out into a meaningless 2.6 score and missed the insight entirely.
The one actionable finding:
boAt’s advantage is not coming from discount framing. Noise actually offers a larger discount (67% versus 60%). It is coming from feature specificity. ASAP Charge gives cold audiences something concrete and surprising to anchor on. Noise’s strongest claims (60 hours, quad mic) are impressive on paper but feel harder to verify without trying the product.
If Noise wants to close the purchase intent gap, the highest leverage change is adding a feature claim that is specific, branded, and verifiable at a glance, the equivalent of ASAP Charge for their product.
Five Use Cases for Growth Marketers
Once you understand what SSR produces, the use cases become obvious.
1. Pre-Traffic Page Testing
Before you put budget behind any page, run it through a persona panel. You’re not looking for a high absolute score, you’re looking for the specific dimension that’s dragging down purchase intent. Is it value for money? Ingredient trust? General product sentiment? Each has a different fix. Knowing which one before you spend is worth the 15 minutes it takes to run.
2. Ad Copy A/B Testing Without Spending
Have two headline variants? Two different offer framings? Two hooks? Run both through the same audience before you commit budget. The variant with the better purchase intent distribution and higher content engagement wins the test, and you haven’t spent a rupee finding out.
3. Competitor Benchmarking
Run your page and your top competitor’s page through identical personas under identical conditions. The gap between the two scores is the insight. More useful than the score itself: which specific dimension is the gap on? If they beat you on ingredient trust but you match them on value for money, you know exactly what to address.
4. Messaging Claim Testing
Pick four different angles for the same product, “clinically proven,” “zero sulphates and parabens,” “works in 7 days,” “trusted by 10,000 parents.” Run all four as separate stimuli through the same audience. See which claim moves purchase intent most and for which persona type. This is message testing at a speed and cost that were previously impossible.
5. Influencer and Collaboration Fit
Build a persona set based on a creator’s audience profile, their demographics, their content consumption habits, their primary concerns. Run your brand’s core positioning through it. See if there’s a natural fit, or if you’d need to adapt your messaging significantly to resonate with their community before spending on a paid collaboration.
The Open Source Skill
The full research pipeline has been packaged into an open source Claude Code skill called /synth-research.
When you run it, Claude walks you through setup interactively, no configuration files, no code to write. You describe what you’re testing, describe your audience, and Claude handles the rest: building the persona prompts, calling the LLM, running SSR, and generating a plain language report with PMF distributions and a specific recommended action.
V1 supports two modes:
| Mode | Personas | Dimensions |
| product | 35 (7 age points x 5 concern types) | Purchase Intent, Product Sentiment, Ingredient Trust, Value for Money |
| ad | 20 (4 concern types x 5 audience mindsets) | Purchase Intent, Content Engagement, Value Proposition Clarity, Brand Sentiment |
What you need to run it:
- Python 3.10+
- A Gemini API key (set in your Claude Code settings)
- Claude Code (CLI or desktop)
Install:
Mac/Linux:cp -r skills/synth-research ~/.claude/skills/synth-research
Windows:xcopy /E /I skills\synth-research %USERPROFILE%\.claude\skills\synth-research
The skill lives in the Smacient marketing skills repository, directly in the synth-research folder: github.com/smacient/marketing-skills/tree/main/skills/synth-research
The underlying SSR library is maintained by PyMC Labs: github.com/pymc-labs/semantic-similarity-rating
How to Get Started
What you need
- Claude Code, Anthropic’s AI coding tool. Requires a Claude subscription (Pro, Max, Team, or Enterprise) or a Console account. Available as a CLI, a desktop app, and extensions for VS Code and JetBrains.
- The Smacient Claude Connector is the data layer that gives Claude access to the SSR engine and drives the Gemini persona generation step. Available at smacient.com.
- A Google Gemini API key is used to generate the persona responses. The free tier is sufficient for most panel runs.
- A Python virtual environment is set up in your project folder since the SSR analysis script runs locally.
Installation
- Get the skill from github.com/smacient/marketing-skills/tree/main/skills/synth-research, or clone the full repo and copy the folder you need:
git clone https://github.com/smacient/marketing-skills
cp -r marketing-skills/skills/synth-research path/to/your/project/.claude/skills/synth-research
- Set your Gemini API key as the GEMINI_API_KEY environment variable in your Claude Code settings.
- Set up a Python virtual environment in your project root and install the dependencies.
- Setup takes about 15 minutes the first time.
Running your first analysis
- Open a Claude Code session in your project folder with the Smacient Claude Connector active.
- Type
/synth-research "your product page or ad" --mode product, for example,/synth-research "https://yourbrand.com/product-page" --mode product. Swap--mode productfor--mode adif you’re testing ad creative instead. - Claude builds the persona panel, generates each persona’s response, and hands the results to the SSR library for classification against your reference sentences.
- Review any borderline or ambiguous personas Claude flags for you, usually a quick check, not a redo.
The script computes the PMF distributions across all research dimensions and builds the full report.
Your report lands as a markdown summary in the chat, with the full PMF breakdown per dimension. The 35-person product mode typically takes about 15 minutes end to end. The 20 persona ad mode is faster, usually closer to 10 minutes. Credit cost scales with panel size, check smacient.com for current pricing.
Get the skill: github.com/smacient/marketing-skills/tree/main/skills/synth-research
Connect the Smacient Claude Connector: smacient.com
FAQs
No. Asking an AI to rate something produces a single number based on a language model trying to be helpful, not a real buying decision. Synthetic consumer research uses distinct personas that respond the way a specific type of customer would, and then measures those responses against reference statements using SSR to produce a probability distribution, not a score.
SSR is a directional signal, not ground truth. It’s built to compress the iteration loop between building something and finding out if it works, not to replace properly designed studies with real participants. For high-stakes decisions like major repositioning or new market entry, real research is still worth the time and cost.
The /synth-research skill defaults to 35 personas for product page analysis (7 age points x 5 purchase concerns) and 20 personas for ad testing (4 concern types x 5 audience mindsets). These give enough spread to see the shape of the distribution, which is the whole point. Fewer personas and you risk missing a bimodal split or a vocal minority.
You get a PMF (Probability Mass Function) for each research dimension, showing what percentage of personas land at each point on a 5-point scale, plus an expected value score for quick comparison. The report lands as a Markdown summary directly in your Claude Code chat.
Claude Code (CLI, desktop app, or VS Code/JetBrains extension), the Smacient Claude Connector, a Gemini API key (the free tier works for most runs), and a Python virtual environment in your project folder. First-time setup takes about 15 minutes.
