Is 22% Customer Disappointment Enough? How Superhuman Measured Product-Market Fit
Short answer: A 22% “very disappointed” score is not enough to justify broad growth on its own, but it is enough to identify which users care and why. Survey recent active users, segment the answers by behaviour and customer type, then verify the strongest segment through retention, use and paid contribution. Treat 40% as an indicative working threshold, not a certificate.
Product-market fit is often described as a feeling: demand pulls the product out of your hands, customers complain when capacity is unavailable, or growth becomes effortless. Those descriptions arrive too late to guide a founder deciding what to change this month.
A survey offers an earlier signal. It is also easy to misuse. One percentage cannot prove a market, and improving the number by deleting inconvenient respondents proves nothing.
What Superhuman did with 22%
Rahul Vohra adapted a question associated with Sean Ellis: how would users feel if they could no longer use the product? The choices were very disappointed, somewhat disappointed or not disappointed.
In Vohra's first-person account of Superhuman's process, the company surveyed people who had used the product at least twice in the previous two weeks. It had between 100 and 200 users available and said results became directionally useful at roughly 40 respondents.
Only 22% initially answered “very disappointed”. Rather than average every user's requests into one roadmap, the team examined who comprised that 22% and what benefit they valued. Narrowing the analysis towards founders, managers, executives and business-development users lifted the score to 33%. Product work then concentrated on reinforcing the benefit loved by strong users while addressing barriers reported by the “somewhat disappointed” group.
Vohra reported that the score reached 58% within three quarters. This is one company's account, not experimental proof that the method caused its growth.
The 40% reference came from Ellis's comparison of nearly 100 startups, as reported by Vohra. Practitioners disagree about how portable that threshold is across products, sample sizes and markets. My view is that it is a useful alarm, not a finish line. Below it, diagnose. Above it, verify with behaviour and economics before accelerating.
Use the Disappointment Segmentation Loop
The Disappointment Segmentation Loop turns the survey from a score into a decision.
| Stage | Question | Evidence produced | |---|---|---| | Qualify | Has this person experienced the core value recently? | Eligible respondent set | | Measure | Would losing the product matter? | Directional attachment score | | Segment | Who cares most, and what job connects them? | Candidate customer segment | | Verify | Do those users stay, use and pay? | Behavioural and economic evidence | | Allocate | What should be strengthened or repaired next? | Bounded product decision |
Repeat the loop with new eligible users. Do not survey the same people until they learn the desired answer.
Qualify respondents before counting them
Someone who registered but never reached the core outcome cannot judge whether losing it would matter. Set a behavioural eligibility rule before sending the question.
For weekly software, that might mean completing the main job twice in 14 days. For a monthly service, it might mean two completed cycles. For an infrequent high-value purchase, the survey may be a poor instrument because customers cannot accumulate enough experience quickly.
Report the denominator. “Half of users would be very disappointed” is meaningless if only four enthusiastic friends replied. Record invited, delivered, started and completed responses, then compare respondents with non-respondents for obvious bias.
Read the reasons before the percentage
Ask four compact questions: how the user would feel without the product, who benefits most, the main benefit and what would improve the experience. The language in the benefit answers tells you what the product already does well.
Group answers by observable attributes such as role, use frequency, current alternative, acquisition source and paid plan. Avoid constructing a segment from flattering adjectives alone. “Busy professionals” is not useful unless you can identify and reach them.
Do not chase the “not disappointed” group. They may be a poor fit. The “somewhat disappointed” users who resemble your strongest users are usually more instructive because one repeated barrier may be holding them back.
Verify attachment with behaviour and money
Survey answers can be polite, strategic or hypothetical. Pair the score with activation, repeated use, retention, referrals and contribution.
A segment that reports strong disappointment but consumes hours of support at an uneconomic price may not be your first profitable market. A quieter segment that renews, expands and refers can be more valuable than its survey language suggests.
The article on [identifying your first profitable customer segment](/market-research/first-profitable-customer-segment/) explains how to add reachability and economics to need.
Worked example: InboxPilot
InboxPilot sells email workflow software to small professional firms for £24 per user each month. These figures are illustrative.
The company has 180 accounts. Its eligibility rule requires at least six active days and 100 processed emails in the previous 14 days. Ninety users qualify, 72 respond and 16 say they would be very disappointed. The score is 16 divided by 72 = 22.2%.
InboxPilot segments the results:
| Segment | Responses | Very disappointed | Score | 90-day paid retention | |---|---:|---:|---:|---:| | Recruitment consultants | 24 | 11 | 45.8% | 87% | | Accountancy staff | 28 | 4 | 14.3% | 61% | | Freelance creatives | 20 | 1 | 5.0% | 50% |
The overall score looked weak, but recruiters show both attachment and retention. Interviews reveal that they value finding candidate conversations within seconds. Their most common barrier is unreliable matching of abbreviated names.
InboxPilot spends 60 engineering hours at £48 an hour improving that job, a cost of 60 x £48 = £2,880. It recruits 20 new recruitment users and measures them after 90 days. If 17 remain, monthly recurring revenue is 17 x £24 = £408. With an 80% contribution margin, monthly contribution is £326.40, giving a simple development-cost payback of £2,880 divided by £326.40 = 8.8 months before acquisition cost.
The company does not abandon accountants immediately. It stops broad acquisition, tests the recruiter improvement and sets a growth gate: at least 40 eligible responses in the new cohort, 40% or more very disappointed, 85% 90-day retention and payback below 12 months.
The survey did not prove product-market fit. It showed where to run a more expensive behavioural test.
Avoid scoring tricks
Do not exclude users after seeing their answers merely to cross 40%. Segmentation is legitimate when defined by a real customer characteristic and followed by a prospective test with new users.
Keep question wording and eligibility stable across cohorts. A change in either can move the score without improving the product. Show raw counts alongside percentages, especially with small samples.
Resist turning the score into a staff target with bonuses attached. Once people are rewarded for the percentage, respondent selection and survey timing can become the work. The business needs more customers receiving value, not a prettier survey.
Run the first loop in six weeks
In week one, define the core value event and recent-use eligibility. Invite every qualifying user, not a hand-picked list. Collect answers in week two and segment them using characteristics you already know.
During week three, interview very disappointed users and similar somewhat disappointed users. Choose one loved benefit to protect and one repeated barrier to remove. Ship the smallest defensible change by week five.
In week six, recruit or observe a new cohort in the chosen segment. Measure the survey only after they qualify, then compare retention, use and contribution. Grow only when words and behaviour point in the same direction.
Related guides
Frequently asked questions
Is 40% a universal product-market fit threshold?
No. It is an indicative benchmark associated with Sean Ellis and popularised through founder case studies, not a law. The result depends on who is asked, how recently they used the product, sample size and product category. Use it consistently to compare qualified cohorts and expose weak attachment. Do not tell investors or staff that crossing 40% proves the market. Retention, repeated use, referral, pricing and contribution must support the result. A high score from a tiny, selected group can be weaker evidence than a lower score across a representative paying cohort.
How many responses do I need?
Vohra said Superhuman found roughly 40 responses directionally useful, but smaller samples have wide uncertainty and should not support a major investment alone. Use raw counts and collect as many eligible responses as practical. Ten out of 20 and 100 out of 200 both equal 50%, but the second is more stable. If you serve a small enterprise market, combine fewer survey responses with deep usage and renewal evidence. Never enlarge the sample with inactive people merely to reach a target number.
Should I survey free users or paying customers?
Survey both only if you can analyse them separately. Paying customers have demonstrated willingness to pay, while active free users may reveal value before pricing or procurement. Combining them can hide important differences. Apply the same core-use qualification within each group, then compare disappointment, retention and support cost. If free users report strong attachment but will not convert, investigate price, buyer authority and the alternative they use. The business decision depends on paid value, not affection alone, unless free use directly supports another documented revenue mechanism.
What if no segment reaches 40%?
Choose the segment with the clearest combination of attachment, repeated use and reach, then remove one repeated barrier. Set a time and spending limit for the next test. If new qualified users still show weak attachment and poor retention, reconsider the problem, product or market rather than endlessly refining survey slices. A low result is useful because it prevents premature scaling. It does not dictate immediate closure, but it should make further hiring and broad acquisition harder to justify until stronger behaviour appears.
Can I use this method before launch?
Not properly, because people must experience the core value before they can judge losing it. Before launch, test the problem through current behaviour, spending, commitments and a usable minimum version. Intent surveys about a concept measure interest, not product-market fit. Once a small group has completed the core job at least twice, the disappointment question becomes materially more informative. If building the experience is expensive, use the guide to [testing a business idea before spending](/business-ideas/test-your-idea-before-spending/) to obtain stronger pre-build evidence.
How often should I repeat the survey?
Repeat it when enough new users have qualified to form a distinct cohort or after a material product or segment change. Monthly or quarterly can work for a frequently used service. Do not ask the same users repeatedly, because familiarity with the question can bias results. Preserve the wording and eligibility rule, and compare cohorts rather than blending every response forever. If the score rises while retention falls, investigate the contradiction. Behaviour should overrule the more flattering measure until you understand the cause.
Comments (0)