If I had $1 for every time a company wanted to treat a small number of survey responses as being statistically representative of their customer base, I’d be a very rich woman! Many of us learned in stats class that n=30 is the absolute minimum for statistical significance. But small sample sizes come with a lot of caveats and issues, including high error rates and a limited to non-existent ability to segment responses.
To help explore the tradeoffs between sample size and error rate, one of my go-to research experts, Joe Hopper from Versta Research, has created this interactive graph. It’s a great tool, and I wanted to share it.
Typically, my rule of thumb for scrappier clients has been at least 100 responses (I can feel my research colleagues wincing!). At that point, the error rate is about 10%, and it is possible to do some crude/modest/rough segmentation. But to feel good about the sample set, you really want to have at least 300 respondents. According to Joe’s chart, at this size, there’s an error rate of about 5.5%.
In fact, for the two most recent research studies my clients conducted, the sample size was 500 for one and 827 for the other! (Error rates were approximately 4.4% and 3.4% respectively) For these projects, a low error rate was critical because my clients intended to use the research results to make important and costly business decisions.
On the flip side, I worked with an early-stage client who was using a quarterly employee Net Promoter Score (eNPS) to track employee sentiment. However, there were only 24 employees in the company. My client didn’t realize that it’s widely understood that eNPS is only useful if at least 100 employees have responded. And when calculating NPS, ideally, you’re working with 1200 responses.
The bottom line is that sample size isn’t just a nice-to-have statistical consideration—it directly impacts the reliability and actionability of your insights. If you don’t have access to a large enough sample for meaningful quantitative research, it’s better to acknowledge that reality upfront rather than pretend small numbers tell a bigger story. In those cases, consider whether you’re truly ready for quant research, or if qualitative methods might better serve your current needs and constraints.