This blog post is guest authored by Dr. Anne Beall, President of Beall Research and Training. Anne is also the author of the succinct and very useful book Strategic Market Research. I asked Anne to write this post because I have found that it is sometimes challenging to convince clients that using a 10-point scale for their customer satisfaction surveys is the best way to go.
As a market-research professional, I have strong feelings about rating scales, and they are the foundation of my work. Using scales that are sub-optimal is similar to using low-grade concrete for building a house. The finished product might look nice, but the structure is built from materials that are unreliable. I recommend using a 10-point scale for several reasons.
1. It requires a smaller sample size in order to have the same degree of precision as a scale with fewer points. According to researchers who analyzed this issue, a 10-point scale requires 71% of the sample size that a 5-point scale requires in order to have the same measurement precision. Thus, if you have the same sample size, you will have 71% of the power to detect a statistical effect with a 5-point scale than with a 10-point one.
2. It is more sensitive at measuring differences than scales with fewer points. For example, with customer satisfaction research, 44% of the respondents using a 5-point scale scored Company A below a 5. However, with a 10-point scale, 82% of respondents scored Company A below a 10. Thus the proportion of customers for whom satisfaction can be improved is larger when using the 10-point scale (The Measurement Imperative by D. R. Wittink & L. R. Bayer in Marketing Research, Fall 2003). Researchers also found that a multiple regression performed on the same data yielded more predictive effects in the 10-point scale data than the same data collected with a 5-point scale.
3. It has greater statistical reliability and validity. Reliability is the extent to which the measurements of a test remain consistent over repeated tests. The reliability of data can be analyzed by looking at how much data changes when you add or subtract different predictors in a statistical analysis. Data collected with the 5-point scale has more instability and the predictive variables are less consistent than the same data collected with a 10-point scale. Given that the data is less stable, we can assume that it is less valid as a result.
So if you want a scale that has the greatest statistical reliability, validity, sensitivity and ability to detect differences, go with a 10-point scale. And to make sure you’re asking great questions overall, this post from our Top Ten Entrepreneurial Research Mistakes series may be helpful.