Marketers often worry about their sample size when preparing market research. They worry that the sample size will be too small to produce accurate results. Clients also press to know if the sample size is adequate. The larger the sample size, the more accurate the results, so the thinking goes.
However, while adequate sample sizes are necessary for valid results, size is not the single marketing research requirement. Marketing researchers face more common pitfalls.
“In truth, there is no magic number that makes a sample good or valid,” says Kevin Lyons, research supervisor at Lipman Hearne Inc. Instead of fixating on the sample size, consider the margin of error, which measures reliability, and the representativeness of the sample.
Common Sampling Errors in Market Research
Seeking ever larger sample sizes. Generally speaking, the larger the sample size, the smaller the margin of error, a measure of confidence in results. However, after a point, larger sample sizes provide little increase in accuracy. Sample quantity passes a point of diminishing returns.
Lack of representativeness of the sample. A representative sample closely matches the overall demographics of the overall audience. Marketers can tap the Census Bureau or other databases to check if a sample is geographically, demographically and, if possible, behaviorally representative. If it’s not, collect more data from the under-represented population or assign additional weight to under-represented demographics, Lyons suggests.
Ignoring the response rates. The number of responses, rather than the sample size, drives the validity of test results, writes marketing strategist Chuck McLeester for Target Marketing. Choose sample sizes based on your expected response rate, not from tradition, your gut or convenience, McLeester urges. Online calculators can compute ideal sample sizes, but the general guideline is that 250 responses will yield 90% confidence that results will vary no more than +/-10%.
Ignoring outliers. Outliers differ markedly from other survey participants and can give wildly different responses that skew results. They may have atypical relationships with the organization. For instance, nonprofits might include board members or staff in surveys. Devise surveys that deliberately exclude such individuals.
Over-defining the population. Over-defining the population can exclude potential customers and lead to biased, inaccurate results, experts warn. For instance, a chip manufacturer defined its sample population as “males aged 25 to 34 who have consumed premium chips in the past two weeks.” If the brand’s top-selling product gets more than 20% of their sales from people outside the defined population, the definition is likely too narrow.
Bottom Line: Although marketers often focus on sample size when conducting surveys, other factors are more likely to lead to inaccurate results.
William J. Comcowich founded and served as CEO of CyberAlert LLC, the predecessor of Glean.info. He is currently serving as Interim CEO and member of the Board of Directors. Glean.info provides customized media monitoring, media measurement and analytics solutions across all types of traditional and social media.
