How to Choose a Sample Size Without Guessing at It

Sample size is not a number you pick because it sounds respectable. It falls out of two decisions: how precise you need the result, and how much variation you expect in the group. Change either and the required size moves.

Most teams skip those decisions and default to whatever survey tool suggests. That is fine until the result lands on a knife's edge and nobody can tell whether the difference is real.

Precision first

Decide how close the estimate must be to be useful. If you are placing a bet on a 2-point difference, a sample that carries a 5-point margin of error cannot support the bet. The margin you can tolerate sets the floor for your size.

Bigger samples shrink the margin, but the returns fall off fast. Cutting a margin in half roughly quadruples the sample. Going from 400 to 1,600 responses buys half the error and four times the cost.

The inputs that actually matter

Plug those into a sample size formula or calculator. The point is not the exact number. It is that you made the trade-off visible instead of inheriting it.

• Desired confidence level, usually 95 percent for product work.
• Tolerable margin of error, stated before data collection.
• Expected proportion, often set to 50 percent when unknown because it is the worst case.
• Population size, which only matters much when the group is small.

When a small sample is enough

Qualitative work does not chase precision. Six to twelve interviews often reveal most themes, because the goal is depth, not prevalence. The same logic applies to usability tests, where five users usually surface most of the serious problems.

The error is applying qualitative logic to a quantitative question, then reporting a percentage from twenty responses as though it were stable.

Key takeaways

• Size flows from precision and expected variation, not preference.
• Halving your margin of error roughly quadruples the sample.
• Set the tolerable error before you collect, not after.
• Small samples suit depth; prevalence claims need room for the margin to breathe.