How to determine population and survey

sample size?
By Gert Van Dessel on February 13th, 2013
367 Comments
inShare3

A survey can only be truly valuable when its reliable and representative for your business.
However, determining the ideal survey sample size and population can prove tricky. In
other words, who will you be surveying and how many people? No idea? No worries. Were
here to help!
Say youre a Market Research Manager at a furniture company and you are planning to
launch a new furniture line bythe end of 2015. However, before you launch the new line you
wish to conduct an online survey on whether your line Fall 2015 is more or less likely to
be a hit or miss on the European Union (EU) market. So far, so good. Yet, the following
question will almost instantly arise: What is the population that I would like to survey?.
Or, who do you need to survey to gain valuable insights in the success of your new furniture
line? In this case the answer is rather straightforward. Assuming that you are launching the
new line on the European market, that minors do not buy furniture and that your furniture is
reasonably priced, your population consists of all adults in the EU.

What is the survey sample size?

For obvious reasons it is impossible to survey those (roughly) 400 million adults in the EU. A
sample of adults living in the EU offers the solution for this issue. A sample is a selection of
respondents chosen in such a way that they represent the total population as good as
possible. However, instantly a new question comes to the forefront: How many people
should my sample consist of?. Using a correct sample size is crucial for your research.
After all, a sample that is too big will lead to the waste of precious resources such as time and
money, while a sample that is too small will not allow you to gain reliable insights.
So, how large should your sample be? Should you survey 1%, 5%, 10%, of the adult
citizens in the EU? Well, this depends largely on how accurate you want your survey data to
be. In other words, how closely you want your results to match those of the entire population.
There are two measures that affect the accurateness of the data.

First of all there is the margin of error (or confidence intervals). In short, this is the
positive and negative deviation you allow on your survey results for the sample. Or, in
other words, the deviation between the opinions of your respondents and the opinion
of the entire population. An example will shed some light on this statistical
explanation. Suppose you set your margin of error on 5%. If lets hope so! 90% of
your survey respondents like the Fall 2015 line, a 5% margin of error means that
you can be sure that between 85% (90%-5) and 95% (90%+5) of the entire
population actually likes the Fall 2015 line.
Second there is the confidence level. This tells you how often the percentage of the
population that likes the Fall 2015 line actually lies within the boundaries of the
margin of error. Or, following on our previous example, it tells you how sure you can
be that between 85% and 95% of the population likes the Fall 2015 campaign.
Suppose you chose the 95% confidence level which is pretty much the standard in
quantitative research1 then in 95% of the time between 85% and 95% of the
population likes the Fall 2015 line2.

How many respondents does your survey require?

Once you have decided how accurate you want your sample data to be, you can start
calculating how many respondents (i.e. people who have completely filled in the survey or
completes as we call them at CheckMarket) you actually need.
Below you find an indicative table on how to calculate your number of completes. Remember
that your population consist of approximately 400 million adults in the EU. As a
consequence, the appropriate number of completes will be found on the last row of the table
below. Depending on the confidence level and the margin of error, the number of
completes will vary. As we chose a margin of error of 5% and a confidence level of 95% for
our Fall 2013 campaign, you need approximately 400 completes (it is advisable to round to
the nearest hundred) for your sample.
Alternatively, on the CheckMarket website, you find an easy sample size calculator to
calculate the number of completes

What about response rate?

Before you start sending out your survey to 400 respondents, remember there is such a thing
as response rate. Response rate is the ratio of respondents that fill in the questionnaire they
received compared to the total number of surveys you send out. For instance, if you send
out your survey to 400 people and you receive 200 filled in surveys, your response rate is
50%.
For an online survey, conventionally, a response rate of 20% is considered as a good
response rate, while a 30% response rate is considered to be really really good. As we
calculated that we need 400 completes, this means that you will definitely have to send the
survey to more than 400 people in order to reach those 400 completes. Obviously, you cannot
predict beforehand what response rate you will achieve. However, assuming that your survey
will achieve a response rate of 20%, we divide the objective of 400 completes by a response
rate of 20%. As a consequence, you will have to send your survey to approximately 2.000
adults in the EU.

In some quantitative research, stricter confidence levels are used (e.g. the 99% confidence
level)
2
To put it more precisely: 95% of the samples you pull from the population.