Sample Size Justification

Students typically struggle with sample size justification, in part because there are 2 types.  One type is based on the population and the second based on a power analysis.  Sample size based on a population is generally not used in dissertations.  It not used in dissertations because the requirement would too exhaustive to stratify the population in terms of geography and the size requirement would be too great. 

The sample size based on a power analysis is used in dissertation and is a required section in your method chapter (and is needed for IRB or URR).  A power analysis essentially says that the researcher has a 80% chance of finding differences or relationships among the variables if they actually do exist.  Sample size based on a power analysis uses the type of statistical analysis you are using such as an ANCOVA, multiple regression, Pearson correlation, etc), the alpha (typically .05), and a small, medium or large effect size.  Effect size has both theoretical and practical considerations.  At a theoretical level, the researcher needs to review other studies that examined the same type of constructs or the same instruments, then see what effect size was found.  If the effect size is not presented, it can be calculated from the means and standard deviations, one-way ANOVAs, frequency counts, correlations, mean gain scores, unstandardized regression coefficients, full sample standard deviations, chi-squares, phi-coefficients, cell frequencies, t-tests, or proportions.  The practical aspect of justifying the sample size is money and time needed to collect data.  For example, if you’re running a multiple regression with 3 predictor variables AND the effect size is small, you’ll need an N=547!   This is in comparison to a regression at a medium effect size with a desired N=76 or a large effect size with an N=34.   Sample size can be calculated by using a free G*Power analysis program or you can purchase a sample size write-up with references from our website by signing on for our Basic Membership for $29.00. 

Sample Size in Plain English

As a dissertation consultant for over 20 years, I consistently see confusion when it comes to answering a simple question—how many participants do in need? The confusion is reasonable because most programs do not even offer a class in sample size and leave it to the graduate student to figure it out on their own. This post will clear it up once and for all.

Two Types of Sample Sizes

There are two types of sample sizes to determine: one sample size determination is used to find the number to have enough participants to be representative of a population, and the other sample size determination is to achieve statistical power. Let’s talk about these two types.

Sample Size for a Population—what researchers and organizations need

This type of sample size determination is an effort to get a representation of the population, such as you see would see in election polling. To determine this sample size, you need to know the population size, confidence interval and confidence level (typically 95%). This is almost never the type of sample size that dissertation students need because you don’t have unlimited time, money, energy to get such as large sample. If you are a funded researcher or organization, and desire this type of sample size, you can view our free calculator at http://www.statisticssolutions.com/products-services/login/free-membership.

Sample Size for Statistical Power—what dissertation students need

Statistical power (also called a power analysis and typically set at .80) is the basically the probability of finding statistical differences in your data if in fact they are there. The .80 is saying that you have an 80% chance of finding difference in you data if differences exist. To assess this type of sample size you need to know a few things. First, you need to know what type of statistical analysis you are going to conduct. That is, the sample size calculation for an ANOVA is different than for a correlation or factor analysis. Second, you need to know the effect size, alpha, and desired statistical power. We decided on the conventional .80 power and alpha is usually set at .05 (you’ll recognize the p = .05 in the articles you’ve been reading for several years). Let’s talk about effect sizes and the three sizes they come in: small, medium, and large. Effect size is this context is the ability to detect differences in the data, so, a bit counter intuitively, a large, easily detected effect requires a small sample size to detect it, while a small, difficult to detect effect in the data requires a larger sample size.

How Do You Decide What Effect Size to Choose?

The next question you should be asking yourself is should I choose a small, medium, or large effect size? There are theoretical and practical considerations here. The theoretical answer is to look at the research previously conducted with your types of research questions, variables, and analyses, to see what effect size was found. The problem is that if a small effect size was found (thus requiring a large sample size) it may be impractical for you to find the 300+ participants! On the other hand, just picking a large effect size willy-nilly isn’t quite correct either. What I find is that most dissertation committees go along with are medium effect sizes. You can try to calculate it for free at G-Power http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/ or if you want to find the appropriate sample size with a simple write up and references, you can go to http://www.statisticssolutions.com/products-services/login/basic-membership (while this one is not free—sorry—it’s cheaper than paying us or others $800 to calculate it).

Sample size note. Having said all of this, you should probably recruit as many participants as you can (hence boosting your statistical power).

If you have any sample size questions, or other questions about your methodology or results chapters, feel free to contact us at http://www.statisticssolutions.com . I hope this helps!

Happy Learning,

Statistics Solutions