MENU

Power Analysis
Power analysis is an important aspect of experimental design. It allows us to determine the sample size required to detect an effect of a given size with a given degree of confidence. Conversely, it allows us to determine the probability of detecting an effect of a given size with a given level of confidence, under sample size constraints. If the probability is unacceptably low, we would be wise to alter or abandon the experiment.

The following four quantities have an intimate relationship:

  1. sample size
  2. effect size
  3. significance level = P(Type I error) = probability of finding an effect that is not there
  4. power = 1 - P(Type II error) = probability of finding an effect that is there

Given any three, we can determine the fourth.

The methods given below use R's Power Analysis (Pwr) package; this implements power analysis as outlined by Cohen (1988). The R functions that are made available here are listed below. A more detailed description is given here.

For each of these functions, you enter three of the four quantities (effect size, sample size, significance level, power) and the fourth is calculated.

The significance level defaults to 0.05, but you can remove it if this is the quantity you would like to calculate.

Specifying an effect size can be a daunting task. ES formulas and Cohen's suggestions (based on social science research) are provided here. Cohen's suggestions should only be seen as very rough guidelines. Your own subject matter experience should be brought to bear.


t-tests

For t-tests, use one of the following two methods (N.B. Cohen suggests that d values of 0.2, 0.5, and 0.8 represent small, medium, and large effect sizes respectively).

pwr.t.test - for t-test

n is the sample size, d is the effect size, and type indicates a two-sample t-test, one-sample t-test or paired t-test:

 (N.B. must be >= 2)
 (N.B. must be <= 1)
 (N.B. must be < 1)

pwr.t2n.test - t-test if you have unequal sample sizes

Enter n1 & n2 and 2 of the 3 other fields to calculate the remaining field, where n1 and n2 are the sample sizes:

 (N.B. must be >= 2)
 (N.B. must be >= 2)
 (N.B. must be <= 1)
 (N.B. must be < 1)
ANOVA

pwr.anova.test - for a one-way analysis of variance

k is the number of groups and n is the common sample size in each group (N.B. Cohen suggests that f values of 0.1, 0.25, and 0.4 represent small, medium, and large effect sizes respectively):

 (N.B. must be >= 2)
 (N.B. must be <= 1)
 (N.B. must be < 1)
Correlations

pwr.r.test - for correlation coefficients

Enter 3 of the 4 fields to calculate the 4th, where n is the sample size and r is the correlation. We use the population correlation coefficient as the effect size measure. Cohen suggests that r values of 0.1, 0.3, and 0.5 represent small, medium, and large effect sizes respectively.

 (N.B. must be >= 4)
 (N.B. must be <= 1)
 (N.B. must be < 1)
Linear Models

pwr.f2.test - for linear models (e.g. multiple regression)

u and v are the numerator and denominator degrees of freedom. We use f2 as the effect size measure (N.B. Cohen suggests f2 values of 0.02, 0.15, and 0.35 represent small, medium, and large effect sizes):

 (N.B. must be <= 1)
 (N.B. must be < 1)
Tests of Proportions

pwr.2p.test - two proportions (equal n)

Enter 3 of the 4 fields to calculate the 4th, where h is the effect size and n is the common sample size in each group (N.B. Cohen suggests that h values of 0.2, 0.5, and 0.8 represent small, medium, and large effect sizes respectively):

 (N.B. must be <= 1)
 (N.B. must be < 1)

pwr.2p2n.test - for unequal n's

Enter n1 & n2 and 2 of the 3 other fields to calculate the remaining field, where h is the effect size and n's are the sample sizes in each group:

 (N.B. must be >= 2)
 (N.B. must be <= 1)
 (N.B. must be < 1)

pwr.p.test - to test a single proportion

Enter 3 of the 4 fields to calculate the 4th, for both two sample and one sample proportion tests, you can specify two.sided, less, or greater to indicate a two-tailed, or one-tailed test. A two tailed test is the default.

 (N.B. must be <= 1)
 (N.B. must be < 1)
Chi-square Tests

pwr.chisq.test - for chi-square tests

w is the effect size, N is the total sample size, and df is the degrees of freedom (N.B. Cohen suggests that w values of 0.1, 0.3, and 0.5 represent small, medium, and large effect sizes respectively):

 (N.B. must be <= 1)
 (N.B. must be < 1)
Contact: help@searchbreast.org