Sample Size and Power Calculator

Every test has a size it needed, and the honest time to work it out is before the data exists. Four things decide it: how small a shift would still matter to you, how often you are willing to raise a false alarm, how often you are willing to miss a real effect, and whether a surprise in either direction would have interested you. This page turns those four into a number of observations, and shows both the textbook answer for a known standard deviation and the larger one you actually need when the spread has to be estimated from the same data.

How it is calculated

N = (z₁₋α/₂ + z₁₋β)² (σ/δ)² two-sided · N = (z₁₋α + z₁₋β)² (σ/δ)² one-sided

The two critical values are what the sample has to clear twice over: far enough from the null value that you would call it significant, and far enough beyond that to be found reliably rather than half the time. Stating the shift in standard deviations, as the handbook recommends, cancels σ out of the expression entirely and leaves the size depending only on how many deviations you want to be able to see. When σ has to be estimated the same formula is used with t in place of z, but the t critical values need degrees of freedom, which need the sample size you are still solving for — so the answer is found by iterating from the normal estimate until it stops growing.

Source: NIST/SEMATECH e-Handbook of Statistical Methods, 7.2.2.2, Sample sizes required — for a one-sided test with α = 0.05 and β = 0.10 detecting a shift of one standard deviation the handbook prints N = (1.645 + 1.282)² = 8.567 ≈ 9, then iterates on the t distribution with N − 1 = 8 degrees of freedom to N = (1.860 + 1.397)² = 10.6 ≈ 11

Questions people ask

Which of the two sizes should I use?

The second one, almost always. The first assumes you already know the population standard deviation, which in practice means a long-running process or a calibrated instrument. If you are going to estimate the spread from the very sample you are planning, the t-based figure is the one that delivers the power you asked for — and it is meaningfully larger at the small sizes where this question usually comes up.

What does "shift to detect" mean, and how do I choose it?

It is the difference you would not want to miss, measured in standard deviations of the thing you are measuring. Choosing it is a judgement about your subject, not a statistical question: what change would be large enough to act on? Working backwards from a sample size you can afford is the common move and it is not illegitimate, as long as you report the shift that size can detect rather than quietly hoping for a smaller one.

Why does halving the shift roughly quadruple the sample?

Because the shift enters squared. The standard error shrinks with the square root of N, so detecting a difference half the size needs four times the observations. This is the single most useful thing to know when planning: precision is expensive and gets worse fast, and a study aimed at a small effect on a budget built for a large one will simply fail to find anything.

What is power, exactly?

The probability of rejecting the null hypothesis when a shift of the size you named is genuinely there. Power of 0.90 means one study in ten will miss it. The convention of 0.80 is a habit rather than a standard, and it means one study in five comes back empty-handed with nothing wrong except the plan. Working out power after the fact from the effect you happened to observe is a different and much less useful exercise, sometimes called observed power; it tells you almost nothing the p-value did not already say.

Does this cover comparing two groups?

Not directly. The formula here is for a test of one mean against a fixed value, which is what the handbook section it comes from covers. Comparing two independent groups of equal size needs about twice this many observations in each group, because the difference of two means carries two lots of sampling error. We would rather point that out than quietly hand you a number from a formula the source does not print.

The handbook prints a table of sizes. Does this page match it?

Mostly. Of the 57 cells in the handbook's two-sided table, 47 come out exactly as its own formula gives them and 10 are printed one larger — eight of those in the column for a half-deviation shift, and two where the formula would give 2 and the table shows 3. This page follows the formula, which is printed on the same page as the table, and rounds up to the next whole observation. If your coursework is marked against the table, expect the occasional difference of one.

Sources

The documents this page reads its numbers out of, linked so you can check them yourself.

1. NIST/SEMATECH e-Handbook of Statistical Methods, 7.2.2.2, Sample sizes required — for a one-sided test with α = 0.05 and β = 0.10 detecting a shift of one standard deviation the handbook prints N = (1.645 + 1.282)² = 8.567 ≈ 9, then iterates on the t distribution with N − 1 = 8 degrees of freedom to N = (1.860 + 1.397)² = 10.6 ≈ 11
2. NIST/SEMATECH e-Handbook of Statistical Methods, 1.3.6.7.1, Cumulative Distribution Function of the Standard Normal Distribution — the two quantities the sample-size formula needs are, in the handbook's words, "critical values from the normal distribution"; this is the table they come from, and its summary prints Zp = 1.645 at p = 0.950 and Zp = 1.282 at p = 0.900

Related tools