T-Test Calculator

A t-test asks one narrow question: could a difference this big have turned up by chance, if there were no real difference at all? It answers in the only currency it has, a p-value, and that answer is worth exactly as much as the assumptions behind it. Paste your numbers, say what you measured, and this page gives you the t statistic, the degrees of freedom it was read against, the p-value, and — the part most calculators leave out — the size of the difference itself, which is usually the number you actually care about.

How it is calculated

t = (x̄ − μ₀) / (s / √N) · t = (ȳ − z̄) / √(s₁²/N₁ + s₂²/N₂) · t = d̄ / (s_d / √N)

Three shapes of the same idea: a difference divided by the standard error of that difference. The one-sample form compares a mean with a value you name; the two-sample form compares two means; the paired form is the one-sample form applied to the differences within each pair, which is why pairing is worth so much when it is available — it removes whatever the two measurements had in common. For two independent samples the degrees of freedom depend on whether you are willing to assume the two spreads are the same. Pooling gives N₁ + N₂ − 2; refusing to assume it gives the Welch-Satterthwaite approximation, which is smaller and usually not a whole number. The handbook rounds it to an integer to use a printed table; this page does not need to, and uses it as it comes.

Source: NIST/SEMATECH e-Handbook of Statistical Methods, 7.3.1, Do two processes have the same mean? — assembly times 32 37 35 28 41 44 35 31 34 38 42 against 36 31 30 31 34 36 29 32 31 give means of 36.0909 and 32.2222 with standard deviations 4.9082 and 2.5386; the handbook prints t = 2.2694 with the Welch-Satterthwaite degrees of freedom approximated as 15.5

Questions people ask

What does the p-value actually mean here?

It is the probability of seeing a difference at least this large if the null hypothesis is true and the assumptions hold. It is not the probability that the null hypothesis is true, not the probability your result was a fluke, and not a measure of how big or important the difference is. A p of 0.04 on a difference nobody would act on is still a difference nobody would act on, and a p of 0.07 on a large difference in a small sample is not evidence of no effect.

Should I pool the variances or use Welch?

Use Welch unless you have a real reason to believe the two spreads are the same. Welch costs a little power when the variances truly are equal and protects you when they are not, and the handbook is explicit that equality "should be tested before this assumption is made". The pooled option is here because textbooks and coursework often ask for it, and because with equal sample sizes and similar spreads the two answers barely differ.

When is a paired test the right one?

When each number in the first list belongs with one particular number in the second: the same person before and after, the same part measured on two instruments, the same field in two seasons. If the lists could be shuffled independently without losing information, they are not paired and the two-sample test is the right one. Treating unpaired data as paired is nonsense; treating paired data as unpaired throws away the very thing that made the design good.

One-sided or two-sided?

Two-sided unless you decided, before seeing the data, that a difference in one direction would not have interested you at all. The one-sided p-value is exactly half the two-sided one, so choosing the direction after looking at the sign of the difference is a way of halving your p-value for free. That is the single most common way this test gets abused.

What does the test assume about my data?

That the observations are independent, and that the sampling distribution of the mean is close to normal — which for reasonable sample sizes follows from the data being roughly symmetric without wild outliers, rather than from the data being exactly normal. Independence is the assumption that gets violated most and repaired least: repeated measurements on the same subject, or readings taken in a run that drifts, are not independent, and no amount of arithmetic fixes that afterwards.

Why is my degrees of freedom not a whole number?

Because you asked for Welch. The Welch-Satterthwaite formula estimates how many degrees of freedom the mixture of two different variances behaves like, and there is no reason for that to land on an integer. Printed tables force you to round it, which is what the handbook does in its own example when it turns 15.5 into 16. Computing the distribution directly removes that step.

What if the calculator says there is nothing to test?

Three things cause it: fewer than two readable numbers, every reading identical so the standard deviation is zero and there is nothing to divide by, or a paired test where the two lists are different lengths. Anything the parser could not read as a number is skipped rather than treated as zero, so a stray label in a pasted column will shrink your sample instead of corrupting it.

Sources

The documents this page reads its numbers out of, linked so you can check them yourself.

1. NIST/SEMATECH e-Handbook of Statistical Methods, 7.3.1, Do two processes have the same mean? — assembly times 32 37 35 28 41 44 35 31 34 38 42 against 36 31 30 31 34 36 29 32 31 give means of 36.0909 and 32.2222 with standard deviations 4.9082 and 2.5386; the handbook prints t = 2.2694 with the Welch-Satterthwaite degrees of freedom approximated as 15.5
2. NIST/SEMATECH e-Handbook of Statistical Methods, 7.2.2, Are the data consistent with the assumed process mean? — particle counts 50 48 44 56 61 52 53 55 67 51 give a mean of 53.7 and a standard deviation of 6.567; against a process mean of 50 the handbook prints t = 1.782 with N − 1 = 9 degrees of freedom
3. NIST/SEMATECH e-Handbook of Statistical Methods, 7.3.1.1, Analysis of paired observations — the differences d = Y − Z have mean d̄ and standard deviation s_d with ν = N − 1 degrees of freedom, and the paired-sample t test statistic is t = d̄ / (s_d / √N)
4. NIST/SEMATECH e-Handbook of Statistical Methods, 1.3.6.7.2, Critical Values of the Student's t Distribution — "The t distribution is symmetric so that t(1−α,ν) = −t(α,ν)", which puts half the curve on each side of zero: a sample mean sitting exactly on the value being tested gives t = 0 and a two-sided p of 1. The same table prints 2.262 at row 9, column 0.975 and 1.746 at row 16, column 0.95

Related tools