Sampling means and unbiased variance

A sample mean varies between samples

A population is the full collection you want to describe. A sample is the part you observe. Its mean is one estimate of the population mean, and a fresh sample can give a different estimate.

Before collecting data, write X₁, …, Xₙ for the observations and X bar = (X₁ + … + Xₙ)/n for their sample mean. They are random variables because the sample has not yet been selected. After observing values x₁, …, xₙ, write x̄ for the resulting numerical mean. The population mean μ is a fixed parameter, even when its value is unknown.

For a small illustration, draw twice independently from values 0, 2 and 4, each equally likely on each draw. Replacement makes the second draw a fresh choice. The nine ordered pairs are equally likely, but their means are not:

For example, the pairs (0,4), (2,2) and (4,0) all give mean 2. An extreme mean of 0 requires both draws to be zero, giving probability 1/9 instead of the single draw’s 1/3. Averaging reduces spread here, but the mean still varies. This small sampling distribution is discrete, not normal.

A random sample needs a selection process

A simple random sample of a stated size gives every possible sample of that size an equal chance under its sampling design. Choosing people who volunteer or measuring only the easiest items to reach does not meet that definition merely because the resulting list is long.

State the population and how observations are selected. To investigate the current output of a production process, a sample confined to its best-performing hour may miss the population you intended to study. A larger sample from that same hour does not remove the selection bias.

The formulas below use independent observations with the same population distribution. Sampling without replacement from a finite population creates dependence; the independent model may be a useful approximation when the sampled fraction is very small, but it is not an exact consequence of simple random sampling. Check the stated sampling model before using an independent-observation formula.

Separate observation spread from mean spread

If X₁, …, Xₙ are independent and identically distributed with population mean μ and finite variance σ², then

The sample mean is centred on μ over repeated samples: it is an unbiased estimator of μ. This does not say that every observed x̄ equals μ.

The standard deviation of the sample mean is its standard error:

σ describes variation among individual observations; σ/√n describes variation among means of n observations. Increasing n does not change the population’s individual-observation spread. Quadrupling n halves the standard error; it does not divide it by four.

For a normal population with mean 50 and variance 81, an independent random sample of 36 has mean distribution X bar ∼ N(50,2.25). Its standard error is 9/6 = 1.5. Thus

Using standard deviation 9 in that probability would ask about one individual observation exceeding 53, not the average of 36 observations.

Decide whether a normal model is exact or approximate

An independent sample from a normal population has a normal sample mean for any sample size. No large-sample approximation is needed for that distribution:

For independent observations from a non-normal population with finite variance, the Central Limit Theorem supports an approximately normal sample mean when n is sufficiently large. It does not say that the individual observations become normal. The required n depends on the population’s shape; strong skewness or rare extreme values can require more observations.

“n ≥ 30” is a common working condition in stated tasks, not a universal guarantee. Increasing n cannot by itself fix dependence, a biased sampling design or a population without the required finite variance. State when a calculated normal probability uses the Central Limit Theorem.

Estimate the unknown population variance

For an independent sample of n observations, x̄ estimates μ. An unbiased estimate of the population variance is

The deviations from the fitted sample mean sum to zero, so they cannot vary independently: once n − 1 are known, the final one is fixed. The divisor n − 1 supplies the unbiased population-variance estimator. Dividing the same squared-deviation total by n instead describes the spread of the observed data as a complete list; it answers a different question.

For data 6, 9, 10, 12 and 13, n = 5, Σx = 50 and Σx² = 530. The mean is 10 and the squared deviations are 16, 1, 0, 4 and 9, totalling 30. Therefore

The variance of the five-value list with divisor n is 6, not the unbiased estimate 7.5. If using a calculator, check whether its spread output uses n or n − 1, and whether it displays a standard deviation or a variance. Square a sample standard deviation to obtain s².

“Unbiased” describes the estimator across repeated independent samples: the average s² targets σ². It does not make this particular s² the exact population variance. Also, although s² is unbiased for σ², taking its square root does not generally give an unbiased estimator of σ.

The estimated standard error is s/square root of n = square root of (s²/n). For these five observations it is square root of (7.5/5) ≈ 1.225. This estimates the spread of a mean; it does not establish that a normal probability calculation is valid for this small sample with an unspecified population shape.

Undo coding for location, not for a shift in spread

For coded observations d = x − a, the mean and unbiased variance estimate are

Adding a back recovers the original mean. It does not change the deviations from that mean, so it is not added to the variance. The units of a mean are the observation units; variance has their square.

Coding can also improve numerical stability. If all observations are close to a very large common value, raw Σx² and (Σx)²/n may be nearly equal. Subtracting prematurely rounded totals can lose the small squared-deviation total. Work with deviations from a nearby a and retain the supplied precision.