Basics of Probability and Statistics Week 1 Challenge

Suppose we draw n independent observations from a population with mean µ and variance σ² this is commonly the case when we are drawing a random sample µ and σ² our parameters and their values are typically unknown. We want to estimate them the sample mean x-bar estimates the population mean µ and the sample variance S² estimates the population variance σ². Ideally, we would estimate σ² with `\sum(x_i-\mu)^2/n`. This is the average squared distance from the true mean and it would be the best estimator of σ². But there is a problem, the population mean µ is usually not known so we cannot use it in this formula and since we cannot possibly calculate this quantity here, `\sum(x_i-\mu)^2/n` the sum of squared distances from the true mean if the true mean is unknown we might try simply replacing the true mean with our best estimate of it the sample mean x-bar `\sum(x_i-\bar x)^2/n`. But there is a problem here, of all the values we could subtract in this formula subtracting x-bar makes this sum as small as it possibly could be. Roughly speaking the sample mean x-bar must fall near the centre of the observations whereas µ the true mean could be any value so this sum here is going to be. Smaller than this sum and so this quantity tends to underestimate the true of the population variance σ². To compensate for that we divide by n-1 this makes the sample variance a little bigger than it would be if we divided by n. It turns out that mathematically this properly compensates for the problem and on average this estimator equals the population variance σ^2.