Central Limit Theorem
The distribution of an average tends to be Normal, even when the distribution from which the
average is computed is decidedly non-Normal. Furthermore, the limiting normal distribution has
the same mean as the parent distribution AND variance equal to the variance of the parent
divided by the sample size.
The Central Limit theorem is the foundation for many statistical
procedures, including Quality Control Charts, because the distribution of the phenomenon
under study does not have to be Normal because its average will be. (see
statistical fine print )
The distribution of an average will tend to be Normal as the sample size increases,
regardless of the distribution from which the average is taken except when the
moments of the parent distribution do not exist. All practical distributions in
statistical engineering have defined moments, and thus the CLT applies.
The Cauchy is an example of a pathological distribution with nonexistent moments.
Thus the mean (the first statistical moment) doesn't exist. If the mean doesn't exist,
then we might expect some difficulties with an estimate of the mean like Xbar.