The central limit theorem of statistics is one of those facts of mathematics – it isn’t at all obvious why it should be true. It is, nevertheless, ubiquitous. Everything from mundane traffic patterns to the intensity level of a laser beam are governed by this law of nature.
The central limit theorem states that the many-times convolution with itself of any distribution with a finite mean and a finite variance will tend to a Gaussian. If the four bits of jargon in that last sentence are unfamiliar, fret not. I’ll see if I can’t try and explain it with an example (which was on a homework for some course. Herr Roy, I want credit for this.)
Let’s say you decide to toss a (fair) coin. You toss this coin a hundred times, note down how it lands each time, and tally up the results. You have a bunch of friends do the same thing, and have them give you their results as well. Or you realise that the number of friends you have isn’t nearly enough for a good experiment, and decide to have MATLAB do the coin-tossing experiment for you.
Either way, you note down how many times your coin landed heads-up out of 100, for each person/trial. This done, you plot a histogram of the results: for how many trials does the number of heads that turned up lie in certain bins? If you did get that statement of the central limit theorem, you would’ve realised that this distribution is just the convolution of a binomial distribution many times over (as many times as your number of trials).
I, obviously, had to go the MATLAB way. Which isn’t really all that bad – it took me four lines of code.
It is worth noting that the number of trials matters a lot more towards getting a decent Gaussian than the number of tosses in each trial. It is also worth remembering that the distribution we started with was a binomial distribution – the number of heads from a (fair) coin tossed a given number of times. Many processes in nature have Gaussian distributions to start with, as well.
It can also be shown that in the limit of a large number of tosses and a fixed mean, the binomial distribution ‘becomes’ a Poisson distribution. Further, around its mean, the Poisson distribution ‘looks’ Gaussian.
Addendum: Anubhab Roy points out that the tails of the distribution formed by many-times convolving a distribution with itself are only approximately described by a Gaussian. He says the Gaussian over-predicts the tails. I did not know this. Thank you, Meister.
[End. Fini. Kaputski. Gaussian]