When samples are taken from a population the results of each sample are likely to be different.
If a sample of the exam results of 20 students from a particular school is taken, the mean and standard deviation of their marks will probably be different if another 20 students were sampled.
However, you would expect that, if the samples were random, the mean exam result of most of the samples would be close to the mean exam result for the whole school.
Sample Means
A spinner with eight evenly spaced numbered sectors (shown below) is spun and the results recorded.
The probability distribution for the spinner is shown below. (Each probability will be 1 / 8 = 0.125)
Number 
1

2

3

4

5

6

7

8

Probability 
0.125

0.125

0.125

0.125

0.125

0.125

0.125

0.125

The mean of this distribution is 4.5 and the standard deviation is 2.29
Experiment The spinner is spun 15 times and the total recorded and divided by 15.
This experiment was simulated on a spreadsheet and repeated 40 times and the results recorded below.
40 samples

Total

Mean

The Mean of the Sample Means The mean value of the means of these samples is 4.47. This value is very close to the mean of the original distribution of 4.5 found above. The Standard Deviation of the Sample Means The standard deviation of the means of these samples is 0.63. The standard deviation of the sample means is much lower than the standard deviation of the original distribution, 2.29, as extreme values are averaged out with each sample. In theory the standard deviation of the sample means is equal to the population standard deviation divided by the square root of the sample size. This is called the standard error of the mean. In this case 2.29/√15 = 0.59, which is close to our value of 0.63 for the 40 samples. 

8

6

6

7

1

1

6

8

4

5

7

2

3

5

1

70

4.67


5

6

6

1

4

4

5

5

5

1

4

6

2

5

6

65

4.33


3

2

1

5

1

4

7

3

2

8

1

4

2

7

7

57

3.80


5

1

8

2

3

8

8

8

7

4

5

7

5

3

7

81

5.40


8

1

4

7

5

6

6

5

7

8

1

4

7

8

6

83

5.53


4

4

1

2

8

5

8

3

8

6

7

4

7

1

1

69

4.60


1

5

4

2

8

7

4

6

5

5

6

4

2

3

4

66

4.40


4

5

3

1

7

1

1

7

2

1

7

7

1

6

3

56

3.73


1

7

3

1

4

5

2

5

7

5

2

6

8

4

3

63

4.20


5

5

7

5

5

6

6

2

3

7

8

2

5

6

4

76

5.07


6

3

3

5

8

3

8

6

5

3

2

4

7

6

5

74

4.93


6

3

4

4

3

8

4

2

3

2

1

4

6

4

8

62

4.13


8

8

5

7

8

3

4

2

1

2

3

8

5

1

8

73

4.87


3

6

1

3

1

4

2

6

5

8

4

2

1

1

8

55

3.67


7

1

5

8

1

4

6

3

3

5

1

1

6

1

5

57

3.80


6

3

1

3

5

7

3

4

4

3

2

7

4

3

6

61

4.07


7

1

4

3

4

1

6

5

7

6

1

4

3

2

4

58

3.87


6

3

6

2

4

6

1

5

1

8

7

1

1

5

5

61

4.07


8

8

4

2

4

2

1

2

5

3

6

1

5

2

4

57

3.80


8

1

6

3

3

6

8

8

3

6

6

1

8

1

5

73

4.87


2

1

4

7

4

5

1

3

5

1

4

8

7

8

1

61

4.07


2

5

4

6

5

3

2

2

5

5

8

8

1

2

8

66

4.40


2

3

8

4

8

5

3

6

2

7

2

3

3

1

6

63

4.20


6

2

6

3

4

2

1

2

8

4

1

3

6

7

2

57

3.80


6

7

6

6

6

6

8

2

5

3

5

4

4

6

3

77

5.13


7

4

4

8

3

3

8

5

1

3

8

8

8

6

2

78

5.20


6

2

2

7

3

6

5

7

7

3

5

1

2

2

3

61

4.07


4

3

6

8

4

8

6

6

8

3

8

6

5

8

6

89

5.93


2

3

8

6

3

5

1

3

7

1

5

4

5

2

6

61

4.07


4

4

4

6

7

2

4

1

2

6

1

3

3

5

5

57

3.80


2

2

5

1

3

3

8

7

4

7

6

4

8

1

5

66

4.40


5

6

8

4

8

5

7

5

7

5

8

7

4

5

5

89

5.93


4

3

6

4

8

4

5

3

2

2

7

5

3

3

3

62

4.13


6

4

7

4

4

3

4

5

4

3

3

5

7

5

7

71

4.73


5

5

5

7

5

1

5

7

6

4

7

4

8

5

5

79

5.27


5

7

5

8

7

2

3

7

8

3

6

5

7

7

2

82

5.47


3

8

8

1

7

5

6

2

4

4

3

6

3

1

1

62

4.13


6

2

8

7

7

3

1

1

6

2

5

2

6

2

4

62

4.13


3

1

6

1

8

6

5

5

7

1

1

1

6

6

8

65

4.33


4

5

4

5

2

6

2

4

2

6

2

5

1

7

3

58

3.87


Mean 
4.47


s.d.

0.63

The experiment was repeated with 500 samples and the distribution of sample mean graphed on a histogram.
The result is shown below.
The mean of the 500 sample means is 4.48 and the standard error is 0.59.
The shape of this frequency graph is very close to a normal distribution and as the number of repetitions is increased the graph becomes more bellshaped like a normal curve. This applies when samples are taken from any type of population or distribution. e.g. rectangular, triangular or normal.
Click here to investigate this closer.
Summary
If a random sample of size n is taken from ANY distribution with a mean of μ and a standard deviation of σ then the distribution of the sample mean will have a :
The standard deviation of the sample means is also known as the standard error of the means. If the original distribution is normal then the distribution of the sample mean will be normal. If the sample size n is large, say greater than 30, then the distribution of the sample mean will be approximately normal for samples taken from ANY distribution.. 