Statistics

Statistics is the branch of mathematics that collects data, presents it, analyses it and draws conclusions from it. It is as old as government: the rulers of Babylonia, Egypt and China counted their people, their land and their harvests to plan taxes and armies. The word itself comes from state.

In the twentieth century statistics became a science of its own. Sir Ronald A. Fisher (1890–1962) laid much of the foundation of modern statistics, and is often called its father; Karl Pearson (1857–1936) and William Sealy Gosset (1876–1937) made other great contributions, and you will meet all three in this unit. Today, with computers and huge amounts of data, statistics is used in almost every field: medicine, farming, sport, weather, business and the artificial intelligence on your phone.

Dispersion

When you study a set of data, you usually begin with a measure of central tendency, the mean, median or mode, which you met in Grade 10 Math. It gives one value for the center of the data. But the center alone doesn’t tell the whole story. Look at the marks of five students in each of three groups:

Group	Marks					Mean
A	48	49	50	51	52	50
B	40	45	50	55	60	50
C	20	35	50	65	80	50

All three groups have a mean of 50, but:

• in group A (low variation), every mark is within 2 of the mean: the students did almost equally well;
• in group B (moderate variation), the marks are spread out a moderate distance from the mean;
• in group C (high variation), the marks are widely spread: some students did very well and some poorly.

Looking only at the mean, you would think the three groups did the same. What makes them different is their spread.

Dispersion (or variation) is how far the values of a data set differ from one another, or from their central value. A measure of dispersion is a number that says how spread out the values are.

Measures of dispersion help us:

• a.to decide whether the data are consistent (steady) or variable;
• b.to compare how reliable two or more data sets are;
• c.to check how well the mean, median or mode represents the data: the smaller the spread, the better one central value describes every value;
• d.to make decisions in education, business, economics and science (which supplier’s parts are more uniform, which medicine works more reliably, which investment is less risky).

The central value describes the middle of the data; the dispersion describes its spread. To understand a data set you need both.

The simplest measure of dispersion is the range, the highest value minus the lowest: 4 for group A, 20 for B and 60 for C. But the range uses only the two end values, and one unusual value changes it completely. This unit builds three better measures: the quartile deviation, the mean deviation and the standard deviation.

Each class includes its lower limit and goes up to, but not including, its upper limit. The class 55–60 kg holds 55, 56, 56.5, 57, 58, 59 and 59.5, but not 60, which belongs to the next class, 60–65.

Data that can take any value in a stretch, decimals as well as whole numbers (heights, weights, times), grouped into classes like these, form a continuous series.

You met the quartiles in Grade 10 Math. They cut the data, in order, into four quarters:

• the first quartile Q₁ has 25% of the data below it and 75% above;
• the second quartile Q₂, the median, has 50% below and 50% above;
• the third quartile Q₃ has 75% below and 25% above.

So the middle half of the data lies between Q₁ and Q₃. The width of that middle half is the interquartile range:

$$\text{Interquartile range} = Q_3 - Q_1$$

Half of the interquartile range is the quartile deviation (Q.D.), also called the semi-interquartile range:

$$\text{Q.D.} = \frac{Q_3 - Q_1}{2}$$

The quartile deviation has the data’s own unit (marks, kg, rupees), so it can’t compare data in different units, or data of very different sizes. For that we divide by Q₃ + Q₁ and get a pure number, the coefficient of quartile deviation:

$$\text{Coefficient of Q.D.} = \frac{Q_3 - Q_1}{Q_3 + Q_1}$$

The quartile deviation ignores the lowest quarter and the highest quarter of the data. That is its weakness (it uses only half the data) and its strength: one very high or very low value can’t change it, and it can be found even when the first or last class is open-ended (“less than 20”, “60 or more”), where the mean can’t.

To find Q₁ and Q₃ of a continuous series, as in Grade 10 Math:

1. Make the cumulative frequency (cf) table. Its last cf is N.
2. Q₁ is the $\left(\frac{N}{4}\right)$th item; Q₃ the $\left(\frac{3N}{4}\right)$th.
3. The quartile’s class is the first whose cf reaches that position.
4. Then

$$Q_1 = L + \frac{\dfrac{N}{4} - cf}{f} \times h \qquad Q_3 = L + \frac{\dfrac{3N}{4} - cf}{f} \times h$$

where L is the lower limit of the quartile’s class, cf the cumulative frequency of the class before it, f the frequency of the quartile’s class and h its width. (Some books write l and i for L and h.)

Worked examples

Question 1

Question 2

Find the quartile deviation and the coefficient of quartile deviation.

Question 3

The marks of 30 students in mathematics are:

12	25	36	45	18	22	33	54	42	29
38	41	15	20	48	52	30	24	19	37
44	21	35	40	14	26	49	31	23	50

The quartile deviation looks at only two values. A measure that uses every value asks: on average, how far is each value from the center?

The mean deviation (MD) is the mean of the distances of the values from a central value, the mean or the median (sometimes the mode). A distance is never negative: 5 below the mean and 5 above it are both 5 away. So each difference is taken as its absolute value, |m − X|, the difference without its sign. (Without the bars, the differences from the mean always add up to exactly 0, the reason the mean is the balance point.)

For a continuous series, with mid-values m:

$$\text{MD from the mean} = \frac{\Sigma f|m - \overline{X}|}{N} \qquad \text{Coefficient of MD from the mean} = \frac{\text{MD}}{\overline{X}}$$

$$\text{MD from the median} = \frac{\Sigma f|m - M_d|}{N} \qquad \text{Coefficient of MD from the median} = \frac{\text{MD}}{M_d}$$

Writing D for the difference, both are $\frac{\Sigma f|D|}{N}$. For a list of values, the mean deviation from the median is never bigger than the one from the mean, because the median is the point from which the total distance is least. In a grouped table the two are usually equal or nearly so, as above.

Worked examples

Question 1

Question 2

Find the mean deviation from the mean, and its coefficient.

Question 3

Find the mean deviation from the median, and its coefficient.

Question 4

A school held a mathematics test of 50 full marks. The marks of 10 students are:

Marks	less than 10	less than 20	less than 30	less than 40	less than 50
Students	1	3	7	9	10

Questions 5 and 6

Here this section is 11.5, its subsections 11.5.1 to 11.5.4, and the box plot 11.6; some editions number them differently.

The mean deviation got rid of the minus signs by ignoring them. The standard deviation gets rid of them by squaring: a square is never negative. Then, to get back to the data’s own unit (marks, not marks²), it takes the square root at the end.

The standard deviation is the square root of the mean of the squared distances of the values from their mean. It is written $\sigma$, the small Greek letter sigma:

$$\sigma = \sqrt{\frac{\Sigma (x - \overline{X})^2}{n}}$$

The square of the standard deviation, $\sigma^2$, is called the variance.

Of all the measures of dispersion, the range, the quartile deviation, the mean deviation and the standard deviation, the standard deviation is the most widely used and the most reliable.

In the next drawing, each mark has a stem from the mean (the red line): the stems are the distances. MD is the stems’ average length; $\sigma$ is the square root of the average of their squares. Try the groups of 11.1, then pull one mark far away and watch $\sigma$ pull ahead of MD.

11.5.1 Uses of standard deviation

The standard deviation is used wherever it matters how steady something is:

• a.Education: to compare how consistent the results of students, classes or schools are.
• b.Weather: to measure how much the temperature or rainfall of a place varies from year to year.
• c.Investment: a share whose price has a large standard deviation is riskier.
• d.Quality control: a factory measures its products (the weight of a packet of noodles, the length of a bolt) and checks that their standard deviation stays small, so the quality is uniform.

11.5.2 Coefficient of standard deviation

The standard deviation has the data’s own unit. To compare spreads in different units (heights in cm and weights in kg), or of data of very different sizes, divide it by the mean:

$$\text{Coefficient of standard deviation} = \frac{\sigma}{\overline{X}}$$

where $\sigma$ is the standard deviation and X the mean. The smaller the coefficient, the more uniform (stable, consistent) the data; the larger, the more variable.

11.5.3 Coefficient of variation

The coefficient of standard deviation is usually less than 1, so for everyday use it is turned into a percentage. The coefficient of variation (C.V.) is the coefficient of standard deviation as a percentage:

$$\text{C.V.} = \frac{\sigma}{\overline{X}} \times 100\%$$

As with the coefficient: the smaller the C.V., the more uniform the data; the larger, the more variable. It is the standard way to say which of two sets is more consistent. (It can be more than 100% when the values are very spread out compared with their mean: Bibek’s scores in the opening come close.)

11.5.4 Standard deviation of a continuous series

In a continuous series each class is represented by its mid-value m, and each squared distance is counted f times. There are four ways of working it out. They give the same answer; choose the one that keeps the numbers easiest.

The actual mean method

$$\sigma = \sqrt{\frac{\Sigma f(m - \overline{X})^2}{N}} \qquad \text{(i)}$$

where m is the mid-value, $N = \Sigma f$ and X the mean. The steps:

• a.Find the mid-value m of each class.
• b.Find fm for each class and their sum $\Sigma fm$.
• c.Find the mean $\overline{X} = \frac{\Sigma fm}{N}$.
• d.Find each class’s distance from the mean, m − X, and square it.
• e.Multiply each square by f: $f(m - \overline{X})^2$.
• f.Add these up: $\Sigma f(m - \overline{X})^2$.
• g.Put the sums into formula (i).

This method is easiest when the mean is a whole number.

The direct method

Opening the bracket in (i) gives a formula that needs no distances at all, only $\Sigma fm$ and $\Sigma fm^2$:

$$\sigma = \sqrt{\frac{\Sigma fm^2}{N} - \left(\frac{\Sigma fm}{N}\right)^2} \qquad \text{(ii)}$$

“The mean of the squares minus the square of the mean.” The steps:

• a.Find the mid-value m.
• b.Find fm for each class.
• c.Find $fm^2$ for each class (m × fm is quickest).
• d.Find $\Sigma fm$ and $\Sigma fm^2$.
• e.Put them into formula (ii).

It suits small mid-values, when the mean is not a whole number.

The short-cut (assumed mean) method

Choose a mid-value near the middle as an assumed mean A, and use the distances from it, d = m − A:

$$\sigma = \sqrt{\frac{\Sigma fd^2}{N} - \left(\frac{\Sigma fd}{N}\right)^2} \qquad \text{(iii)}$$

The steps:

• a.Find the mid-value m.
• b.Choose a mid-value as A and find d = m − A.
• c.Find fd for each class.
• d.Find $fd^2$ for each class (d × fd).
• e.Find $\Sigma fd$ and $\Sigma fd^2$.
• f.Put them into formula (iii).

Moving every value by the same amount doesn’t change the spread, so the standard deviation of the d’s is the standard deviation of the m’s. That is why the method works. It is useful when the actual mean is an awkward decimal.

The step deviation method

When the classes are wide and the numbers large, divide the distances by the class width h as well: $d' = \frac{m - A}{h}$, small whole numbers like −2, −1, 0, 1, 2. Then multiply the answer by h to get back to the data’s unit:

$$\sigma = \sqrt{\frac{\Sigma fd'^2}{N} - \left(\frac{\Sigma fd'}{N}\right)^2} \times h \qquad \text{(iv)}$$

The steps:

• a.Find the mid-value m.
• b.Choose a mid-value as A and find $d' = \frac{m - A}{h}$.
• c.Find fd′ for each class.
• d.Find $fd'^2$ for each class (d′ × fd′).
• e.Find $\Sigma fd'$ and $\Sigma fd'^2$.
• f.Put them into formula (iv).

Back to the batters

The two batters of the opening both average 30 runs. With the standard deviation:

Batter	Scores	Mean	$\sigma$	C.V.
Aarav	28, 31, 30, 29, 32, 30, 31, 29	30	1.22	4.1%
Bibek	0, 72, 5, 64, 3, 55, 1, 40	30	29.03	96.8%

Aarav’s squared distances from 30 add up to 12, so $\sigma = \sqrt{\frac{12}{8}} = \sqrt{1.5} \approx 1.22$; Bibek’s add up to 6740, so $\sigma = \sqrt{842.5} \approx 29.03$. Aarav is about 24 times as consistent. A captain who needs a steady 30 sends in Aarav; one who needs a big score to win, and can risk a duck, might choose Bibek. Statistics gives the numbers; the decision still depends on the match.

Question 1

Question 2

Question 3

Question 4

Find the standard deviation and its coefficient.

Question 5

The marks of 40 students in a mathematics exam of 50 full marks are:

Marks	less than 10	less than 20	less than 30	less than 40	less than 50
Students	5	11	25	35	40

Question 6

In a factory, the daily wages of the workers range from Rs 600 to over Rs 1,200:

Daily wage (Rs)	Workers
more than 600	100
more than 700	80
more than 800	55
more than 900	25
more than 1000	15
more than 1100	8
more than 1200	4

Question 7

The monthly salaries of the workers in two shoe factories are:

	Factory A	Factory B
Workers	40	50
Mean monthly salary	Rs 35,000	Rs 45,000
Standard deviation	Rs 40	Rs 35

Question 8

The scores of 36 Grade 10 students in a mathematics exam are:

35	62	72	50	70	74	34	60	36
21	14	75	48	28	38	64	21	45
57	40	19	34	30	47	20	75	51
45	56	22	29	22	13	53	66	62

A mean, or even a mean and a standard deviation, can hide what the data really look like. A whisker box plot (box-and-whisker plot) draws the whole spread in one picture, and puts two data sets side by side so they can be compared at a glance. You met it in Grade 9.

A whisker box plot shows a data set by its five-number summary: the smallest value (minimum), the first quartile, the median, the third quartile and the largest value (maximum). The box, from Q₁ to Q₃, holds the middle 50% of the data; the two whiskers, from the smallest value to Q₁ and from Q₃ to the largest, hold the lowest and the highest 25% each.

Read it like this:

• the line inside the box is the median, the typical value;
• the width of the box is the interquartile range, Q₃ − Q₁: a narrow box means consistent data;
• the whole width, whisker end to whisker end, is the range;
• a long whisker on one side means the data stretch far that way: a few values much higher (or lower) than the rest.

In Activity 1, half of the students are between 114 and 145 cm, and the median, 126 cm, is nearer the bottom of the box than the top: in the middle half, the heights above the median are more spread out than those below it.

Steps to draw a whisker box plot:

1. Arrange the data in ascending order.
2. Find the smallest value, Q₁, the median, Q₃ and the largest value. For n values in order, Q₁ is the $\frac{n+1}{4}$th, the median the $\frac{n+1}{2}$th and Q₃ the $\frac{3(n+1)}{4}$th, as in Grade 10 Math.
3. Draw a scale and mark the five values on it.
4. Draw a box from Q₁ to Q₃ with a line across it at the median, then the whiskers from the smallest value to Q₁ and from Q₃ to the largest.

11.6.1 Comparing two data sets with box plots

To compare two data sets with box plots, draw them on one scale and compare:

• the medians, for which is higher in a typical case;
• the boxes’ widths (interquartile ranges), for which is more consistent;
• the whiskers, for the extremes: the guaranteed minimum, the best case.

Question 1

The batteries of two brands of mobile phone were tested to see how long they last after a full charge.

Question 2

The yield of paddy from a field over ten years with two fertilizers, A and B:

Question 3

The life of two kinds of battery:

Question 4

The numbers of mobile phones sold by two shops over one week are:

Shop	Mon	Tue	Wed	Thu	Fri	Sat	Sun
A	8	5	14	9	11	7	18
B	10	12	8	16	6	3	2

In the SEE’s Optional Mathematics paper (75 marks, 3 hours), Statistics is a content area of its own. CDC’s specification grid gives it 7 marks, in three kinds of question:

Group	Kind of question	Statistics’ marks	Levels
A	multiple choice, 1 mark	1	understanding 1
B	within the content area: one question on statistics alone, in three 1-mark parts	3	knowledge 1, understanding 1, higher ability 1
C	across content areas: parts of a longer question joining statistics with another area	3	application 3

In CDC’s model paper, statistics turns up in all three groups: what a higher coefficient of variation means (group A, 1 mark); two box plots on one scale, read for the median, then a coefficient of quartile deviation, then a choice between the two data sets with a reason (group B, 1 + 1 + 1); and, inside a question that goes on to continuity, the coefficient of variation from a grouped table (group C, 3 marks).

What each kind asks, and where its marks are:

• 1-mark questions: what Q.D., MD, $\sigma$, variance or C.V. means; “smaller C.V., more uniform”; $\sigma$ from the variance; Q.D. from Q₁ and Q₃. Work them on rough paper.
• Knowledge: a formula written exactly, or a value read off a box plot (a median, a quartile, the smallest value).
• Understanding: a coefficient from given values, $\frac{Q_3 - Q_1}{Q_3 + Q_1}$ or $\frac{\sigma}{\overline{X}} \times 100\%$.
• Higher ability: a choice with a reason drawn from the numbers: the narrower box or the smaller C.V. is more consistent; the higher minimum is safer.
• Application (group C): the full table, m, fm, $fm^2$ (or d, fd, $fd^2$), then X, $\sigma$ and C.V. Write the table neatly: most of the marks are in it.

Here is a question in the style of group B, written for this book, with how it is marked.

These questions join the unit’s measures together, as the SEE’s longer statistics questions do.