Exam practice

Features: the morning’s humidity, the cloud cover at 9 a.m., the wind speed and the month.

Label: whether it rained that afternoon (yes or no).

A feature that might help: the air pressure in the morning, or whether it rained the day before. A feature that would not help: the day of the week, or the name of the person who wrote the record. (Any sensible answers, with a reason: a helpful feature changes with whether it rains.)

A model’s job is to answer for new things it has never seen, so it must be tested on things that are new to it. A nearest-neighbor model gives the label of the closest example. For one of its own training examples, the closest example is that same example, at no distance, so it always gives that example’s own label and scores 100%, however badly it would do on anything new. Like a student who has learned the practice answers by heart, a score on what it has already seen shows only that it remembers. So the examples are split before training, and a test set is kept apart and used only for checking.

• i.102 ÷ 120 × 100% = 85%.
• ii.Rice: 87 ÷ 90 × 100% ≈ 96.7%. Maize: 15 ÷ 30 × 100% = 50%.
• iii.The total of 85% hides a model that is very good on rice and right only half the time on maize: it is biased against maize, probably because there were few maize examples in its training data. A maize farmer shouldn’t rely on it. The team should collect more maize examples and report the accuracy for each crop.

A model is biased when it works worse for one group of people or things than for others because of how it was made, usually because of its data.
Example: a mango model trained only on kinds of mango that turn yellow when ripe learns that green means not ripe. A Bambaiya mango stays green when it is ripe, so the model calls ripe Bambaiya “not ripe”: in the widget it was right about every other mango and half the Bambaiya. Nothing in the program was wrong; the training data was one-sided, with no Bambaiya in it. (Other good examples: voice typing trained mostly on English or on one accent; face systems trained mostly on lighter-skinned faces; photo models trained mostly on photos from rich countries, which called a bar of soap from a home in Nepal “food”.)

• i.Before: daytime 47 ÷ 50 = 94%, night 21 ÷ 50 = 42%, all (47 + 21) ÷ 100 = 68%. After: daytime 46 ÷ 50 = 92%, night 44 ÷ 50 = 88%, all (46 + 44) ÷ 100 = 90%.
• ii.Yes. Night accuracy rose from 42% to 88%, and the gap between day and night fell from 52 to 4 percentage points. Daytime fell only a little (94% to 92%, one more photo wrong out of 50), so the change helped far more than it cost. Over all, accuracy rose from 68% to 90%.
• iii.If the test photos were in its training data, the model could simply remember them, and the test would look better than the model really is. Only photos it has never seen show how it will do on new night photos.

1. Collecting: a voice model recorded only in Kathmandu. Does the data include everyone the model will be used for?
2. Labeling: a labeler who mixed up two leaf diseases. Who labeled it, and were the labels checked?
3. Choosing features: “has internet at home” as a feature for a scholarship. Could this feature treat a group unfairly?
4. Testing: testing only on the kinds of mango it learned from, or giving only the total. Was it tested on everyone, and scored group by group?
5. Using: a face model from another country used unchanged in Nepal. Is it used on the people and things its data came from?

Nonexamples are things that are not what the model is looking for, shown to it with that label, so that it learns what its answer is not.
For a momo-finder: a plate of idli, a bowl of rice, a white folded cloth, a plate of sel roti, a dumpling of another kind (any things that are not momos).
Look-alikes are the most useful because they are near the edge between “momo” and “not momo”. They teach the model exactly which differences matter, so it doesn’t call every round white thing a momo.

A “model” that says “real” to every message, without looking, is also right 98 times out of 100: 98% accuracy, while catching no scam at all. So 98% alone says nothing about whether it catches scams.
Ask: how many of the scams in the test set did it catch, and how many real messages did it wrongly call scams? Was it tested on messages it never saw in training? Was the test set like the bank’s own messages, including scams written in Nepali?

A model has no opinions, but it copies the patterns in its data, so it can be unfair without anyone wanting it to be. If its examples leave a group out, it works worse for that group: face systems trained mostly on lighter-skinned faces were wrong for up to 34.7% of darker-skinned women and at most 0.8% of lighter-skinned men (Gender Shades, 2018). If the labels carry people’s mistakes or one-sided ideas, the model repeats them. And it repeats them for everyone, quickly, in the same confident way. So a model is only as fair as the data and the decisions behind it, and it must be tested group by group to find out.

A good plan includes:

• Collecting: record people from every province, in the hills, the mountains and the Terai; women and men; children, adults and older people; speakers whose first language is Maithili, Bhojpuri, Tamang, Newari or another, as well as Nepali; in quiet rooms and noisy streets, on cheap phones too. Ask everyone’s permission and keep their recordings private.
• Labeling: have what was said written out by careful people who know the speaker’s way of speaking, with a second person checking a sample of every labeler’s work, and the same spelling rules for everyone.
• Testing: keep apart a test set of recordings from speakers who are not in the training data, from every group; report the accuracy for each group (province, age, first language), not only the total; collect more examples for any group that does badly and test again; record the results before and after.