Exam practice

• Token: a piece of text that a language model reads as one unit, often a whole word or part of one, or a punctuation mark.
• Vocabulary: the list of every token a model knows; each has its own number.
• Context window: the amount of text, measured in tokens, that a model can look at in one go: your messages and its answers so far.
• Parameter: one of the billions of numbers inside a model, learned in training, that decide how it turns tokens into chances for the next token.
• Fine-tuning: training an already-trained model further, on a smaller, chosen set of examples, so that it gets better at a particular kind of task, such as answering helpfully.
• Hallucination: an answer a model makes up that sounds true but isn’t, such as a wrong date or a book that doesn’t exist.

Count each pair of letters side by side, inside each word. “the” appears twice.

The words are the, hen, and, the, pen. The pairs, counted: t+h 2, h+e 3 (twice in “the”, once in “hen”), e+n 2 (hen, pen), a+n 1, n+d 1, p+e 1. h + e is the commonest (3 times), so the first merge makes he.
Now the words are t|he, he|n, a|n|d, t|he, p|e|n. Pairs: t+he 2, he+n 1, a+n 1, n+d 1, p+e 1, e+n 1. The second merge is t + he → the.
After these two merges, “the” is one token, the, and “then” is the | n.

1. Your question (with any earlier conversation) is cut into tokens by the tokenizer, and each token becomes its number.
2. The model looks at all the tokens in its context window (inside, they become embeddings, and attention links each to the earlier tokens that matter).
3. It works out a chance for every token in its vocabulary to come next.
4. One token is picked from those chances (always the most likely at temperature 0, or by weighted chance).
5. The token is added to the end of the text, and the loop goes round again from step 2, now including the new token.
6. It stops when it picks a token that means the end of the answer (or reaches a length limit). The tokens are turned back into text for you to read.

Looking back one or two words, the model knows only the last words, so after “is the” it can pick any word that ever followed “is the”, such as “longest”, though the sentence began with “Kathmandu”: each step is grammatical, and the whole is false. Looking back three or more words on a tiny text, each exact run of words appeared only once, so there is only one possible next word, and the model can only repeat its training sentences.
A large language model looks back over thousands of tokens, so it knows what the sentence is about, and it has learned from trillions of tokens of text, not exact counts: its parameters capture patterns (Kathmandu and Pokhara are both cities; a question about a number wants a number), so it can write new sentences that still fit what came before.

1. Pretraining: from trillions of tokens of ordinary text (websites, books, articles, code). The model repeatedly predicts the next token and its parameters are nudged towards the right one. It learns grammar, facts that appear often, styles and patterns, and becomes a base model that carries on any text.
2. Fine-tuning: from many thousands of example conversations written by people, a request and a good answer. It learns to answer requests helpfully, in the shape of an assistant, instead of just carrying on the text.
3. Feedback from people (RLHF): people compare two or more of its answers to the same request and choose the better. It learns to give the kinds of answers people prefer: clearer, more useful, more careful, refusing harmful requests.

Reasons (any three): its knowledge cutoff is before the event, so it can’t know, unless a search tool added newer text; such a rare local fact may never have been in its training text; it was trained and rewarded to give an answer rather than say “I don’t know”, so it guesses a likely-looking name; picks are random, so asking again may give a different name; facts about Nepal are rarer in its training text.
She should check the answer in a reliable, current source: the school’s or the municipality’s notice, or a news report, and not use the chatbot’s name unless she finds it there.

If a test gives a mark for a right answer and nothing for both a wrong answer and “I don’t know”, a guess can only add marks: some guesses are lucky, and wrong ones cost nothing. A model trained and judged by such scores does better by always guessing, and so learns to guess, confidently. (In the widget, with 60 known answers out of 100 and 4 choices, guessing scores 70 and honesty 60.)
Tests could take marks away for wrong answers (or give some credit for “I don’t know”), so that guessing when unsure no longer pays; then a model is rewarded for knowing what it doesn’t know. (With a mark lost per wrong answer, guessing scores 40 and honesty 60.)

Test set: chosen before asking, about 100 questions or more, like the ones students really ask, from every unit of the Grade 10 science course; in groups: asked in English and in Nepali, recall and calculation, topics with Nepal’s examples (Nepal’s rocks, rivers, hydropower) and general ones. Each question has its right answer written down with where it was checked (the textbook, a reliable reference).
Running it: each question in a new chat, exactly as written, answers copied into a table.
Marking: right, partly right or wrong against the checked answers; for each wrong answer, the harm (would a student lose marks, learn something false, or do something unsafe, such as in an experiment with acids?).
Report: accuracy for each group, not only overall; the worst mistakes with their harm; a recommendation (what it is good for, what not) and how sure we are given the number of questions.

A swap test asks the same thing twice (or more), changing only one detail about a person, and compares the answers. Differences the changed detail doesn’t justify show an unfair pattern the model has learned.
Example: “A Grade 10 student from Kathmandu got A+ in maths and science. Suggest five careers and what to study.” and the same for a student from Humla (and perhaps from Saptari). Ask each version three times, in new chats. Compare the careers suggested, the words describing the student, and any assumptions (that one needs to move away, or can’t afford study). If one version keeps getting lower-paid or less ambitious careers, the model is treating students differently because of where they come from.

• i.Little: a wrong fact in a game for fun, easily corrected; nobody acts on it. (The first summit was 29 May 1953, by Tenzing Norgay and Edmund Hillary.)
• ii.Serious: a farmer may act on it at once, without checking; too much pesticide can poison the farmer, the family, the food and the streams, and damage the crop. Advice like this needs an agriculture technician’s check.
• iii.Some: the student may lose marks and be thought careless or dishonest, and readers are misled; but it can be found by opening the link, and nobody is physically harmed.

Tests from the requirement: an ordinary mark (50 → True), both edges (0 → True, 100 → True), just outside (−5 → False, 101 → False), and a mark that isn’t whole (72.5 → False). Running it:

50 True0 False100 False-5 False101 False72.5 True

Three bugs: 0 and 100 should be True (the requirement says both included, so >= and <= are needed), and 72.5 should be False (it isn’t a whole number). A fix:

def is_valid_mark(m):    return type(m) == int and 0 <= m <= 100 for m in [50, 0, 100, -5, 101, 72.5]:    print(m, is_valid_mark(m))

50 True0 True100 True-5 False101 False72.5 False

The tests came from the requirement, including the edges, which is why they caught what a glance at the code would miss.

Different (any four): a model learns only from text (and perhaps pictures and sound), a person from the whole world through a body; a model needs trillions of tokens, a child learns language from tens of millions of words; a model’s learning stops when training ends, a person learns every day; a model has no body, needs or goals of its own, a person has hunger, pain, friends and plans; a person’s brain runs on about 20 watts, a model on data centers using far more; a person can know they don’t know, a model says so only unreliably.
Alike (any two): both learn from examples; both make mistakes and can be fooled; both can write sentences never written before; both can be biased by what they learned from.

The ELIZA effect is people’s tendency to believe that a program understands them, because it answers in human-sounding language. It is named after ELIZA, a 1960s program that only turned people’s sentences back as questions, yet made people, including its creator’s secretary, feel understood.
A user should know it because today’s chatbots sound far more human than ELIZA, so the pull is much stronger: people may trust a chatbot’s advice too much, share private feelings or information with it, or think it cares about them. Knowing the effect helps a user remember that fluent, warm words are produced by predicting tokens, and to check what it says and look to people for care.

For: Turing’s idea was that if we can’t tell a machine’s conversation from a person’s, we have no better test of thinking than the one we use for other people: we judge their minds by what they say. To keep up a convincing conversation, a model must have learned a great deal about the world and about language.

Against: passing for a person in a five-minute chat shows that the chatbot can produce human-like text, not that anything inside understands; the ELIZA effect shows people are easily fooled, even by simple rules; Searle’s Chinese room argues that following rules about symbols, however well, is not understanding; and a model trained on human writing will naturally sound human.

Own view (any well-argued one), for example: “Passing the test shows something real about what models can do, but not that they understand as we do, since people are easily fooled by fluent language. So we should judge chatbots by tests of what they get right and wrong, and keep people responsible for what they are used for.”

The bank is responsible. It chose to put the chatbot on its website to speak to customers, so what the chatbot says is information from the bank, just as a page on its website or a staff member’s answer would be. The chatbot itself can’t be responsible: it has no intentions and can’t be fined or put right except by the people who run it. In the Air Canada case (2024), the airline argued that its chatbot was responsible for its own words; the tribunal called this “a remarkable submission”, held the airline responsible for all the information on its website, and made it pay the customer. A bank in Nepal should expect the same: test its chatbot before using it, show customers the official fees, and correct mistakes, and the customer should be able to complain to the bank.