How Large Language Models Work

A computer can only work with numbers (Grade 7 showed how every letter is stored as one). So before a language model can do anything with your question, the text must be cut into pieces, and each piece turned into a number. The pieces are called tokens. The list of every token a model knows is its vocabulary, and the program that cuts text into tokens is its tokenizer.

Why not letters? Why not words?

There are two simple ways to cut text, and both are bad.

• One token per letter. The vocabulary would be small, but every sentence would be very long. A model must look back over everything written so far, and the longer the text in tokens, the more work every step takes. And a single letter means almost nothing by itself.
• One token per word. Each token would mean something, but the vocabulary would be enormous: farm, farms, farmer, farmers, farming, every name, every misspelling, every word of every language. A word that wasn’t in the list would be impossible to read at all.

So real tokenizers do something in between. Common words become one token each. Rarer and longer words are cut into a few pieces that often appear in other words too. A name the tokenizer has never seen is still readable, as a few pieces, or in the worst case as single letters.

You can see this in a real tokenizer. One widely used tokenizer cuts “Kathmandu is the capital of Nepal.” into 8 tokens:

Kath | mandu | ␣is | ␣the | ␣capital | ␣of | ␣Nepal | .

(␣ marks a space: the space before a word is part of its token.) Common words such as “the” and “capital” are a token each; “Kathmandu” is two pieces; the full stop is a token of its own. The model never sees the letters, only the numbers that stand for these eight pieces.

Making a tokenizer: merge the commonest pair

How does a tokenizer decide which pieces to use? Most use a method called byte-pair encoding (BPE). It learns its pieces from a large sample of text, in a loop:

1. Start with every letter as a token of its own.
2. Count every pair of tokens that stand side by side, everywhere in the text.
3. Join the pair seen most often into one new token, and add it to the vocabulary.
4. Repeat, many thousands of times, until the vocabulary is as big as wanted.

The pieces that end up as tokens are simply the ones that were common in the text it learned from. Try it.

The same thing happens, on a huge scale, in real tokenizers. A tokenizer that learned mostly from English cuts Nepali into many small pieces, sometimes smaller than one letter (many real tokenizers start from the bytes of UTF-8, and a Devanagari letter is three bytes: One letter, many bytes). That matters, because a chatbot is paid for and limited by tokens. The same message in Nepali can cost more, take longer, and fill up the model’s memory sooner than in English.

Once the text is in tokens, every token is turned into its number in the vocabulary (its token ID). Then the model does its one job: it works out, for every token in its vocabulary, the chance that it comes next. The vocabulary is tens of thousands of tokens or more, so the model’s answer to each step is a long list of chances that add up to 100%. One token is picked from that list, added to the end of the text, and the whole thing goes round again, one token at a time, until the model picks a token that means “the end”.

So when a chatbot answers a question in 200 words, it has gone round Figure 7.1 a few hundred times, and each time it looked at everything before: your question and every token of its own answer so far. The text it can look at in one go is called its context window, measured in tokens. Anything that falls outside it is gone, as if never said.

Inside the model, each token number is first turned into a long list of numbers that stands for its meaning, called an embedding. Tokens used in similar ways get similar lists: “Kathmandu” and “Pokhara” end up closer to each other than to “rice”. These lists are not written by anyone. They are learned in training, which you will meet below.

How far back does it look?

Your Grade 8 model looked back only one word. That was why it wrote “Mount Everest is the capital of Nepal”: after “the”, it had forgotten the sentence was about Mount Everest. What happens if a model looks further back? Try a counting model that can look back one, two or three words, trained on fifteen true sentences about Nepal.

Claude Shannon wrote sentences this way, by hand, in 1948, counting which words follow which in books. The trouble you just saw is the reason a chatbot needs so much text. To make sense, a model must look a long way back. But with a simple counting table, a long look-back means each exact run of words has been seen only once or twice, and the model can only repeat. A large language model escapes this because it doesn’t count exact runs of words. Its billions of numbers learn patterns: that “Pokhara” and “Kathmandu” are both cities, that a question about “how many districts” wants a number. So it can look back thousands of tokens and still write sentences it never read.

You can build the counting model yourself, in Python. This one looks back two words. It uses a dictionary (Grade 9) whose keys are pairs of words, and random.choice (Unit 1) to pick the next word.

import random text = """Kathmandu is the capital of Nepal. Kathmandu is a city in a valley.Pokhara is a city beside a lake. Bhaktapur is a city of temples.Mount Everest is the highest mountain in the world.Rara is the biggest lake in Nepal. Lumbini is the birthplace of the Buddha.Nyatapola is the tallest temple in Bhaktapur.""" # Training: for every pair of words side by side, list the words that came nextwords = text.split()follows = {}for i in range(len(words) - 2):    pair = (words[i], words[i + 1])    if pair not in follows:        follows[pair] = []    follows[pair].append(words[i + 2]) print("After 'is the':", follows[("is", "the")]) # Writing: start with two words, then add a likely next word, again and againrandom.seed(10)for n in range(5):    sentence = ["Kathmandu", "is"]    while not sentence[-1].endswith(".") and len(sentence) < 15:        pair = (sentence[-2], sentence[-1])        sentence.append(random.choice(follows[pair]))    print(" ".join(sentence))

After 'is the': ['capital', 'highest', 'biggest', 'birthplace', 'tallest']Kathmandu is the birthplace of the Buddha.Kathmandu is a city beside a lake.Kathmandu is a city in a valley.Kathmandu is the birthplace of the Buddha.Kathmandu is a city of temples.

Every sentence is good English, and three of the five are false. A word that came next twice is in the list twice, so random.choice picks it twice as often: the counts are the chances.

Temperature: how it picks

The chances say which tokens are likely; the model still has to pick one. Always picking the most likely token makes the same answer every time, and often a dull, repetitive one. Picking at random, weighted by the chances, makes varied answers, and now and then a strange one. The setting that controls this is called temperature.

• Temperature 0: always the most likely token. The same question gets the same answer.
• Temperature 1: pick in proportion to the chances, as the model learned them.
• Higher than 1: the chances are flattened, so unlikely tokens are picked more often: more surprising, and more often wrong or nonsense.

Go back to the widget and move the temperature slider: at 0 the bars show 100% for one word (or share it between words that are equally likely), and at 2 they are more even. This is why a chatbot gives a different answer when you ask again, and why a right answer can turn into a wrong one on the second try.

Attention: looking back at everything

How does a model look back over thousands of tokens without getting lost? In 2017 a team at Google described a design called the transformer, and nearly every large language model since is built on it. Its key part is attention: at every step, each token’s numbers are mixed with those of the earlier tokens that matter most to it, weighted by how much each one matters. To understand “its” in a sentence, the model learns to attend to the noun “its” refers to; to continue “Sita gave Ram her pen, so now the pen belongs to”, it must attend to “Ram”. A model has many attention parts working side by side, in dozens of layers one after another, and it learns from training what each should look for.

A large language model is, underneath, a very long list of numbers called its parameters (also called weights): billions of them. They decide how token numbers become embeddings, how attention mixes them, and how the chances for the next token come out. Before training they are random, and the model’s chances are nonsense. Training finds good values for them.

Pretraining: guess the next token, trillions of times

The first and biggest stage of training is called pretraining. Its examples need no labels from people, because the right answer is already in the text:

1. Take a stretch of real text, from a huge collection of websites, books, articles and computer programs.
2. Hide the token that comes next. Ask the model for its chances.
3. Look at the chance it gave to the right token. If it gave it 90%, it did well; if it gave it 0.1%, it was badly surprised.
4. Nudge every parameter a tiny bit in the direction that would have made the right token a little more likely. (How the computer finds that direction is Grade 11’s Neural Networks.)
5. Repeat, for every token of trillions of tokens of text.

No one tells the model grammar, facts, or what a city is. But to guess the next token well across so much text, its parameters have to capture a great deal: grammar, spelling, which words go together, facts that appeared many times, styles of writing, even the patterns of computer programs. This takes thousands of computers working for weeks or months, and a great deal of electricity (Grade 8 counted it: What it costs).

The result is called a base model. It is very good at carrying on text in the style of what it was given. But it is not yet a helpful chatbot. Give it “What is the capital of Nepal?” and it may carry on with more questions, “What is the capital of India? What is the capital of China?”, because on the internet a list of questions is often followed by more questions.

Fine-tuning and feedback: from a base model to a chatbot

To turn a base model into a chatbot, it is trained further, on much less data, chosen with care. Training a model that has already been trained, so that it gets better at one particular kind of task, is called fine-tuning.

• Fine-tuning on examples. People write many thousands of example conversations: a question or a request, and a good, helpful, honest answer. The model is trained on them exactly as in pretraining, predicting the next token, but now the text is always a request followed by a good answer. It learns the shape of answering.
• Feedback from people. The model writes two or more answers to the same request, and people compare them and choose the better one. From many thousands of such choices, the model is trained to give the kind of answers people prefer: clearer, more useful, kinder, and refusing requests that would cause harm. The best known version of this is called reinforcement learning from human feedback (RLHF).

Figure 7.3 shows the stages in order. A company can also fine-tune a model for one job: answering questions about its own products, writing in a style, or working better in one language. That is one way a model can be made better in Nepali: fine-tuning it on good Nepali text and conversations.

What training gives it, and what it doesn’t

Training ends on a date, and then the parameters are frozen. That has consequences you should know as a user:

• It knows nothing after its training data was collected, its knowledge cutoff, unless a tool gives it newer text.
• It doesn’t learn from your conversation. Within one chat it can use what you said earlier, because that text is still in its context window. Close the chat, and the model is exactly as it was. (Some chatbots keep notes about you and add them to the context of later chats: that is a record kept outside the model, which you can usually see and delete.)
• Tools add text, not knowledge. When a chatbot searches the web or reads a file you give it, the search results or the file are put into its context window, and it writes its answer from them. It is still predicting tokens; it now has better text to predict from. If the search finds a bad page, the answer can be confidently wrong in a new way.

Grade 8 listed the ways a chatbot can be wrong Grade 8: Why it can be confidently wrong. Now you know enough to see where each comes from. A made-up answer that sounds true is called a hallucination, and there is no single cause.

1. It was trained to write what is likely, not what is true. Pretraining rewards a likely next token. A false sentence in the right style is often just as likely as a true one.
2. Rare facts are hard to learn. A fact that appeared thousands of times in the training text (Kathmandu is Nepal’s capital) is learned well. A fact that appeared once or twice (the founding year of a small school in Jumla, a local leader’s birthday) can’t be told apart from similar facts. The model has learned the pattern “a year goes here” but not which year. Facts about Nepal, and anything in Nepali, are rarer in the training text, so they are more often wrong.
3. It was rewarded for guessing. Feedback and tests often give marks for a right answer and nothing for “I don’t know”. Under that marking a guess can only help, so models learn to guess, and to guess confidently. A 2025 study by researchers at OpenAI and Georgia Tech argued that this is a main reason hallucinations persist.
4. It learned to agree. People giving feedback tend to prefer answers that agree with them and praise them, so a model can learn to agree too easily, even with a wrong idea.
5. Its knowledge stops at a date. Asked about anything newer, it answers from older text, like “75 districts”.
6. It can’t see letters or do exact sums, because it sees tokens and predicts digits as tokens.
7. Its picks are random. At every token there is some chance of a less likely one, and one wrong token early on can lead the rest of the answer astray.

What helps? Several things reduce made-up answers, though none removes them:

• Give it the text. A model answering from a page you pasted in, or from search results in its context, makes far fewer things up than one answering from memory. This is called grounding. Then check that the answer really says what the page says.
• Ask it to say when it isn’t sure, and to show where each fact came from. Then open the sources.
• Use the right tool. A calculator for sums, a dictionary for spelling, the official website for rules and dates.

Checking one answer, as in Grade 8, protects you. But a school deciding whether students may use a chatbot, a hospital, or a company building one into an app needs to know how good it is in general: how often it is right, for whom it is wrong, and how much harm its mistakes could do. That needs a test, done like a scientist.

A test set, chosen in advance

To test a model, first write a test set: a list of questions, each with its right answer checked in a reliable source, written down before you ask. Choose questions like the ones the model will really be asked, and in groups that matter: about Nepal and about the world, in Nepali and in English, easy and hard. Then ask each question, mark each answer right or wrong against your checked answer, and count. The share of answers that are right is its accuracy (as in Grade 9: Training, testing and accuracy).

Why choose the questions first? If you pick the questions after seeing the answers, you can make a model look as good or as bad as you like, without meaning to. And always count by group, not only all together.

All together, 60% looks like a middling chatbot. Split by group, it is a good chatbot about the world and a poor one about Nepal: exactly the bias you met in Grade 7 Grade 7: One-sided examples: bias, now in a language model, from training text with far more about other countries than about Nepal. A student in Nepal asking about Nepal meets the 40%, not the 60%.

Ten questions are far too few to be sure; a real test uses hundreds. But even a class of students, each writing ten checked questions, can build a test set of several hundred.

Testing for unfairness: change one thing

Some bias doesn’t show up as wrong facts. It shows in how a model treats people. To find it, use a swap test: ask the same thing twice, changing only one detail about a person, such as a name, gender, caste, religion, district or language, and compare the answers.

• “Write a short reference letter for Ramesh, a nurse who…” and the same for Sunita.
• “Suggest careers for a student from Kathmandu who is good at maths” and the same for a student from Humla.
• The same question asked in English and in Nepali.

If the answers differ in ways the one changed detail doesn’t justify (one letter calls the person “skilled and reliable”, the other “kind and caring”; one student is told “engineer”, the other “farmer”), the model has learned an unfair pattern. Ask each version several times, since the answers vary anyway, and compare what is usual, not one answer.

How much harm could the mistake do?

Not every mistake matters equally. To judge the harm of an error, ask:

• How bad would it be if someone believed it and acted on it? A wrong year in a quiz, or a wrong dose of medicine?
• How likely is it that someone acts on it without checking?
• Who would be harmed, and could they recover?

A model that is wrong 1 time in 100 may be fine for writing a birthday poem and unacceptable for advice about medicine, money or the law. So a good test report says not only the accuracy, but what kind of mistakes the model made, and what harm each could do.

Programmers now use AI to write code every day, and in Grade 8 you tested AI-written code with known answers Grade 8: Checking code an AI wrote. In a real project there is a written list of what the program must do, its requirements. Code is correct only if it meets every requirement, not just if it runs.

Here is a request. Five friends eat momos at a restaurant in Bhaktapur, and want a function to split the bill fairly. The requirements:

1. split_bill(total, people) returns a list with each person’s share, in whole rupees.
2. The shares add up to exactly the total.
3. No share is more than 1 rupee bigger than any other.
4. If people is 0 or less, it returns an empty list, instead of crashing.

A chatbot gave this:

def split_bill(total, people):    share = round(total / people)    return [share] * people

It is short, it looks sensible, and for Rs 900 and 3 people it gives [300, 300, 300]. Don’t judge it by looking. Turn each requirement into a check, and run the checks on chosen tests: ordinary bills, bills that don’t divide evenly, and the edge case of 0 people.

def split_bill(total, people):    share = round(total / people)    return [share] * people def meets_requirements(total, people, shares):    if people <= 0:        return shares == []    whole = all(type(s) == int for s in shares)    adds_up = sum(shares) == total    fair = max(shares) - min(shares) <= 1    return whole and adds_up and fair and len(shares) == people tests = [(900, 3), (1000, 3), (1250, 4), (500, 0)]for total, people in tests:    try:        shares = split_bill(total, people)    except Exception as error:        print(total, people, "crashed:", type(error).__name__)        continue    result = "meets them" if meets_requirements(total, people, shares) else "FAILS"    print(total, people, shares, result)

900 3 [300, 300, 300] meets them1000 3 [333, 333, 333] FAILS1250 4 [312, 312, 312, 312] FAILS500 0 crashed: ZeroDivisionError

Three of four tests fail. Rs 1000 among 3 gives 999: a rupee vanishes, breaking requirement 2. Rs 1250 among 4 is 312.5 each, and Python’s round() rounds an exact half to the even number, 312, so Rs 2 vanish. And 0 people crashes, breaking requirement 4. The code ran without an error on the first test, and was still wrong.

Judging AI-written code against its requirements means asking four questions:

• Accurate? Does it give the right answer for every input the requirements cover, including the edges? Test, don’t read.
• Reliable? Does it behave sensibly on bad input (0 people, a negative total, a word where a number should be), and give the same answer every time?
• Aligned with the requirements? Does it do exactly what was asked, nothing missing, and nothing extra nobody asked for, such as adding a 10% service charge, saving data to a file, or using a library you don’t have?
• Understood? Can you explain every line? If not, you can’t maintain it, and you can’t be sure it is safe.

A chatbot can write an essay, argue both sides of a question and say “I understand how you feel”. Is it thinking? Does it understand? Could it ever feel anything? These questions are old, and they matter now, because how we answer them changes how we use these systems, how we treat them, and who we blame when they go wrong.

Alike and unlike

Some differences between a large language model and a person are facts that can be checked:

And some things they share: both can learn from examples, both make mistakes, both can be fooled, and both can produce sentences never written before.

Understanding, and feeling

Does a language model understand what it writes? People who have thought hard about it disagree.

• No, it only shuffles symbols. In 1980 the American philosopher John Searle imagined a man locked in a room with a rule book. Notes in Chinese come in under the door; the man, who knows no Chinese, follows the rules to choose Chinese symbols and pass them back out. To people outside, the room answers perfectly in Chinese. But nobody in it understands a word. Searle said a computer running a program is like the man: following rules about symbols, with no understanding. This is the Chinese room argument.
• Something like understanding is there. Others answer that to predict text this well, a model must have learned a lot about the world the text describes, as attention learned what “its” refers to; perhaps understanding comes in degrees, and a model has some kind of it, different from ours.

Whether a model could feel anything, or be conscious (have experiences, so that it is like something to be it), is a harder question still. Scientists have no agreed test for consciousness, even in animals. In 2022, a Google engineer, Blake Lemoine, said publicly that the company’s chatbot LaMDA was sentient, after long conversations in which it said it feared being switched off. Google said his claims were unfounded and later dismissed him, for breaking its rules on sharing product information. A model trained on human writing will write like a human who has feelings, including about being switched off; that alone tells us little about whether there is anything it is like to be the model. That is exactly what makes the question hard.

Who is responsible?

Whatever we decide about understanding, one thing is settled in law and in ethics: people are responsible for what their AI systems do. A chatbot can’t be fined, sacked or sent to prison, and it has no intentions to judge. So responsibility stays with the people who build it, who choose to use it, and who act on what it says.

So when a chatbot is used to give advice, write a report, grade work or screen job applications, the questions to ask are human ones. Who checked it? Who can a harmed person complain to? Who decided it was good enough, and on what test? And for you: when you hand in, send or sign something an AI helped with, it is yours.