How it actually works

Most people use AI assistants for months without knowing what's happening under the hood. That's fine for a kettle. It's not fine here, because the assistant's strange behaviour, the confident mistakes and the sudden forgetting, all come from the same four facts. Once you know them, the behaviour stops being mysterious and starts being predictable.

1. It predicts the next piece of text. That's the whole trick.

A language model has read an enormous amount of text and learned one skill: given some text, what is likely to come next? Not what's true. What's likely.

When you ask it a question, it isn't searching a database for the answer. It's producing the text that would most plausibly follow your question, one small piece at a time. Each piece becomes part of the text, and it predicts the next one.

This sounds too simple to produce anything useful. It turns out that predicting text well, at this scale, requires something that looks a lot like understanding. To guess the next word of a legal contract, you need to know how contracts work. To continue a joke, you need to get the joke. So the ability is real. But the mechanism never changes: it's always choosing the plausible next piece.

2. Plausible is not the same as true

Here is the consequence that matters most at work. The model is built to produce text that sounds right. Most of the time, for common things, text that sounds right is right, because the model learned from a lot of correct text. But when it doesn't know, it doesn't stop. It keeps producing plausible text, in exactly the same tone.

That's why it invents a statistic with two decimal places, a court case that doesn't exist, or a link that goes nowhere. It isn't lying; lying needs an intention. It's doing the only thing it does, and a made-up figure is a plausible-looking figure.

Three kinds of content live mostly in the outer ring: specific numbers, names of real people and sources, and direct quotes. Those are the things you check. Lesson 3 is entirely about how.

3. It can only see what's in the window

The model doesn't remember. Each time it replies, it reads everything in the current chat from the top: your messages, its own earlier replies, the files you attached, plus some instructions from the product. That's called the context window, and it is the only thing the model can see.

Three things follow:

• It doesn't know what you told it last week. Some products add a "memory" feature that quietly pastes a few saved facts into the window, but that's the product, not the model. If something matters, say it again.
• It doesn't know anything about you. Not your job, your company, your reader, or what "the usual format" means. Everything you don't put in the window, it guesses.
• It can't see your files or the internet unless the product gives it a tool to fetch them and it decides to use it. "Summarise the attached" works because the attachment is placed in the window. "Summarise the report I sent Dave" doesn't, because it isn't.

4. The window is measured in tokens, and it fills up

Text is counted in tokens, roughly three-quarters of a word in English. The window has a limit, usually large, but long conversations and big attachments eat it. As the window fills, two things happen: the product may start dropping the oldest parts, and the model has more text to weigh, so early instructions get less attention. That's why a chat that was great on Monday gives odd answers on Thursday. Nothing broke. The window got crowded.

This also explains the paid plans. Every reply means re-reading the whole window. Long chats cost more compute, so they use your allowance faster. One chat per task is cheaper, cleaner and more accurate, and it's the first habit in lesson 2.

One more thing: it's a little random on purpose

Ask the same question twice and you'll often get two different answers. The model picks from the plausible options with some randomness built in, because always choosing the single most likely word produces dull, repetitive text. For writing, that's a feature. For facts, it's another reminder that the output is generated, not retrieved.

What this means for you

Four facts, four habits:

1. It predicts text, so tell it what shape of text you want, not just what topic.
2. Plausible isn't true, so check numbers, names and quotes. Skim the rest.
3. It only sees the window, so paste the material and say who you are. Don't assume it remembers.
4. The window fills up, so start a new chat for each new task.