The Transformer - A Friendly Guide

Why This Guide Exists

The Transformer is the engine inside ChatGPT, Gemini, Claude, and virtually every modern AI that reads or writes text. Famous explanations like The Illustrated Transformer are great, but they still lean on vectors, dot products, and dimensions.

This guide goes one level simpler. No math. No dimensions. Just one small example, followed from start to finish.

Our running example:

Input  (English):  Python program to print hello world
Output (code):     print("Hello World")

1. The Black Box

From the outside, the Transformer is just a translation machine:

Something goes in, something comes out. Now let’s open the box.

2. Inside: A Reading Team and a Writing Team

The Transformer is a company with two departments:

Both departments are actually stacks — the original design used 6 encoder layers and 6 decoder layers, each refining the work of the one below it. Like six rounds of editing: each pass makes the understanding a little sharper.

3. Words Become Numbers (With Seat Numbers)

Computers cannot read words, only numbers. So before anything else:

1. Embedding: each word is swapped for a list of numbers. Similar words get similar numbers (“print” and “display” end up near each other).
2. Positional encoding: a “seat number” is mixed in, so the model knows where each word sits. Without it, “program to print” and “print to program” would look identical.

Analogy: every word gets a name badge (its meaning) and a cinema ticket (its seat number).

4. Self-Attention: Words Talk to Each Other

This is the heart of the Transformer. Before deciding what a word means, the model lets every word look at every other word in the sentence.

Consider: “The animal didn’t cross the street because it was too tired.”

What does “it” refer to — the animal or the street? You know instantly. Self-attention is how the model figures it out: while processing “it”, it looks around, finds “animal” highly relevant, and blends some of “animal” into its understanding of “it”.

Every word plays three roles at once:

Each word’s question (Query) is matched against every word’s label (Key). Good matches get high attention; then the word absorbs a blend of the matching words’ information (Values).

Analogy: a group discussion. Everyone asks a question, everyone offers what they know, and each person walks away with a personalised summary of the most relevant answers.

After the discussion, each word goes through a small feed-forward network — a moment of thinking alone to digest what it just heard. Discussion, then reflection. That pair repeats in every layer of the stack.

5. Many Discussions at Once (Multi-Head Attention)

One discussion isn’t enough. “It” needs to know what it refers to (the animal) but also how it feels (tired). So the Transformer runs several attention discussions in parallel — the original used 8 — each free to focus on different things: one on grammar, one on meaning, one on nearby words. The results are merged into one summary.

Analogy: eight breakout rooms discussing the same sentence from different angles, then combining their notes.

6. The Writer’s Two Glances

The decoder (writing team) has the same tools, plus one rule and one extra habit. Before writing each word, it glances at two places:

Glance 1 — at its own draft (masked self-attention). The decoder re-reads what it has written so far — but is forbidden from peeking ahead. During training the full correct answer is on the table, so without this rule the model could just copy the next word and learn nothing.

Analogy: in an exam you may re-read your own answers, but not the answer key.

Glance 2 — at the reading team’s notes (cross-attention). The decoder checks the encoder’s understanding of the input. About to write what goes inside the brackets? It looks back and finds “hello world” in the English sentence.

7. Picking the Next Word: A Confidence Vote

At the top of the decoder sits the final step: turn all that understanding into an actual word.

The model gives a score to every word it knows (tens of thousands), and softmax turns the scores into percentages that add up to 100%:

After "<start>", the model votes:

  print    96%   ← chosen
  display   2%
  code      1%
  ...everything else shares the remaining 1%

Analogy: a multiple-choice question with 50,000 options, where the model writes its confidence next to each one — then picks the favourite.

8. The Loop: One Word at a Time

The encoder reads the input once. The decoder then loops, adding one word per step, feeding each new word back in, until it produces a special <end> token:

Final output: print("Hello World")

This is exactly what a chatbot is doing when its answer appears word by word on your screen.

9. How It Learns

Everything above described a trained Transformer. But fresh out of the box, all its internal weights are random — its first attempts are garbage. Here’s how it gets good.

Step 1: Prepare a quality dataset

As always, everything starts with good data: thousands of quality input/output pairs.

Input:  "Python program to print hello world"
Output: print("Hello World")

Input:  "Python program to add two numbers"
Output: a + b

... thousands more pairs ...

The dataset is the answer key. We know what the model should say for every input.

Step 2: Let it guess (badly)

The untrained model runs the full pipeline and votes on the first word. With random weights, the vote is nonsense:

Wanted:   print   →  100%
Got:      print   →    3%,  banana → 7%,  the → 5%, ...

Step 3: Measure the miss (cross-entropy loss)

The cross-entropy loss is a single number measuring the gap between what we wanted and what we got. Confident and right → tiny loss. Confident and wrong → huge loss.

Step 4: Nudge every weight — in both teams

The loss flows backwards through the whole machine (backpropagation), nudging every learnable weight a tiny bit:

• the Query, Key, and Value matrices in every attention head, in every encoder and decoder layer
• the feed-forward networks
• the normalization layers and the final voting layer

One crucial point: a bad output is never just the decoder’s fault. The encoder and decoder are trained together, and the blame — and the correction — is shared equally between them. If the translation is wrong, maybe the writing was sloppy, or maybe the reading notes were bad in the first place. Both get tuned.

Repeat millions of times. The random weights slowly become the “knowledge” that makes attention meaningful and the votes accurate.

Analogy: a student doing thousands of practice problems with an answer key. Every mistake produces a small correction — not just to the final answer, but to the whole way of reading the question.

10. What Is the “Context Limit”?

You’ve heard it: “this model has a 128k context window”. The context limit is the maximum number of tokens the model can consider at once — your prompt plus its answer so far. Here is what that limit means inside each layer:

So the context limit boils down to three things:

1. Attention is quadratic. The “everyone talks to everyone” meeting becomes impossibly expensive as the room fills up. This is the fundamental reason context windows exist.
2. Seat numbers have a trained range. The model has only practised with positions up to its limit; beyond that, the seat numbers mean nothing to it.
3. Memory during generation. While writing, the model keeps every previous token’s Key and Value on hand (the “KV cache”) so old tokens don’t need re-reading. Longer context → more to keep in memory.

Analogy: a meeting room. Ten people can all genuinely listen to each other. Ten thousand people cannot — the cross-talk (attention) explodes, the seat numbering scheme runs out, and nobody can keep notes on everyone. The context limit is the room’s fire-code capacity.

That’s also why long-context research focuses exactly here: smarter seat numbers that generalise to unseen positions, and cheaper attention where not everyone talks to everyone.

11. Summary Cheat-Sheet

The one-sentence version: a Transformer reads by letting every word talk to every other word, writes by voting on one word at a time, and learns by comparing its votes against a quality answer key and nudging every weight — in both the reader and the writer — until the votes come out right.

Where to Go Deeper

• The step-by-step series: Part 6: Transformer Introduction through Part 13: Sentence Embeddings with Sentence Transformers
• Jay Alammar’s The Illustrated Transformer — the classic visual walkthrough, one notch more technical than this guide
• The original paper: Attention Is All You Need (2017)