How Large Language Models Actually Work: A Visual Guide to Transformers

The Core Idea: Predict the Next Word

At their heart, large language models do one thing: predict the next word in a sequence. That's it. Given "The cat sat on the", they predict "mat" (or "floor", or "lap"). The magic is that to do this well at scale, they must develop a deep internal model of language, facts, reasoning, and even something that looks like common sense.

Tokens: The Building Blocks

LLMs don't see words — they see tokens. A token is a chunk of text, typically 3-4 characters. "Hello world" might tokenize as ["Hello", " world"]. The model has a vocabulary of 50,000-200,000 tokens. Everything — every word you type, every response you get — starts and ends as a sequence of token IDs.

The Transformer Architecture

Introduced in the 2017 paper "Attention Is All You Need," the transformer is the engine behind every modern LLM. It has three key components:

1. Embedding Layer

Each token is converted into a high-dimensional vector — typically 4096 to 8192 numbers. These vectors capture semantic meaning. The word "king" sits close to "queen" in this space, while "banana" is far away. This is the model's vocabulary of meaning.

2. Attention Mechanism

This is the breakthrough. Attention lets each token "look at" every other token in the sequence and decide which ones are relevant. When processing "The cat sat on the mat because it was tired," the model uses attention to figure out that "it" refers to "the cat" (not "the mat"). It does this by computing mathematical relationships between every pair of tokens — all in parallel.

3. Feed-Forward Networks

After attention gathers context, feed-forward layers process each token independently through a series of mathematical transformations. These layers store the model's factual knowledge — dates, names, relationships, concepts. In a 70-billion-parameter model, these layers contain billions of learned patterns.

Training: The Secret Sauce

Training happens in two phases. Pre-training: the model reads trillions of words from the internet, books, and code, learning to predict the next token. This takes months on thousands of GPUs and costs millions of dollars. Fine-tuning: the model is then trained on curated examples of helpful responses, learning to be useful, harmless, and honest.

A 70B parameter model trained on 15 trillion tokens costs approximately $4-6 million in compute alone. The resulting model file is around 140GB.

Inference: How a Prompt Becomes a Response

When you type a prompt, here's what happens:

1. Your text is tokenized into a sequence of token IDs
2. Each token flows through all layers of the transformer — embeddings → attention → feed-forward — dozens of times
3. The final layer outputs probabilities for every token in the vocabulary
4. A sampling strategy (like temperature-based sampling) picks the next token
5. That token is appended to the sequence, and the process repeats — this is autoregressive generation

Why Size Matters (Until It Doesn't)

Bigger models consistently perform better — up to a point. The shift in 2024-2026 has been toward reasoning models that "think" before responding, using chain-of-thought processing. Models like o1, Claude 3.5, and DeepSeek-R1 spend extra compute on internal reasoning steps. A well-trained 8B reasoning model can outperform a naive 70B model on complex tasks.

Running Models Locally

Thanks to quantization (compressing model weights from 16-bit to 4-bit precision) and tools like Ollama and llama.cpp, you can now run capable models on a gaming PC. A quantized 8B model needs about 5GB of VRAM and runs at 30-50 tokens per second on an RTX 3070. The gap between cloud and local AI is closing fast.