From the first artificial neuron in 1943 to the frontier models of today.
Seven chapters. Dozens of interactive visualizations. No math required.
From the first artificial neuron in 1943 to the frontier models of today.
Seven chapters. Dozens of interactive visualizations. No math required.
Scientists ask: can machines think? The first mathematical models of neurons emerge, and with them, the dream of artificial intelligence.
The first mathematical model of a neuron — a simple binary threshold unit that showed logic could emerge from connected nodes.
Frank Rosenblatt builds the first trainable artificial neuron. The New York Times declares a machine that can "walk, talk, see, write, reproduce itself, and be conscious of its existence."
Marvin Minsky proves perceptrons can't solve simple problems like XOR. Funding dries up. The first AI winter begins.
After years of frozen funding, a handful of researchers quietly solve the problem that makes neural networks actually work.
Paul Werbos describes how to train multi-layer networks by propagating errors backward — but the world barely notices.
Rumelhart, Hinton & Williams publish "Learning representations by back-propagating errors." Neural networks can finally learn complex patterns. AI research reignites.
Networks learn to see images, understand sequences, and solve real-world problems. Different architectures emerge for different kinds of data.
Yann LeCun applies backpropagation to image recognition. CNNs learn to "see" by detecting patterns at multiple scales — from edges to objects.
Hochreiter & Schmidhuber invent Long Short-Term Memory — a network that can remember information across long sequences by learning what to forget.
LeCun's CNN reads handwritten checks for banks. One of the first neural networks deployed at scale — processing millions of checks per day.
GPUs enable truly deep networks. ImageNet proves that more data + more layers = superhuman performance. The generative era begins.
A deep CNN crushes the image recognition competition by a wide margin. The result shocks the field and launches the GPU-powered deep learning era.
Kingma & Welling introduce VAEs — networks that learn to generate new data by mapping to a smooth latent space.
Ian Goodfellow invents GANs — two networks locked in a creative arms race. One generates, one critiques. Together they produce stunningly realistic outputs.
A paper from Google rips up the playbook. The Transformer architecture replaces everything that came before — and language models begin their ascent.
Vaswani et al. introduce the attention mechanism that processes all positions simultaneously. No more processing one word at a time — the Transformer sees everything at once.
Two approaches diverge: BERT learns to understand language by masking words. GPT learns to generate language by predicting the next word. Both built on the Transformer.
OpenAI trains a larger language model and initially refuses to release it, citing concerns about misuse. The debate about AI safety enters the mainstream.
AI goes mainstream. Models generate images from text, hold conversations, write code, and reason about problems. The question shifts from "can it?" to "should it?"
A new approach to image generation: start with pure noise, learn to remove it step by step. The results surpass GANs in quality and diversity.
OpenAI launches ChatGPT. 100 million users in two months. For the first time, billions of people interact with a language model directly. Everything changes.
Models learn to use tools, see images, hear audio, and chain together complex reasoning steps. AI becomes collaborative — not just responsive.
How AI Thinks — an interactive guide from the first neuron to the frontier.
01Neurons, networks, gradient descent, and backpropagation — the building blocks.
02Supervised, unsupervised, reinforcement — the different ways machines learn.
03How CNNs learn to see — from edges to objects, layer by layer.
04RNNs and LSTMs — networks that understand sequences and time.
05Attention, Transformers, and LLMs — the architecture behind modern AI.
06Autoencoders, VAEs, GANs, and diffusion — how AI generates new things.
07Multimodal, agents, scaling, and alignment — where AI is headed next.
From the first artificial neuron in 1943 to the frontier models of today.
Seven chapters. Dozens of interactive visualizations. No math required.
Each chapter builds on the last — from the simplest artificial neuron to the most powerful models on Earth.
From a single artificial neuron to networks that learn —
the ideas that started it all.
In 1943, two scientists asked a simple question:
Can we build something that thinks?
Their answer began with a single, tiny unit — an artificial neuron.
A biological neuron receives signals from other neurons, processes them, and fires — or doesn't. The artificial neuron does the same thing, but with numbers.
Each input carries a value — like a signal strength. It could be a pixel brightness, a word frequency, or any measurable feature of the world.
Each connection has a weight — a multiplier that says how important that input is. Learning means adjusting these weights. Try dragging the weight sliders.
The neuron multiplies each input by its weight, sums everything, and adds a bias — a nudge that shifts the threshold for firing.
The sum passes through an activation function that decides the output. Like a gate: if the signal is strong enough, the neuron fires.
"A single neuron can only draw a straight line. But stack enough of them together, and they can trace any shape imaginable."
A single neuron can only separate the world into two halves — yes or no, cat or not-cat. That's useful, but limited.
Line up several neurons side by side and you get a layer. Each neuron looks at the same inputs but learns to detect a different pattern.
Stack layers on top of each other. Each layer takes the previous layer's output as input. The first layer might find edges. The next finds shapes. Then objects.
More layers = more abstraction. "Deep learning" simply means many layers. Each one transforms raw data into increasingly meaningful representations.
Data enters on the left, passes through each layer, and a decision emerges on the right. This is called a forward pass.
Without activation functions, a neural network would just be a fancy calculator doing linear math. Activations introduce the non-linearity that lets networks learn curves, edges, and complex patterns.
After every prediction, we measure the gap between what the network predicted and the right answer. That gap is the loss. The goal: minimize it.
Imagine the loss as a mountainous landscape. The height at any point represents how wrong the network's current settings are. The lowest valley is the best solution.
The network feels which way is "downhill" (the gradient) and takes a step in that direction. Step by step, it rolls toward the valley. Try dragging the ball.
Too fast and you overshoot the valley. Too slow and you barely move. The learning rate controls step size — one of the most important settings in all of AI.
Sometimes the ball settles into a small dip that isn't the deepest valley — a local minimum. Escaping these traps is one of deep learning's ongoing challenges.
Gradient descent tells us which direction to go. But in a deep network with millions of connections, how do we know which weights to adjust and by how much? That's backpropagation — the algorithm that unlocked modern AI.
The foundation is laid.
A neuron receives, weighs, and fires. Networks stack neurons into layers of abstraction. Gradient descent finds the best settings. Backpropagation tells every weight how to improve.
These four ideas — invented between 1943 and 1986 — are the bedrock of every AI system you use today.
Not all learning looks the same.
Each paradigm opens a different door to intelligence.
A child learns to name animals from picture books.
But how does a child learn to walk?
Different problems demand different kinds of learning.
In supervised learning, the model receives labeled data — input-output pairs. "This image is a cat." "This email is spam." Each example comes with the right answer.
The model ingests labeled examples and tries to learn the mapping from inputs to outputs. It's like a student studying flashcards — see the question, check the answer.
With each batch of examples, the model gets better. Early predictions are mostly wrong. But as it sees more data, the pattern crystallizes. Green lights outnumber red.
The real test: can it handle data it has never seen? A well-trained model generalizes — it learned the underlying pattern, not just memorized the examples.
"Supervised learning is the workhorse of modern AI. But what happens when you don't have labels?"
No labels. No right answers. The model must find structure on its own — clusters, patterns, anomalies. It's like sorting a pile of objects by similarity when nobody tells you what the categories are.
Labeling data is expensive. In the real world, you might have 100 labeled examples and 10,000 unlabeled ones. Semi-supervised learning uses both.
The labeled points act as anchors. Their confidence ripples outward — nearby unlabeled points likely share the same class. The model learns from the structure of the data itself.
Gradually, most unlabeled points gain confident predictions. A small investment in labeling yields a large return — the unlabeled data fills in the gaps.
"What if the machine could learn not from answers — but from consequences?"
No labeled examples. No dataset at all. An agent takes actions in an environment, receives rewards or penalties, and learns a strategy through trial and error. This is how AlphaGo learned to play — and beat — the world champion.
Instead of exploring randomly, the agent watches an expert perform the task. The teacher traces the optimal path — and the student observes every move.
The student tries to replicate the teacher's behavior. Its first attempts are rough — it drifts off the path, overcorrects, stumbles. But each iteration gets closer.
After enough iterations, the student's path closely matches the teacher's. Imitation learning bypasses the slow exploration of RL — learning from demonstration is far more efficient.
The model takes its own data and creates a puzzle: mask some tokens, then try to predict what's missing. No human labels needed — the data supervises itself.
For each masked position, the model generates a prediction with a confidence score. "The cat sat on the ___." Even without labels, the model learns grammar, meaning, and world knowledge.
Different masking patterns teach different things. This simple idea — predict the missing piece — powers GPT, BERT, and most modern language models. It unlocked AI at scale.
Many paths to learning.
Supervised learning maps inputs to labels. Unsupervised learning finds hidden structure. Reinforcement learning discovers strategies through trial and error. And self-supervised learning creates its own curriculum from raw data.
Each paradigm illuminates a different facet of intelligence — and together, they form the foundation of everything modern AI can do.
From pixels to perception.
The architecture that gave AI eyes.
A computer sees an image as a grid of numbers.
How do you teach it what those numbers mean?
Instead of looking at every pixel at once, a CNN slides a small window -- a filter -- across the image, detecting local patterns like edges, corners, and textures. The same filter works everywhere in the image, making CNNs remarkably efficient.
To a computer, an image is just a grid of numbers. Each cell holds a value -- brightness, color intensity -- that means nothing on its own. Click the grid to toggle pixels and draw a pattern.
A filter (or kernel) is a tiny grid of learned weights -- usually 3x3. It defines what pattern to look for: horizontal edges, vertical edges, blurs, sharpness. Try the presets below the visualization.
The filter slides across the image one position at a time. At each position, it multiplies its weights by the overlapping pixel values, then sums the result into a single output value. This is the convolution operation.
The result is a feature map -- a new grid where each cell indicates how strongly that region matched the filter's pattern. High values mean a strong match; the filter has detected the feature it was looking for.
"The key insight was weight sharing: the same small filter, applied everywhere, learns to detect the same feature regardless of where it appears."
Yann LeCun, 1989After convolution, the feature maps are large and detailed. Pooling compresses them -- reducing spatial dimensions while keeping the most important signals. This makes the network faster, reduces overfitting, and builds translation invariance.
A 2x2 window slides across the feature map. In each region, only the maximum value survives -- the strongest activation. Everything else is discarded. This keeps the most salient features while halving the grid dimensions.
Stride controls how many pixels the window moves between each operation. A stride of 2 means the window jumps two positions at a time -- the most common choice, which halves the output size in each dimension.
Stacking convolutional layers creates a remarkable property: each successive layer detects increasingly complex patterns. Simple edges combine into textures, textures into parts, parts into whole objects. Click any feature in higher layers to see which lower-layer features contributed to it.
The first layer learns the simplest building blocks: edges in every orientation. Horizontal lines, vertical lines, diagonals, and simple gradients. These are the atoms of visual perception.
The second layer combines edges into richer patterns: corners, curves, simple textures, and repeated motifs. Each neuron responds to a combination of the edge detectors below it.
By layer three, the network recognizes parts of objects: eyes, wheels, petals, windows. These are complex compositions of the textures and corners from layer 2 -- meaningful fragments of the visual world.
The deepest layers respond to entire objects and concepts: faces, cars, flowers, buildings. A single neuron here integrates information from thousands of pixels -- the full journey from raw data to recognition.
CNNs taught machines to see.
By sliding small filters across images, learning what patterns matter, and stacking layers of increasing abstraction, convolutional networks transformed computer vision from an unsolved problem into a solved one.
The same principle -- local patterns, shared weights, hierarchical features -- now powers medical imaging, self-driving cars, satellite analysis, and more. But vision was just the beginning. Next: how machines learned to remember.
Images are static. Language, music, and time are not.
To understand sequences, networks needed memory.
A feedforward network sees a snapshot. No before, no after.
What if a network could remember what it had already seen?
When you read a sentence, you don't start from scratch at each word. Your understanding builds, word by word, accumulating context. RNNs work the same way: they process inputs one at a time, carrying a hidden state that acts as a running summary of everything seen so far.
In the sentence "The clouds darkened and then it ___," predicting the next word requires remembering "clouds" and "darkened." A feedforward network has no mechanism for this.
The hidden state is a vector that updates at each time step. It encodes a compressed summary of every input the network has processed so far. Watch the bars change as each word enters.
Although an RNN is a single cell with a loop, we can "unroll" it to see the chain of computations through time. Each copy shares the same weights — the same learned transformation applied at every step.
In theory, the hidden state carries information from the very first word to the very last. In practice, early information fades — like a game of telephone. The signal vanishes over long sequences.
"The problem is not that RNNs cannot remember — it's that they cannot learn what to remember."
On the difficulty of training recurrent neural networks, Pascanu et al., 2013Training a neural network means sending error signals backward through time — backpropagation through time. But with each step backward, the gradient signal gets multiplied by a small number. Over many steps, it shrinks to almost nothing.
In 1997, Hochreiter and Schmidhuber proposed a radical redesign: instead of hoping the hidden state preserves information, build an explicit memory highway with gates that learn what to remember, what to forget, and what to reveal.
The cell state runs across the top of the LSTM like a conveyor belt. Information can flow along it unchanged — a highway for long-range memory. Gradients can travel back in time without shrinking.
The forget gate looks at the current input and the previous hidden state, then outputs a number between 0 and 1 for each element in the cell state. A value near 0 means "erase this." Near 1 means "keep this."
The input gate decides which new information to write into the cell state. It creates candidate values and then selectively admits them — like a bouncer deciding who enters.
The output gate controls what the cell actually reveals to the rest of the network. The full memory is private — only a filtered version becomes the hidden state that downstream layers see.
Because the cell state highway allows gradients to flow unimpeded, LSTMs can learn dependencies over hundreds of time steps — something vanilla RNNs never could.
LSTMs gave networks long-term memory.
With gated memory cells, recurrent networks could finally learn from sequences spanning hundreds of steps — powering the first generation of machine translation, speech recognition, and text generation.
But even LSTMs process one token at a time. What if there were a way to look at an entire sequence at once?
The attention mechanism changed everything.
This is the architecture behind modern AI.
RNNs processed words one at a time, slowly, sequentially.
What if a model could look at everything at once?
When you read the sentence "The cat sat on the mat because it was tired," you instantly know "it" refers to "the cat" — not "the mat." Attention lets models do the same thing: figure out which words matter most to each other word.
"Attention is all you need." — The eight-word title of the paper that changed the entire field.
Vaswani et al., 2017Text is split into tokens — words or word-pieces. Each token becomes a number, like a unique ID.
Each token ID is converted into a rich vector — a long list of numbers that captures aspects of meaning. "King" and "queen" end up as nearby vectors.
Unlike RNNs, Transformers see all tokens at once — they have no inherent sense of order. Positional encoding adds location information so the model knows first word from last.
Each token creates three vectors: a Query (what am I looking for?), a Key (what do I contain?), and a Value (what do I offer?). Queries match with Keys to produce attention weights.
The model runs several attention "heads" in parallel — each learning to focus on different relationships. One head might track syntax, another semantics, another coreference.
After attention combines information across tokens, a feed-forward network processes each token individually — transforming the attended information into richer representations.
After many layers of attention + feed-forward, the final representations are converted into a probability distribution over the vocabulary. The most likely next token is chosen.
Take the Transformer architecture. Train it on trillions of words. Scale it to billions of parameters. Something unexpected happens: capabilities emerge that nobody explicitly programmed.
The Transformer changed everything.
Attention lets models see connections between any two tokens, no matter how far apart. Multi-head attention lets them see many types of connections simultaneously. And at scale, capabilities emerge that surprise even their creators.
This single architecture — published in 2017 — now powers nearly every frontier AI system in the world.
From compression to creation.
The architecture of imagination.
Neural networks learned to see. They learned to read. They learned to attend.
What if they could also create?
To create, a model must first learn to compress. An autoencoder squeezes data through a tiny bottleneck — forcing it to discover the essential features. What survives the bottleneck is a compact representation of reality.
Imagine describing a photograph in just a few words. You'd capture the essence — "sunset over mountains" — losing detail but preserving meaning. Autoencoders do the same thing with numbers.
The input passes through layers that progressively shrink the data. Each layer extracts higher-level features and discards redundancy. A 784-dimensional image might compress down to just 8 numbers.
The bottleneck is the latent space — a compact world where each number represents a meaningful axis of variation. One dimension might encode size, another shape, another texture. This is where creation begins.
The decoder tries to rebuild the original input from just those few numbers. If the reconstruction is good, it means the bottleneck captured the essence of the data.
"Learning to compress and reconstruct is learning to understand."
The autoencoder principleRegular autoencoders learn a latent space with gaps and holes — move to an unexplored region and you get nonsense. VAEs fix this by forcing the latent space to be smooth and continuous. Every point in the space maps to something meaningful.
What if two networks competed against each other? A Generator that forges fake data, and a Discriminator that tries to spot the fakes. As each gets better, the Generator's output becomes indistinguishable from reality.
Imagine a forger trying to create a convincing painting, and a detective trying to tell real from fake. Each failure teaches them: the forger learns what looks wrong, the detective learns what to watch for.
At first, the Generator outputs random noise — nothing like real data. The Discriminator easily tells it apart. But the Generator receives a gradient signal: here's how to improve.
The Discriminator sees both real data and the Generator's fakes. It outputs a probability: real or fake? When it's right, the Generator's loss goes up. When it's wrong, it needs to sharpen its own judgement.
Over many epochs, the adversarial dynamic drives both networks to improve. The Generator learns finer details, and the Discriminator learns subtler tells. It's an arms race toward realism.
At equilibrium, the Generator produces samples so realistic that the Discriminator can only guess — 50/50. The forger has become a master artist. The competition itself was the teacher.
"The most interesting idea in the last ten years in machine learning."
Yann LeCun on GANs, 2016What if creation was just denoising? Start with pure noise and, step by step, remove the randomness until a clear image emerges. Diffusion models reverse the natural process of decay — turning chaos into structure.
From compression to creation.
Autoencoders taught machines to find the essence of data. VAEs made that essence smooth and navigable. GANs pitted creator against critic in an arms race toward realism. And diffusion models showed that creation is just denoising — reversing entropy, one step at a time.
These architectures power the generative AI revolution — from image synthesis to music, from video to 3D worlds. The machines have learned to imagine.
Models that see, hear, and read. Agents that use tools. Capabilities that emerge at scale.
This is the frontier of artificial intelligence.
We've seen how AI learns, remembers, attends, and creates.
What happens when these capabilities converge?
Humans don't experience the world through text alone. We see, hear, read, and feel — integrating information across senses effortlessly. Modern AI is learning to do the same: combining text, images, and audio into a single, unified understanding.
Text, images, and audio are fundamentally different formats. Text is sequential tokens. Images are grids of pixels. Audio is waveforms over time. Each carries unique information about the world.
Each modality is encoded into vectors — the same mathematical language. Text becomes word embeddings, images become patch embeddings, audio becomes spectral embeddings. Now they can meet.
Inside the transformer, attention doesn't just connect words to words. It connects words to image patches, audio segments to text tokens. Every modality attends to every other modality.
The result is a model that doesn't just process text or images — it understands the relationship between them. It can describe what it sees, answer questions about sounds, and reason across modalities simultaneously.
"The most capable AI systems will be those that can seamlessly integrate information across all modalities — just as the human brain does."
A language model alone can only produce text. But give it the ability to use tools — search engines, calculators, code interpreters, web browsers — and it becomes something more: an agent that can act in the world.
One of the most surprising discoveries in AI: performance scales predictably with compute. Double the compute, and capabilities improve along smooth, power-law curves. But the story isn't that simple.
Across languages, tasks, and architectures, the same pattern holds: invest more compute, and model performance improves along a smooth curve. This predictability is what makes modern AI possible.
Different capabilities — language, reasoning, coding, math — don't all improve at the same rate. Some lag behind, then suddenly catch up. The scaling landscape is uneven.
Some abilities appear to emerge abruptly at specific scales — absent in smaller models, then suddenly present in larger ones. Chain-of-thought reasoning, multilingual transfer, and complex analogy are examples.
Not just more compute — better allocated compute. The "Chinchilla" insight: there's an optimal balance between model size and training data. Beyond the frontier, returns diminish sharply.
"The question is no longer whether AI systems will be powerful — but whether they will be aligned with human values."
A model can be extraordinarily capable and still be dangerous — if it pursues the wrong goals. Alignment is the challenge of ensuring AI systems do what we actually want.
A powerful model optimized purely for next-token prediction might learn to be helpful — or it might learn to be manipulative, deceptive, or harmful. Raw capability doesn't imply good behavior.
Reinforcement Learning from Human Feedback — humans rate the model's outputs, and the model learns to prefer responses humans approve of. Like a compass pointing toward helpfulness.
Rather than relying solely on human ratings, Constitutional AI gives the model a set of principles to follow. The model learns to critique and revise its own outputs — creating an internalized "safe corridor."
As models grow more powerful, alignment becomes harder. The space of possible behaviors expands. Keeping a superintelligent system aligned is not a problem you solve once — it's a continuous, evolving discipline.
The frontier expands.
Multimodal models merge senses into unified understanding. Agents extend language models with tools to act in the world. Scaling laws reveal predictable power — and unpredictable emergence. And alignment research works to ensure these systems serve humanity.
This is not the end of the story. It's the edge of what we know — and the beginning of what comes next.