Uses a neural network to invent 1,024 new human-readable symbols from scratch — arguing that notation itself is part of intelligence.
Notational Intelligence, Linus Lee | Compile 26 · Linus Lee
17 min total·Actually worth watching closely: ~6 min·3 must-watch clips
- 0:00 – 3:20Listen
Opening claim: intelligence lives in the joint system of people and symbols
Introduces notational intelligence: long addition isn't pure mental arithmetic and the chalkboard isn't a calculator — the computing happens in the combination of the brain and the shapes you write down. And instances of notation far outnumber computers, so their impact on augmenting human intelligence may be greater.
Intelligence isn't only in the brain or the machine — it's in the combined system of person plus written symbols.
A purely verbal argument; the long-addition example is easy to picture, no need to watch the screen.▶ Jump to 0:00Speaker · Linus Lee - 3:23 – 6:20Skim
Three properties of good notation, and the calculus notation duel
Lays out three properties of good notation — abstraction, suggestiveness, natural transformation — and validates each with the history of Leibniz's dy/dx beating Newton's dot notation.
Leibniz won because you can operate mechanically on the symbols, the way you manipulate a ratio, and it still means something mathematically.
The side-by-side of the two notations on the slide is worth a glance (around 256s); the rest is fine to just listen to.▶ Jump to 3:23Speaker · Linus Lee - 6:20 – 8:40Skim
The power of flat: 2D notation and the coordinate plane
Argues that flatness is notation's key advantage — you can move it around, copy it and print it; the coordinate plane turns an equation into a visible line, putting physical spatial intuition to work on relationships between numbers.
An operation in graphical space (sliding the line up and down) corresponds directly to a meaningful operation on the ideas (moving the y intercept).
The coordinate-plane sketch at 447s is a standard slide diagram — one glance is enough to build the intuition.▶ Jump to 6:20Speaker · Linus Lee - 8:40 – 10:25Listen
Notations are invented: the arrow is only about 300 years old
Uses the arrow's appearance in 1737 (before which pointing out a direction meant drawing a hand) and the birth of programming languages to show that notation systems are invented cognitive tools — and so can go on being invented.
Even something as self-evident as the arrow is a recent invention — the space of new notations is nowhere near exhausted.
Mostly spoken history; nothing critical on screen.▶ Jump to 8:40Speaker · Linus Lee - 10:25 – 12:50Listen
Experiment setup: inventing from scratch, unconstrained by human notation
Sets out the goal and premises: unlike prior work that simulates the evolution from pictograms to written language, here the same deep learning tools are used to imagine totally new notations built from scratch; inventing one takes an input domain (what to represent) and the constraints of the medium (how it can be shown).
Inventing a new notation = input domain + the constraints inherent to the medium; you need both.
Conceptual setup for the experiment, mostly spoken — fine to treat as a podcast.▶ Jump to 10:25Speaker · Linus Lee - 12:50 – 14:55Watch
The trick revealed: an autoencoder whose middle layer is an image
A one-hot vector goes through a ResNet generator to produce a 32x32 image, and a decoder tries to read back what the generator wrote down; end-to-end gradient descent makes the generator and decoder learn to talk to each other in images.
Define the symbol as the image bottleneck of an autoencoder, and the notation system becomes a trainable object.
The architecture diagram from 771s is the key to the whole experiment — much faster to see than to hear.▶ Jump to 12:50Speaker · Linus Lee - 14:55 – 17:11Watch
Results and the fix: 1,024 symbols and human visual constraints
The model learns 1,024 distinguishable symbols, evolving from noise, simple to complex; it also reveals that naive training yields the degenerate "turn on the nth pixel" solution, so scale, rotation and brightness invariance have to be encoded as constraints of the medium.
For machine-invented symbols to mean anything to people, the invariances of human vision have to be written into the training constraints.
The symbol alphabet and the training evolution from 901s are the visuals most worth seeing with your own eyes.▶ Jump to 14:55Speaker · Linus Lee