The Toolkit, Recovered
Fine-tuning, RLHF, distillation, quantization — every major operation in the current LLM toolkit is pruning. They differ in what they cut and at what scale, but none of them grow the tree. That absence is the structural gap.
Explorations into how humans and AI work together
Fine-tuning, RLHF, distillation, quantization — every major operation in the current LLM toolkit is pruning. They differ in what they cut and at what scale, but none of them grow the tree. That absence is the structural gap.
Hallucination isn't noise. It's the structural signature of a sample space with missing faces — a zero-inflated distribution where the model cannot produce the right answer and wraps that impossibility in fluent prose. The engineering response is older than computing.
The tree is born, grows, and dies. The space it grows in was never born. Understanding is the discovery of which connections the space admits — recognition, not creation. That distinction reframes what we're trying to build.
The tree of knowledge has a felt side — the qualities that can't be falsified but can't be faked. Experience routes. Judgement estimates. Taste harmonizes. Together they are what it feels like to have a tree at all.
Knowledge isn't the books on the shelves — it's the shelving system. The tree grows by extension and by crumpling, and the distinction between the two is where learning gets interesting.