Volume V · Thinking Before Acting
A Chain, Then a Tree
The last ten lectures were about the machine room — chips, interconnects, parallelism, serving, the honest gap between advertised and realized compute. None of it asked what the model sitting on top of all that silicon actually does with a prompt. This lecture is where the series leaves the machine room and walks into the reasoning loop running on top of it: not new hardware, not a new architecture, just a different way of asking the same model a question, that turns out to change what it can do.
The mental model
Chain-of-thought commits to one reasoning path, token by token, with no way to backtrack once it's started down that path. Tree of Thoughts spends extra compute to buy something CoT structurally cannot offer: the ability to explore several paths and abandon the ones that go bad. Commit versus explore is the tradeoff, and it recurs through every later lecture on agent design.
Chain-of-thought: reasoning as a prompt, not a mechanism
Chain-of-thought (CoT) prompting, introduced by Wei et al. (arXiv:2201.11903, NeurIPS 2022), makes no change to model architecture, training procedure, or weights at all. It is a change to the few-shot exemplars in the prompt: instead of showing the model examples that go straight from input to output, the exemplars show intermediate natural-language reasoning steps between the question and the final answer. The model, conditioned on that pattern, produces its own intermediate reasoning steps before answering — and the paper reports substantial gains on arithmetic, commonsense, and symbolic reasoning benchmarks from this change alone.
One qualifier the paper itself is explicit about: these gains are reported to emerge mainly at sufficient model scale — CoT prompting a small model does not reliably produce the same benefit. The mechanism, whatever it is, appears to depend on capacity that only larger models have.
Tree of Thoughts: buying the ability to backtrack
Tree of Thoughts (ToT), Yao et al. (arXiv:2305.10601, NeurIPS 2023), states its own motivation directly: token-level, committed left-to-right decoding is insufficient for tasks that need exploration or strategic lookahead — exactly the structural limit Figure 11a shows. ToT generalizes CoT's single chain into a tree of "thoughts," where a thought is a coherent intermediate reasoning unit — larger-grained than a token, smaller than a full solution. The language model plays two roles in this process: it generates candidate next thoughts, and it self-evaluates them, and those evaluations feed a search algorithm — breadth-first or depth-first search — that can look ahead along a branch and backtrack out of one that isn't working.
Commit versus explore — a tradeoff this volume keeps returning to
CoT is cheap: one generation pass, one path, no self-evaluation step, no search algorithm to run. ToT is expensive by comparison — it requires generating and evaluating multiple candidate thoughts at each level of the tree, and running an explicit search procedure on top of the language model calls. What ToT buys with that extra cost is the one thing CoT structurally cannot offer: the ability to notice a reasoning path is bad and abandon it before finishing it, rather than discovering only at the end that an early step was wrong.
This tradeoff has a name, and it recurs
"Commit to one path cheaply" versus "explore several paths and prune the bad ones at a compute cost" is not unique to reasoning prompts. Volume VII's look at multi-agent systems and Volume VIII's benchmarks both revisit some version of this same tradeoff — a cheaper, more committed strategy against a more expensive one that can recover from its own mistakes. Treat CoT-versus-ToT as the clearest, smallest instance of a pattern that shows up again later in this series.