Two ways to be wrong

We already know that letting a model write its steps before answering helps — that was thinking longer at answer-time. And we just gave it a way to reach outside itself.

Take one question that needs both, and try each half alone.

Why each half fails

Thinking alone has no reality check. Every step comes out of the weights. When step three is a false memory, there is nothing in the process that could disagree with it — so the model reasons forward from the error, and the tidy chain of steps makes the wrong answer look more trustworthy, not less.

Acting alone has no aim. Every observation is real, so nothing is invented. But with no thought between calls, the query never improves and the results are never combined. It repeats itself and then guesses.

Line the two up and the fix writes itself. One has judgement and no facts; the other has facts and no judgement. So alternate them:

thought → action → observation → thought → …

Each thought decides the next action. Each observation constrains the next thought. This is ReAct (Yao et al., 2023), and it is the standard shape of an agent.

What the loop buys

Watch the fourth trace in the widget. Two moments matter, and neither is available to either half alone.

A thought heads off the Casino Royale 1967-versus-2006 ambiguity before it becomes a bad query — reasoning improving an action. Then an observation changes what the model goes looking for next — an action correcting the reasoning.

There is a third benefit that is easy to undervalue: you can read it. When the agent gets something wrong you can see whether the query was bad, the page was bad, or the inference from the page was bad. A model that emits only an answer gives you nothing to debug.

Go deeper: it is not uniformly better, and that is worth knowing

On HotpotQA, a multi-hop question benchmark, the exact-match scores run roughly: standard prompting 25.7, chain-of-thought 29.4, act-only 25.7, ReAct 27.4. ReAct alone lands below plain chain-of-thought. Combine them — ReAct plus self-consistency — and it jumps to 35.1.

Where interleaving wins outright is in environments where actions change the world rather than just retrieving from it. On ALFWorld, a text household simulator, ReAct reaches around 71% against roughly 45% for act-only; on the WebShop browsing benchmark, about 40% against 29%.

That difference makes sense. In pure lookup, a model’s memory is already a decent first draft. Once actions have consequences and the state moves, having a plan stops being optional.

Which raises the question the loop has quietly been assuming: what is a tool, from where the model is sitting?

Sources · 3

Full bibliography →