Shreshth Rajan, June 2026.
A search agent runs queries, reads results, and calls tools to decide what to do next. We build one by taking a text model and teaching it, afterward, to act.
Two things keep breaking. Agents get worse after about five turns, even when every fact they need is still on screen. And sampling more candidate answers stops helping: the right one shows up more often and the agent still misses it.
Acting on partial information needs a running summary of what's true right now, and a way to score a candidate that isn't just your own confidence. A text model has neither. It rereads the transcript, and it ranks by its own likelihood.
The summary. A running summary is a short note you update once per observation. A transcript agent rereads everything and works out the current state from scratch, every turn, from a history that keeps filling with facts that are no longer true.
So I built a task that tests nothing else. Four people have locations, updates arrive in order, and some overwrite earlier ones. The question asks where one person is now, and their last update sits early so the distractors come after it. Reading the newest fact gives the wrong answer.
# Same model, same facts, same information. Only the representation differs.
def answer_transcript(model, facts, query): # work the state out every turn
ctx = "\n".join(facts) # context grows with every update
return model(f"Facts:\n{ctx}\nQ: {query}")
def answer_note(model, facts, query): # carry a short note instead
note = init_note() # ~20 tokens, fixed size
for fact in facts:
note = model(f"Note:\n{note}\nUpdate: {fact}\nRewrite the full note.")
return model(f"Note:\n{note}\nQ: {query}") # never sees the transcriptRereading the transcript gets 43% right after 8 updates and 12% after 128. Keeping a twenty-token note and rewriting it each time gets 95 to 100% at every length, on 21 tokens of context instead of 1073, with the same model in both cases.
The facts are in front of it either way, so what the transcript version loses is the rereading, not the information. A bigger model would degrade later, not differently. And my note is just a prompt, not something the model learned, so a trained version should do better. MEM1 trains one and reports exactly that.
The score. Picking the best of N candidates needs a score. Training a separate one is how OpenAI first beat fine-tuning on grade-school math.
A text model doesn't come with one. Its only free score is its own likelihood, and ranking by likelihood just hands back its first choice. But sampling only helps because the first choice is often wrong, so the free scorer is the worst one available: it makes the same mistakes as the thing it's checking. Process reward models and generative verifiers are better, but they're fine-tuned from the same backbone on the model's own answers, so the mistakes stay correlated.
I took one scorer and slid it toward the model's own confidence, changing nothing else. Pure scorer: best-of-N on GSM8K rises to 70% as N grows. Pure confidence: it flattens, then drops to 34%, worse than taking one sample. The crossover is near 0.6 correlation between the two sets of mistakes, and past that more search makes you worse. A perfect scorer gets 97%.
I built that correlation by hand, so it shows the mechanism, not that real scorers sit at 0.6. It's the actor-critic problem under another name, and Phasic Policy Gradient keeps policy and value apart for the same reason.
Calling these one problem is my argument, not my measurement. A text model learns to predict text. It doesn't learn to track state and it doesn't learn to judge. Post-training asks it to do both with the only materials it has, and both break down over long runs. Adding memory or a verifier afterwards inherits the problem, because neither was ever learned.
I'd train both alongside generation: a recurrent note that has to reproduce the current state from a fixed-size carry, and a scorer trained on wrong answers drawn from outside the model's own samples, so its mistakes differ. I haven't done it, and that's the experiment that would settle this.
Setup. Summary study: Qwen2.5-1.5B-Instruct on a synthetic four-entity state-tracking task, 8 to 128 updates, transcript-conditioned answering against an incrementally rewritten note, accuracy and context measured at answer time. Score study: best-of-N on GSM8K with a fixed verifier whose score is blended with the generator's normalized log-likelihood as \(V_\rho = (1-\rho)\tilde v + \rho\tilde s\), so quality is constant and only correlation varies. Code and data: github.com/ShreshthRajan.