A new study from researchers at the Meridian Institute for AI Evaluation suggests that even the longest-context language models struggle to use information buried deep in very long prompts.
The team tested twelve leading models on a retrieval-and-reasoning benchmark spanning documents from 8,000 to 200,000 tokens. Accuracy held steady up to about 32,000 tokens, then declined steadily. Past 64,000 tokens, most models answered correctly barely half the time, with performance worst for facts located in the middle of the prompt.
The findings echo earlier “lost in the middle” research but extend it to today’s much longer context windows and to multi-step reasoning, not just simple retrieval. The authors argue that raw context length is a poor proxy for usable context, and propose a new metric, effective reasoning span, that they say better predicts real-world performance on long documents.
For developers, the practical takeaway is familiar: chunking, retrieval, and careful prompt design still matter, even with 128K and 200K context windows on the label.
The paper is available as a preprint, and the evaluation harness has been released as open source.
