Memory Is an Unsolved Research Problem That Model Scaling Cannot Fix

There is a widespread assumption in AI that better memory will arrive as a byproduct of model progress. Bigger context windows, better attention, more parameters. If a model can hold a million tokens, why bother with retrieval at all? Just put everything in.

The evidence increasingly shows this assumption is wrong. Not slightly wrong. Directionally wrong. Scaling context makes memory worse, not better.

Every model gets worse with more context

The research is converging. RULER found that effective context sits at roughly 50-65% of marketed capacity (Hsieh et al., 2024). Lost in the Middle showed over 30% performance degradation when relevant information is positioned in the middle of the input (Liu et al., 2024). NoLiMa found that when literal keyword overlap is removed, most models drop below 50% of their short-context baselines at just 32K tokens (Modarressi et al., 2025). Chroma Research tested 18 production LLMs and found monotonically decreasing performance in every single one as input length grew, coining the term "context rot" (Hong et al., 2025). A separate study found catastrophic collapse at as low as 40% of maximum context length (arXiv:2601.15300).

18 out of 18 models. Monotonically decreasing. Not some models under certain conditions. All of them, always.

Adding tokens to a context window does not just fail to help beyond a threshold. The additional tokens actively degrade performance. The model attends to more noise, loses coherence in the middle of the input, and produces worse answers than it would with less.

Context windows are containers. They are not memory.

Memory is a distinct research discipline

The frontier labs are focused on reasoning benchmarks, model scale, and general capability. Memory sits at the periphery of their research agenda, something to be handled by making context windows bigger. But if larger context actively degrades recall, then the dominant approach to AI memory is architecturally self-defeating.

Memory requires its own architectures, its own evaluation frameworks, and its own research investment. It will not emerge as a side effect of making the next model bigger.

Consider what human memory does. It does not store everything and search through it. It reconstructs (Bartlett, 1932). It encodes selectively, prioritising what matters (Tulving, 1972). It consolidates during sleep, replaying and reorganising experiences into coherent knowledge (Walker & Stickgold, 2004). It abstracts, forming general concepts from specific instances. It forgets strategically, letting unimportant details decay so that important ones are easier to find.

None of this has a meaningful analogue in current AI systems. Context windows approximate storage. Retrieval-augmented generation approximates search. But the active processes (consolidation, abstraction, reorganisation, anticipation, reflection) are entirely absent.

A system that retrieves 5-10K precisely targeted tokens will outperform one that stuffs 200K of loosely relevant context, because the model has less noise to contend with. But even precise retrieval has limits. Some questions can only be answered by knowledge that has been pre-consolidated from fragments scattered across hundreds of conversations. No retrieval query, however well-constructed, will find what was never linked together in the first place.

What solving memory requires

We believe memory is the core of cognition. Not a storage problem, but the foundation of how intelligent systems learn, reason, and adapt over time.

Our research programme at Hyperplane Labs is organised around six cognitive capabilities that current AI systems lack.

Perception. Comprehension of new inputs. What it means, what matters, how it connects to what is already known.

Consolidation. Periodic reprocessing to extract patterns, merge redundant memories, strengthen important ones, and let unimportant details decay.

Abstraction. Moving from specific instances to general knowledge. Concept formation over retrieval.

Reorganisation. Restructuring existing knowledge when new information arrives. Rethinking and learning based on new inputs.

Anticipation. Surfacing information before it is asked for, based on patterns in how the system is being used.

Reflection. Self-directed review of what the system knows, identifying gaps, contradictions, and decay.

Each of these draws on established cognitive science and addresses specific, measurable failure modes observed in current AI memory systems. Together they form a research agenda that goes beyond retrieval, toward systems that actively maintain, improve, and reason over their own knowledge.

Memory is not solved. Model scaling will not solve it. It requires dedicated research.

References

Bartlett, F.C. (1932). Remembering: A Study in Experimental and Social Psychology. Cambridge University Press.

Hong, K., Troynikov, A., and Huber, J. (2025). "Context Rot: How Increasing Input Tokens Impacts LLM Performance." Technical Report, Chroma.

Howard, M.W. & Kahana, M.J. (2002). "A distributed representation of temporal context." Journal of Mathematical Psychology, 46(3), 269-299.

Hsieh, C-P. et al. (2024). "RULER: What's the Real Context Size of Your Long-Context Language Models?" COLM.

Liu, N.F. et al. (2024). "Lost in the Middle: How Language Models Use Long Contexts." TACL.

Modarressi, A. et al. (2025). "NoLiMa: Long-Context Evaluation Beyond Literal Matching." ICML.

Stickgold, R. and Walker, M.P. (2013). "Sleep-dependent memory triage." Nature Neuroscience, 16(2), 139-145.

Tulving, E. (1972). "Episodic and semantic memory." In Organisation of Memory, Academic Press.

Walker, M.P. and Stickgold, R. (2004). "Sleep-dependent learning and memory consolidation." Neuron, 44(1), 121-133.

arXiv:2601.15300 (2026). "Intelligence Degradation in Long-Context LLMs."