From raw interaction to reusable knowledge: Rethinking memory for AI agents - Microsoft Research
Overview
Resources Resources
Publications
Code & data
People
Microsoft Research blog
Research areas: Intelligence Research areas: Intelligence
Artificial intelligence
Audio & acoustics
Computer vision
Graphics & multimedia
Human-computer interaction
Human language technologies
Search & information retrieval
Details
Research areas: Systems Research areas: Systems
Data platforms and analytics
Hardware & devices
Programming languages & software engineering
Quantum computing
Security, privacy & cryptography
Systems & networking
Research areas: Theory Research areas: Theory
Algorithms
Mathematics
Research areas: Other Sciences Research areas: Other Sciences
Ecology & environment
Economics
Medical, health & genomics
Social sciences
Technology for emerging markets
Resources Resources
Publications
Code & data
People
Microsoft Research blog
Research areas: Intelligence Research areas: Intelligence
Artificial intelligence
Audio & acoustics
Computer vision
Graphics & multimedia
Human-computer interaction
Human language technologies
Search & information retrieval
Research areas: Systems Research areas: Systems
Data platforms and analytics
Hardware & devices
Programming languages & software engineering
Quantum computing
Security, privacy & cryptography
Systems & networking
Research areas: Theory Research areas: Theory
Algorithms
Mathematics
Research areas: Other Sciences Research areas: Other Sciences
Ecology & environment
Economics
Medical, health & genomics
Social sciences
Technology for emerging markets
Academic programs
Events & academic conferences
Microsoft Research Forum
Behind the Tech podcast
Microsoft Research blog
Microsoft Research Forum
Microsoft Research podcast
Microsoft Research Labs Microsoft Research Labs
Africa
AI for Science
AI Frontiers
Asia-Pacific
Cambridge
Health Futures
India
Montreal
New England
New York City
Redmond
Other labs Other labs
Applied Sciences
Mixed Reality & AI - Cambridge
Mixed Reality & AI - Zurich
All Microsoft
Global
Microsoft Security
Azure
Dynamics 365
Microsoft 365
Microsoft Teams
Windows 365
Tech & innovation Tech & innovation
Microsoft AI
Azure Space
Mixed reality
Microsoft Holo Lens
Microsoft Viva
Quantum computing
Sustainability
Industries Industries
Education
Automotive
Financial services
Government
Healthcare
Manufacturing
Retail
Partners Partners
Find a partner
Become a partner
Partner Network
Microsoft Marketplace
Marketplace Rewards
Software development companies
Resources Resources
Blog
Microsoft Advertising
Developer Center
Documentation
Events
Licensing
Microsoft Learn
Microsoft Research
Global
Microsoft Security
Azure
Dynamics 365
Microsoft 365
Microsoft Teams
Windows 365
Tech & innovation Tech & innovation
Microsoft AI
Azure Space
Mixed reality
Microsoft Holo Lens
Microsoft Viva
Quantum computing
Sustainability
Industries Industries
Education
Automotive
Financial services
Government
Healthcare
Manufacturing
Retail
Partners Partners
Find a partner
Become a partner
Partner Network
Microsoft Marketplace
Marketplace Rewards
Software development companies
Resources Resources
Blog
Microsoft Advertising
Developer Center
Documentation
Events
Licensing
Microsoft Learn
Microsoft Research
From raw interaction to reusable knowledge: Rethinking memory for AI agents
By
Ke Yang
,
Research Intern
Michel Galley
,
Senior Principal Research Manager
Chenglong Wang
,
Senior Researcher
Jianfeng Gao
,
Technical Fellow & Corporate Vice President
Jiawei Han
,
Chair Professor, University of Illinois Siebel School of Computing and Data Science
Cheng Xiang Zhai
,
Professor, University of Illinois Department of Computer Science
Today’s AI agents store long interaction histories but struggle to reuse them effectively.
Raw memory retrieval can overwhelm agents with lengthy, low-value context.
Plug Mem transforms interaction history into structured, reusable knowledge.
A single, general-purpose memory module improves performance across diverse agent benchmarks while using fewer memory tokens.
It seems counterintuitive: giving AI agents more memory can make them less effective. As interaction logs accumulate, they grow large, fill with irrelevant content, and become increasingly difficult to use.
More memory means that agents must search through larger volumes of past interactions to find information relevant to the current task. Without structure, these records mix useful experiences with irrelevant details, making retrieval slower and less reliable. The challenge is not storing more experiences, but organizing them so that agents can quickly identify what matters in the moment.
In our recent paper “Plug Mem: A Task-Agnostic Plugin Memory Module for LLM Agents,” we introduce a plug-and-play memory system that transforms raw agent interactions into reusable knowledge. Rather than treating memory as text to retrieve, Plug Mem organizes that history into structured knowledge designed to support decisions as the agent acts.
Stay connected to the research community at Microsoft.
Cognitive science offers a useful framework here. It distinguishes between remembering events, knowing facts, and knowing how to perform tasks. Past events provide context, but effective decisions rely on the facts and skills extracted from those events.
This distinction motivated a shift in how we decided to design memory for AI agents. Plug Mem implements this shift by converting the agent’s interaction history, such as dialogues, documents, and web sessions, into structured, compact knowledge units that can be reused across tasks.
A key difference between Plug Mem and conventional AI memory systems is what gets stored. Traditional approaches store text chunks or named entities (references to people, places, and concepts). Plug Mem uses facts and reusable skills as the fundamental building blocks of memory. This design reduces redundancy, increases information density, and improves retrieval precision. It’s built around three core components:
Structure. Raw interactions are standardized and transformed into propositional knowledge (facts) and prescriptive knowledge (reusable skills). These knowledge units are organized into a structured memory graph, enabling knowledge to be stored in a form designed for reuse.
Retrieval. Rather than retrieving long passages of text, Plug Mem retrieves knowledge units that are aligned with the current task. High-level concepts and inferred intents serve as routing signals, surfacing the most relevant information for the decision at hand.
Reasoning. Retrieved knowledge is distilled into concise, task-ready guidance before being passed to the base agent, ensuring that only decision-relevant knowledge enters the agent’s context window.
Figure 1 illustrates how these components work together.
Figure 1. Plug Mem organizes different types of agent interactions into a knowledge-centric memory graph, enabling structured retrieval and reasoning.
Most AI memory systems are built for one job. A conversational memory module is designed around dialogue. A knowledge-retrieval system is tuned to look up facts. A web agent’s memory is optimized for navigating pages. Each performs well in its target setting but rarely transfers without significant redesign.
Plug Mem takes a different approach. It is a foundational memory layer that can be attached to any AI agent without needing to modify it for a specific task.
To test Plug Mem, we evaluated the same memory module on three benchmarks that each make different demands on memory:
Answering questions across long multi-turn conversations
Finding facts that span multiple Wikipedia articles
Across all three, Plug Mem consistently outperformed both generic retrieval methods and task-specific memory designs while allowing the AI agent to use significantly less memory token budget in the process.
We wanted to evaluate whether the right information was reaching the agent at the right moment, without overwhelming the model’s context window, which has limited capacity. To do this, we introduced a metric that measures how much useful, decision-relevant information a memory module contributes relative to how much context it consumes.
When we plotted utility against context consumption, Plug Mem consistently came out ahead: it delivered more decision-relevant information while consuming less of the AI agent’s context than other approaches, as shown in Figure 2. These results suggest that transforming experience into knowledge—rather than storing and retrieving raw logs—produces memory that is more useful and efficient.
Figure 2. Across all three benchmarks, Plug Mem delivered more useful memory with less of the agent’s context window.
Why general-purpose memory can outperform task-specific designs
General-purpose memory modules can outperform systems tailored to specific tasks because the decisive factor is not specialization but whether memory can surface the right knowledge precisely when the agent needs it. Structure, retrieval, and reasoning each play a distinct role, and getting all three right matters more than optimizing for a single use case.
Plug Mem is not meant to replace task-specific approaches. It provides a general memory foundation upon which task adaptations can be layered. Our experiments show that combining Plug Mem with task-specific techniques yields further gains.
As AI agents take on longer and more complex tasks, its memory needs to evolve from storing past interactions to actively supplying reusable knowledge. The goal is for agents to carry useful facts and strategies from one task to the next rather than starting from scratch each time.
Code and experimental results are publicly available on Git Hub (opens in new tab) so that others can reproduce the results and conduct their own research.
Plug Mem: A Task-Agnostic Plugin Memory Module for LLM Agents
Chair Professor, University of Illinois Siebel School of Computing and Data Science
Professor, University of Illinois Department of Computer Science
Agent Lightning: Adding reinforcement learning to AI agents without code rewrites
Introducing KBLa M: Bringing plug-and-play external knowledge to LLMs
Key Takeaways
-
Resources Resources
Publications Code & data People Microsoft Research blog -
Research areas: Intelligence Research areas: Intelligence
Artificial intelligence Audio & acoustics Computer vision Graphics & multimedia Human-computer interaction Human language technologies Search & information retrieval -
Research areas: Systems Research areas: Systems
Data platforms and analytics Hardware & devices Programming languages & software engineering Quantum computing Security, privacy & cryptography Systems & networking -
Research areas: Theory Research areas: Theory
Algorithms Mathematics -
Research areas: Other Sciences Research areas: Other Sciences
Ecology & environment Economics Medical, health & genomics Social sciences Technology for emerging markets
