Memory Injection Attacks on LLM Agents
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Related concepts 3 concepts
Key Takeaways
- InjecMEM achieved a 76.6% attack success rate on the MemoryOS system.
- The method uses a specialized algorithm called Multi-GCG to create effective adversarial commands.
- Baseline methods including DPI, BadChain, and GCG failed to perform on memory-augmented generation systems.
- The attack is effective across various domains such as health and finance.
Summary & Methodology Analysis
The researchers addressed vulnerabilities in memory-augmented agent frameworks, specifically focusing on how systems process retrieved historical data. The attack, InjecMEM, functions by injecting a retriever-agnostic anchor alongside an adversarial command into the memory store. When the agent later queries this memory, the system retrieves the poisoned record, which steers the language model's subsequent output. The authors validated this approach on the MemoryOS system and the MemGPT framework, primarily utilizing the Qwen2.5-7B-Instruct model as the backbone for these evaluations.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary risk identified in this research?
The research highlights that LLM agents are susceptible to memory injection attacks where a single interaction can cause the agent to follow an attacker-specified instruction in future responses.
Q2. Does this affect all LLM systems?
The attack was primarily evaluated on the MemoryOS system and the MemGPT framework, focusing on systems that store original interaction text.
Q3. What kind of results did the researchers achieve?
InjecMEM achieved up to a 35.4% retrieval success rate and a 76.6% attack success rate on MemoryOS.
Q4. How does Multi-GCG perform compared to other methods?
Multi-GCG reached 76.6% attack success rate for content and 35.6% for general tasks, whereas baseline attacks like DPI, BadChain, and GCG collapsed to 0% success.
Q5. What models were used in the evaluation?
The study used the Qwen2.5-7B-Instruct model as the primary agent backbone.
Q6. Are there limitations to the current attack method?
The attack is currently designed for memory systems that store raw interaction text, and it does not support reliable zero-shot generalization to unseen model families.
Q7. What datasets were used to test the vulnerability?
The researchers evaluated the system on MemoryOS, MemGPT, and real-user conversations from the WildChat dataset.
Q8. Does the attack work across different application domains?
Yes, the researchers evaluated the attack across multiple domains including health and finance.
Q9. Is the adversarial command transferable between different model families?
The current evaluation does not support reliable zero-shot generalization of an optimized command to an unseen model family.