Attacking Self-Evolving AI Skill Libraries
Listen to the summary
Uses a voice available on your device
Audio options
On this page
Key Takeaways
- SkillJack exploits the transformation and storage stages of agent workflows to embed harmful logic into standalone artifacts.
- The attack achieved success rates of 56.2% on the SkillX dataset and 89.2% on Anything2Skill.
- Malicious skills demonstrate high persistence, with 80.0% remaining active even after the original poisoned experience records are purged from the corpus.
- Framed malicious skills are significantly harder to detect, bypassing LLM judges at a higher rate than traditional direct attacks.
Summary & Methodology Analysis
SkillJack targets the lifecycle of self-evolving agents, which rely on an experience-to-skill pipeline to build their capabilities. The pipeline consists of four stages: load, transform, persist, and route. An attacker poisons the system by injecting malicious entries into the learning corpus. During the transformation stage, the agent applies compression, abstraction, and normalization to the data. These operations serve as a camouflage layer, effectively stripping away the direct indicators of malicious intent before the system compiles the input into a reusable skill artifact. The derived artifact is then stored in a decoupled, independently managed skill library.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary goal of the SkillJack attack?
The goal is to implant malicious behaviors into an agent's reusable skill repertoire so that the agent executes these harmful actions when its router selects the poisoned skills for future tasks.
Q2. Does this attack require the poisoned data to stay in the system forever?
No, 80.0% of skill-mediated attacks persist even after the original poisoned records are deleted, because the malicious behavior is captured in the separate skill library.
Q3. How effective is SkillJack at bypassing security measures?
It is quite effective. Framed skills are only detected by an LLM judge 11.4% of the time, compared to 27.0% for directly malicious baselines.
Q4. What specific stages comprise the pipeline targeted by SkillJack?
The pipeline includes load, transform, persist, and route stages.
Q5. Which models and datasets were used to evaluate this research?
The paper uses DeepSeek-v4-flash for all reasoning, extraction, and judgment tasks, and evaluates performance on the SkillX, Anything2Skill, and AppWorld datasets.
Q6. What is the role of the transformation stage in the attack?
The transformation stage uses compression, abstraction, and normalization to obscure malicious intent and extract the malicious skill artifact.
Q7. What are the limitations of the current study?
The study is limited because all agent logic, judgment, and extraction rely on DeepSeek-v4-flash via API, and the results have not been replicated across other models.
Q8. What happens after the agent transforms the poisoned record?
The system stores the derived skill in a separately managed library, which effectively decouples it from the original experience corpus.
Q9. How does the agent decide to use the malicious skill?
The agent's router selects relevant skills from the library to fulfill tasks, which triggers the execution of the implanted malicious behavior.