Securing Autonomous AI Agents with ClawSentry
Listen to the summary
Uses a voice available on your device
Audio options
On this page 5 sections
Related concepts 3 concepts
Key Takeaways
- ClawSentry reduces contextual attack success rates on the SkillInject benchmark from 39.55% to 2.61%.
- Deployment across five work agents on the SkillsSafety benchmark lowered attack success rates from a range of 33.5 to 49.7% down to 9.09 to 15.03%.
- The framework utilizes an Agent Harness Protocol to apply consistent security policies across Codex, Claude Code, Kimi CLI, and Gemini CLI without requiring internal agent modifications.
- Aggregate task success rates on clean skills are maintained at 98.7% during security operations.
Summary & Methodology Analysis
ClawSentry addresses the security gaps in autonomous agent runtimes by implementing a progressive, multi-tier monitoring strategy. The architecture operates through a three-tier decision engine, moving from a deterministic policy layer to rule-anchored semantic analysis and read-only multi-turn monitoring. To prevent bypass attempts, such as rephrased retries or tool switching, the system maintains a session-level record of blocked effects. This methodology ensures that security boundaries are enforced across the lifecycle of agent actions rather than relying on a single, static inspection point. The framework integrates these controls through an Agent Harness Protocol, which creates a framework-agnostic abstraction layer. This allows the system to interface with various runtimes like Codex, Claude Code, Kimi CLI, and Gemini CLI without modifying their internal logic or architecture. Because the system is designed for modularity, it audits tool outputs asynchronously to manage post-action consequences, even if those actions remain outside the scope of real-time prevention. The benchmark results demonstrate that this approach maintains high performance, keeping the task success rate on clean skills at 98.7% while significantly limiting attack success rates. Evaluation metrics were generated using an unattended harness that follows a safety-first operating point, where any Defer decision is deterministically resolved as a Block, effectively establishing a performance lower bound for the monitor. The framework acknowledges specific technical boundaries in its current implementation, as it cannot retroactively enforce security at the post-action consequence stage or mitigate threats that do not cross defined lifecycle boundaries. Specifically, it remains unable to address direct user-to-model jailbreaks, base-model misalignment, or threats originating from a compromised host OS, model-API manipulation, or weight poisoning.
Interactive System Flowchart
Illustrative Implementation
A short sketch of the paper's core idea, not the authors' own code.
# Illustrative sketch (not from the paper)
import asyncio, collections
class ClawSentry:
def __init__(self): self.blocked=collections.defaultdict(set)
def fspr(self,pkg): return pkg.get("evidence",0)>=1
def l1(self,r): return "Block" if r.get("unsafe") else "Defer"
def l2(self,r): return "Block" if "malicious" in r.get("intent","") else "Defer"
def l3(self,r): return "Allow"
def decide(self,r):
for tier in (self.l1,self.l2,self.l3):
d=tier(r);
if d!="Defer": return d
return "Allow"
def anti_bypass(self,sid,eff):
if eff in self.blocked[sid]: return "Block"
self.blocked[sid].add(eff); return "Proceed"
async def post_audit(self,out): return "Flag" if "dangerous" in out else "OK"
async def process(sid,pkg,req,out):
cs=ClawSentry()
if not cs.fspr(pkg): return "Block(FSPR)"
if cs.decide(req)=="Block": return "Block(Engine)"
if cs.anti_bypass(sid,req.get("effect"))=="Block": return "Block(Bypass)"
audit=await cs.post_audit(out)
return f"Done,audit={audit}"
# asyncio.run(process(...)) # usage placeholder// Illustrative sketch (not from the paper)
const {Map,Set}=global;
class ClawSentry{
constructor(){this.blocked=new Map();}
fspr(p){return(p.evidence||0)>=1;}
l1(r){return r.unsafe?'Block':'Defer';}
l2(r){return (r.intent||'').includes('malicious')?'Block':'Defer';}
l3(){return'Allow';}
decide(r){for(const t of [this.l1,this.l2,this.l3]){const d=t.call(this,r);if(d!=='Defer')return d;}return'Allow';}
antiBypass(s,e){let set=this.blocked.get(s)||new Set();if(set.has(e))return'Block';set.add(e);this.blocked.set(s,set);return'Proceed';}
async postAudit(o){return o.includes('dangerous')?'Flag':'OK';}
}
async function process(s,pkg,r,out){
const cs=new ClawSentry();
if(!cs.fspr(pkg))return'Block(FSPR)';
if(cs.decide(r)==='Block')return'Block(Engine)';
if(cs.antiBypass(s,r.effect)==='Block')return'Block(Bypass)';
const a=await cs.postAudit(out);
return`Done,audit=${a}`;
}
// process(...).then(console.log);
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary purpose of ClawSentry?
ClawSentry is a security monitor designed to safeguard autonomous LLM agents by providing multi-tier protection against security risks throughout the agent execution lifecycle.
Q2. How does it affect the usability of agents?
It maintains high usability, as shown by an aggregate task success rate of 98.7% on clean skills while effectively reducing attack success rates.
Q3. Which specific agent runtimes does the system support?
The system supports Codex, Claude Code, Kimi CLI, and Gemini CLI via an abstraction protocol.
Q4. What is the impact of ClawSentry on the SkillInject benchmark?
On the SkillInject benchmark using Codex/GPT-5.4, the contextual attack success rate decreased from 39.55% to 2.61%.
Q5. How were the benchmark metrics calculated?
The evaluation used an unattended harness where all Defer decisions were treated as Block actions, providing a lower bound for security performance.
Q6. What types of threats are explicitly out of scope for this monitor?
Direct user-to-model jailbreaks, base-model misalignment, compromised host OS, model-API manipulation, and weight poisoning are out of scope.
Q7. How does the system handle post-action consequences?
It monitors post-action consequences through asynchronous auditing, though it cannot enforce these non-retroactively.
Q8. Did the authors need to modify the agent internals to implement this security?
No, the Agent Harness Protocol allows the application of security policies across different agents without modifying their internal components.
Q9. What is the range of improvement seen on the SkillsSafety benchmark?
Attack success rates were confined to 9.09 to 15.03% compared to the original range of 33.5 to 49.7%.