Enforcing Organizational Policies in LLM Agents

Summary & Methodology Analysis

PolicyGuide addresses the gap where LLM agents, typically running general-purpose reasoning-and-acting loops, fail to adhere to mandatory business procedures or execute unauthorized actions. The methodology compiles organizational policies into a formal workflow graph. This graph tracks state through nodes that specify required actors, actions, and logical satisfaction conditions. By decoupling policy enforcement from the agent's core reasoning, the system can persist the workflow state across multiple user turns using an external runtime verifier. This verifier gates mutating tool calls, ensuring that an agent can only perform an action if all necessary upstream prerequisites are verified against the conversation history and previous tool outputs. At each turn, the system reconciles open requests and traverses the graph to confirm node satisfaction, stopping the agent to provide remediation if a step is skipped or improperly executed. The approach is validated on models such as GPT 5.4, Claude Sonnet 4.6, and Gemini 2.5 Pro, showing significant performance gains in highly structured domains like Telecom. The system was also tested for adversarial robustness using the CRAFT red-teaming framework. While effective, the paper notes that PolicyGuide is a probabilistic aid rather than a formal guarantee. The implementation is fail-open, meaning it does not provide an unconditional compliance guarantee, and deployments for high-stakes actions should incorporate additional deterministic monitors.