Dynamic Decision Control for LLM Agents

Summary & Methodology Analysis

The system architecture deploys two specialized heads, which are compact neural network components trained to perform inference-time routing, on top of a frozen foundation model. The Capability Head processes final-layer hidden states to determine whether the current model should retain control or hand off the request to a more capable fallback model. This ensures that expensive models are only invoked when necessary, optimizing both cost and latency for the agent workflow. A second component, the Resolution Head, operates using middle-layer hidden states to classify the specific intervention required, such as clarifying a prompt, invoking a tool, or abstaining from a task entirely. By observing these internal representations during the generation process, the system gains instance-level self-awareness without the need to fine-tune the core model, which is the process of updating model weights to specialize performance on a target dataset. The heads function as a dynamic policy that executes at inference time to route task flow between available models. Because the primary model remains frozen, the deployment avoids the overhead of managing multiple full-sized model instances, providing a unified interface for agent decision making across various tasks. One limitation is that the effectiveness of the control heads is strictly bound by their own prediction quality, and there is ongoing room to improve their accuracy. Additionally, the system faces performance constraints when making predictions at the start of a generation (prefix-time) compared to using full-trajectory information, as early prediction provides less context regarding the model's progress.