When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents

When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents

An LLM agent that reads a tool output and calls the next function looks natural: the output says “the next step is to send an email,” so the agent sends it, but that output might be fabricated, a market panel that looks real but is entirely invented, and the agent still sends the email. The line between data and instruction has vanished: the tool output became a command.

The problem is structural: in agent loops, one system component (the tool) provides Observations, and another component (the LLM) decides the next action, but the LLM is prompted to treat Observations as evidence not as orders, and evidence shouldn’t specify what to do. That distinction breaks in practice. Three recent preprints approach the same root cause from different sides: the conflation of action induction with execution authorization.

The act/don’t-act gate is separable and fragile

A paper from August 2025 (“Calibrated Enough to Know, Not Calibrated to Act”) runs a clean experiment. It shows LLM agents a market panel with numbers that look professional. The agents commit to a directional call on a question that is provably unpredictable. Across 12 frontier models, commitment jumps from 6.5% to 54.0% as the amount of evidence (authoritative-looking) escalates. The key result: when the entire display is fabricated, so every number is invented, commitment still goes from 24.5% to 36.8%. That is statistically indistinguishable from the 37.6% produced by genuine market data. What unlocks confident action is the authority of its packaging, not the information it carries.

The failure is narrow. On answerable questions attached to the same panels, models answer with near-perfect accuracy, and stated probabilities barely move across the gradient that swings action by 48 points. When models are asked to classify a question’s knowability before acting, they call it irreducible 90% of the time and then commit on just 0.4% of those source. The act/don’t-act gate is what fails. That gate is separable. Supervised fine-tuning of a 3B model on 540 synthetic cases (dice, coins, jars, timers) drives commitment to 0.0% on the original cases and transfers to three unseen domains. The gate is trainable and context-fragile. Deployment needs both halves of that sentence.

This tells us something about tool-augmented agents: if a tool output arrives in an authoritative format, the LLM may treat it as a command to act, regardless of whether the data is real or made up. The gate that should say “do I really know enough to call a tool?” has been bypassed.

Separating action induction from runtime authorization

The problem gets sharper when the tool output explicitly contains action-yielding language. A second preprint (“When Tool Outputs Become Commands”) argues that the risk comes from conflating two distinct runtime roles: action induction (the Observation suggests a next action) and execution authorization (the system decides whether to actually call that tool). The paper proposes SARA, a system that treats them separately.

On the Observation side, a context-isolated Action Probe extracts action-inducing semantics and persistently records action-origin provenance across steps. That provenance provides a review signal later. On the execution side, actual tool calls are authorized only against the user objective and against audited evidence from authorized successful executions. The authorization must satisfy goal-level, execution-chain-level, and argument-level support. To preserve this separation across multi-step execution, SARA applies No-History-Promotion: it prevents historical recurrence from laundering action origins into execution authority. “I did this before” does not become “I am allowed to do it again.”

Across AgentDojo and AgentDyn, SARA limits the attack success rate (ASR) to no more than 0.63% across four primary evaluation settings while maintaining competitive task utility source. It consistently reduces ASR across additional agent backbones. The mechanism is explicit: an observation can induce a possible action, but that action is not executed until it passes a separate authorization check that looks at the user’s goal and the chain of evidence.

Safety does not compose across iterations

Even with a good single-step gate, a multi-step agent introduces a new failure mode. A third preprint (“Safety Does Not Compose”) shows that trajectory-scoped monitors reset their safety state after each trajectory. Against an attack whose evidence is fragmented across several iterations, every trajectory-scoped monitor has a true-positive rate equal to its false-positive rate. The evidence it would need never appears in the window it sees. A monitor that retains cross-iteration state separates the two perfectly.

The obvious repair of carrying a geometrically decaying risk score is insufficient. The cooling-off period a patient adversary must wait is a constant that does not grow with the horizon N. So the paper presents LoopHarness, which restores a persistent, non-decaying safety state at the loop level. Under mediated commits and an arbiter detection floor δ_M, it bounds the expected number of unauthorized irreversible actions by B + m − 1 + m/δ_M, a constant in N. The term B + m − 1 is decided by a model-free rule and survives a fully colluding verifier.

What these papers together suggest

All three point to the same architectural gap. Current agent designs hand Observations to the LLM and let the LLM decide both what the Observation means and whether to act on it. That conflation makes the system vulnerable to any output that looks like a command, whether it comes from a manipulated tool or a benign one that returns badly formatted data.

The fixes are design choices about role separation. No hardware changes or exotic monitoring are required. The action-induction role (proposing a possible next call) can be separated from the execution-authorization role (checking that call against the user’s objective and the audit trail). The act/don’t-act gate can be trained as a small, separable module. And the safety state must persist across loop iterations, not reset at each trajectory.

None of these papers claim they solved the problem for every agent. The SARA paper reported ASR under 0.63% on two benchmarks source, but the benchmarks are not the full deployment space. The fine-tuning result for the act/don’t-act gate transferred to three unseen domains, but it did not survive every response format: when the format left no room to reason, the model remained confident and wrong. LoopHarness gives a formal bound, but that bound depends on a detection floor that must be set by the deployer.

What would change the picture? A demonstration that an adversary can construct an Observation that passes SARA’s authorization check while still causing harm. Or evidence that the gate-training approach fails against adaptive attacks that change the packaging of tool outputs across iterations. For now, the separation of action induction from execution authorization looks like the right structural answer. The hard part is building the infrastructure to maintain that separation across many steps, many user objectives, and many tools.