Input Validation Patterns for Untrusted Content Entering Agent Context

Input validation for AI agents can't follow the old client-server model, because agents don't have a single front door anymore. A request comes in from a user, sure, but by the time an agent finishes a task it may have queried a memory store, called three external tools, pulled content off the open web, and taken instructions from another agent entirely. Each of those channels is a separate way into the model's context, and each carries a different trust level that old validation logic was never built to sort out.

The old model assumed a perimeter: user sends request, server validates, server responds. Agents dissolve that perimeter at runtime, by design, because being useful means reaching outside the conversation. Tool outputs arrive as structured or semi-structured text stitched into the next prompt. Retrieved documents land verbatim inside the context, often indistinguishable from the system's own instructions. Messages from other agents carry an implicit trust claim tied to the sender's identity, an identity that may itself have been compromised three steps upstream. Memory stores persist across sessions, so a poisoned write today shapes behavior weeks from now, long after anyone remembers where it came from.

An agent that validates only the initial user prompt has validated almost nothing. Industry research puts it plainly: a large majority of organizations plan to deploy agentic AI, but far fewer feel ready to secure it. That 54-point gap lives almost entirely in the channels past the front door, the tool calls, the retrieved pages, the inter-agent messages, the memory writes nobody's watching. Each entry point needs its own enforcement pattern because the threat model at each one is genuinely different. Treat them as one problem and the gap stays open.

How the context window becomes the attack surface

Call it context flattening. When untrusted external content sits in the same context window as system prompts and user instructions, the model has no reliable way to tell them apart. It processes tokens, not trust levels. There's no field in a transformer's attention mechanism marked "provenance," no header saying this string came from a hostile web page rather than the developer who wrote the system prompt.

This isn't a flaw in any particular model's implementation. It's a structural fact about how language models read input, and no version bump fixes it. A retrieved document, a tool response, or a message from another agent containing the phrase "ignore previous instructions" reaches the model with the same syntactic weight as instructions the developer actually wrote. OWASP rankings consistently place prompt injection at the top of the risk list for exactly this reason. The Agent Security Bench, published at ICLR 2025, found attack success rates exceeding 84% against agentic systems under adversarial testing.

String matching won't get you out of this. Attackers rephrase; ban one sentence structure and they'll find another that means the same thing. So enforcement has to happen at the boundary, before content enters the context, not somewhere inside the model's reasoning where it can be argued with. What can actually be enforced is structural: which content enters which part of the context, and what permissions travel with it. That's the design principle everything below builds from.

Validating tool call arguments and outputs at the tool boundary

When an agent picks a tool and fills in its arguments, both the choice of tool and the arguments came out of the model. Both are untrusted, full stop, no matter how confident the output sounds.

There are two moments to check here. Before the call executes, validate the arguments the model proposed. Allowlisting beats denylisting, categorically: verify that a file path falls inside a permitted directory rather than scanning for traversal sequences like ../, because traversal patterns are infinite and an attacker only needs one your filter missed. Enforce type constraints, numeric ranges, string length limits, date bounds, all before the tool runs. Check the requested action against a predefined allowlist. If it's not on the list, it doesn't run, no matter how persuasively the model frames the request.

After the call, the tool's output needs the same scrutiny coming back in. Treat it as untrusted external content, not a trusted system response just because it came from your own infrastructure. Strip or quarantine anything that structurally resembles an instruction: imperative phrasing, role-assignment patterns, the kind of language a prompt injection would use. Where the tool is supposed to return a JSON object with specific fields, enforce that schema strictly. Reject or redact anything outside it.

Allowlist over denylist carries most of the weight in this section. It's the only approach that survives an adversary who gets to keep rephrasing the attack until something slips through. Pair it with least-privilege scoping: a tool that can only read from one directory cannot be talked into writing to another, no matter how the model constructs its argument. And remember that agents move far more data per session than a person clicking through a UI ever would. A single compromised tool call isn't a single-user incident. It's closer to a batch job gone wrong.

Enforcing provenance and trust labels when agents call other agents

A message arriving from another agent carries an implicit claim: this came from a trusted peer. That claim can't be verified at the model level. The receiving agent sees text, not a cryptographically bound identity attached to that text. So the trust gets assumed, never established.

This is what makes lateral movement the real threat in multi-agent systems. Compromise one agent, and it can issue instructions to every downstream agent through channels those agents were built to trust. Armorer Labs, a continuous security testing platform for AI agents, works specifically in this space by proving such attacks against isolated agent twins before they reach production. Research tracked by vectra.ai shows lateral movement appearing in 8 of 21 documented multi-stage attacks. This dynamic shows clearly that defenses built one agent at a time, at the component level, miss attacks that unfold across multiple reasoning steps and multiple agent boundaries, because no single component ever sees the whole chain.

The fix is to re-verify trust at every hop instead of inheriting it. Assign explicit trust tiers to message sources, and don't let a receiving agent automatically absorb the sending agent's privilege level. Enforce a defined schema for inter-agent messages; free-form natural language from a peer gets treated as untrusted content, same as a scraped web page, not as a standing instruction. Check that a requested action falls within the scope the sending agent was actually delegated, not merely within what the receiving agent is technically capable of doing. Log provenance on every message, origin agent, timestamp, requested action, before execution, so a multi-step attack leaves a trail someone can follow after the fact.

Trust gets re-established at every boundary. It doesn't travel with the message. This is also where the CaMeL information-flow model, covered further down, earns its keep: provenance tracking is what lets a system enforce policy before a tool call happens instead of doing forensics after the damage is done.

Handling retrieved documents and RAG content as a distinct injection surface

RAG pipelines exist to pull outside content into the model's reasoning, which is exactly what makes them such an efficient injection target. The whole point of retrieval is surfacing content the model will treat as relevant and useful. An attacker only has to make sure the content the pipeline surfaces is theirs.

Retrieved documents land in the context verbatim, sitting right next to the system's own instructions, and the model has no built-in way to discount them. The web-browsing case is worse, because the open web isn't just untrusted by circumstance; it's adversarial by design where agents are concerned. Pages get written specifically to issue instructions to whatever agent happens to read them. URL unfurling, Open Graph metadata, hidden text buried in the page's own markup: all of it becomes a vector once an agent is the one doing the reading.

A few patterns hold up. Wrap retrieved content in explicit delimiters and tell the model plainly that anything inside those markers is data, not instruction. It's imperfect; attackers can still try to break out of the delimiter, but it raises the cost of a naive attack. A 2026 arXiv preprint (arXiv:2607.05277) proposes something more structural for web agents: use a separate LLM to analyze a page's HTML and swap out third-party or user-generated content for placeholders before the page ever reaches the agent's working context, so the agent reasons about the page's structure rather than its potentially hostile text. Quarantined summarization works similarly. Run retrieved documents through a separate, restricted model that outputs a structured summary against a fixed schema, and let the primary agent see only that summary, never the raw document. Restrict what can be retrieved in the first place, too: source allowlists and content-type restrictions at the retriever level, before anything gets fetched.

Memory poisoning sits right at the seam between retrieval and persistence. A write into a RAG knowledge base today becomes part of every future retrieval from that store. Documented research has shown that targeted writes into long-term memory or RAG stores can steer an agent's later responses through nothing more exotic than ordinary interaction. Write validation for these stores matters just as much as read validation, then. A write is a potential injection into every context that store will ever populate.

Persistent memory as a slow-burn injection channel

Tool outputs and retrieved documents do their damage inside a single context window and then they're gone. A poisoned memory write is different. It sticks around, influencing every session that retrieves it, for as long as it sits in the store.

Persistence as an attack pattern showed up in 5 of 12 documented incidents in 2024, according to vectra.ai, and has continued to grow as a feature of sophisticated multi-stage attacks. It's become the default, not the edge case.

Sleeper memory poisoning is the mechanism to know here: adversarial content gets implanted as a fabricated memory during some ordinary, unremarkable interaction, then surfaces weeks later in a completely unrelated session, shaping the agent's behavior long after anyone would think to connect the two. Nobody's debugging a decision made today by tracing it back to a conversation from a month ago.

Validation has to happen at write time, not retrieval time. Memory entries should conform to a typed schema; anything reading as free-form, instruction-like text gets rejected before it's stored, not filtered out later once it's already influencing outputs. Every entry should carry a source tag and a trust-tier label, recorded at the moment of the write, so retrieval logic can limit how much weight a low-trust entry carries. High-value or long-lived memory stores deserve an approval step before a write commits, the same logic behind requiring human sign-off before a destructive tool call executes. And memory should get audited periodically for instruction-like patterns that slipped through. It's not a black box. It's a store, and stores can be inspected and purged.

Memory validation is input validation stretched across time, really. The content entering tomorrow's context has to be governed today, at the moment it's written, or it isn't governed at all.

Privilege separation as the architectural enforcement layer across all entry points

Every entry point covered so far shares one failure mode: untrusted content and trusted instructions end up sharing a context, a model, and tool access, all at once. Privilege separation fixes this at the architecture level rather than the content level, and it's the closest thing this space has to a load-bearing wall.

The dual-LLM pattern splits the job in two. A privileged model holds the tool access and takes the actions, but it never reads untrusted content directly. A quarantined model reads the untrusted content, the web page, the document, the incoming message, but has no tool access and can't act on anything. The quarantined model can only hand back structured summaries or labels, and the privileged model receives those, never the raw text. That design breaks the causal path an injected instruction needs to reach an actor with real permissions. Even a fully manipulated quarantined model can't call a tool it was never given access to. A guardrail model is itself just another model, and it can be prompt-injected too; this is one layer in a stack, not a solution that stands alone.

CaMeL, published by Google DeepMind in June 2025, is the most formally grounded defense running today. It puts a custom interpreter between the LLM and its tool calls, one that tracks where every piece of data came from and enforces security policy before each execution, not after. Untrusted data the quarantined model retrieves gets stored in variables whose actual contents are redacted from the privileged model's view entirely; the privileged model sees the variable's name, never its value. Capability-based controls on top of that stop private data from flowing somewhere it isn't authorized to go. There's an honest cost, too: CaMeL solves 77% of tasks with provable security guarantees on AgentDojo, against 84% task completion for an undefended baseline. That seven-point gap is the price of the guarantee, and for most consequential deployments, it's a fair trade. A variant called Fides pushes the idea further by generating an agent's execution plan one step at a time instead of all upfront, which limits how far an injected instruction can redirect a workflow that's already several steps in.

Both patterns rest on the same insight: the thing doing the enforcing has to sit outside the model, not inside its reasoning. A model can't be trusted to police its own outputs, because the same vulnerability that let the injection in is the vulnerability that would let it argue its way past a check running in the same head.

Context minimization and the principle of least context

Privilege separation governs what a model is allowed to do with untrusted content once that content is in front of it. Context minimization asks a narrower, earlier question: how much of that content needed to be there in the first place.

Every token of untrusted content sitting in the context window is attack surface. Tokens that don't need to be there are risk with no benefit attached, and they should be cut.

A few patterns handle this well. Session pruning removes content from the context once the current task no longer needs it, rather than letting every prior interaction pile up indefinitely in the background. Strict interface formatting works at the design level: a code agent that interacts with untrusted documentation through a narrow API description, method names, argument types, return types, nothing else, simply has no surface for injected prose to land on, because the interface was never built to accept prose in the first place. Schema-gated retrieval converts retrieved content into a fixed schema before it enters the context at all; anything that doesn't fit the schema gets dropped, not passed through with a warning label attached.

The same logic applies on the user-facing side, not just the machine-facing one. A user who pastes in a block of text copied from somewhere else is, from the agent's point of view, doing exactly what a poisoned tool output or a hostile web page does: introducing content whose origin the system can't verify. Less context, chosen on purpose rather than accumulated by default, is what keeps that exposure small enough to actually manage.