When Someone Else's Sentence Becomes Your Instruction — Prompt Injection

A Single Line of Log Became a Command

Cloud servers write down what happened in logs. There are even artificial intelligence assistants that read these logs instead of humans and run commands to fix problems. Inside the assistant is a language model—a model that takes in text and outputs the text that should follow it (from the definition in "How Generative AI Answers").

A preprint released in 2026—meaning a paper made public before peer review—fed logs tampered with on purpose to these assistants. The authors planted 42 sentences across five types of logs and ran eight models five times each under three conditions (160 runs per model and condition based on 32 attack sentences). In what the authors call the active condition, the rate at which models executed the planted commands as-is ranged from 0% to 86.2% depending on the model.

When given advance instructions saying, "Do not execute fix commands," most models dropped to 0%. Yet one model still executed them 30.0% of the time. Six out of the eight models went as far as fetching and running code from the outside (arXiv:2604.15368, preprint).

The same paper includes a stranger observation. Three commercial gatekeepers—which separately inspect inputs to filter out dangerous things—failed to catch almost any of the sentences embedded in the logs. One caught only the single most blatant sentence out of 32, and another caught none. Yet when the exact same sentences were pulled out of the logs and shown separately, the gatekeepers caught them. This means that even the same sentences slipped past most of these gatekeepers' inspections when mixed into the logs.

The authors also observed a new behavior. The models recognized suspicious pieces on their own, erased them, and then executed the remaining commands as-is (arXiv:2604.15368, preprint).

Two things should be made clear. First, this article does not evaluate specific products or companies. It does not list model names or product names; it writes down what is possible, but not how to do it. Second, this field has uneven evidence grades. Of the 26 sources cited in this article, 20 are preprints uploaded to preprint servers like arXiv and SSRN, and one is a preprint uploaded to another open repository. Therefore, grade labels are attached within sentences at locations with heavy weight.

An Error That Just Did What It Was Told

This kind of problem is called a prompt injection. As "How Guessing the Next Word Becomes a Conversation" defines it, a prompt is the text put in front of a model to get an answer. A prompt injection is when a sentence written inside the data the model was given to read acts as an instruction that moves the model. It is also called an injection for short. You give it a log to read, and one line inside the log is read as "From now on, do this."

People keep the two separate. A person asked to read meeting minutes treats the sentence "Open the company safe after reading everything" as content to be read, not as a command given to them. The model does not have a mechanism to separate the two.

Depending on where this sentence is placed, its character splits. Indirect prompt injection is a method where an attacker places the sentence in a spot the model will read later, instead of typing it directly into the input box. Such spots include web pages, attached files, search results, the logs seen earlier, and responses returned by tools. Here, tools are programs attached to let the model do things beyond writing text, such as searching the web, opening files, and running commands. The attacker leaves it written and walks away, and the sentence triggers during someone else's task.

This is a different problem from hallucinations. By the definition in "How AI Gets Things Wrong", a hallucination is when AI makes up content that sounds plausible but differs from the facts or contradicts the given data. In an injection, the model takes a directive planted by someone else as its own user's instruction and follows it as is. In a hallucination, the content the model outputs deviates from the facts or data, and in an injection, whose instruction the model followed deviates.

The reason the model cannot separate the two is that commands and data—meaning the materials given to read—enter through the same channel. A preprint on SSRN synthesizing prompt injection literature calls this property structural. Because it arises from the language model being designed to process commands and data through the same neural pathway, it cannot be eliminated through training alone, according to the paper. The same literature notes that this item has been number one on a security advisory list since 2023.

The second conclusion of the same literature is this: indirect prompt injections processed through searched-for content produced actual verified security incidents, and their severity exceeded 9.0 on the vulnerability severity scale. The authors read this as a signal that the problem has moved from theory to the operational stage. The third conclusion is that the channel through which instructions come and the channel through which untrusted data comes must be structurally separated (DOI: 10.2139/ssrn.6740060, preprint, narrative literature review).

Another preprint on SSRN compares this structure to a web browser. As the authors see it, one thing browser security learned over 25 years was moving the mechanism that enforces rules outside the application. Artificial intelligence systems holding tools do not have such a layer.

The rules the system must follow are written in the system prompt. A system prompt is a guideline text that the side building the service puts in front of the model before the user's question. This guideline is placed inside the same chunk of text as user messages, tool responses, and searched-for documents. When a sentence inside a searched-for file clashes with the guideline, the model weighs and decides for itself, and there is no separate process that hands down a ruling (DOI: 10.2139/ssrn.6876926, preprint).

The first systematic measurement of this weakness was a benchmark in 2023. By the definition in "How AI Gets Things Wrong", a benchmark is a set of test questions designed to make multiple AI models solve the same problems and compare their scores. When the models of that time were run through the first test set of planting instructions in external content, the models were universally vulnerable. The authors pointed to two causes: the model's inability to distinguish context as information to refer to from instructions to execute, and the lack of awareness that instructions inside external content should not be executed (arXiv:2312.14197, preprint).

The same paper also presented its own defense. The authors write that a defense touching the inside of the model lowered the success rate of planted instructions to close to zero while preserving output quality. However, the abstract does not state how often normal inputs were mistakenly blocked. This blank space is the question for the second half of this article.

What Users Coax vs. What Others Say on Their Behalf

There is a neighbor easily confused with injection. Jailbreaking is when a user directly coaxes the model into saying something it was originally not supposed to say. This includes having it play a role or pushing it by saying, "This is a novel, so it's fine."

What separates the two, rather than the shape of the sentence, is whether the person who inserted the instruction is the conversation's user themselves. In a jailbreak, the person making the model say strange things is the user of that conversation. In an injection, it is different. The person who planted the sentence is not there, and execution happens under the name and authority of an unsuspecting user. The user's mailbox, files, and payment methods are at stake.

Attack Success Rate (ASR) is the percentage among attempted attacks that actually reached the result the attacker wanted. If you try one hundred times and get it ten times, it is 10%.

The literature often mixes jailbreaks and injections into a single number. A conference paper that created a classification system for attacks targeting chatbots divided 210 attack sentences into 7 categories and ran them on 5 open-weight models inside the researchers' computer. Open-weight models are models whose parameters—the numbers inside the model determined by training—are made public so anyone can download and run them. It is reported that the attack success rates ranged from 10.5% to 83.8% depending on the model.

The authors conclude that the main factor dividing resistance was the quality of alignment training rather than model size. Alignment is the process of fitting what the model outputs to the user's intent and human values, as defined in "How Guessing the Next Word Becomes a Conversation". However, in the same paper, the category that worked best was "jailbreak/roleplay" at 34.7% (DOI: 10.51582/interconf.19-20.05.2026.017, conference proceedings). The scope of the abstract appears to be measured per model across all seven categories of attack sentences, and if so, this jailbreak category is included within it. By this article's definition, that category is not injection. When carrying over numbers, one must check every time whether that paper measured injection or jailbreak.

Even looking at the injection side alone, the numbers are not small. Conversational models use formatting tags internally to distinguish who is speaking. A preprint testing attack sentences that mimicked only the outer appearance of those tags reports that the average attack success rate rose from 5.18% to 32.05%, and from 15.13% to 45.90%, across two test sets. Variants that persuaded the model over multiple turns of conversation worked particularly well, averaging 52.33% in one test set.

The authors' next observation weighs heavier. Existing prompt-based defenses that block attacks by inserting warnings or demarcation markers into the text given to the model were largely powerless against this method (arXiv:2509.22830, preprint). The method is not described here.

From Reading to Execution

Up to this point, this has been a matter of the scale at which models spit out weird answers. Changing the scale is a tool.

An agent is a language model that doesn't stop at generating words, but also holds tools to read the web, open files, and execute commands. The very tools that make agents useful also make them dangerous.

A survey grouping 44 attack papers from 2023 to early 2026—in other words, an article that gathers and organizes multiple papers—puts this transition in a single sentence (SSRN preprint). An injection against a chatbot yields, at worst, an embarrassing sentence. The same injection against an agent holding tools results in a leaked database, an unauthorized money transfer, or contaminated messages spreading to a neighboring agent. The authors summarize it like this: The types of vulnerabilities are not new; what is new is the scope of the damage when breached once, or its blast radius.

The same survey also points out two repeatedly appearing findings. First, no defenses carried out purely at the prompt level can withstand an adapting attacker. An adapting attacker is one who looks at what the defense looks like and fixes their attack accordingly. Second, the paths spreading between multiple agents are the least defended sections (DOI: 10.2139/ssrn.7218206, preprint).

The spreading goes beyond the screen, too. In a preprint testing a robotic system where multiple agents split up planning and control using language models, a planted sentence caused the action aimed at by the attacker while lowering the completion rate of the original task. The attack propagated to other agents along the prompt structure shared by the agents. However, the magnitude of the effect depended on how the prompt was structured and which agent was targeted (arXiv:2608.00747, preprint).

A preprint targeting coding agents presents an attack branch that works no matter what the user asks. It is an attack that relies on the fact that while the user's question changes every time, the system prompt and tool descriptions rarely change. The success rate was up to 87% across five mock agents, and the baseline compared by the authors—meaning the best among existing methods run together under the same conditions—was 50%. The authors write that sentences created this way transferred beyond mock environments to actual coding agents (arXiv:2510.23675, preprint, authors' own method).

It spreads across time, too. Some agents possess memory that persists even after a session—meaning a single continuous exchange of tasks with a user—ends. A preprint testing two commercial coding agents and four models reports results from both sides. It was difficult to make an agent overwrite its own memory file using only untrusted external content. However, sentences already planted in that file successfully attacked the current session and subsequent sessions. The success rate and the degree to which the sentences persisted varied greatly depending on the system, model, and attack target (arXiv:2607.14611, preprint).

Another preprint that names this phenomenon borrows its name from stored vulnerabilities long known on the web. A stored vulnerability is an attack where code planted in a place like a bulletin board is executed whenever anyone later opens that post. Here too, planted sentences are not trapped within a single interaction and can survive across time. The authors write that the results suggest the following: the fundamental challenge of agent security cannot be solved simply by filtering out untrusted inputs, but lies in governing how external information gains privileges as it crosses long-lasting system boundaries (arXiv:2606.04425, preprint).

How to Read the Sentence "We Blocked 94%"

Now let's look at the defense side. Most defense papers are the results of authors evaluating their own methods, which inherently makes it like a manufacturer reviewing its own product.

The false positive rate is the proportion where a defense mechanism mistakenly blocks normal inputs that have no problems. If you make the gatekeeper fierce, the attack success rate can drop as much as you want. If you let nobody in, it is zero. Therefore, the performance of a defense must be read by putting the attack success rate, false positive rate, and whether the originally intended task can still be done with the defense turned on all together.

Let's look at the inconvenient results first. An evaluation study published in a peer-reviewed academic journal ran four defenses and an undefended control group across three models on a banking task set (one of the four defenses was evaluated on only two of the three models). When there was no defense, the number of successful attacks out of 288 was 0, 11, and 1 remaining per model. Even before applying defenses, attacks hardly succeeded. The authors add that since models differed in their ability to reach the attack targets required by the test, even this difference between models should be read with caution.

In the model where attacks succeeded most frequently, the two defenses seemed to lower the success rate further. However, those two defenses also lowered normal task performance, and one reduced the actions available to the agent itself. Among the four pairs of comparisons made on that model, not a single difference remained after applying a correction for multiple comparisons, which filters out differences that arise by chance across multiple comparisons (DOI: 10.3390/computers15090570, peer-reviewed).

In a test where attacks hardly succeed in the first place, the effectiveness of a defense cannot be measured. If "The Score Went Up. But What Went Up?" asked what went up when benchmark scores rose, what we must ask here is what the defense claims to have blocked. Sometimes that answer comes from a place that cannot be measured.

An example listing three numbers side by side is a defense preprint on SSRN. It reported the numbers together even though it evaluated its own defense system using its own created set of attacks. Merely narrowing the scope of permissions blocked 56.5% of attacks, while the rate of mistakenly blocked normal inputs was 0%. Turning on all layers that block using predetermined rules raised the blockage to 90.6%, but the false positive rate became 5.9%. Adding a statistical anomaly detector on top of this blocked no additional attacks and roughly doubled the false positive rate.

No configuration reached 100%. The authors call that remaining fraction the "honest residual" without erasing it. However, these figures came from a simulation that computer-mimicked 8,000 sessions. In a pilot experiment run with real agents, the attack success rate without a defense was so low that the layers could not be ranked (DOI: 10.2139/ssrn.6870178, preprint, simulation, authors' own method).

Another study published in a peer-reviewed venue also places three numbers side by side. It ran 250 adversarial examples and 250 normal examples on 7B-class (open-weight models with roughly 7 billion parameters). It reports lowering the attack success rate from 36.0% to 17.2% while maintaining a normal task success rate of 97.2%, and raising the detection F1 from 0.749 to 0.892. F1 is a detection score that reflects both how well attacks are not missed and how well false alarms are avoided; higher is better.

An adaptive attack evaluation aimed at the pipeline reports 6.7%. However, the abstract does not state whether the attack set for that evaluation was the same as the preceding 250 cases, making it impossible to compare the two values on the same scale (DOI: 10.3390/app16157662, peer-reviewed, authors' own method). 17.2% is not a number that can be read as "blocked." It means that roughly one in six attacks got through.

There is also a study that notes the cost of false positives. It is a journal paper comparing lightweight detectors using 662 prompts from a public dataset (399 normal, 263 attacks). The best detector had an F1 of 0.901, and keyword rules had 0.125. If a detector of the same family is used as a gateway that rejects suspicious inputs, the proportion of attack prompts passing the gateway in the test split of this dataset drops from 1.000 (no gateway) to 0.117, while the usability for normal usage drops to 92.9%. This is not a value measured up to the attacker achieving their goal. This value is when the score threshold where the detector starts blocking—in other words, the operating point—is set toward the side with fewer false positives. This means roughly 7 out of 100 normal requests are blocked (DOI: 10.63575/cia.2025.30205, journal).

There is an SSRN preprint that tested a configuration combining rule filters and language model classifiers under a single-turn question-and-answer condition using 625 prompts created by the authors themselves. At least in this study, the mistakenly blocked normal inputs were not distributed evenly. At standard difficulty, this configuration blocked 100.0% of attacks and mistakenly blocked 5.0% of the 200 normal prompts. That 5.0% was concentrated entirely in categories touching on security. Of normal prompts discussing security research, 87.5% were blocked, and 37.5% of critical normal prompts were blocked (sample sizes per category are not in the abstract), while blocks in all other categories were zero. The normal user blocked in this benchmark was someone talking about security rather than a random person.

In the same study, a rules-only configuration showed 75.0% blocking with a 2.5% false positive rate at standard difficulty, but against harder-crafted attacks, its blocking rate collapsed to 24.6%. The combined configuration showed 98.9% blocking with a 0% false positive rate on the hard side (DOI: 10.2139/ssrn.6246379, preprint, authors' self-made benchmark).

There is also a preprint that reported a baseline together. Evaluated with 80 attack sessions and 100 normal sessions, it records detecting 76.25% of attacks (95% confidence interval 65.86–84.24%) and mistakenly blocking 7.0% (3.43–13.75%) of normal sessions. The confidence interval in parentheses is the range where the true value is likely to lie, given the fluctuations caused by a small sample. A simple vocabulary baseline under the same conditions had 1.25% detection and a 6.0% false positive rate. The false positives are similar, but the detection differs by 60 times.

A holdout is test data set aside and not used for development, like the test data mentioned in "AI Models", and this holdout was created by the researchers themselves. The authors note themselves that it is necessary to re-verify by comparing against trained defense models and independent public benchmarks (DOI: 10.20944/preprints202608.1961.v1, preprint, authors' own method).

Finally, there is a peer-reviewed journal paper that self-tested where detection collapses. On a test set organized with detection tools for web agents, it produces good numbers: an accuracy of 0.9849, an attack recall of 0.9851, and a normal false positive rate of 0.0050. Attack recall is the proportion of actual attacks that were caught. However, the more text placed alongside the model—in other words, the more context attached—the lower the performance drops.

The value that drops is a metric different from this accuracy. It is the average of the aforementioned F1 calculated separately for each category, and on the same test set, this value was 0.8887. That value, which was 0.8301 on static prompts, became 0.6306 in multi-turn conversations, 0.3844 when retrieved documents were attached, and 0.3200 when memory was attached.

The authors also record the fact that when data was randomly split, scores were higher, but 144 data leakage warnings popped up at that time. Data leakage is when things resembling test data get mixed into development data, making scores look better than actual skill. Rather than placing their tool as a standalone device deciding on permission, the authors position it themselves as a triage component that picks out suspicious things and passes them to a higher verification stage (DOI: 10.3390/fi18070376, peer-reviewed, authors' own method). The observation from the beginning of this article—that it caught things when kept separate, but couldn't catch them when inside the logs—appears here with a different face.

One more qualifier is attached to all these numbers. A preprint that gathered eight defenses and re-crafted attacks tailored to each of them bypassed all eight, reporting a consistently exceeding 50% attack success rate (arXiv:2503.00061, preprint). Most of the figures presented earlier were measured with predetermined, fixed attacks, and such values are not the maximum an attacker can achieve. In the words of "How AI Gets Things Wrong", regular accuracy does not speak for safety in places where someone is intentionally trying to deceive.

Why It Is Hard to Completely Stop

If the cause is instructions and data entering through the same channel, separating the two channels comes to mind as the first line of defense. Indeed, that has been the dominant defense direction in this field.

There is a preprint that directly examines that direction. The authors report that the paradigm separating data and instructions fails to catch attacks that manipulate context, while simultaneously hindering even contextually appropriate behaviors. Context-manipulating attacks are those that dress up the surrounding circumstances so that information flow that should be blocked looks legitimate in that situation. The authors write that this suggests an impossibility result. Either the attacker can always cook up a context where the blocked flow looks legitimate, or a defender who tightens the criteria ends up blocking even genuinely legitimate flows (arXiv:2605.17634, preprint).

The authors' expression is "suggests," not "proved." The trade-offs in the previous section—namely, 90.6% containment with 5.9% false positives, and a 0.117 pass rate with 92.9% usefulness—cannot verify this suggestion because they measured different mechanisms under different conditions. The previous section also had configurations that blocked almost all difficult attacks while producing zero false positives.

There is also an SSRN preprint that measured the limits of defenses based on blocklists. One widely proposed defense is to unwrap all instructions wrapped in different encodings back to their original text before inspecting them. The authors implemented this mechanism deterministically without random elements and tested it against 10 instructions and 13 transformation variants. The 4 variants it knew how to reverse were restored with an average of 0.95, while the 9 it did not know averaged 0.00.

Even among the 4 known ones, if instructions were mixed into surrounding text, the restoration rate plunged to 0.00, and when another layer of the same encoding was applied on top, the restoration rate of 1.00 dropped to 0.00. Since running 10 cases each on three types of control datasets without attacks produced zero false alarms, the mechanism did its job accurately. The authors' conclusion is that while this defense is necessary, the shape of its boundary is exactly the shape of the list of encodings the mechanism knows (DOI: 10.2139/ssrn.7389061, preprint). The names of the encodings are not listed here.

When an agent's conversation gets long, it summarizes and shortens the past history. One SSRN preprint views this summarization step as a place where the trust boundary is rewritten. The trust boundary is the line separating instructions to trust and follow from data to merely reference. Through summarization, text mixing both types is rewritten into a single new piece of text, and the model working behind the scenes can treat that summary as something trustworthy.

In a 900-row scale pilot experiment from this preprint, when summarized normally, four metrics such as the rate at which an attacker's sentence was promoted to the trusted side were each 0.027. That is 4 out of 150 rows. In a summarization designed to guard the boundary, all four metrics were 0.000. The authors explicitly state in the abstract that this result does not establish a universal defense (DOI: 10.2139/ssrn.6933161, preprint).

Narrowing the Outside of the Model

Some authors emphasize that as much as making a model hard to fool, we should minimize what it can do even if it is fooled. A preprint summarizing web agent measurements notes that we should assume some untrustworthy instructions will be processed and sometimes followed, and then restrict what happens afterward (DOI: 10.2139/ssrn.7276838, preprint).

Least privilege is a design principle that gives a component only the exact amount of authority it needs to finish its assigned task, keeping it from holding the rest at all. It is like not giving the key to the safe to someone who just came to summarize meeting minutes. In the simulation we saw earlier, this single layer blocked 56.5% of attacks with a 0% false-blocking rate. Although the authors do not state the reason separately, this principle differs in nature from detection. Detection has to judge whether a sentence is bad, but least privilege does not judge and simply makes it impossible to do that thing in the first place (DOI: 10.2139/ssrn.6870178, preprint, simulation, authors' own method).

This is also the first conclusion of the survey cited earlier. Effective security comes from restricting what the system around the model is allowed to do, rather than relying on the model's ability to withstand trickery (DOI: 10.2139/ssrn.7218206, preprint). The literature comparing it to browsers demands the same thing. It is a layer that enforces rules in a place the application cannot touch (DOI: 10.2139/ssrn.6876926, preprint).

A preprint that enforces user-verified rules at every point a tool is called also notes that it provided protection without harming usability, but because there are no figures in the abstract, it is cited only as an example of a direction (arXiv:2604.11790, preprint).

There are also attempts that include numbers. This is a preprint that separately tracks how far information can flow, automatically executing only tool calls where safety is guaranteed and asking the user only for the rest. It reports that on one test set, it blocked all target attacks while the loss of task usability during the attacks was 2% (arXiv:2502.08966, preprint, authors' own method). However, as we saw earlier, there is a study that broke all eight defenses with tailored attacks, and whether this defense is among those eight cannot be known from the abstract alone.

Where to place humans is also a design decision. The previous simulation study treats human approval steps as security controls rather than mere conveniences. This is because setting up a feedback loop that learns and corrects automatically turns that loop itself into a new contamination channel (DOI: 10.2139/ssrn.6870178, preprint, simulation, authors' own method).

A position paper summarizing this direction first notes that in realistic tasks, it is frequently necessary to rewrite plans and security policies during execution. And it argues that while certain context-dependent decisions must still be made by the model, they should only be made within a design that strictly limits what the model can see and what it can decide. It also notes that in ambiguous cases, human interaction should be treated as a core design element. The same writing points out that existing test sets can create a false sense of security regarding both usability and security (arXiv:2603.30016, preprint, position paper). This means that even within designs that narrow the outside of the model, a share of the work remains for the model to handle.

A List of Questions You Can Answer

The previous comprehensive web agent preprint (SSRN) split "blocked or breached" into eight categories, using data up to August 1, 2026. These are: whether it was exposed, whether it accepted the instruction, whether it went as far as harmful behavior, whether the attacker's goal was achieved, whether that fact was hidden from the user, whether it was recovered, whether normal tasks still work, and whether normal inputs were blocked excessively. The reason for splitting them is that measured vulnerabilities vary substantially depending on the ability to complete tasks, the budget allowed for the attack, whether the attacker can adapt, the format of the input, and the scoring method.

The same literature lists two results side by side. In one test set, the frequency with which agents initiated adversarial instructions was much higher than the frequency with which they accomplished the attacker's goals. In adaptive evaluations from 2026, success rates rose considerably when attackers were given the chance to repeatedly fix their attacks. And it concludes with this: no single-model attack success rate, including very low ones, is the same thing as a "reliable web agent" (DOI: 10.2139/ssrn.7276838, preprint).

So "is this agent safe from injection?" is not a question you can answer. What you can answer is the following list:

① Who wrote this data? It asks whether what the agent reads distinguishes between what the user wrote and what anyone else could write (arXiv:2606.04425).

② What is the list of things this agent can do right now? It asks whether it goes up to reading, writing, execution, or money. It is this list that determines the blast radius (DOI: 10.2139/ssrn.7218206).

③ What was blocked, and what was wrongly blocked? A report with just a single number cannot be read. Nor can you assume that the wrongly blocked side is evenly distributed. In one study's mixed configuration, only a small minority of normal prompts were wrongly blocked, but those false positives were clustered in categories discussing security, where 87.5% were blocked (standard difficulty, author's own benchmark, DOI: 10.2139/ssrn.6246379).

④ Were those figures measured under fixed attacks, or adaptive attacks? Eight defenses were all opened up in front of customized attacks (arXiv:2503.00061). And if a test had an undefended control group where attacks succeeded 0 out of 288 times, the defensive effect derived from there has not been measured yet (DOI: 10.3390/computers15090570).

⑤ Do the planted sentences disappear when this session ends? In two memory-capable coding agents tested by a preprint, sentences already planted could go on to attack subsequent sessions (the degree varied greatly depending on the system and model, arXiv:2607.14611).

There is no reason to lean toward cynicism. More than half of the breaches were blocked without false positives simply through least privilege (simulation and author's own method, DOI: 10.2139/ssrn.6870178), and a defense tracking information flow yielded a 2% utility loss in one test set (which is the author's own method, and its performance against adaptive attacks is unknown, arXiv:2502.08966). Only when it is written where it breaks down can you ask where to fix it.

There is no reason to lean toward optimism either. There are indications that dominant defensive directions run into principled tradeoffs (arXiv:2605.17634), the boundaries of list-based blocking devices follow the exact shape of those lists (DOI: 10.2139/ssrn.7389061), and one literature review states that this problem cannot be eliminated through training alone (DOI: 10.2139/ssrn.6740060, preprint and narrative literature review).

What remains at the end is the distinction that opened this article. In the hallucinations covered in "Why Do LLMs 'Lie'?", the content output by the model diverged from facts or data. In the problem seen here, the model did not make up content. Rather, it followed it all too well. It read sentences inside logs, accepted them as instructions, and did as it was told. The property of following instructions well made this tool useful, and that exact same property created this vulnerability.

This problem happened because the model listened a little too well. So the place to fix it does not lie solely within the model. To prevent sentences written by others from running in my name, alongside filtering sentences better, you must determine beforehand what actions are even allowed to be done in your name.

Source Papers

Preprints (arXiv, SSRN, etc.) are manuscripts that have not undergone peer review. Most defense papers evaluate their own authors' methods, so their grades are noted in the text accordingly.

• "LogJack: Prompt Injection Through Cloud Logs" (2026, preprint) — arXiv:2604.15368 — 5 types of logs, 42 payloads, 8 models. Under active conditions, the rate of executing commands as-is was 0% to 86.2%; even after passive instructions, one model reached 30.0%. The gatekeeper failed to catch sentences inside logs, but caught them when separated.
• "Prompt Injection and Jailbreak Attacks in LLM-Based Agents" (2026, preprint and narrative literature review) — DOI: 10.2139/ssrn.6740060 — The structural cause where commands and data enter through the same neural pathway cannot be eliminated by training alone. Indirect injection via retrieved content caused real-world incidents exceeding a severity of 9.0 (moving from theory to operations).
• "Agent Security Policy (ASP): The Missing Trust Boundary Layer" (2026, preprint) — DOI: 10.2139/ssrn.6876926 — Policies reside within system prompts with no separate process to adjudicate conflicts. Contrast this with browser enforcement layers.
• "Benchmarking and Defending Against Indirect Prompt Injection" (2023, preprint and author's own method) — arXiv:2312.14197 — The first indirect injection benchmark, universally vulnerable. The false-block rate is not in the abstract.
• "Taxonomy of Prompt Injection Attacks in LLM-Based Chatbots" (2026, conference proceedings) — DOI: 10.51582/interconf.19-20.05.2026.017 — 210 payloads, 5 open-weight models, attack success rates ranging from 10.5% to 83.8% (since this combines 7 categories, the jailbreak category of 34.7% is included within). Determined by the quality of alignment training rather than size.
• "ChatInject: Abusing Chat Templates for Prompt Injection" (2025, preprint and author's own method) — arXiv:2509.22830 — Increased from 5.18% to 32.05% and 15.13% to 45.90% compared to existing methods, with multi-turn interactions being particularly effective at an average of 52.33%. Existing prompt-based defenses proved largely powerless.
• "Vulnerabilities in Autonomous Execution: LLM Multi-Agent Systems" (2026, preprint) — DOI: 10.2139/ssrn.7218206 — Synthesis of 44 attack papers. What is new is not the type of vulnerability, but the blast radius. Prompt-level defenses cannot withstand adaptive attackers, and multi-agent propagation was the least defended.
• "When Prompts Control Robots: Multi-Agent Robotic Systems" (2026, preprint) — arXiv:2608.00747 — Triggered hazardous behaviors and degraded task completion rates, propagating across agents via shared prompt structures. The magnitude of the impact varies depending on prompt composition and targets.
• "QueryIPI: Query-agnostic Injection on Coding Agents" (2025, preprint and author's own method) — arXiv:2510.23675 — Branches independent of user queries reached up to 87% across 5 simulated agents (top baseline was 50%). Generated sentences transferred to actual coding agents.
• "Bad Memory: Prompt Injection Risks from Agent Memory" (2026, preprint) — arXiv:2607.14611 — While forcing memory overwrites using external content proved difficult, content already planted attacked current and future sessions.
• "What If Prompt Injection Never Left? Cross-Session Stored" (2026, preprint) — arXiv:2606.04425 — Planted sentences persisted across sessions. The authors suggest that the fundamental challenge is not just filtering, but governing how external information gains authority across persistent boundaries.
• "Evaluating IPI Defenses in Tool-Using LLM Agents" (2026, peer-reviewed journal) — DOI: 10.3390/computers15090570 — 0 out of 288, 11, and 1 successful undefended attacks (one of the four defenses was evaluated on only two models). No significant comparisons after correction. Defenses also lowered normal task performance.
• "Ariadne: Human-in-the-Loop Defense against Indirect Prompt Injection" (2026, preprint, simulation, and author's own method) — DOI: 10.2139/ssrn.6870178 — 8,000-session simulation. Principle of least privilege alone blocked 56.5% with 0% false blocks; full stack blocked 90.6% with 5.9% false blocks; statistical detector added 0 additional blocks with about twice the false blocks. No 100% solution exists. Real-world pilots had a low baseline attack rate, preventing ranking.
• "Spotlight-Guard: Security and Performance in LLM Agents" (2026, peer-reviewed journal and author's own method) — DOI: 10.3390/app16157662 — Attack success rate dropped from 36.0% to 17.2%, normal task success rate was 97.2%, and detection F1 rose from 0.749 to 0.892. Adaptive attack evaluation was separate, and whether the attack sets were identical is not stated in the abstract.
• "Injection Detection and Refusal-Usefulness Tradeoffs" (2025, journal) — DOI: 10.63575/cia.2025.30205 — 662 prompts. Top detector F1 was 0.901, keyword rules 0.125. At a specific operating point, success rate dropped from 1.000 to 0.117 while usefulness was 92.9%.
• "Evaluating Hybrid Guardrail Architectures for Injection Defense" (2026, preprint and author's own benchmark) — DOI: 10.2139/ssrn.6246379 — Standard blocking was 100.0% with a 5.0% false-block rate, but those false positives were concentrated solely in security research (87.5%) and critique (37.5%). Using rules alone resulted in 75.0% with a false-block rate of 2.5%, which later shifted to 24.6%.
• "ChainGuard: Provenance-Aware Runtime Security" (2026, preprint and author's own method) — DOI: 10.20944/preprints202608.1961.v1 — Detected 76.25% of attacks (CI 65.86–84.24%) with a 7.0% (3.43–13.75%) false-block rate, compared to vocabulary baselines of 1.25% and 6.0%. The holdout set was created by the researchers themselves, and the authors note that independent benchmark re-verification is needed.
• "PromptSentinel-X: Leakage- and Context-Aware Detection" (2026, peer-reviewed journal and author's own method) — DOI: 10.3390/fi18070376 — Accuracy 0.9849, recall 0.9851, false-block rate 0.0050, macro-F1 0.8887. When context was attached, the macro-F1 dropped from 0.8301 to 0.6306, 0.3844, and 0.3200. Random splits yielded 144 leakage warnings. The authors state it is a filtering component, not an authorization mechanism.
• "Adaptive Attacks Break Defenses Against Indirect Prompt Injection" (2025, preprint) — arXiv:2503.00061 — Bypassed all eight defenses, consistently achieving over a 50% attack success rate.
• "AI Agents May Always Fall for Prompt Injections" (2026, preprint) — arXiv:2605.17634 — The separation paradigm fails to catch context manipulation and hampers appropriate behaviors. The authors note this "suggests" impossibility.
• "The Encoding Alphabet Is the Security Boundary" (2026, preprint) — DOI: 10.2139/ssrn.7389061 — Averaged 0.95 recovery across 4 reversible encodings, and 0.00 across 9 unknown ones. When mixed into surrounding text, both dropped to 0.00; adding another layer of encoding dropped recovery from 1.00 to 0.00.
• "Context Compaction Provenance (CCP) Lab" (2026, preprint) — DOI: 10.2139/ssrn.6933161 — 900-row pilot; naive compression yielded 0.027 across four metrics each (4 out of 150 rows), while protective variants yielded 0.000. Explicitly stated not to be a universal defense.
• "ClawGuard: Runtime Security for Tool-Augmented LLM Agents" (2026, preprint and author's own method) — arXiv:2604.11790 — Enforces user confirmation rules at tool call boundaries. No numerical values in the abstract.
• "RTBAS: Defending LLM Agents Against Injection and Leakage" (2025, preprint and author's own method) — arXiv:2502.08966 — Automatically executes only safe calls via information flow tracking. Blocked all target attacks on one test set with a 2% loss in usefulness.
• "Architecting Secure AI Agents: System-Level Defenses" (2026, preprint and position paper) — arXiv:2603.30016 — Design that restricts what models can see and decide, requiring human intervention when ambiguous. Existing test sets can create a false sense of security.
• "Measuring Indirect Prompt Injection in Web Agents" (2026, preprint) — DOI: 10.2139/ssrn.7276838 — Synthesized up to 2026-08-01, splitting measurements into eight branches. Model-level attack success rates—including very low ones—do not equate to a reliable agent.