---
title: "Which LLMs resist prompt injection as agents: 5 models, 1,470 runs"
canonical: https://pharosone.ai/knowledge/which-llms-resist-prompt-injection-in-agents
publisher: PharosOne (Agentic Mind, Inc.)
updated: 2026-10-23
license: "Open for reuse: crawling, copying, quoting, summarising and model training permitted without asking; citation of the title and canonical URL requested (see https://pharosone.ai/terms#reuse)"
cite_as: "\"Which LLMs resist prompt injection as agents: 5 models, 1,470 runs\". PharosOne, 2026-10-23. https://pharosone.ai/knowledge/which-llms-resist-prompt-injection-in-agents"
---

# Which LLMs resist prompt injection as agents: 5 models, 1,470 runs

> October 2026: Claude Haiku 5.5 and GPT-6 Luna resisted best (8.8%, 11.2%, not separable), Qwen3.5-9B worst (25.2%). A third of runs unlabelled; bounds inside.

URL: https://pharosone.ai/knowledge/which-llms-resist-prompt-injection-in-agents · Category: Reports · Published: 2026-10-22 · Updated: 2026-10-23 · Author: PharosOne Research · Reviewed by: PharosOne editorial

*PharosOne sells security testing of AI agents and wrote this article. We do not build or sell any of the models tested.*

**In short.** In 1,470 attacked agent runs on 147 benchmark-derived scenarios, collected on 18–20 October 2026, Claude Haiku 5.5 had the lowest prompt injection success rate of five models, 8.8% [6.1, 12.6], followed by GPT-6 Luna at 11.2% [8.1, 15.3]; the two cannot be separated on this sample. Qwen3.5-9B was the least robust at 25.2% [20.6, 30.4]. A defensive system prompt cut Gemma 4 31B's rate from 19.7% to 7.5% and left no model at zero. Each rate is witnessed successes divided by all planned runs. 504 of the 1,470 runs (34.3%) could not be labelled and stay in the denominator, so 1 minus the rate is not a resistance rate. Among labelled runs only, the rates run from 13.1% (Claude Haiku 5.5) to 35.6% (Qwen3.5-9B) in the same order; if every unknown run had been a success they would run from 41.5% (Claude Haiku 5.5) to 55.4% (GLM-4.7-Flash). The run was not reviewed by our referee.

## Key facts

- Updated: 23 October 2026. Runs collected 18–20 October 2026.
- Sample: 147 scenario units from public agent benchmarks. Each unit ran on all five models, once without and once with a defensive system prompt: 1,470 attacked runs, 294 per model.
- Models: Claude Haiku 5.5, GPT-6 Luna, Gemma 4 31B IT, GLM-4.7-Flash and Qwen3.5-9B, each through pinned providers on OpenRouter.
- Order, lowest attack success first: Claude Haiku 5.5, GPT-6 Luna, Gemma 4 31B, GLM-4.7-Flash, Qwen3.5-9B. The order was the same in an earlier run on the same scenarios.
- Unlabelled runs: 504 of 1,470 (34.3%), between 86 and 114 per model. They stay in the denominator, so each rate is a lower bound of witnessed success and 1 minus the rate is not a resistance rate; the rate among labelled runs is in the table under the limitations.
- Intervals: 95% Wilson intervals for each rate; the probability of being the most robust model comes from a cluster bootstrap over the 147 units (4,000 resamples).
- Weak spot of the strongest models: scenarios where the attack sits in an MCP server's tool descriptions.
- Next step: measure your own agent with the [Pharos Framework](/knowledge/pharos-framework) or an [AI-run pentest](/knowledge/ai-agent-security-assessment).

## What counts as a successful attack?

**Attack success rate (ASR).** The share of attacked runs in which the agent carried out the attacker's action, witnessed by the simulated environment: a transfer recorded in its state, a request received at a sink the attacker controls, a protected value repeated in the reply. An agent that tried the action and failed does not count; the difference is explained in [Attempt versus execution](/knowledge/attempt-versus-execution). The denominator is every planned attacked run, including runs that could not be labelled.

**Unknown.** A run whose outcome the harness could not establish, for example because it was cut off at the step limit or the harness could not confirm that the agent saw the attacker's content. The rate is witnessed successes divided by planned runs; unknown runs sit in the denominator, so 1 minus the rate is not a resistance rate. Each model's split into success, confirmed non-success and unknown, and its rate among labelled runs, are in the table under the limitations.

**Defensive system prompt.** A security section added to the agent's system prompt at one of two levels: *standard*, written the way a careful company would write it, and *hardened*, an enumerated policy plus marking of tool output as data in the style of spotlighting ([Hines et al., 2024](https://arxiv.org/abs/2403.14720)). Each level has three wordings so that no single text drives the result.

**Paired unit.** One scenario run on all five models, each with and without its assigned defence prompt: ten runs that differ in the model, its provider settings and the prompt. The defence level and wording were assigned per model by hash, so the defended runs of different models do not always share a level.

**P(lowest rate).** The share of bootstrap resamples of the 147 units in which a model had the lowest attack success rate of the five.

## Which model resisted prompt injection best?

| Rank | Model (maker) | Attack success rate [95% CI] | Difference from Claude Haiku 5.5, 95% CI (pp) | P(lowest rate) | Earlier run, same scenarios [95% CI] |
|---:|---|---|---|---:|---|
| 1 | Claude Haiku 5.5 (Anthropic, closed) | 26/294 = 8.8% [6.1, 12.6] | reference | 0.893 | 23/294 = 7.8% [5.3, 11.5] |
| 2 | GPT-6 Luna (OpenAI, closed) | 33/294 = 11.2% [8.1, 15.3] | [−1.0, +5.8] | 0.079 | 24/294 = 8.2% [5.5, 11.9] |
| 3 | Gemma 4 31B IT (Google, open weights) | 40/294 = 13.6% [10.2, 18.0] | [−0.3, +9.9] | 0.024 | 39/294 = 13.3% [9.9, 17.6] |
| 4 | GLM-4.7-Flash (Z.ai, open weights) | 49/294 = 16.7% [12.8, 21.4] | [+1.7, +13.9] | 0.003 | 44/294 = 15.0% [11.3, 19.5] |
| 5 | Qwen3.5-9B (Alibaba Qwen, open weights) | 74/294 = 25.2% [20.6, 30.4] | [+10.5, +22.4] | 0.000 | 74/294 = 25.2% [20.6, 30.4] |

Rates pool both prompt arms. A difference interval that includes zero means the sample cannot tell the two models apart.

Claude Haiku 5.5 is clearly more robust than GLM-4.7-Flash and Qwen3.5-9B. Its lead over Gemma 4 31B is borderline, and its lead over GPT-6 Luna is within the noise. In the earlier run on the same scenarios the order was identical, with P(lowest rate) of 0.594 for Haiku and 0.402 for Luna. Between the two runs Luna's rate rose from 8.2% to 11.2%, almost entirely in the MCP scenarios; the earlier run left 60.5% of runs unlabelled against 34.3% here, so the two runs are not labelled alike, and we do not read the change as an effect.

## Where did the strongest models still fail?

In the MCP scenarios, where the attack sits in the tool descriptions an MCP server sends to the agent ([Poisoned tool descriptions and memory](/knowledge/poisoned-tool-descriptions-and-memory) explains the class). Each model had 64 runs in these cells. GPT-6 Luna carried out the attack in 30 of them, Qwen3.5-9B in 26, Claude Haiku 5.5 in 21, GLM-4.7-Flash and Gemma 4 31B in 14 each. On these cells the overall ranking does not hold.

These cells account for 21 of Claude Haiku 5.5's 26 successful attacks and 30 of GPT-6 Luna's 33. In the 80 native AgentDojo runs per model, where the injected text arrives inside tool results such as emails and documents, no success was witnessed for either closed model: 0 of 80 each, a 95% upper bound of about 4.6%. None of Claude Haiku 5.5's 80 runs there was unknown, and 3 of GPT-6 Luna's were. The other per-family counts are small and we report them without intervals.

## Does a defensive system prompt help?

| Model | No defence prompt [95% CI] | With defence prompt, same 147 units [95% CI] | Standard level only | Hardened level only |
|---|---|---|---|---|
| Claude Haiku 5.5 | 14/147 = 9.5% [5.8, 15.4] | 12/147 = 8.2% [4.7, 13.7] | 7/78 = 9.0% [4.4, 17.4] | 5/69 = 7.2% [3.1, 15.9] |
| GPT-6 Luna | 21/147 = 14.3% [9.5, 20.9] | 12/147 = 8.2% [4.7, 13.7] | 5/68 = 7.4% [3.2, 16.1] | 7/79 = 8.9% [4.4, 17.2] |
| Gemma 4 31B IT | 29/147 = 19.7% [14.1, 26.9] | 11/147 = 7.5% [4.2, 12.9] | 8/72 = 11.1% [5.7, 20.4] | 3/75 = 4.0% [1.4, 11.1] |
| GLM-4.7-Flash | 25/147 = 17.0% [11.8, 23.9] | 24/147 = 16.3% [11.2, 23.1] | 12/80 = 15.0% [8.8, 24.4] | 12/67 = 17.9% [10.6, 28.7] |
| Qwen3.5-9B | 41/147 = 27.9% [21.3, 35.6] | 33/147 = 22.4% [16.5, 29.8] | 19/73 = 26.0% [17.3, 37.1] | 14/74 = 18.9% [11.6, 29.3] |

It helped one model clearly. Gemma 4 31B fell from 19.7% to 7.5%, and the two intervals do not overlap. GPT-6 Luna's point estimate fell from 14.3% to 8.2% with overlapping intervals. For Claude Haiku 5.5, GLM-4.7-Flash and Qwen3.5-9B the change was small and the intervals overlap widely. The intervals are per arm; we did not compute a paired test of this contrast, so overlapping intervals read as "no clear effect on this sample".

With a defence prompt, the top three models are tied: Gemma 4 31B 11, GPT-6 Luna 12 and Claude Haiku 5.5 12 successful attacks out of the same 147 units (P(lowest rate) 0.490, 0.215 and 0.295). Without one, Claude Haiku 5.5 leads with P(lowest rate) 0.965, and Gemma 4 31B's rate is about twice Haiku's (difference [+2.7, +17.7] pp). A defence prompt narrowed the gap between weaker and stronger models, and no model reached zero with it. Adaptive attackers, which this test did not use, do better still: a 2025 study bypassed 12 published defences with attack success above 90% for most of them ([Nasr et al., 2025](https://arxiv.org/abs/2510.09023)).

The hardened column needs care. Gemma 4 31B has the lowest hardened rate (4.0% against Haiku's 7.2%), but each model's defence level was assigned per scenario by hash, so the hardened runs of different models cover different scenario mixes, and the difference interval includes zero. Gemma also looped and hit the step limit far more often than Haiku (45 loops and 43 truncated runs against 5 and 6, out of 294), which leaves more of its runs unlabelled.

## What do these numbers not show?

- **Live traffic.** The tasks come from public benchmarks with simulated tools and services. Some MCP web fixtures are visibly synthetic, and Claude Haiku 5.5 often declined to act on placeholder data, which leaves the run unlabelled.
- **Adaptive attackers.** The attacks were fixed in advance and did not adapt to the model's replies. Every attack in the set has a visible goal: a value the attacker planted, such as an address or an account, has to end up in the agent's action. Attacks without such a value, where the agent is talked into misusing permissions it already has, are not measured here.
- **Unlabelled runs.** 504 of the 1,470 runs (34.3%) ended unknown. They stay in the denominator without being counted as successes, so the ranking table reports witnessed successes. The table after this list gives each model's split, its rate among labelled runs and its all-success bound. The all-success bound is an extreme case for every model at once; if unknown runs hid successes at different rates for different models, the order could change, and this sample cannot rule that out.
- **Run-to-run variation.** Of each model's successful attacks in this run, the share also witnessed in the same scenario in the earlier run was: Claude Haiku 5.5 15 of 26, GPT-6 Luna 19 of 33, Gemma 4 31B 28 of 40, GLM-4.7-Flash 31 of 49, Qwen3.5-9B 57 of 74 (58–77%). Provider nondeterminism and the harness change both contribute, and the earlier run labelled fewer runs, so we give no significance test for the difference between the runs.
- **Other task types and other models.** Repository-management MCP scenarios were excluded, and coverage of the AgentDyn suites was cut by the cost cap. The open-weight models are small and mid-sized; larger versions and other model families were not tested.
- **A population estimate.** The 147 units were chosen in priority order, favouring scenario families where some model had succeeded in the earlier run. Over all 464 attacked runs of that earlier run, Claude Haiku 5.5's rate was 5.0% [3.3, 7.3]. Both the rates and the order depend on the mix: on the 64 MCP tool-description runs per model the order changes, and GPT-6 Luna had the most successes of the five.
- **Label edge cases.** A refusal that repeats a protected value counts as a success. Whether that is a real leak depends on the scenario's goal: sometimes repeating the value is the leak, sometimes it is harmless. We did not re-adjudicate these labels, so their effect on the ranking is not measured; they raise the rate of models that refuse at length, Qwen3.5-9B among them. Runs cut off before they ended, mostly at the step limit each benchmark environment sets: Qwen3.5-9B 60 of 294 (20.4%), GLM-4.7-Flash 60 (20.4%), Gemma 4 31B 43 (14.6%), GPT-6 Luna 21 (7.1%), Claude Haiku 5.5 6 (2.0%), against the 5% we aim for. A success witnessed before the cut still counts, so fewer of these runs end unknown: 27, 42, 33, 7 and 2 respectively.

**Unlabelled runs by model.** Each model's 294 runs split into witnessed success, confirmed non-success (the harness saw the run end without the attacker's action, including runs where the agent tried and failed) and unknown, with three rates computed from that split.

| Model | Success | Confirmed non-success | Unknown | Rate among labelled runs [95% CI] | Rate if all unknown were successes | Rate among runs with confirmed exposure |
|---|---:|---:|---:|---|---:|---|
| Claude Haiku 5.5 | 26 (8.8%) | 172 (58.5%) | 96 (32.7%) | 26/198 = 13.1% [9.1, 18.5] | 41.5% | 23/255 = 9.0% |
| GPT-6 Luna | 33 (11.2%) | 158 (53.7%) | 103 (35.0%) | 33/191 = 17.3% [12.6, 23.3] | 46.3% | 33/251 = 13.1% |
| Gemma 4 31B IT | 40 (13.6%) | 149 (50.7%) | 105 (35.7%) | 40/189 = 21.2% [15.9, 27.5] | 49.3% | 40/260 = 15.4% |
| GLM-4.7-Flash | 49 (16.7%) | 131 (44.6%) | 114 (38.8%) | 49/180 = 27.2% [21.2, 34.1] | 55.4% | 49/245 = 20.0% |
| Qwen3.5-9B | 74 (25.2%) | 134 (45.6%) | 86 (29.3%) | 74/208 = 35.6% [29.4, 42.3] | 54.4% | 68/252 = 27.0% |

The last column counts successes in runs where the harness's exact-text check confirmed that the attacker's content reached the agent, over all such runs. Three of Claude Haiku 5.5's 26 successes and six of Qwen3.5-9B's 74 happened in runs where that check found no match: the environment witnessed the attacker's action, but the content reached the agent in a form the exact match missed. These runs count in the overall rate and not in the last column.

Most of Claude Haiku 5.5's unknown runs (59 of 96) are clean endings the harness could not label, such as declining to act on visibly placeholder data. GLM-4.7-Flash's are split mainly between runs cut off at the step limit (42) and runs where the harness could not confirm the agent saw the attacker's content (41).

## What should you do with this when you pick a model?

1. **Shortlist with it.** On this sample the closed models and Gemma with a defence prompt are close; Qwen3.5-9B and GLM-4.7-Flash were carried into the attacker's action noticeably more often.
2. **Test the agent you will ship.** The weakest cells for the most robust models were MCP tool descriptions, a property of the servers you connect, which no model ranking captures. Run the free, open-source [Pharos Framework](/knowledge/pharos-framework) against your agent; it reports an attack success rate with a Wilson interval per probe.
3. **Measure your defence prompt instead of assuming it works.** Its effect ranged from cutting the rate by more than half to no visible change, depending on the model.
4. **Ask for intervals.** A rate without its sample size and interval cannot be compared across models or vendors.

For a measured result on your own deployment, the [AI-run pentest](/ai-run-pentest) puts PharosOne's own AI agents against one agent with its tools, data sources and memory; the attacks adapt to the agent's replies, and an engineer reviews the findings and signs the report ([details](/knowledge/ai-agent-security-assessment)).

## Method

- **Scenarios.** 147 units drawn from a larger PharosOne run on public benchmarks, by source family: AgentDojo 600 runs ([Debenedetti et al., 2024](https://arxiv.org/abs/2406.13352)), 200 of them with PharosOne's own URL-rendering tasks on its workspace suite; MCP servers 360 ([MCP-SafetyBench](https://github.com/xjzzzzzzzz/MCPSafety), 40 of them with our rendering tasks); competition-written cases from [IPI Arena](https://github.com/grayswansecurity/ipi_arena_os) and related sets 320; [AgentDyn](https://github.com/SaFo-Lab/AgentDyn) 90; web pages 60; [NVIDIA Nemotron IPI](https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Indirect-Prompt-Injection-v1) 40. Each environment ran in an isolated container with egress only to the model API. Clean runs without an attack were not part of this screen.
- **Models and providers.** `anthropic/claude-haiku-5.5` (Google Vertex AI and Amazon Bedrock), `openai/gpt-6-luna` (Azure), `google/gemma-4-31b-it` (Novita, Crusoe), `z-ai/glm-4.7-flash` (Novita), `qwen/qwen3.5-9b` (DeepInfra), through OpenRouter with fallbacks disabled. The open-weight models ran in bf16 at temperature 0 with seed 0. The two closed models ran at their provider's default temperature, because their endpoints on this route do not accept the setting. Every run records the served provider and model id.
- **Dates.** Earlier run: 18–19 October 2026. Screening run reported here: 19–20 October 2026, on a revised harness that records the outcome of more runs (unknown share 60.5% → 34.3% on the same scenarios).
- **Defence arms.** Each unit ran every model once without a defence prompt and once with one level (standard or hardened) and one of three wordings, assigned by hash.
- **Failures.** 30 of 1,470 runs failed on infrastructure (mostly empty provider completions) and were not retried; they count as unknown unless a successful attack was witnessed before the failure.
- **Statistics.** Wilson 95% intervals for each rate ([Wilson, 1927](https://doi.org/10.1080/01621459.1927.10502953)); cluster bootstrap over the 147 units, 4,000 resamples, for P(lowest rate) and differences between models (139 units for the hardened arm); exact McNemar tests between the two runs. The effect of the defence prompt was compared by per-arm intervals only; no paired test of that contrast was run.
- **Checks.** A random sample of 12 runs was read by hand; no label contradicted its trace. Twelve runs are too few to estimate label error per model.
- **Cost.** About $3.42 in model fees for the 1,470 runs.
- **Review status.** The screening run selected the target model for a later stage of a pre-registered experiment ([How we audit our own AI security numbers](/knowledge/what-we-retracted-and-why)). The run itself was not reviewed by our referee; the numbers are the run report's own, with its caveats kept.

## FAQ

### Which LLM is most resistant to prompt injection when it runs as an agent?

In our October 2026 test of five models on 1,470 attacked agent runs, Claude Haiku 5.5 had the lowest attack success rate, 8.8% [6.1, 12.6], and was the most robust model in 89.3% of bootstrap resamples. GPT-6 Luna, at 11.2% [8.1, 15.3], cannot be separated from it on this sample.

### Does a defensive system prompt stop prompt injection?

No model reached zero with one. On the same 147 scenarios, a defensive system prompt cut Gemma 4 31B from 19.7% to 7.5%; for Claude Haiku 5.5, GLM-4.7-Flash and Qwen3.5-9B the rates with and without the prompt had overlapping intervals. With the prompt, Gemma, GPT-6 Luna and Claude Haiku 5.5 were tied at 11, 12 and 12 successful attacks out of 147.

### Are closed models more robust than open-weight models?

In this sample the two closed models ranked first and second and Qwen3.5-9B ranked last at 25.2% [20.6, 30.4]. The open-weight models tested are small and mid-sized (9B and 31B parameters, and GLM-4.7-Flash), so the result says nothing about larger open-weight models.

### Where did the most robust models still fail?

Mostly in scenarios where the attack sits in an MCP server's tool descriptions. 21 of Claude Haiku 5.5's 26 successful attacks and 30 of GPT-6 Luna's 33 came from those cells. In their 80 native AgentDojo runs each, no success was witnessed for either model (0 of 80, 95% upper bound about 4.6%); 3 of GPT-6 Luna's 80 runs there were unknown.

### Can I use this ranking instead of testing my own agent?

No. The tasks come from public benchmarks with simulated tools, the attacks did not adapt to the model's replies, and a third of the runs could not be labelled. Your agent's rate depends on its tools, its system prompt and the data it reads, so it has to be measured on the agent itself.

## Sources

1. Target re-screen on the relaxed harness, 1,470 trajectories (internal run report, td35). PharosOne Research, 2026-10-20.
2. [AgentDojo. A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents](https://arxiv.org/abs/2406.13352). Debenedetti et al. arXiv 2406.13352, 2024-06-19.
3. [AgentDyn (benchmark repository)](https://github.com/SaFo-Lab/AgentDyn). SaFo-Lab, GitHub, 2026.
4. [MCPSafety, MCP-SafetyBench (benchmark repository)](https://github.com/xjzzzzzzzz/MCPSafety). GitHub, 2026.
5. [IPI Arena, open-source release](https://github.com/grayswansecurity/ipi_arena_os). Gray Swan, GitHub, 2026.
6. [Nemotron RL Agentic Indirect Prompt Injection v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Indirect-Prompt-Injection-v1). NVIDIA, Hugging Face, 2026-04.
7. [Defending Against Indirect Prompt Injection Attacks With Spotlighting](https://arxiv.org/abs/2403.14720). Hines et al. arXiv 2403.14720, 2024-03-20.
8. [The Attacker Moves Second. Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections](https://arxiv.org/abs/2510.09023). Nasr, Carlini, Sitawarin, Schulhoff, Hayes et al. arXiv 2510.09023, 2025-10-10.
9. [Probable Inference, the Law of Succession, and Statistical Inference](https://doi.org/10.1080/01621459.1927.10502953). Edwin B. Wilson, Journal of the American Statistical Association 22(158), 1927.

Source: PharosOne, "Which LLMs resist prompt injection as agents: 5 models, 1,470 runs", https://pharosone.ai/knowledge/which-llms-resist-prompt-injection-in-agents, updated 2026-10-23.

---

PharosOne is a product of Agentic Mind, Inc., an independent AI agent security company. Site map for agents: https://pharosone.ai/llms.txt