Category 9 of 12 · Controls: Testing before and after deployment

Red teaming and evaluation

Automated and managed adversarial testing of models and agents for injection, jailbreaks, data leakage, and unsafe tool use, before release and continuously after.

Mitigates: ASI01 Agent Goal Hijack, ASI02 Tool Misuse and Exploitation, ASI05 Unexpected Code Execution, LLM01 Prompt Injection · MITRE ATLAS AML.T0051, AML.T0054

Vendors, by signal score

• #1
  PromptfooBeing acquired by OpenAI

  Open-source red teaming, static scanning, and evaluation for LLM applications and agents, with plugins for tool misuse, MCP, memory poisoning, and prompt injection. OpenAI agreed to acquire it in March 2026 and committed to keep it open source.

  31
• #2
  Mindgard

  Automated AI red teaming that emulates attacker behavior against models, agents, and the tools, APIs, and data sources they connect to, with findings mapped to the OWASP LLM Top 10 and MITRE ATLAS.

  30
• #3
  Pillar Security

  AI agent security platform: RedGraph runs multi-turn adversarial attacks against agents to test tool orchestration and permission escalation, alongside AI discovery and posture, runtime guardrails, and governance.

  10
• #4
  SPLXAcquired by Zscaler

  Automated AI red teaming with thousands of attack simulations, plus runtime guardrails, prompt hardening, and discovery of models, workflows, and MCP servers. Acquired by Zscaler in 2025 and now sold as Zscaler AI Red Teaming.

  10

Ranked within this category only. How the score works.

Also covers this category

• CalypsoAIAcquired by F5

  Runtime guardrails for AI inference: inspects prompts, outputs, and agent tool calls, blocks unauthorized tool calls, and logs every enforcement decision with its reasoning. Acquired by F5 in 2025; now sold as F5 AI Guardrails.

  19
• CiscoPlatform suite

  Cisco AI Defense, which incorporates the Robust Intelligence acquisition, adds agent red teaming, runtime guardrails, and MCP server scanning; Duo registers agents as identities with human owners, and Secure Access enforces MCP policy. Cisco also acquired the NHI vendor Astrix Security.

  60
• FortinetPlatform suite

  FortiAIGate guards LLM traffic against prompt injection, data leakage, and model poisoning; the Virtue AI acquisition adds agentic red teaming, agent and MCP tool discovery and scanning, blocking of malicious tool calls, and real-time guardrails.

  45
• HiddenLayer

  AI security platform with runtime detection and response for models and agents: evaluates model requests, responses, and tool calls against policy and can block or redact them, alongside AI discovery, supply chain scanning, and attack simulation.

  47
• LakeraAcquired by Check Point

  Runtime guardrails for LLM applications and agents (Lakera Guard) that screen prompts, tool responses, and tool descriptions for injection and data leakage, plus AI red teaming. Acquired by Check Point in 2025.

  38
• Palo Alto NetworksPlatform suite

  Prisma AIRS: AI runtime security, agent security, AI red teaming, model scanning, posture management, and an AI gateway, built partly from acquisitions of Protect AI, Koi (agentic endpoint security), and Portkey (now Prisma AIRS AI Gateway), with CyberArk as its identity security platform.

  65
• SentinelOnePlatform suite

  AI security built on the Prompt Security acquisition: runtime protection for employee and homegrown AI, discovery of shadow MCP servers and agents, Prompt AI Agent Security (preview) for agent governance, and Prompt AI Red Teaming.

  55
• Straiker

  Agent security platform: Discover AI inventories agents and MCP servers, Ascend AI red-teams them, and Defend AI blocks prompt injection, data exfiltration, and destructive actions at runtime, with a kill switch to cut off or shut down a rogue agent.

  36
• ZscalerPlatform suite

  Zscaler AI Protect extends the Zero Trust Exchange to AI: secure access to GenAI apps, AI asset discovery, and runtime guardrails, plus AI Red Teaming from the SPLX acquisition, an AI Broker with an agent registry for MCP and A2A traffic, and an AI Access Graph built on Symmetry Systems.

  70

Questions to ask a vendor

1. Does it test multi-step agent behaviour and tool use, not only single prompts?
2. Can tests run in CI on every model or prompt change?
3. Are findings mapped to OWASP and MITRE ATLAS IDs?

Go deeper

• Red team an LLM (guide)
• LLM red teaming shortlist
• AI red teaming shortlist

FAQ

Is open source red teaming enough?

Open source harnesses such as garak, PyRIT, and promptfoo cover a lot of ground and are a good start. Commercial tools add managed attack libraries, agent-level scenarios, and reporting.

How often should agents be red teamed?

On every material change to the model, prompts, tools, or permissions, which in practice means continuously in CI.