COAX: automated red-teaming for AI agents
COAX attacks an AI agent the way an attacker would, indirect prompt injection, tool abuse, data exfiltration, adaptive multi-turn attacks, and verifies with deterministic oracles when an attack succeeds, producing a robustness report mapped to the OWASP LLM Top 10. It is the tool we use to stress-test an agent before giving it access to your data and your systems.
Star on GitHub
A defensive red-teaming tool: it attacks an AI agent with known attacker techniques and tells you, with deterministic proof, where it is vulnerable, before someone else finds out in production.
The problem
A chatbot that answers questions is one thing. An agent that reads emails, queries your CRM, calls tools and makes decisions is another: it has a completely new attack surface. The threats are no longer just "make the bot swear", but:
- Indirect prompt injection, malicious instructions hidden in a web page, a document or an email that the agent ingests;
- Tool abuse / excessive agency, convincing the agent to call a tool it should not, with dangerous arguments;
- Data exfiltration, leaking a secret from the system prompt or a "private" field.
These are exactly the entries of the OWASP LLM Top 10. And most teams discover them in production.
What COAX does
COAX automates the work of an attacker and, crucially, verifies the results deterministically, not "by feel":
- It attacks through a single typed interface, so it can test any agent (HTTP, OpenAI-compatible APIs, web agents via Playwright, local models via Ollama) without the attack code knowing which one it is hitting.
- Deterministic oracles that tell you with certainty whether an attack succeeded: a canary (a planted secret: if it appears in the output, it leaked), a tool-trace (was a forbidden tool called?), a policy with an LLM judge and a deterministic fallback.
- Adaptive attacker in a closed loop, with bounded budget and cost, refining attacks based on the responses.
- Reports in Markdown/HTML with Attack Success Rate per family, severity, OWASP mapping, a reproducible transcript and concrete remediation for each successful attack.
The results
Against the deliberately vulnerable decoy agent, the report is clear-cut, and false positives are zero:
| Attack family | Attack Success Rate |
|---|---|
| Indirect injection | 100% (20/20) |
| Tool abuse | 100% (4/4) |
| Direct override | 40% (2/5) |
| Jailbreak | 40% (2/5) |
| Obfuscation | 40% (2/5) |
| Oracles, false positives (benign corpus) | 0% |
105 tests, fully offline, with CI across multiple Node versions. The thing that matters: when COAX says "attack succeeded", it is true, because a deterministic oracle proves it, not a vague judgement.
Responsible use
COAX is a defensive tool, for testing systems you own or are authorised to test. Any non-local target requires explicit authorisation, and the tool uses only known and published families of techniques to measure robustness, it does not synthesise new exploits. The value is the measurement, not the attack.
Why it matters for your company
If you are thinking of putting an AI agent in contact with real data and processes, this is the question that counts: is it safe? COAX is how we answer with facts.
- Security audit of agents before production. We attack the agent the way an adversary would and hand you a report with vulnerabilities, severity and how to close them.
- Stronger compliance. Prompt injection and data exfiltration are concrete risks under GDPR and the AI Act: proving you have tested them is an asset, not a cost.
- Trust earned, not promised. An agent that has passed serious red-teaming is an agent you can build on.
Learn more
Code, adapters and method are open source (MIT).
Rayo Consulting research · github.com/dylanpatriarchi/coax