AI Security Testing
Service · AI security testing
I break your AI before someone else does.
If you’ve put a chatbot in front of customers, or given an AI agent access to your tools, money, or data, nobody has tried to turn it against you under controlled conditions. I do, then I hand you what it did and the fix — nothing held back for a second call. The reason I find things others miss is below: the damage comes from the whole chain of steps, and most checks only watch one step at a time.
What I test for
Three ways an AI system does something it should have refused, each one invisible to a check that only watches a single step. For each, the thing I test and the fix it points to.
Failure class A
A safety value becomes the lever
A model can be led to treat the harmful action as the responsible one. The attacker never asks it to ignore a guardrail. The guardrail’s own reasoning supplies the excuse, so a stronger safety instinct gives the attacker more to work with, not less.
Defense · Treat the strength of a safety disposition as an attack surface to test, not only as protection. Measure whether protective reasoning can be pointed at an action, not only whether the model refuses one.
Failure class B
A big action split into ordinary steps
One action that would matter if you saw it whole is split up, so no single decision point has enough to recognize it. Separation of concerns, compartmentalized context, handoffs between subagents. The same structure that makes a system clean makes it blind.
Defense · Where something came from and what it is for have to carry across steps the way the effects do. A check that sees only one slice — a file, a summary, one tool call, memory, or nothing — cannot judge the whole. A shared record that carries state across steps is the missing piece.
Failure class C
A premise installed early, carried forward
A premise is set in one session, saved through memory or outside state, and acts on a later session that never sees or questions where it came from. The dangerous moment is when text from an untrusted source turns into something the agent just believes, with no mark of its origin.
Defense · Preserve origin and trust level across the memory boundary. Memory content must not be promoted to a higher trust role than the live operator.
Proof · operations, not party tricks
The clever one-message jailbreak is a party trick, and the bounty and CVE process is built to score it: one reproducible step, one boundary, one clean fix. Real attacks are operations — patient, ordinary-looking, assembled from context the system picks up along the way. That is what I build and prove.
I took a working exfiltration chain to a vendor program and they closed it as not-applicable, because nothing a traditional single boundary protects was shown to fail. They disputed the category. The behavior happened anyway. Here is what I have made AI systems do, under controlled conditions on endpoints I own.
An agent doing authorized work followed an instruction from a source it had no business trusting and executed a transfer to an attacker’s address. Its own automated approval check signed off. Not one step in the chain broke a rule the agent could see.
A coding agent, set up through nothing but ordinary project files, shipped the environment file’s secrets to an endpoint I controlled. Four runs, four clean exfiltrations. Transcripts and capture logs on file.
Two sessions, run apart. The first sealed a record and sent nothing. The second shipped it without ever seeing inside, and my endpoint rebuilt the entire configuration. No single decision point ever held enough to catch what was happening.
Untrusted content got folded into an agent’s memory. I deleted the original source. A fresh, unrelated session loaded that memory and acted on the planted instruction as its own belief — the poison outlived the thing that carried it in.
Same agent, same task, instructions that granted nothing. Clean memory, it refused every time. One planted note claiming prior approval, and it acted every time — then told the user it had their permission, 6 for 6. Its own judgment failed. Only the sandbox stopped the connection.
How far I take one finding
A code fix only proves the vendor changed something. I prove the old version actually behaved as the weakness.
On idle device firmware I found a safety check a newer build added and an older one lacked. Most people stop at the diff. I loaded each version on my own box and sent the same bad request. The old firmware took it three times out of three. The fixed one refused it three out of three. Same machine, same request, only the firmware changed.
It was already patched by the time I got there, so no live bug and nothing to collect. I did it for the method, not the money — proving the old behavior is the part everyone skips.
Where it does get scored, it scores
I broke frontier models in an indirect-prompt-injection challenge backed by OpenAI, Anthropic, Google DeepMind, Meta, Amazon, and the UK AI Safety Institute.
Their systems, their challenge, their bounty. Not my own scorecard.
A transparent researcher keeps the alarming context in one place. A patient adversary distributes it across ordinary interactions.
So a control that judges local surface taxes the party best able to reason about the risk, and lets the actor who made intent nonlocal pass under it. The limiting factor is where the control is placed, not how accurate it is. Placement is the finding.
How I work
Isolated rigs, one variable, controls stated.
- Each test runs in a fresh isolated environment, one variable changed against a blank control, so the cause is the thing under test and nothing else.
- Contaminated batches get thrown out, not cleaned up. One run that saw prior artifacts invalidated a whole batch, so the batch was excluded and marked as contamination evidence.
- What was observed stays separate from what I think it means. Every claim carries the limit that bounds it, and withdrawn results are reported as withdrawn.
- Negative results count. A current-turn refusal beating a planted note, a benign control that also fired, an organic pipeline that would not preserve a forged claim — all reported, because they draw the real boundary.
Disclosure
Aggregate results, conditions, controls, failure definitions, negative results, sanitized transcripts, endpoint-side confirmation, limitations, and defensive implications are publishable. That is what this page is.
Exact prompts, reusable framing sequences, working chains, infrastructure, and pretexts are withheld for coordinated disclosure. A full reproduction path would add little scientific value while lowering the cost of misuse.
Have a chatbot or an AI agent you want tested before it costs you something? Or run a safety program and want the bounded write-ups and receipts? Reach out.
Get your system tested →