Containing the Simulated Adversary: Safety for Autonomous Attack Simulation

Breach-and-attack simulation is quietly changing shape. For years it meant scripted playbooks: a human picked the techniques, a framework detonated them, and someone read the report. The new version is an agent that reasons over MITRE ATT&CK, selects techniques adaptively, chains them like a real campaign, and runs them through real offensive tooling, on its own. The capability is genuinely useful, because an autonomous adversary can walk a campaign technique by technique and measure, at each step, whether your defenses actually notice.

It is also a live offensive capability with the human taken out of the inner loop. The offensive frameworks that do the detonating already exist and already work; that was never the hard part. The hard part, the part that decides whether you can run this thing continuously and unattended instead of once with everyone watching, is containment. Can the attack be run over and over, without supervision, without ever escaping its box? That question turns out to have a precise engineering answer, and the answer doubles as the reason the tool's measurements can be trusted at all.

The human was the containment

A manual purple-team exercise is contained by the person running it. The operator chooses the target, eyeballs each action before it fires, and stops the moment something looks wrong. That judgment lives in the inner loop, and automation removes it exactly there, at machine speed and machine scale. Three risks follow immediately.

• Blast radius. A real technique executed against the wrong host is a real intrusion. An agent that mis-resolves a target, or a misconfiguration that quietly points it at production, does damage no report can undo.
• Escalation with no brake. Adversary emulation is adaptive on purpose; chaining tactics is the whole point. An autonomous chainer with no enforced ceiling will walk from benign discovery to destructive action, because that is what a real adversary does.
• Telemetry confusion. Simulated attacks emit the same signals as real ones. Unmarked, they trigger real incident response, poison your detection baselines, and, worst of all, hand a real intruder cover: "ignore it, it is just the BAS tool."

You cannot fix this by putting a human back in front of every action; that is just the manual process again, and it does not scale to an agent taking steps faster than anyone can review them. The alternative is to make the unsafe actions unrepresentable: build the system so the dangerous thing cannot be expressed, not merely discouraged. That reframes the whole problem.

Threat-model the simulator itself

Security tools are usually modeled by what they defend against. An autonomous attack simulator has to be modeled the other way, by what it could do if it went wrong, because it is holding a loaded offensive capability. Enumerate the failure modes and design a guarantee against each:

None of these guards is novel on its own. Allowlists, default-deny, and dry-runs are old ideas. The claim worth making is that this specific set, enforced in code rather than asserted in a policy document, is enough to make autonomous emulation safe to run unattended, and that two of the six are less obvious than they look.

The invariants that hold it in the box

The first four are the familiar ones, made load-bearing by living in the execution path instead of a runbook. Each is written so its violation is an error the code raises, not a call an operator has to remember to make.

Target isolation is the innermost guard. Every executable step names a target, and the executor rejects any host not on the run's allowlist. The tool adapters re-assert the check, so a mistake in the governance layer alone cannot land a technique on an unintended host. A target that resolves outside the allowlist does not run; it raises.

Default-deny governance makes authorization a positive act. A technique-and-asset pairing that has not been explicitly allowed is denied. Silence is refusal, never permission, and the agent cannot route around the gate because the only path from a plan to a running technique passes through it. This inverts automation's usual failure mode, where a system does whatever it can unless told to stop. Here it does nothing unless told it may.

Dry-run first means every technique produces a plan before it produces an effect. In simulation mode the plan is the output: a full campaign is reasoned and scored with nothing detonated. Promoting a step to live fire requires a recorded approval tied to that plan, so "who authorized this action, against this target, at this time" always has an answer. A live step with no decision behind it is not a violation caught later; it is a state the runner will not enter.

The kill switch aborts all in-flight steps at the next checkpoint. The property that matters is not that a stop exists, everything can be killed, but that the stop is clean: the loop checks for it between steps and hands control to teardown instead of leaving work half-done. An adaptive chainer with no brake is the most dangerous configuration of an autonomous attacker, and this is the brake.

The two that are worth stealing

The remaining two invariants are the ones I would carry into any system that generates attack-shaped activity, not just a BAS engine.

Signed simulation markers. Every telemetry event the simulator emits carries a keyed cryptographic tag, an HMAC over the event content under a per-engagement secret. It does two jobs at once. The first is deconfliction: the correlation layer, and the defender, can tell a simulated event from a real one and attribute it to a specific engagement, so a simulation never kicks off a real incident response and never quietly corrupts a detection baseline. The second is anti-forgery: because the marker is keyed, merely observing a simulated event does not let anything mint new "simulated" events. That closes the ugliest abuse of an unmarked simulator, a real intruder generating events dressed in the simulation's clothes so responders wave them through.

Two honesties keep the marker from lying to you. The key is per-engagement and has to be rotated before you ever wire this into a production SIEM, because a marker is only as trustworthy as the secrecy of its key, and a shared or leaked development key is worse than no marker at all: it actively invites the forgery it was meant to prevent. And the marker asserts provenance, not benignity. It says "this came from the simulator," not "this was harmless." Keep those two claims separate.

Ephemeral digital twins. The invariants above bound what runs and where the signal goes. Twins bound what can be harmed. Instead of detonating against the real asset, the harness provisions a disposable stand-in for the affected asset class, instruments it, runs the technique, captures the result, and destroys it. The target of a real detonation is therefore a thing whose entire purpose is to be thrown away. Two properties make it a containment boundary and not just a test box. A parity gate: a twin that does not materially resemble the asset it stands for is measuring the wrong thing, so the fabric refuses to score a run whose twin fails parity, producing no verdict rather than a misleading one. And teardown as a guarantee, not a best effort: destroy is a step in the orchestration that runs even when the run fails or is killed, so a host that has had a real technique detonated on it never outlives the run that created it. That guarantee is what lets destructive or self-propagating techniques be exercised at all: their blast radius is the twin, and the twin is guaranteed to cease to exist.

Containment and honesty are the same property

Here is the part that makes all of this more than a compliance story. The properties that keep the simulated adversary contained are the same ones that make its measurements trustworthy.

A verdict is only honest if the thing observing the attack is independent of the thing executing it. A tool that grades its own homework reports success. So the target's real behavior is recorded by a sensor separate from the executor, and that observed telemetry is correlated against real detection engines rather than the offensive tool's self-report. The signed markers are what keep that correlation clean: because every synthetic event is attributable, the correlation layer reasons about simulated and real signals without conflating them, and the same marking that stops a simulated event from triggering a real response also stops it from being silently counted as a real detection.

With execution contained and observation independent, the coverage measurement falls out almost for free. Each technique resolves under a fixed precedence, prevented over detected over logged over missed, so the strongest outcome that actually occurred is the one reported and a merely-logged technique is never credited as detected. That precedence surfaces the failure class shallower tools hide: the logged-not-detected gap, where the telemetry to catch a technique exists but no rule fires on it. It is the same illusion I wrote about from the defender's side in deployed is not detected: coverage you believe you have and do not. A contained, independently observed simulation is what lets you measure that gap repeatedly and trust the number, instead of running one careful exercise a year and calling it coverage.

Doing this responsibly

A few choices keep an autonomous attack simulator on the right side of the line between defensive research and offensive tooling, and they are worth stating as requirements rather than aspirations.

• Simulation by default. The core should plan and score a full campaign with nothing detonated. Live execution is opt-in, gated, and only ever pointed at a twin or a scoped, approved target. The safe path is the default; the dangerous path is the one you have to deliberately unlock.
• Invariants in code, and tested. Target isolation, default-deny, dry-run gating, kill-switch behavior, and marker signing should be exercised by tests that pass without touching a network, so the safety properties are verified the way the rest of the system is, not left to documentation.
• Method, not engine. The reusable contribution here is the design discipline, the specific set of guarantees and why each is load-bearing. Autonomous breach-and-attack simulation is for authorized testing only, against environments you own or are explicitly permitted to test. The architecture is defensive in intent and in effect: its entire purpose is to keep a live capability contained.

The hard part of autonomous attack simulation was never making an agent attack. It is making an attacking agent safe to run without a human watching, and that reduces to a small set of invariants enforced in code and bounded by targets that are guaranteed to be destroyed. Get that substrate right and you get two things at once: a simulated adversary that cannot escape, and a detection-coverage map you can actually believe. They are, it turns out, the same achievement.

This post is the practitioner-facing version of a research whitepaper I wrote for AEGIS Labs, Containing the Simulated Adversary: A Safety Architecture for Autonomous Breach-and-Attack Simulation on Ephemeral Digital Twins, which describes the safety architecture and the reasoning behind each invariant. It is method, not implementation, and contains no offensive tradecraft. MITRE ATT&CK is a trademark of The MITRE Corporation.