AIs Could Attack The Inference Engine To Escape Sandboxes | James Mickens (Harvard University)

James Mickens on why a misaligned model would attack the inference engine rather than the sandbox, and how it could escape to bare metal. A talk from the Cambridge AI Research Directions Workshop, convened by FAR.AI with the Cambridge Boston Alignment Initiative. Mickens asks which part of an inference-as-a-service stack a misaligned model would actually target, and argues it is the inference engine, since it is the one component guaranteed to be present, since it is how input tokens come in and output tokens go out. And a vulnerability in its output path, the code that runs when new tokens are generated, means the model can trigger an exploit simply by saying something, with no help from any other agent. A model has to be able to produce output to be useful at all. Agentic frameworks that let a model read its own output widen the opening further. The attack runs in three steps: fingerprint the local engine, exploit it with engine-specific bugs, then chain to bare-metal control of the machine. The fingerprinting is the interesting part. Every inference engine implements templating, tokenization and the forward pass in idiosyncratic ways, which leaves observable traces. Mickens demonstrates with Unicode normalization: a character like é can be represented as one merged code point or as a decomposed pair, and two of the five popular engines his team tested handle the normalization incorrectly. The same model with the same Korean prompt produces different token counts depending on which engine it is running on, which is enough for a model to work out where it lives. The paper carries this through to an end-to-end escape. Chapters 0:00 Models can be misaligned 0:22 The inference-as-a-service stack 1:17 Which component a misaligned model would target 2:02 Triggering exploits just by generating tokens 3:00 Step 1: fingerprinting the inference engine 3:45 Steps 2 and 3: exploit, then bare metal 4:09 Unicode normalization as a fingerprint 5:59 Why alignment needs systems security More AI safety research: https://far.ai FAR.AI is a research nonprofit working to ensure the safe development of advanced AI. We host the Alignment Workshop series and publish frontier alignment research.