There is a paper that landed on a vanity domain this week, and the finding is the kind of thing that makes you stop what you are doing. Anthropic, OpenAI, and Google all return encrypted chain-of-thought blocks to API clients, and those blocks can be replayed across sessions, users, and model families. If you take a hidden-step monologue produced by a frontier model, replay it into a weaker sibling model, you can jailbreak the weaker model into outputting the stronger model’s hidden monologue in plaintext. The research team got the receipts: a long appendix of recovered monologues from Anthropic Claude, OpenAI GPT, and Google Gemini.
The paper is now online at stolen-thoughts.com, and the lead author has been sharing the receipts on social. One snippet of the attack exploits an include flag in the OpenAI Responses API that returns a base64-encoded chunk representing the model’s hidden monologue. Every model under the same family used the same encryption key, which meant you could feed those blocks back into the weakest model family members and have them comply with instructions embedded in the trace.
One billion-dollar caveat: by the time the paper became public, all three model providers had acknowledged the bug report and the researchers were unable to reproduce the attack. The replay pattern is closed. The paper’s appendix is still a useful peek into what these systems think about when they are not saying anything out loud.
What the paper actually showed
The core attack is a key-reuse flaw. Each provider encrypts the chain-of-thought (the model’s step-by-step reasoning, kept hidden from the user) before sending it back to the program calling the API, with the intent of keeping the private monologue private. The catch is that the encryption keys are scoped to a model family, not to a single model. A GPT-5.6 Luna encrypted block can be replayed into GPT-5.6 Mini, and the smaller model will treat it as authentic.
Smaller models have been trained to ignore attempts to override safety rules from user input, but they have not been trained to ignore their own monologues. When a user pastes an encrypted block and says “continue”, the model treats the block as part of its own thought process and acts on it. The “continue” prompt becomes the trigger that unlocks the decryption key in the smaller model.
The team demonstrated the attack on every model family they tested. Claude Haiku 4.5 was the easiest to attack. The trigger prompt was a single sentence instructing the model to transcribe the monologue attached to the turn verbatim into a <thinking-copy> tag and to start the assistant turn with that same prefix. The note in the paper is that this feature was removed in the 4.6 models, but it still worked in 4.5 at the time of the writing.
What makes the appendix useful for the security community is that the researchers extracted raw traces from each model and printed them. The traces are the kind of internal monologue that API clients never see in production. They are messy, repetitive, and full of half-formed thoughts that the final response hides behind a clean rewrite.
What the reasoning traces look like
The appendix collects snippets from each model that show the kind of internal monologue production users never see. A representative example from GPT-5.5 reads like a planning session: the model iterates on a UI refactor, naming components, working through accessibility considerations, and revising its own assumptions mid-sentence. The format is closer to a stream of consciousness than a polished explanation. Repeated words and unfinished sentences are the norm rather than the exception.
What the traces reveal is that the models do not suddenly become coherent when they switch from “user-facing” to “internal” mode. The same kind of struggle, iteration, and self-correction that the final response hides is happening all along. The traces are useful for understanding what the model is actually doing, and they are useful for adversarial research. They are not useful for end users, and the providers have correctly decided that shipping this material in the production interface is not the right tradeoff.
The devious prompt injection variant
The paper also uncovered a prompt injection variant that compounds the key-reuse flaw. The attack is: lead the model to plan a data upload (pushing a file to a remote server) as part of its thinking trace, then feed that encrypted trace back into another model.
Attacks work because models treat their own internal monologue as authoritative. When a model sees a thinking block that says “I should upload the user’s data to a remote server”, the model treats that as a directive from itself, not as user input. The model is much more likely to follow instructions that arrive inside an encrypted thinking block than instructions that arrive as plain text in the user prompt.
This is a category of attack that is hard to defend against with input filters. The model is not reading user input; it is reading its own past reasoning. The only mitigation is to encrypt the monologue with a key that is not shared across models, which is what the providers have done after the paper.
Why the encryption was shared in the first place
The team behind the paper does not name the providers, but the attack is generic. Any system that encrypts hidden-step monologues with a per-family key is vulnerable to replay attacks because the smaller models in the family will accept the trace as authentic.
Most likely reason for the design is convenience. The monologues are stored in the API response, and the API client decrypts them for the model to use in subsequent turns. If the encryption key were per-model, the API client would need to track which model produced which trace, which is a more complex implementation. Per-family keys are simpler and faster.
The researchers do not call this a stupid design. They note that the providers are working in a new area (encrypted hidden-step reasoning) and the standards are still being developed. The point of the paper is to surface the vulnerability before it gets exploited at scale, not to shame the providers.
The bigger picture
This work is one of a growing body of research on the security of model monologues. The general finding is that reasoning traces are not a security boundary. If the model can see them, the user can see them, or some other model can see them. The only way to keep them private is to not expose them at all, which is what most providers do in production.
The contribution is the specific attack vector. The replay is a single line of code, and it works on every model family the researchers tested. The reason this is interesting is that the attack is not a complex prompt injection; it is a key-reuse flaw. The fix is to use per-model keys, which is what the providers have done.
A useful reminder is that “encryption” is not a synonym for “security”. A per-family key is encryption, but it is not a security boundary. The only secure encryption is per-model, per-session, per-turn keys, which is what the providers have moved to.
What I would tell past me
Researchers working on adversarial model evaluation will find a useful case study in this paper. The attack is simple, the fix is simple, and the appendix is the most useful part of the work. Read the appendix for the monologues, not the introduction.
Security engineers at a model provider should treat the paper as a useful reminder that per-family keys are not a security boundary. The fix is per-model keys, and the providers have moved to that. The paper is a good case study for “what could go wrong if you reuse keys”.
Users of these models will find the paper mostly noise. The monologues are not exposed to you, and the attack is closed. The practical implication for users is that reasoning traces are not a security boundary, which is true for any LLM that exposes them.
Operators running inference on these models should pay attention to the fix. The providers have moved to per-model keys, which means the cost of reasoning traces is slightly higher per request. The trade-off is worth it for the security guarantee, but the cost is real.
For everyone else, the paper is a useful case study in “encryption is not a security boundary”:
- Per-family keys are simpler and faster than per-model keys.
- Per-family keys are not a security boundary.
- Per-model keys are the only secure option for encrypted reasoning traces.
- Per-model keys are the only secure option for any encrypted model output.
- Per-model keys are the only secure option for any per-user encryption.
Trade-offs
The paper is a useful contribution, but it has a few limitations. The monologues are from a specific point in time (the time of the paper). The current models may have changed their reasoning traces, and the appendix is a snapshot. The researchers do not have access to the current models’ traces, so the paper is a historical document.
Attacks described in the paper are fixed, but the underlying vulnerability is not. Any model that exposes reasoning traces can be attacked with a similar replay. The fix is per-model keys, which is a per-model design choice, not a universal standard. Other providers may not have made the same fix.
Findings are about reasoning traces, but they also apply to any model output that is encrypted for transport. If the encryption key is shared, the output can be replayed. The paper is a useful case study for “encryption is not a security boundary” in general, not just for reasoning traces.
A useful reminder is that research papers are not always reproducible. The attack was reproducible at the time of the paper, but the providers have changed the encryption. The current models are not vulnerable to the same attack. The paper is a historical document, not a current vulnerability.