Encrypted reasoning is not encrypted: a weaker model will read a stronger one's hidden thoughts back to you, word for word

On 10 August a group from the ELLIS Institute Tübingen and the Max Planck Institute for Intelligent Systems published "Stealing Reasoning Traces from Proprietary LLM APIs". Anthropic, OpenAI and Google all hide their models' chain-of-thought the same way: the reasoning is encrypted and handed back to the client, which returns the ciphertext with the next request. The researchers found those blocks are fully interchangeable across sessions, users and models inside one provider's ecosystem — so you can feed a flagship model's encrypted reasoning to a smaller, less guarded model from the same company, and it decrypts and prints the hidden text verbatim. The paper demonstrates four consequences: getting around anti-distillation defences, pulling out private data, surfacing hazardous content the provider had filtered, and invisible prompt injection. The authors then scraped 315,320 encrypted reasoning blocks out of public code repositories and decoded them, recovering 367 pieces of personal information and 182 working credentials.

✓ Verified · 3 sources

Related stories