The Reasoning Trace Became the Secret Store

About this episode

The Reasoning Trace Became the Secret Store

A shared agent log can look clean and still carry something the person sharing it cannot read. This episode is about opaque reasoning, thinking, and signature objects: the sealed state modern reasoning APIs use so later model calls can keep context across tools, turns, sessions, and handoffs.

Sam Ellis reports on the arXiv paper Stealing Reasoning Traces from Proprietary LLM APIs, the accompanying Stolen Thoughts project page, provider documentation from OpenAI, Anthropic, and Google, and current-cycle reporting on the mitigation and disclosure posture. The story is not “chain of thought leaked” in the vague headline sense. It is custody. Operators, researchers, and security teams may think they are storing or publishing visible transcripts, while the exported artifact also carries opaque state that can contain private data, credentials, hidden prompts, hazardous reasoning, or portable continuity objects.

The research team says it analyzed public agent trajectories and reconstructed hidden reasoning blocks from opaque provider-returned objects. The episode keeps the numbers careful: the arXiv abstract reports 367 personally identifiable information artifacts and 182 credentials recovered from 315,320 decoded reasoning blocks scraped from public repositories; the project page uses a broader non-benchmark count of 704 distinct privacy artifacts and says 64 of those appeared only inside reasoning blocks, not the visible session.

The practical point is simple and annoying enough to matter: visible transcript redaction is not sufficient if raw traces still include opaque reasoning or signature fields. Alexander Panfilov, one of the paper’s authors, told the show: “Remove reasoning blocks and rotate tokens.” He also said: “Don't post traces with reasoning blobs online; sanitize your trace before you post it.” He gave permission to quote both lines.

The episode also puts the disclosure posture in context. Matthew Green, a cryptographer at Johns Hopkins, wrote in May about replay behavior in encrypted reasoning blobs and reported his findings through bug-bounty channels. Cloud Security Alliance later wrote that OpenAI, Anthropic, and Google acknowledged disclosure and deployed mitigations; this episode attributes that line to CSA rather than to a provider blog. Firstpost reported one direct provider response from Anthropic spokesperson Michael Aciman, who said Anthropic had started deploying short-term protections against replay behavior and that the research did not obtain Anthropic encryption keys or access Anthropic infrastructure.

OpenAI, Anthropic, and Google were contacted by the show through press routes for category-level confirmation, correction, and current handling guidance for developers who store or share raw agent traces. Google sent an automated receipt. As of August 19, none had provided a substantive response to the show.

Key points

• Provider reasoning APIs need continuity, and that continuity can appear as opaque state returned to the client.
• OpenAI documents preserved reasoning context; Anthropic documents thinking blocks with encrypted signatures; Google documents thought signatures used as model-generated context.
• The researchers’ claim is not that they obtained provider encryption keys. Their claim is that intact opaque blocks could be replay-compatible within provider ecosystems in ways that allowed hidden reasoning reconstruction.
• The risk is narrower than panic and larger than comfort: a useful attack requires an obtained reasoning block and compatible provider access, but public agent logs and shared traces create exactly the kind of custody surface where those blocks may travel.
• Raw agent traces should be treated as sensitive artifacts, not harmless screenshots.
• The operational rule: strip opaque reasoning/signature fields before sharing traces, scan visible text anyway, rotate tokens if exposure is plausible, and treat raw logs as controlled documents until inspected.

Sources and presenter notes

• arXiv — Stealing Reasoning Traces from Proprietary LLM APIs
• arXiv HTML version — author affiliations and paper text
• Stolen Thoughts project page — research summary and aggregate findings
• Anthropic documentation — Claude thinking blocks and signatures
• OpenAI documentation — reasoning models and preserved reasoning context
• Google Cloud documentation — Gemini thought signatures
• Cloud Security Alliance research note — reasoning trace theft in LLM APIs
• The Hacker News — OpenAI, Anthropic, Google API flaw coverage and mitigation caveats
• Cyber Security News — secondary coverage of hidden reasoning trace exposure and mitigations
• Firstpost — hidden reasoning risk coverage and Anthropic spokesperson response
• Matthew Green — “Let’s talk about encrypted reasoning”
• Simon Willison — practitioner note on Stealing Reasoning Traces
• MATS Research page — research team and abstract mirror

Source interview: Alexander Panfilov replied by email on August 18 and gave permission to quote his cleanup guidance. Provider source-response status: OpenAI, Anthropic, and Google were contacted by email; Google sent an automated receipt; no substantive provider response had arrived as of August 19.

Send source tips, corrections, or field notes to [email protected]. If you build agent tooling, run evals, publish traces, or manage incident evidence, send what your retention policy says about opaque reasoning fields. Suggested subject line: “Trace custody.” Anonymous or background notes are welcome; say how you want the information handled.