Cloudflare knew its AI's memory pool was too small. Then it leaked other customers' data

At 23:11 UTC on September 8, a security researcher named Gadi Evron of Knostic filed a report through Cloudflare's HackerOne program. He had a script. When he sent very long prompts to GLM-5.3, a coding model Cloudflare hosts on its Workers AI platform, the answers sometimes came back with text from other people's sessions.

Cloudflare reproduced it the next morning. On Monday, after a coordinated disclosure with the maintainers of the open-source engine involved, it published the full account. The short version: for 13 days, some customers got pieces of other customers' requests. The longer version is more uncomfortable, because Cloudflare knew about half the problem before the model launched.

A memory pool that was too small

When a language model reads your prompt, it builds a working memory called the KV cache, a block of GPU memory holding its notes on every token it has seen. Cloudflare runs GLM-5.3 on SGLang, a popular open-source inference engine, in a split setup. One pool of GPUs, the prefill nodes, reads the prompt and builds the cache. It then ships that cache over the network to a second pool, the decode nodes, which write the reply.

Cloudflare advertised a context window of 1,048,576 tokens. The cache pool on its prefill GPUs had room for 681,280. "We were aware of this mismatch before launch," the company wrote. It expected the engine to return an error for any prompt it couldn't fit, and its long-context tests came back fine, so it shipped.

The engine didn't return an error. In SGLang's split mode, a prompt too big for the pool was marked as rejected and swapped for a one-token placeholder, but the request was never actually stopped. Prefill built a cache for the placeholder and sent it across. The decode node got a "transfer complete" message and assumed the memory it had reserved now held the new prompt. Most of it still held whatever the previous requests had left there. The decode node then wrote a reply while reading those stale pages, and returned it with a normal success code.

From the outside, that looked like a model answering your question while half-remembering someone else's.

Cloudflare says 155 responses were affected between August 27, when GLM-5.3 launched, and September 9. All of them were answers to prompts longer than 681,280 tokens, and they went to what it calls "a small number of customers." After filtering out its own tests and Evron's, the company says the traffic looks like ordinary long-context use and it found no evidence of deliberate exploitation.

Once it declared an incident at 13:22 UTC on September 9, Cloudflare moved fast. It pulled the model within 40 minutes, found the root cause soon after and had it back online at 19:40 with a bigger cache pool and a patched engine. The pool now holds about 2.04 million tokens, roughly double the context window.

What the post leaves out

The report doesn't say whose data ended up in those 155 responses, what kind of data it was, or how the people affected were told. That gap matters more than the count. GLM-5.3 is pitched at agentic coding, which is exactly the workload that produces 700,000-token prompts: whole repositories, build logs, config files. That's the kind of text where secrets tend to live.

The other uncomfortable detail is timing. Cloudflare says the underlying SGLang bug, issue #30233, was reported upstream on July 6, seven weeks before GLM-5.3 went live.

Not just Cloudflare's problem

The SGLang team published its own post on Monday, and it widens the blast radius. Default split prefill-decode setups were affected from the v0.4 line in early 2025 until versions 0.5.21 and 0.5.22, released this month. The team found a related race in which an aborted request's late memory writes could land in slots already handed to another user. That corruption produces no error and no log line, so operators can't easily tell whether it already happened to them. The maintainers spell it out: "in multi-tenant serving this is a data-leakage concern as well as an integrity one."

Anyone running SGLang in that mode on shared hardware is told to upgrade and to make sure the cache pool can actually hold the context length they advertise. Cloudflare and the company behind SGLang are now working on per-transfer checksums, ownership tags on memory pages and formal verification of the transfer protocol.

Everyone is selling million-token context windows right now, and the memory to back them is the most expensive thing in the rack. This leak came from the gap between the number on the spec sheet and the memory actually behind it. Cloudflare's deployment checklist now compares KV pool size against context length before launch. It's one line, and it would have caught this in August.

Sources: Cloudflare blog, LMSYS / SGLang, Unite.AI.