Cloudflare Discloses GLM-5.3 KV Cache Bug That Leaked Data Across Workers AI Tenants

Cloudflare’s October 26 disclosure of cross-request data exposure in its GLM-5.3 Workers AI model is a case study in how inference optimizations can silently break tenant isolation. Between August 27 and September 9, 2026, a configuration mismatch plus a bug in the SGLang engine allowed 155 responses— all to prompts longer than 681,280 tokens— to include fragments of other customers’ recent requests. Cloudflare disabled the model, expanded GPU memory pools, patched rejection handling with upstream SGLang, and restored service on September 9. No other Workers AI models were affected, and the company found no evidence of deliberate exploitation.

The technical failure chain is worth understanding because it is not a classic SQL injection story. Cloudflare deployed GLM-5.3 with a advertised 1,048,576-token context window while the KV cache pool on prefill GPUs only accommodated 681,280 tokens. Operators believed over-length prompts would error cleanly; long-context tests succeeded on shorter paths, so the mismatch stayed latent. When a prompt exceeded the pool, SGLang in disaggregated prefill/decode mode marked the request rejected, substituted a one-token placeholder, yet still transferred partial KV state to decode nodes. Decode assumed prefill had overwritten cache pages; stale pages from prior sessions remained readable during generation.

That means the vulnerability required extremely long prompts— beyond what most applications send— but anyone hitting the threshold faced genuine confidentiality risk. Cloudflare notes that affected traffic patterns resemble ordinary long-context use rather than a sustained attack campaign. Still, for legal teams reviewing AI vendors, “low probability” cross-tenant leaks are not acceptable when prompts may contain source code, patient summaries, or unreleased financial models.

Remediation combined capacity and logic fixes. Cloudflare resized pools to roughly 2.04 million tokens, about twice the advertised context, so valid prompts always fit without hitting the rejection path. Engineers also hardened the rejection path so aborted prefills cannot leave stale KV pages attached to live decode sessions. SGLang maintainers published coordinated guidance urging operators to upgrade to v0.5.22 or later and to treat disaggregated mode as sensitive to pool sizing changes.

For developers on Workers AI, the actionable checklist starts with inventory: did any production workflow call @cf/zai-org/glm-5.3 during the window with prompts approaching a million tokens? If yes, open a support ticket and document what categories of data were included. Even without a public list of affected accounts, regulatory frameworks like GDPR may require breach assessments when personal data could have crossed tenants.

For platform engineers self-hosting SGLang, the lesson generalizes. KV cache sizing is not a performance knob alone; it is a security boundary when multi-tenant schedulers reuse memory arenas. Any change to quantization, tensor parallelism, or CUDA graphs can shrink effective pool size silently. Continuous integration should include adversarial tests that send monotonically growing prompts until rejection, verifying that rejection zeroizes or isolates cache lines rather than reusing them.

Security researchers will ask why the gap persisted from launch until external report on September 8. Cloudflare’s timeline shows awareness of mismatch before launch but mistaken confidence in error handling. That is a human process failure layered atop a software bug. Vendor questionnaires should now ask explicitly how prefill/decode deployments test over-length prompts under load, not just happy-path latency benchmarks.

Compliance officers comparing cloud AI providers should weigh disclosure quality. Cloudflare published detailed mechanics, coordinated with upstream, and quantified affected responses. That transparency helps customers decide whether to continue long-context workloads on shared infrastructure or reserve them for single-tenant dedicated clusters with contractual isolation guarantees.

Model risk teams inside banks should revisit policies that allowed employee experimentation on shared hosted LLMs without logging prompt lengths. A single engineer testing a million-token document summarization pipeline could have triggered the bug without malicious intent. Logging token counts and model IDs is now baseline hygiene.

From a product strategy angle, the incident reinforces why enterprises pay premiums for private endpoints even when public APIs look cheaper. Isolation failures are rare but catastrophic for trust. Cloudflare’s brand damage is limited because scope was narrow and remediation fast, yet procurement committees will cite the incident in RFPs for years.

Looking forward, expect certification efforts— similar to SOC 2 extensions— that audit inference schedulers for cache reuse patterns. Open-source engines will add debug modes that paint KV pages with canary values between requests to detect stale reads in staging. Customers should demand release notes that call out memory pool changes as loudly as kernel security patches.

If you build on Workers AI, diversify models for different context lengths rather than pushing every document through the longest-context endpoint. Smaller models with verified isolation on shorter windows may be safer for mixed-trust data than frontier models tuned for maximal tokens.

Finally, treat this disclosure as a reminder that AI security is supply-chain security. Your application’s safety depends on scheduler bugs three layers below your SDK. Pin engine versions, subscribe to vendor incident RSS feeds, and rehearse incident comms before regulators ask what you knew and when. The GLM-5.3 episode ended without known mass exploitation, but the next cache bug might not be so constrained.

Engineering managers should add a quarterly review of maximum prompt sizes per route, tied to alerts when production traffic crosses eighty percent of the verified safe threshold for each model ID. That simple metric would have flagged Cloudflare’s mismatch earlier if customers had visibility into pool limits, which vendors should publish in documentation rather than hiding in support tickets.

Legal teams drafting AI addenda to vendor contracts should require notification within seventy-two hours when cross-tenant isolation failures are even suspected, plus forensic summaries detailed enough to support regulatory filings without NDA walls. Cloudflare’s public write-up sets a high bar competitors will be measured against.

For open-source contributors, prioritize tests that simulate disaggregated prefill aborts under memory pressure; many projects lack fuzzing at that layer because it is hard to reproduce without multi-GPU labs. Cloud funding for such tests would improve the whole ecosystem, not just one cloud.

Educators teaching cloud security should use this incident as a midterm case: students trace data flow from client SDK through scheduler to KV pages and propose monitoring signals. The exercise connects abstract “tenant isolation” to concrete byte reuse.

If your organization paused GLM-5.3 after the disclosure, re-evaluate now that pools were expanded and rejection paths hardened, but keep canary prompts in staging that verify isolation after every engine upgrade. Trust but verify on every release train.

Readers following this story should bookmark primary sources and revisit them after the next earnings cycle, regulatory comment period, or patch release. Headlines move faster than institutions, but your decisions should not. Write down what would change your mind— a court ruling, a revised error rate, a confirmed exploit— and wait for that signal instead of reacting to social posts alone. Teams shipping products should translate news into tickets: one for legal review, one for documentation, one for customer comms, even if you decide no code changes are required. That discipline turns news consumption into operational resilience rather than anxiety scrolling.