Cloudflare shows how it squeezes trillion-parameter open models onto fewer GPUs
Cloudflare published the playbook it uses to serve heavyweight open models like Moonshot's Kimi K2.6 and Z.ai's GLM 5.2 on its Workers AI platform. Storing the attention cache in 8-bit instead of 16-bit doubles context capacity to 1.37 million tokens and delivers 41% higher throughput at roughly 30% lower cost per token; compressing GLM's weights to 4-bit shrinks the model from 705 GB to 421 GB. Accuracy on benchmark suites was indistinguishable from the originals, the company says.
Why it mattersThe frontier of AI economics is not only better models — it is serving the same models cheaper. Techniques like these are a big reason heavyweight open models keep getting cheaper to run, and Cloudflare published them openly for any infrastructure team to copy.
Read in the app — free, in 9 languages
Related stories
TikTok put a shopping chatbot inside the video you are watching.2026-10-06Meta's assistant keeps an hourly file on everyone in your life.
2026-10-05One command restores the Apple Intelligence off switch Apple deleted.
2026-10-05OpenAI will ship a Codex upgrade daily for 28 days or reset limits.
2026-10-05Meta open-sourced the firmware for building your own Muse gadget.
2026-10-04