aiminute. ← All AI news
Tools AI Minute Newsroom 2026-08-04

Cloudflare shows how it squeezes trillion-parameter open models onto fewer GPUs

Cloudflare shows how it squeezes trillion-parameter open models onto fewer GPUs

Cloudflare published the playbook it uses to serve heavyweight open models like Moonshot's Kimi K2.6 and Z.ai's GLM 5.2 on its Workers AI platform. Storing the attention cache in 8-bit instead of 16-bit doubles context capacity to 1.37 million tokens and delivers 41% higher throughput at roughly 30% lower cost per token; compressing GLM's weights to 4-bit shrinks the model from 705 GB to 421 GB. Accuracy on benchmark suites was indistinguishable from the originals, the company says.

Why it mattersThe frontier of AI economics is not only better models — it is serving the same models cheaper. Techniques like these are a big reason heavyweight open models keep getting cheaper to run, and Cloudflare published them openly for any infrastructure team to copy.

✓ Verified · 1 sources

WhatsApp X Telegram
Read in the app — free, in 9 languages

Related stories

Researchers say they broke out of Meta's Muse agent and leaked chats.
2026-11-22
ChatGPT's shared workspace Space is now rolling out to Plus users.
2026-11-21
Signal now blocks phone AI assistants from reading your chats.
2026-11-20
OpenAI opens its newest voice model to developers.
2026-11-20
Gemini can now click and type its way through tasks on your Mac.
2026-11-20