The Qwen4 preview is open: six billion parameters active per token, and on Alibaba's own table it fixes more real bugs than Opus 4.6

Alibaba's Qwen team has opened the weights of Qwen3.8-Flash-Next, the model it put a countdown clock on this morning. It is a multimodal mixture-of-experts system: 125 billion parameters in the main network, plus a 51-billion-parameter n-gram embedding table and a 4-billion-parameter multi-token prediction layer — about 180 billion in total in BF16, of which roughly 6 billion are active for any given token. The architecture is new: a hybrid of Gated DeltaNet and Qwen Sparse Attention, 512 experts with ten routed and one shared, a gated residual, and a training recipe mixing the Muon and AdamW optimisers. Context is 262,144 tokens natively and extends to a million. Qwen's own published table puts it at 62.5 on SWE-bench Pro against 53.4 for Claude Opus 4.6 Max, 81.0 on SWE-bench Multilingual against 77.5, and 58.7 on DeepSWE 1.1 — the lab's numbers, on the lab's harness, not yet independently reproduced. The team says training cost about a ninth of what Qwen3.7-Plus cost. The weights are on Hugging Face and ModelScope under a qwen-community-1.0 licence: 131 files, roughly 360GB, with an FP8 build that halves the memory. Qwen calls it an early preview of the architecture Qwen4 will be built on, and says a hosted version follows at $0.16 per million input tokens and $0.47 per million output.

✓ Verified · 4 sources

Related stories