Research · 2026-10-24 · All research

We cut a 321B coding model in half to run it on a home PC. Here is what broke, and how we fixed it.

GLM-5.3-Flash has 288 experts in every layer. We kept 152, chosen by optimization, and published the result as open weights. Along the way our first release broke arithmetic, and the smaller files fell into a reasoning loop that one setting removes.

Photo: Jakub Pabis / Pexels

GLM-5.3-Flash is one of the strongest open-weight models for coding and agentic work. It is also 321 billion parameters: 341 GB at 8 bits. Almost nobody can run that at home.

It is a mixture-of-experts model. Every layer holds 288 experts, and for each token the router uses only a few of them. Most experts are rarely needed for code. So the question is simple: how many can we remove, and which ones, before the model stops being the same model?

Our answer is GLM-5.3-Flash-Coder-RCO-152E: 152 of the 288 experts in every layer, published as open weights under the same MIT license as the original. The 3-bit file is 75 GB and runs on a 24 GB GPU with 64 GB of RAM.

The first attempt, and why we replaced it

Our first release kept 144 experts per layer, chosen with a published saliency score (REAP) measured on Python agent sessions. On Python agent tasks it held up well.

Then we ran a broader regression test, and it failed in ways a coding benchmark does not show. It could get 17 x 23 wrong. It looped on questions in other programming languages. The cause was the calibration: the experts that handle arithmetic, other languages and general reasoning were rarely used in Python agent sessions, so the score ranked them low and we cut them.

We said so on the model page, took the files down, and rebuilt. The lesson is general: a pruned model keeps what you measure it on, and quietly loses the rest. If the test set does not cover a skill, the pruning will not protect it.

How we chose the experts this time

Two changes.

The calibration data. Instead of one kind of session, a mix: agentic coding sessions written by the full model, code in 21 programming languages, math, and 15% general text. Everything outside our evaluation sets.

The selection method. Instead of scoring each expert on its own, we used RCO, an optimization method from IST Austria. It treats the set of kept experts as one decision and directly minimizes the difference between the pruned model's output and the full model's output on the calibration data.

At the same 152 experts, the optimized mask matched the score-based mask on code and agentic text, had 22% lower error on general text, and produced far fewer loops: 4 problems in our 30-run loop test, against 22 for the first release.

A short healing step followed: a small adapter on the attention and shared-expert weights, trained to match the full model's output distribution.

What it keeps

At 8 bits, against the full model with the same prompts and default sampling:

Arithmetic, executed code and tool calls pass at every published size.

The loop that only appears after quantization

The 8-bit model was clean. The 4-bit and 3-bit files were not, and the failure took a while to understand.

On harder coding exercises the model would start reasoning normally, hesitate, write "hmm", and then never stop writing "hmm" until it ran out of tokens. No code at all. Worse, once one attempt had looped, the next attempts, which see the previous reasoning in their history, looped again within a few thousand characters.

On Aider Polyglot, with up to ten attempts per exercise, the exercises on which each file looped:

The obvious fixes did not work. A presence penalty stopped the repetition, but the model then kept reasoning without ever ending. A cap on reasoning length helped on one exercise in three, because the loop often started long before the cap.

What worked was much narrower: forbidding the three "hmm" tokens of this tokenizer. With that one setting, the 3-bit model solved all three exercises it had looped on, two of them on the first attempt. The exact setting is on the model page; it is a single --logit-bias flag in llama.cpp.

We think the underlying cause is that the quantized model was never trained on its own outputs: once it drifts into a hesitation state, nothing pulls it back. That is what we are testing next, by training the 2-bit version on its own generations with the full model as the teacher.

Try it

• Weights: huggingface.co/lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E-GGUF
• Recipe, mask, scripts and evaluation harness: github.com/lapadatbogdan/GLM-5.3-Flash-Coder-RCO-152E

This research used compute from EuroHPC JU AI Factory allocations on the LUMI supercomputer. RCO and GSQ are methods from IST Austria; REAP is from Lasby et al., 2025.