Research · 2026-10-24 · All research

+23 points on a 70B open model, without new human data. What RLVR does, and what it doesn't.

We raised Apertus-70B from 41.6% to 64.8% on held-out coding problems in one round, by letting the model learn only from its own answers that passed the tests. We also found where the method stops.

Photo: Nemuel Sereti / Pexels

Apertus-70B is the fully open model from the Swiss AI Initiative: weights, training data and recipe are all public. Its authors aligned it on human preferences. It had never been trained with reinforcement learning from verifiable rewards, the technique behind most of the recent progress in coding and mathematics.

That made it a clean test. If we apply the method to a model that has never seen it, how much does it move, and what exactly moves?

In one round, on held-out coding problems, it moved from 41.6% to 64.8%. This post is about how, and about the three places where the method stopped working, because those are as useful as the headline.

First, what did not work: teaching by example

The intuitive way to make a model better at reasoning is to show it good reasoning. We took high-quality worked solutions from stronger models and fine-tuned Apertus on them.

It got worse, and it got worse the more we trained. On a set of recent competitive programming problems, the base model solved 24%. After 8,000 examples it solved 14%. After 37,000 it solved 10%.

We checked the data, and the data was clean. The problem was what the model learned from it. It learned to sound like the stronger model: confident, well-structured explanations, followed by code that did not work. Imitation teaches style. It does not teach the ability underneath.

What worked: learn only from your own answers that pass

The alternative flips the source of the training data. Instead of learning from someone else's solutions, the model:

1. generates several attempts at each problem,
2. has every attempt checked automatically, by running the tests or comparing the final answer,
3. keeps only the attempts that passed,
4. is trained on those.

Nothing enters the training set unless it was verified correct, and everything in it came from the model itself. That is the whole idea behind RLVR: the reward is not a judge's opinion, it is a test that cannot be argued with.

On grade-school mathematics (GSM8K), the first round took Apertus from 65.2% to 74.6%, a gain of 9.4 points. Same mechanics as the imitation experiment. The only difference was where the data came from.

The number that told us where to go next

Before investing in coding, we asked a simple diagnostic question: is the model unable to write this code, or just unreliable at it?

The two look identical if you only let the model answer once. They look very different if you let it try 24 times. On a standard coding benchmark, Apertus solved 38.2% on the first try, but 62.4% of the problems at least once in 24 attempts.

That 24-point gap is the key. The ability was already there; the model just did not produce it consistently. It meant we did not need expensive additional pre-training. We needed to turn "sometimes" into "reliably".

The result: +23.2 points in one round

We built a pool of 8,445 verified Python problems, each with its own tests, and removed anything that overlapped with our evaluation sets. Apertus generated attempts, every attempt was executed against its tests, and 11,727 correct solutions became the training set.

On problems it had never seen, from the same distribution:

That is the largest gain of the whole project, from a single round, with no new human-written data.

Where it stops

Three findings define the limits of the method. We think they matter more to anyone planning to use it than the headline does.

The gain belongs to the kind of problem you train on. The same improved model, evaluated on a coding benchmark written in a different style, did not improve at all: 38.2% before, 37.8% after. The model became much better at the problems it practised, not at coding in general. The practical rule is simple and strict: train on the distribution you will be evaluated on, and for a business, that means on your own tasks.

It works once, not repeatedly. A second round on the coding pool did nothing: 64.8% to 64.0%. On mathematics, the second round added only 1.4 points. The method converts the gap between "sometimes" and "reliably" into performance, and it does most of that in the first pass. The size of the gain is roughly the size of the gap you start with. Pushing past that point needs harder problems or a different technique, not more of the same.

More training on the same attempts makes it worse. When we reused one batch of generated attempts for several training passes, performance fell steadily, from 76.0% to 74.8%. After the first pass the model is no longer the one that produced those attempts, so it is learning from stale data. The fix is to regenerate fresh attempts every round rather than reuse old ones.

What it means for a business

The method is general. Any open model, any task where an answer can be checked automatically: code that must pass tests, figures that must match a source, a document that must satisfy a set of rules. Most companies have more of those than they think.

It also sets honest expectations. RLVR will not turn a model into something it could not already do on a good day. What it does is make the good day the normal day, on exactly the work you care about. For most business uses, that is the difference between a demo and a tool.

Where this ran

Training a 70-billion-parameter model this way means generating and checking hundreds of thousands of attempts. We ran it on LUMI, the EuroHPC supercomputer hosted by CSC in Kajaani, Finland, on AMD MI250X accelerators, under a competitively evaluated allocation from the EuroHPC Joint Undertaking (AI Factory Playground, EHPC-AIF-2026PG01-685).

We acknowledge the EuroHPC Joint Undertaking for awarding this project access to the EuroHPC supercomputer LUMI, hosted by CSC (Finland) and the LUMI consortium, through the EuroHPC AI Factory Playground Access call.