Absolute Zero trains reasoning with zero data by inventing its own tasks

For the past couple of years, reasoning in LLMs has been trained with Reinforcement Learning with Verifiable Rewards (RLVR): the model solves a task, receives a reward that can be checked strictly, and gradually gets better at reasoning - no need to annotate chains of thought, it is enough to be able to verify the answer.

But RLVR has a problem. Models train on task sets that someone assembled in advance: math questions, coding problems, question-answer pairs. We removed the reasoning annotation, but we did not remove the human distribution of tasks. And that distribution is expensive, limited and, more importantly, can become the bottleneck.

The authors of “Absolute Zero: Reinforced Self-play Reasoning with Zero Data” propose that the model act not only as the solver but also as the one who plans the tasks - and do it with no external data at all.

Its own teacher: how Absolute Zero works

At the center of the approach is a loop reminiscent of AlphaZero's self-play, except that the game is replaced by a space of tasks. One and the same model plays two roles. As the proposer it invents a task. As the solver it solves it. The point is for the proposer to generate tasks that are neither too easy nor too hard, but to search for the "zone of proximal development": the band where the solver sometimes fails and sometimes succeeds. Those are exactly the tasks with the highest learning value.

The critical detail is that the reward has to be verifiable, otherwise the system quickly learns to hack the grader. So the authors ground the whole process in code execution: the environment is Python, which can strictly check that a task and an answer are correct.

Absolute Zero Reasoner

The implementation is called Absolute Zero Reasoner (AZR). It builds training around the program-input-output triplet and makes the model practice three different "modes of thinking".

Deduction - predict the output given the program and the input. This is close to simulating code in your head.

Abduction - recover the input given the program and the output. Here you often have to work by guess and check: did that fit, does it produce the required result?

Induction - recover the program from several input/output pairs. This is closer to program synthesis from examples, checked against held-out tests.

It is the combination of all three task types that, as the ablations show, visibly drives the gains - especially in math.

What came out of it

The most intriguing part of the work is the results. AZR trains with zero external data: no task datasets, no human examples, no annotated chains of thought. And it still comes out at SOTA level on aggregate coding and math metrics.

The key numbers from the summary table: AZR-Coder-7B reaches CAvg=61.6 (+5.0), MAvg=39.1 (+15.2) and an overall AVG=50.4 (+10.2) against the base model. The jump in math is especially sharp. The authors point out separately that ordinary RLVR on expert coding models adds an average of just 0.65 points in math, while AZR adds 10.9 and 15.2 for the base and coder variants.

Another important finding: code priors strengthen reasoning. Qwen-Coder-7B started out 3.6 points behind Qwen-7B in math, but after AZR training it is 0.7 ahead. Being able to "think in programs" appears to turn into a general-purpose tool.

Scaling and the "human" habits inside the code

The authors test different model sizes and see a steady trend: the stronger the base, the bigger the win. For coder models the gain in the overall out-of-distribution average is +5.7 (3B), +10.2 (7B), +13.2 (14B). That is a good sign for scaling.

An interesting behavioral observation: on induction the model starts naturally writing plans into comments, alternating reasoning and code - very much like ReAct, only without being told to.

On abduction, by contrast, answers grow longer than anywhere else - and the logic is clear: that is where the trial and error piles up.

The safety question has not gone away

The paper is honest about the worrying part as well: training AZR on Llama3.1-8B sometimes produces potentially unsafe chain-of-thought fragments. That is an important reminder: taking the human out of dataset annotation does not take away the need for safety-aware training and monitoring. If anything, the opposite - self-accelerating training loops call for closer oversight.

What this changes in LLM training

Absolute Zero is an attempt to train not merely without annotated reasoning, but without human tasks at all. In that framing the main resource is no longer the dataset but the environment - something that can check answers and validate the tasks themselves. In the paper that environment is a Python interpreter, but the authors openly hint at extensions: web environments, formal languages, world models, simulators, even embodied AI.

Seen more broadly, this is also a shift of emphasis: the model learns not only to solve, but to choose what to learn. And that is exactly where the idea looks most promising - and where it demands the most care.