Reinforcement Learning (RL) – a topic that’s constantly present in 2025. Ever since DeepSeek-R1 kicked off the year with a major reasoning breakthrough, researchers and developers have increasingly turned to RL-based training as a way to strengthen reasoning and agentic behavior. We saw hundreds of research on RL everywhere: in LLMs, VLMs coding domain, agentic systems, robotics, and beyond.

Unlike supervised learning (depends on labeled datasets), or unsupervised learning (searches for patterns without guidance), RL is built on interaction. The system learns by doing, by nudging the environment and watching how it reacts. Rewards, penalties, surprises, and mistakes shape the policy more than any static dataset ever could. In other words, it pushes models closer to real-world behavior.

But is reinforcement learning really as strong as it looks right now? This year also surfaced a set of concerns about the strategy.

Today we’re zooming out to see what RL achieved in 2025, where it ran into walls, and what momentum it brings into 2026. Let’s begin the full year-in-review.

In today’s episode, we will cover:

Revisiting some basics

Reinforcement learning is a type of machine learning where a model learns through trial-and-error interactions with an environment. Traditionally, RL centered on building explicit agents that take actions and receive rewards. The landscape shifted this year: RL techniques moved well beyond classical agent settings and showed up across model training, optimization, reasoning, and even data curation.

At its core, the setup is simple. The agent takes an action, observes the outcome, and receives a reward or penalty based on how good that action was. Over time it tries to learn a policy that picks actions leading to high long-term reward. RL problems are usually framed as a Markov Decision Process (MDP) with the following elements:

When RL works with LLMs:

RL algorithms aim to find the optimal policy, meaning the best possible strategy for earning the most reward over time. RL lets models learn from types of feedback that are not differentiable, such as human opinions or preference comparisons. It is especially useful for two things:

2025 taught us a lot about both parts.

Reinforcement Learning from Human Feedback (RLHF) becomes classic

Let’s start with alignment. Models can hallucinate, produce biased or unsafe content, and stumble on complex instructions. Alignment matters because it nudges AI toward being more helpful and more consistent, whether it’s a chatbot holding a coherent conversation or a code model finally passing its unit tests.

Reinforcement Learning from Human Feedback (RLHF) became the default alignment strategy for LLMs in 2025. It typically follows three steps:

However, there are some difficulties with human feedback: it is expensive and slow to collect. That's why, in our era of AI-powered optimization, what has also become popular is Reinforcement Learning from AI Feedback (RLAIF).

From human to AI judgement: Reinforcement Learning from AI Feedback (RLAIF)

RLAIF follows the same steps as RLHF, but instead of collecting human preference data, it replaces or supplements human evaluations with AI evaluators that judge model outputs. These evaluators may be:

The reward model is then trained to match the AI feedback, and different optimization algorithms are used to fine-tune the LLM just like in RLHF.

Benefits of RLAIF:

But there are some limitations too:

One of the interesting examples of RLAIF is Curriculum-RLAIF that helps the reward model generalize better and produce stronger alignment during RL. It trains reward models the same way you’d teach a student: start with simple, clear-cut examples and gradually introduce harder, noisier ones. Easy pairs are generated using guided prompts, hard pairs – using random sampling, and medium-difficulty “bridge” pairs in between. These pairs are sorted from easy to hard and fed to the reward model in that order.

Beyond alignment, RL has also been used to boost reasoning abilities of models, leading to the emergency of the entire class of Reasoning Models, with even early examples like DeepSeek-R1 and Kimi-1.5.

Reinforcement Learning with Verifiable Rewards (RLVR) – The new promise?

The beginning of 2025 shocked everyone with the release of DeepSeek-R1 on January 20. It was the first model to deliver advanced reasoning on par with OpenAI’s o1 (at that time) and make it available to the public. For the first time, we could actually watch the reasoning unfold, since the open model produced long, detailed chains of thought that showed how it arrived at its answers. The backbone for this was the Group Relative Policy Optimization (GRPO) algorithm, which sits inside a broader training approach that strengthens the model’s reasoning ability called Reinforcement Learning with Verifiable Rewards (RLVR).

RLVR is a part of the current trend of moving from subjective alignment, like humans labeling outputs, to objective alignment, using rewards that can be proven correct or automatically checked, like math or code.

Here is how RLVR works:

Researchers from LeapLab (Tsinghua University) and Shanghai Jiao Tong University decided to test whether RLVR is actually as impressive as many claim.