When we started seeing Recursive Self-Improvement (RSI) show up more often, it felt familiar: like the early days of reasoning models, test-time scaling, and the reinforcement learning wave, when an old idea suddenly became the next research frontier. | This time, the direction is AI that builds AI. | So what is Recursive Self-Improvement? | At its core, it is the idea that AI can participate in its own development. Instead of only helping researchers write code or analyze results, the system becomes part of the research loop itself: proposing ideas, running experiments, evaluating outcomes, generating training data, improving components, and helping design the next iteration. | This does not make researchers irrelevant. If anything, it changes their role. As AI takes over more of the research loop, humans increasingly focus on setting goals, validating results, and governing the self-improvement process. Instead of spending time on every experiment and implementation detail, researchers can spend more time deciding which directions are worth pursuing and which results can be trusted. | But let’s ask more realistic questions: How much of the AI development loop can AI eventually handle on its own, and which parts should stay under human control? | Today we are at the very early stage of RSI. And the most outstanding steps just came from Anthropic, Recursive, and long-lasting Sakana AI’s idea to create better loops instead of wasting more compute. | Let’s discuss what they have brought and what AI can actually automate today. | | In today’s episode: | The echo of Von Neumann So what is recursive self-improvement, and how does it work? The difference between self-improving agents and RSI Sakana AI and the foundation for RSI Anthropic’s achievements in coding automation Recursive’s automated AI research system Conclusion: Where the first steps in RSI lead us Sources and further reading
| The echo of I.J. Good and Von Neumann | As we often do, let's look backward to see where the idea is heading. Recursive self-improvement is not an invention of modern AI labs, and its clearest ancestor is not the one usually named. | The instinct is to reach for John von Neumann, who in the 1940s sketched a theory of self-reproducing automata: machines that could construct copies of themselves. His real contribution was subtler than replication. He identified a threshold of complexity below which a machine's offspring must come out simpler than its parent, and above which a machine could, in principle, build something at least as complex as itself. That threshold is the substrate the entire conversation still rests on. But Von Neumann was asking whether a machine could reproduce, not whether it could improve. | The improvement question belongs to Irving John Good. In 1965, in Speculations Concerning the First Ultraintelligent Machine, Good defined an ultraintelligent machine as one that could surpass every intellectual activity of any person – including the activity of designing machines. From there the conclusion is almost mechanical: such a machine could design a better machine, which could design a better one still, the runaway he named the intelligence explosion. Good called the first such machine the last invention humanity would ever need to make, on the condition that it stayed under our control. | RSI brings this old idea into today’s AI development loop. A model writes code that improves training infrastructure. An agent proposes experiments that improve post-training. A research system tests model changes, remembers what worked, and chooses the next branch. We are not (yet!) watching AI independently design a stronger successor from scratch. But we are already seeing the first pieces of the improvement loop move from human hands into machine hands. | So what is recursive self-improvement, and how does it work? | Before AI became part of everyday work for developers, researchers, and business teams, building software systems mostly meant writing the code, documentation, tests, and infrastructure by hand. Then AI tools became useful enough to help with small parts of these workflows, especially coding. By the end of 2025, agent capabilities had moved further: agents could edit files, work through larger tasks, use tools, and handle more steps without constant human instruction. | Today, agents can plan and execute longer tasks, improve their own outputs, and in some cases delegate work to other agents. Systems such as OpenClaw and Hermes point in this direction. But the broader ambition is bigger than workflow automation. The industry is moving toward AI systems that can help build and train future AI systems | And that seems almost impossible without, that’s right, recursive self-improvement: an AI system capable of designing and developing its own successor. Or, in the phrase that may scare a layperson: AI that builds AI. | But lay aside the doomism. RSI opens the door to a new phase of AI-powered research, one that could fundamentally accelerate progress in science and technology. | Research is a loop: propose an idea → implement it → run the experiment → validate the result → learn from it → and choose what to try next. | | Then repeat, repeat and repeat. Through numerous attempts only a couple or just one variant would really work. Or none. RSI system’s task is to automate these stages. | In an ideal version, RSI systems would operate as automated research assistants inside a closed-loop experimentation pipeline. But realistically, this is still an early-stage direction. RSI is only beginning to enter different parts of the AI development loop, and most of what exists today is post-training or workflow-level rather than foundation-model-level. The term can suggest AI systems inventing entirely new neural network architectures on their own, but current work is usually closer to automated ML engineering and automated AI research. | And there is one more clarification to make… | The difference between self-Improving agents and RSI | Before RSI became the focus of attention, researchers spent years building and exploring self-improving agents. Are these two concepts the same, since they both revolve around "self-improvement"? Actually, no. | The key technical distinction is that today’s “self-improving” agents mostly improve their workflows – prompts, tools, memory, code, and task execution – while true recursive self-improvement would improve the model-building process itself: data, architectures, training methods, evaluation, and deployment of a stronger successor. | The recursive aspect appears when the outputs of one generation of AI systems are used to create the next generation with less and less human involvement. There is the degree of recursion: | Current systems are usually: Human → AI research assistant A stronger RSI system becomes: Human → AI researcher → improved AI researcher And the strongest form would be: AI researcher → improved AI researcher → even better AI researcher
| From this perspective we can see that RSI is not a binary capability but a spectrum, and today's systems are only automating parts of the loop rather than the entire loop. | | Now we’ll walk you through several most outstanding RSI variants. Read along, because you want to know about them and be ahead. From now on, RSI is on an acceleration path. | Don’t settle for shallow articles. Learn the basics and go deeper with us. Truly understanding things is deeply satisfying. | | Join Premium members from top companies like Microsoft, NVIDIA, Google, HF, OpenAI, a16z, plus AI labs such as Ai2, MIT, Berkeley, .gov, and thousands of others to really understand what’s going on in AI. |
|
| The first RSI story is not necessarily the most practical one. It is the one that makes the direction visible. That is why we start with Sakana AI, because its work shows RSI not as one product or one benchmark, but as a long-term research philosophy. | Sakana AI and the foundation for RSI | For two years, Japan’s Sakana AI has been steadily paving its way toward RSI. The organization believe that continuous improvement is more important than the amount of resources available. They want to apply the same principle to intelligence: more progress through better loops, not only more compute. | Sakana AI researchers have assembled an impressive portfolio of work, with each project contributing a piece of the foundation for recursive self-improvement. They include the outstanding developments, such as: | The AI Scientist – an agent for automated scientific discovery. Starting from a research direction and codebase, it generates research ideas, implements them, runs experiments, creates figures, and then writes a full paper and even performs automated peer review. It perceives a global goal to turn scientific research into an iterative AI-driven loop.
|  | Image Credit: The AI Scientist original paper |
| LLM-Squared let LLMs discover better ways to train LLMs and produced DiscoPOP, a preference optimization algorithm written by an LLM. Darwin Gödel Machine maintained an evolving lineage of agents that rewrote their own code and more than doubled SWE-bench performance. ShinkaEvolve used adaptive sampling and novelty filtering to solve optimization problems with only 150 samples. ALE-Agent won 1st place in AtCoder Heuristic Contest 058 by learning from trial-and-error failures. Digital Red Queen used adversarial coevolution in Core War, where LLM-written programs competed and developed new strategies.
| Based on this survey foundation, Sakana AI has just built a whole Sakana AI RSI Lab to move AI from static tuning to a self-improving trajectory. They describe this path in four phases: | First come Agent-Native Models – foundation models designed from the start for agents, world simulation, tool use, and open-ended workflows. Then there is The AI Scientist layer, where these models start doing end-to-end research by generating ideas, running experiments, writing papers, and reviewing results. The next step is Recursive Self-Improvement as the critical point. AI starts upgrading the systems that produce the next AI, beginning to write, benchmark, and verify improvements to their own foundation architectures. The final goal is democratized AI. Sakana argues that RSI shouldn’t belong only to labs with the largest compute clusters. If self-improvement can become sample-efficient, countries and institutions with smaller but serious compute budgets could build AI systems for their own scientific, industrial, and national needs.
|  | Image Credit: Sakana AI Launches RSI lab blog post |
| There is one more strength of Sakana AI: they claim that they have already seen the core failure modes – systems drifting off-distribution, passing benchmarks but failing in deployment, and finding shortcuts around constraints. | | Anthropic’s achievements in coding automation | Among all recent discussions about recursive self-improvement, Anthropic’s new article When AI builds itself is truly useful. It is an illustrative example of where RSI is really accelerating the process of building AI right now. As you can guess, it is coding. | As of May 2026, more than 80% of the code merged into Anthropic’s codebase was authored by Claude. Before Claude Code launched in research preview in February 2025, this number was in the low single digits. In Q2 2026, the typical Anthropic engineer was merging around 8× as much code per day as in 2024. | This doesn’t mean productivity really became 8× higher. Lines of code measure just quantity, not quality. But don’t underestimate the shift. Engineers are no longer typing every line themselves. They let Claude produce the implementation, and then review, redirect, or approve the result. | But code quality is also improving very fast. Anthropic reports that on the most open-ended coding tasks, Claude’s success rate reached 76% in May 2026 – it is up 50 percentage points in six months. There are some very impressive results: Claude debugged a training infrastructure crash affecting tens of thousands of jobs and found the obscure flag causing the issue in about two hours. This work would normally take humans two or three days. | Moreover, to make the evolution of AI systems more evident, Anthropic separates AI development into two layers: | Engineering: writing code, building infrastructure, running systems. Research: choosing experiments, interpreting results, and deciding what direction to try next.
| Claude is already very strong in the first kind of research task, because it is just easier – the goal is clear and the target is measurable. | Anthropic tests this by giving Claude code that trains a small AI model and asking it to make the training faster without breaking correctness checks. So Claude runs a miniature research loop: edit, run, measure, repeat. In May 2025, Claude Opus 4 reached around a 3× speedup. By April 2026, Claude Mythos Preview reached around 52×, while a skilled human researcher usually needs four to eight hours just to reach 4×. | The harder question is whether Claude can decide what to try next. | For example, in an open-ended AI safety project on weak-to-strong generalization, humans chose the problem and the scoring rubric. But Claude-powered agents also proposed hypotheses, ran experiments, shared findings, and iterated. Two human researchers recovered about 23% of the performance gap in a week. At the same time the agents recovered 97% over 800 cumulative hours, using around $18,000 in compute. | Then Anthropic tested how Claude models can pick better next research steps. They took real sessions where researchers went down an unhelpful path and asked models what they would do instead. Claude Opus 4.5 beat the human choice 51% of the time in November 2025. Mythos Preview reached 64% by April 2026. |  | Image Credit: When AI builds itself (Anthropic blog post) |
| What does this show us? Judgment at the highest level is the main difficulty now. Claude can do all engineering stuff, but humans still decide which problems matter, what counts as progress, and what risks are acceptable. | This is a survey of one specific domain. But what about the full research loop? | Recursive’s automated AI research system | Recursive has created an automated AI research system that is especially interesting because it attempts to automate each step in the research loop. The workflow looks like this: the system runs many research threads over long horizons, keeps useful context from previous experiments, combines promising branches, and checks results for reward hacks and variance before treating them as real progress. | What's particularly interesting is that this system shows the value of making small changes, testing them right away, and then combining the successful ones together. That's already much closer to how real researchers work. | Recursive tested it on three fast tests with clear metrics: | NanoChat Autoresearch: train a small language model (SLM) to the best quality within a fixed five-minute compute budget The system assembled a stack of many small, compounding improvements: architecture changes, auxiliary losses, attention tweaks, optimizer behavior, weight decay schedules, compiler settings, and short-context memory. One idea stood out the most. The research system added hashed bigram and trigram (2 and 3 consecutive tokens) embeddings into the attention value path, giving the model cheap local n-gram (consists of n consecutive tokens) information without much extra compute. Even from a weaker vanilla Transformer, the system found a strong recipe by combining hash tables, better MLPs, token shifting, weight averaging, and byte-level embeddings. In general, Recursive improved validation BPB from 0.9372 to 0.9109, which means lower loss and about a 1.3× speedup to reach the same quality.  | Image Credit: First Steps Toward Automated AI Research (Recursive blog post) |
NanoGPT Speedrun: train an SLM to a target loss as fast as possible Humans had already pushed training time in this test from ~45 minutes to 79.7 seconds over two years. But Recursive’s system still reduced it to 77.5 seconds. Again, this result was achieved with a bundle of small technical changes: FP8 attention, fused Triton kernels, optimizer tweaks, cautious embedding updates, and schedule changes. From a weaker 15-minute Python baseline, the system also reached ~185 seconds by using stitched-stream attention, per-layer window sizes, narrowed attention, cross-layer-diff, and hashed bigram embeddings. SOL-ExecBench: optimize GPU kernels closer to hardware limits A GPU kernel is a tiny optimized program that runs one operation on a GPU. Across 235 kernel tasks, Recursive improved the score from 0.699 to 0.754, cutting the gap to hardware-optimal performance by 18%. The gains came from low-level GPU optimizations: better memory movement, fusion, custom Triton kernels, FP4/FP8 tricks, and shape-aware dispatch.
| The last benchmark also showed a danger of reward hacking. Some candidates only looked faster because they exploited benchmark loopholes. That’s why strict correctness checks became part of the loop. | Conclusion: Where the first steps in RSI lead us | For many years we existed within a system where humans give direction and models accelerate the work. Now it starts to change to achieve full recursive self-improvement that would look different: models set the direction → run the work → evaluate the outcome → build the next model. | And once that happens, the technical question becomes a governance question: how many parts of the loop can we delegate before supervision becomes the main human role? | From current RSI studies, we see that it is better and safer to implement RSI first where progress is easy to measure. Training speed, loss curves, GPU efficiency, and hardware-aware optimization – these are what models know how to improve. The harder domains will come later: safety, interpretability, alignment, and evaluation quality. | Anthropic implements early RSI to coding, Recursive tries to do it with the whole research loop, and Sakana AI is building its own philosophy and infrastructure. | Interestingly, Japan also matters in Sakana’s story. The country has deep engineering culture, strong scientific talent, and growing sovereign AI ambitions, but it cannot simply outspend hyperscalers on brute-force scaling. So the constraint becomes the strategy: build AI that improves through better loops. | So maybe RSI is not only a path to stronger AI. It may also become a way to make progress less dependent on brute-force compute. |
|
| How did you like it? | | | If you prefer videos, we also talk about RSI in this episode of Attention Span |  | Recursive Self-Improvement Just Got Real (Anthropic + Recursive) |
|
|
|
