In this guest post, Prasenjit Sarkar from Sonar highlights findings from the new report The Coding Personalities of Leading LLMs. He explains the Engineering Productivity Paradox: AI boosts code output but not real velocity, since all that code still needs review and verification. A very interesting read.

Technology leaders are in a race to harness the power of AI to boost engineering productivity. We see the potential everywhere – from AI coding assistants generating over 30% of new code at Google to the promise of agentic workflows transforming the entire software development lifecycle (SDLC).  But many leaders are also discovering what we call the Engineering Productivity Paradox: despite a massive increase in the volume of AI-generated code, overall engineering velocity is not increasing at the same rate. 

The reason is simple: all that AI-generated code has to be reviewed and verified by humans. And just like human developers, AI models have their own unique styles, strengths, and weaknesses. Measuring their performance on benchmarking challenges alone isn't enough to understand the quality, security, and maintainability of the code they produce. 

Just recently, Sonar published its latest report, “The Coding Personalities of Leading LLMs,” which goes beyond today’s LLM benchmarks to reveal a more nuanced view of LLM performance. Part of our ongoing The State of Code series, the report analyzes the code generated by six leading LLMs to uncover their distinct “coding personalities.” This new framework for evaluating these powerful tools will help you understand the hidden risks and opportunities of AI-assisted development.

Key findings from the LLM report

Our analysis, which used the SonarQube Enterprise static analysis engine to assess over 4,400 Java programming assignments completed by six leading LLMs, revealed several critical insights for any organization adopting AI. 

1. Each LLM has a unique coding personality

Just as every developer has a distinct style, so does every LLM. Our report identifies measurable "coding personalities" based on traits like:

Our report introduces "coding archetypes" to bring these personalities to life:

Understanding these traits is like understanding the work style of a new hire – it’s critical for knowing how to manage their output and integrate them into your team.

>Dive into the report now to learn more about these archetypes<

2. LLMs share impressive strengths

Our research confirms that the models have powerful capabilities that can speed up the initial stages of development.

3. LLMs have common blind spots

While the models we studied are incredibly capable of solving complex problems, our analysis found they share a consistent set of fundamental flaws.

4. Newer models can be risky

One of the most surprising findings from our analysis is that a model “upgrade” can conceal an increase in real-world risk.  When we compared Claude 3.7 Sonnet with its successor, Claude Sonnet 4, we saw this paradox in action. The newer model showed a 6.3% improvement on benchmark pass rates, but the bugs it introduced were over 93% more likely to be of ‘BLOCKER’ severity. 

In its effort to solve more complex problems, the newer model generates more sophisticated – and more fragile – solutions. This shows why relying on performance benchmarks alone can be misleading; it's essential to analyze the quality and risk profile of the code, not just its functional correctness.

Similarly, GPT-5's improved functional correctness and major reduction in 'BLOCKER' vulnerabilities come at a cost, introducing a new, more complex risk profile. This is because the model's attempts at sophisticated solutions result in a far higher rate of code smells and advanced "Concurrency / Threading" bugs than its peers.

Vibe, then verify: How Sonar helps you lead in the AI era

These findings don’t diminish the transformative potential of AI; they clarify the path forward. As developers increasingly "vibe" with AI to accelerate creation, success comes from a “trust but verify" approach. True engineering productivity requires a partner that brings confidence to both sides of the equation – fueling the vibe while fortifying the verification. This is where Sonar becomes an essential partner.

Sonar helps you solve the Engineering Productivity Paradox, enabling your teams to safely adopt AI without sacrificing speed or quality. Our platform is the industry standard for integrated code quality and code security, providing a consistent verification layer for all code, whether it’s written by a human or an AI.

No matter which coding personality you "hire" – from an ambitious “senior architect” like Claude Sonnet 4 to a speedy “rapid prototyper” like OpenCoder-8B – Sonar ensures the final output meets your organization's standards. We help you:

As AI rewrites the rules of software development, leaders can't afford to be surprised by the hidden risks. Relying on performance benchmarks alone is like hiring a developer based only on a resume – it doesn't tell you anything about their real-world habits or the quality of their work.

Download The Coding Personalities of Leading LLMs to see the complete analysis, explore all the coding archetypes, and get the data you need to make informed decisions about integrating AI into your SDLC.

*This post was written by Prasenjit Sarkar, Solutions Marketing Manager at Sonar, and originally published here. We thank Sonar for their insights and support of Turing Post.