Intro

82% of organizations currently don’t measure the impact of AI tools, which is alarming to me.

The reason why that is alarming is: How do they actually know if it’s useful or not? Just adopting a new tool, because it’s a shiny new thing, is never a good idea.

This has been an interesting insight from the AI Impact Report done by LeadDev, which I closely looked at.

Some other interesting insights:

• 60% of organizations believe the lack of metrics on AI impact as a key challenge.

• 59% of respondents feel more productive using AI coding tools.

• 85% are prioritizing internal engineering processes in their AI investment strategies.

• 54% of respondents expect hiring of junior engineers to decrease.

In this article, I am closely examining the report and sharing my thoughts on the data.

This is an article for paid subscribers, and here is the full index:

- The survey was conducted between 30 May and 21 June 2025 and is based on 883 respondents
1. AI tools adoption in engineering teams
1.1 66% of engineering teams have either selectively or widely adopted AI tools
1.2 Internal engineering processes have been the biggest use case for AI
1.3 Cursor is the most popular AI dev tool
1.4 Automated code generation, summarization and writing documentation → top use cases for AI
1.5 There is no “100X” productivity with AI
2. The impact of AI on engineering roles
2.1 The most impacted are junior roles
2.2 Managing AI Agents and Prompt Engineering are the most popular skills to learn right now
2.3 Critical thinking will be the #1 important skill in the next 3 years
2.4 48% have reported significant changes to team structures
🔒 3. AI challenges and best practices
🔒 3.1 Lack of clear metrics to evaluate AI’s impact has been the biggest challenge
🔒 3.2 82% of engineering organizations currently don’t measure the impact of AI tools
🔒 3.3 Development time per feature is the most common metric for the impact of AI tools
🔒 3.4 Guidelines about ethical usage of AI are not present in more than 50% of companies
🔒 Last words

If you like such breakdowns of reports with interesting data, you’ll like these 2 articles as well:

The State of Engineering Leadership in 2025 Gregor Ojstersek · Jun 15 Read full story

The State of Software Development in 2025 Gregor Ojstersek · Aug 3 Read full story

The survey was conducted between 30 May and 21 June 2025 and is based on 883 respondents

Here are the full demographics:

Now, let’s get to specific insights.

1. AI tools adoption in engineering teams

1.1 66% of engineering teams have either selectively or widely adopted AI tools

Which makes a lot of sense. It helps with productivity, as we’ve seen in this article.

20% of respondents mentioned that their teams are using AI for Proof of concepts and pilot projects only, and 13% of engineering teams are still exploring.

This is actually 33% and I think the number is quite high. I’d personally expect this number to be lower, and my recommendation would be to move from proof of concepts to at least try to add a certain AI tool in the overall engineering process.

Only 2% of teams don’t plan to use AI tools, which is quite low in my opinion, but very interesting, as I thought more teams have issues with the protection of their data and overall security concerns.

1.2 Internal engineering processes have been the biggest use case for AI

Especially AI coding tools like Cursor or GitHub Copilot have become a standard in our industry, and many teams are using them. I’ve personally started with GitHub Copilot and then switched to Cursor after.

Additionally, it is also interesting to note that 45% of AI usage is applied to non-engineering processes. I’ve personally seen a lot of AI adoption in marketing, legal, and HR departments.

1.3 Cursor is the most popular AI dev tool

Speaking about Cursor and GitHub Copilot, they are the two most popular AI dev tools. And then followed by Copilot (I believe this answer is a mix of Microsoft Copilot and GitHub Copilot), as the vendor is not specified.

And only after it’s ChatGPT, which I believe is the most popular AI tool worldwide for general purposes.

I am personally using a combination of Cursor and ChatGPT for any coding I am doing and then Perplexity for deeper research, so the results don’t surprise me.

1.4 Automated code generation, summarization and writing documentation → top use cases for AI

This is great data, as we can see what specific use cases companies are using AI for.

And yes, the top 3 are all great ways to use AI; however, I want to stop a bit at summarizing meetings, as it’s quite common these days to have many AI note takers as part of meetings, and it’s something we need to be careful with.

I’ve seen in meetings where people are reluctant to be open about things with AI note takers present, which can be problematic when we want to make decisions or get something done.

So, definitely monitor for such behavior, and my take is that we shouldn’t use them if they actually make us less productive.

Also, it’s great to see 17% using AI for Code reviews, as I’ve personally tried both CodeRabbit and Korbit AI, and I believe using AI for code reviews is great, as long as the accountability is still on human engineers, not on AI.

1.5 There is no “100X” productivity with AI

We can see based on the data → AI makes you more productive, but not close to what some people had shared with their takes. I’ve debunked the myth of AI increasing productivity by 100x in this article:

Enforcing the use of AI in engineering teams - good or bad thing? Gregor Ojstersek · Apr 13 Read full story

2. The impact of AI on engineering roles

2.1 The most impacted are junior roles

This is what I’ve already noticed based on the report from SignalFire. Both Big Tech and Startups reported that they have decreased hiring for entry-level roles.

Read my full thoughts on this here:

Decrease in Entry-Level Tech Jobs Gregor Ojstersek · Jun 4 Read full story

2.2 Managing AI Agents and Prompt Engineering are the most popular skills to learn right now

To become great at Prompt Engineering, here is an in-depth guide that I did in collaboration with the Product Lead from OpenAI Miqdad Jaffer:

Guide to Prompt Engineering Gregor Ojstersek and Miqdad Jaffer · Jul 2 Read full story

To add, AI Evals are becoming more and more important, which is a mix of all the skills mentioned below, as it’s impossible to build a great AI product without a good Evals process. Here are 3 articles that would help you with that:

• AI Evals: How To Systematically Improve and Evaluate AI

• Guide to Rapidly Improving AI Products Part 1

• Guide to Rapidly Improving AI Products Part 2

I’ll look to create more articles related to specific skills mentioned below in the future, so definitely stay tuned!

2.3 Critical thinking will be the #1 important skill in the next 3 years

Definitely agree with the importance of critical thinking as just blindly agreeing or trusting AI to be correct is a recipe for disaster. So, being curious and looking to either confirm or deny is crucial.

The following skills are System Design, having good domain knowledge, and overall communication skills. And then also debugging and code reviewing.

Fully agree with these 5 skills, and to help you with all of them, here are 100+ resources:

100+ resources to become a great engineering leader Gregor Ojstersek · January 14, 2024 Read full story

I will do an updated 2025 edition of the resources in the near future.

2.4 48% have reported significant changes to team structures

And the most common have been dedicated AI/ML teams, followed by reorganization of teams to support AI projects, and then integrating AI specialists into already established engineering teams.

Definitely agree that adjusting team structures is the way to go, so that they can support the projects. However, I’d be careful with creating specialized AI/ML teams too early.

As we noted in the article Biggest Mistakes Engineering Leaders Make With AI → Data scientists and MLEs need data to work with. If you’re still in product discovery, there’s often little for them to do.

Focus on building the product first, secondly build the data foundations, and thirdly optimize your AI system.

And only when you are the the third part, that’s when you should hire MLEs or data scientists to optimize the system.

3. AI challenges and best practices...