This week’s newsletter is sponsored by Larridin, an AI-native developer intelligence platform.

Measure AI’s impact on engineering.

Claude writes code 10x faster than a human. So why is engineering productivity up 20%, not 200%?

Engineers spend only 30–40% of their time coding. The rest goes to planning, deployments, support, reviews, and everything in between.

Most of this work has been invisible and hard to measure. Larridin changes that by showing how engineering work flows across your org, uncovering blockers, and helping teams fix what slows them down.

Larridin gives AI-native teams a full view of developer productivity, from traditional engineering metrics to AI-native insights like Agent Effectiveness.

Unlock AI-native engineering potential

Thanks to Larridin for sponsoring this newsletter, let’s get back to this week’s thought!

Intro

I recently had the pleasure of speaking with Vrushank Vyas, Head of GTM and CS at Portkey. We were planning to meet on-site at their offices in San Francisco, but the timing didn’t work in our favor at this time, so we talked over Zoom regarding how Portkey works and builds its product.

The especially interesting part he mentioned is that every engineer is expected to wear a product hat and have complete ownership of making decisions end-to-end.

It makes a lot of sense since their product is infrastructure and engineering-heavy, so you need to technically understand the product in order to make good decisions.

In today’s article, we are going over in detail how a company, which is planned to be acquired by Palo Alto Networks, builds its product. Their processes, culture, focus, hiring, etc, and much more!

P.S. This article is part of the articles where I speak with a relevant company and go through their engineering culture and processes. I recently spent time in San Francisco talking with different companies, and I am sharing how these companies work.

If you like these kind of articles, you’ll like these as well:

• How OpenAI’s Codex Team Works and Leverages AI

• Inside OpenAI’s Engineering Culture: How the Teams Behind ChatGPT Work

• How an AI-Native Startup From SF Works and Builds Its Product

Let’s start.

How big is Portkey, and what do they do?

The company started a bit more than 3 years ago. And in total, there are around 32 people at this time:

• 24 engineers

• 8 people outside of engineering (0 product managers and 1 product designer)

They have about 4 people in the US, and the rest of the team is in India. The way they work fundamentally differentiates from a company I recently did a deep dive into, where everyone is on-site working together in the offices in SF.

Their main focus is on building what Vrushank called plumbing infrastructure for AI. A platform that equips teams that are building AI products with everything they need: AI gateway, observability, guardrails, governance, and prompt management.

Their tech stack and (AI coding) tools

Vrushank mentioned that they are really heavy on Node.js with TypeScript, where most of the things they build are on this stack. He mentioned examples in particular, such as their Gateway service and their Control Panel.

Regarding the tools for tasks and project management, they use Linear and Notion.

An important thing he mentioned is that they are trying to move away from separate tools as much as possible, and have as many things inside the main repository as markdown files. The reason is that AI agents can much better interpret that and use markdown instead of having the context in many different places.

They made an internal bet that before the end of the year, they’ll move off from some of the tools they use right now, and just use simple .md inside the GitHub repo.

For AI coding tools, they used to work a lot more with Cursor, now they are moving a lot more towards Claude Code, where almost everyone in the company has built custom skills they use daily.

They also built 2 custom AI agents

Vrushank mentioned they are always trying out what may be a fit for them and might do quick proof of concepts, but he mentioned 2 notable, useful AI agents they use daily:

1. Task-tracking agent based on the discussions with customers

An agent that reviews all transcripts and either drafts the needed email automatically or sends them a Slack message for approval. They just click approve, and it sends the next steps to the customer. It’s built on top of Claude Agent SDK.

2. A pricing agent

They call it Dagger. It scans 1600+ LLMs across 80+ AI providers, which include providers like OpenAI, Anthropic, Amazon Bedrock, Microsoft Azure, Google Vertex AI, Fireworks AI, and Together AI.

The challenge is keeping up with constant changes, like new APIs, features, and pricing updates. The gateway has to support all of these changes while also generating accurate metrics for every request, including cost.

To handle that, they’ve built a pricing agent on top of the Claude Agent SDK. It runs continuously, monitors provider websites and API changes, detects pricing or API updates, and automatically opens PRs with the changes.

They are very selective about what they build

Every feature has to justify its place. Prototyping is fast, they can build something in a day or two, but deciding what to build and use in production is where they put in a lot of thought.

Their philosophy is that the core production traffic their customers run through Portkey must always work reliably. They only build new features when they solve a real, broadly relevant production need.

When a customer requests something new, they first look at whether the existing product already solves it. If it does, they guide the customer to that solution. If it doesn’t, and it’s a valid requirement for an infrastructure platform, then they build it.

Vrushank mentioned that the key is that every feature has to scale. They process trillions of tokens every day, and all 100+ enterprise customers run on the exact same product.

They don’t maintain custom builds or separate release branches. So before shipping anything, they make sure it can reliably handle production scale, from hundreds of millions of requests to trillions of tokens per day.

An example of a feature they said no to

They had customers ask them to build AI evals inside their platform so they could track the accuracy of their AI products in the platform. Vrushank mentioned that they could have built it and likely won more customers, but they’ve chosen not to.

They’ve been very deliberate about staying focused on production traffic. Their view is that evals are still primarily part of the development workflow, while their focus is on running production AI traffic reliably at scale.

They know that the moment they’d build evals, it becomes an entirely separate product with its own heavy feature demands and roadmap. Rather than spreading themselves too thin, they’ve stayed disciplined and focused on solving the production infrastructure layer really well.

They have a strong sense of direction, but not a detailed roadmap

They focus more on direction, instead of planning out the entire roadmap for the year. Instead, they like to operate around strategic themes.

Vrushank mentioned that in AI, things move too quickly for a fixed long-term roadmap, so they focus on identifying important shifts and building infrastructure products around them.

For example, they started with the AI gateway, then built the MCP gateway, and are now building the agent gateway. They’ve even considered a sandbox gateway. Their focus is less about committing to a detailed roadmap and more about following the broader theme of becoming the infrastructure layer for AI systems.

Customer demand plays a role as well. They evaluate whether something fits their long-term direction and whether it strengthens their position as the AI gateway company. This has helped them avoid distractions their competitors have pursued, like RAG libraries and RAG agents. Their focus is on building the best AI gateway possible.

Beyond that, they see opportunities to add value on top of the data they already generate, like helping customers decrease the token spend by optimizing model selection with recommendations on when a smaller model could handle a request just as effectively as a larger one.

But these are still near-term explorations, not fixed roadmap commitments.

Their engineering culture is highly cross-functional

Everyone wears multiple hats. Engineers can be forward-deployed, product-focused, and directly involved in customer support. The same person who builds a feature might also handle the UI, write the documentation, support customers using it, and jump on a call to debug issues directly.

They also put a big focus on creating great documentation, and the engineers building the product are also responsible for documenting and supporting it.

At the same time, they’re not trying to create a “hustle” environment or push extreme work hours. People go home at a normal time and maintain balance outside of work.

The expectation is simply that engineers take ownership end-to-end.

Vrushank also mentioned that it’s a culture built for people who thrive on variety, whether that’s building features, debugging customer issues, writing docs, or jumping into product discussions. It’s not for everyone, but for people who enjoy broad ownership and wearing many hats, it’s a very strong fit.

They are intentionally building what they describe as a “barbell” engineering organization

On one end are engineers deeply focused on reliability, making sure the infrastructure is stable, scalable, and continues to perform well.

On the other end are engineers who move fast and turn ideas into reality quickly. If they spot an opportunity, they can prototype and ship it immediately.

Vrushank mentioned a good example: Their rankings page. It started as a simple idea to make use of the data Portkey already had, but building it required real engineering work, from anonymizing enterprise data and masking sensitive information to ensuring that the data updates reliably every day.

What made this a great example is the ownership behind it. The engineer who built it went above and beyond to realize it. On his own initiative, he bought an LED display, connected it with an Arduino, and turned it into a live leaderboard running 24/7 in the office, showing real-time model usage trends to anyone walking by.

That’s what they mean by the barbell approach. One side is deeply technical infrastructure engineers focused on reliability, and the other is builders who can quickly create bold, visible new products and experiences.

We’ve already mentioned in one of the previous articles that being a great generalist or extreme specialist is the best way to get more opportunities. Being in the middle, not so much.

About 40% of their code is AI-generated

So, this number might be something that a lot of companies have higher, but it definitely makes a lot of sense. And the reason why it makes sense for me is that they are building a product that is infrastructure-heavy.

Which means that the changes are more complicated and one mistake can be very costly for the business. That’s where AI doesn’t help as much.

A similar thing was also mentioned to me by Emma Tang, who is leading the data platform team at OpenAI. I recently spoke with her while visiting OpenAI’s offices in San Francisco.

She mentioned that they don’t see as much AI productivity increase inside her team, in comparison to other, more product-focused teams, because of the nature of the work they are doing.

Portkey’s hiring process

Because of the nature of their work (infrastructure-heavy) and expectations of being able to have end-to-end ownership, they mostly hire engineers who are more senior.

Their hiring process is quite thorough, with most engineers going through 5-6 rounds, including an on-site interview.

It typically starts with an initial screening, followed by a lighter technical exercise, then a more challenging exercise. Candidates are then invited for an on-site “super day”, where they work directly with engineering team members on practical problems.

The final stage is usually a conversation with the founder, focused on alignment and culture fit, before moving to an offer.

They’re also fully open to candidates using AI during technical interviews. Since AI is part of how engineers work day to day, they see no reason to restrict it in a realistic technical setting like pair programming or problem solving.

But during behavioral or leadership conversations, that’s not a great look if a candidate uses AI to answer questions.

For technical tasks, though, using AI effectively is seen as a positive signal because it reflects how engineers actually build and solve problems in practice.

Last words

What stood out most about Portkey is their focus on staying in their lane and being good at it, at the same time, their focus on product engineering and end-to-end ownership.

A big thanks to Vrushank Vyas for sharing how they work. Looking forward to meeting the team in person next time.

Liked this article? Make sure to 💙 click the like button.

Feedback or addition? Make sure to 💬 comment.

Know someone that would find this helpful? Make sure to 🔁 share this post.

Whenever you are ready, here is how I can help you further

• Join the Cohort course Senior Engineer to Lead: Grow and thrive in the role here.

• Interested in sponsoring this newsletter? Check the sponsorship options here.

• Take a look at the cool swag in the Engineering Leadership Store here.

• Want to work with me? You can see all the options here.

Get in touch

You can find me on LinkedIn, X, YouTube, Bluesky, Instagram or Threads.

If you wish to make a request on particular topic you would like to read, you can send me an email to info@gregorojstersek.com.

This newsletter is funded by paid subscriptions from readers like yourself.

If you aren’t already, consider becoming a paid subscriber to receive the full experience!

Check the benefits of the paid plan

You are more than welcome to find whatever interests you here and try it out in your particular case. Let me know how it went! Topics are normally about all things engineering related, leadership, management, developing scalable products, building teams etc.