Intro

How do the teams behind ChatGPT actually work?

From the outside, it’s easy to imagine thousands of people, complex hierarchies, and long product roadmaps. In reality, the organization is much smaller and operates very differently from most traditional tech companies.

Teams are highly technical, roles often overlap, and many ideas start from the bottom up rather than from top-down planning. Engineers frequently own problems end-to-end, research and product development happen side by side, and experimentation is a core part of the process.

In this article, we’ll look inside OpenAI’s engineering culture and explore how the teams building ChatGPT are structured, how ideas turn into real products, and what principles guide the way they build AI systems.

I recently had the pleasure of speaking with Sulman Choudhry, Head of Engineering, ChatGPT.

We talked about the structure of the teams building ChatGPT, their process, culture, hiring, growing inside the company, onboarding, and knowledge sharing.

In today’s article, you can expect to read about the structure of the teams building ChatGPT, their process, and culture. And next week, on Wednesday, we’ll go through hiring, growing inside the company, onboarding, and knowledge sharing.

This is an article for paid subscribers, and here is the full index:

- The structure of teams that are building ChatGPT
- They have far more engineers than other roles
- An interesting and unique aspect of how they work
- They hire engineers in two extremes
- They try to create an environment where people can experiment, take risks, and explore ideas
- Turning an idea into an actual product, how do they know when it makes sense?
- Most of the ideas come from the bottom up
🔒 The scaling challenge when something becomes successful
🔒 They don’t work with a detailed roadmap, instead a lot gets figured out in real time
🔒 4 core pillars of OpenAI’s engineering culture
🔒 Variance is a big part of their engineering culture
🔒 Last words

Let’s start!

The structure of teams that are building ChatGPT

A key thing inside the engineering organization that’s building ChatGPT is the clear distinction between horizontal infrastructure teams and vertical product teams.

Horizontal teams focus on shared infrastructure. These are the teams responsible for things like reliability, quality, developer efficiency, and latency.

They try to anticipate infrastructure needs before they become obvious. For example, as ChatGPT has become more multimodal, with voice and video, they hired the author of the WebRTC standard to lead the team responsible for their real-time communication infrastructure.

Vertical teams, on the other hand, focus on specific user problems and metrics. They are organized around experiences or use cases, such as improving the core product experience, building for power users, or developing new model capabilities.

The second important thing is that they try to make teams as end-to-end as possible.

A big lesson Sulman mentioned from building AI-native products is that it’s valuable to blur the line between research and engineering. In many cases, software engineers are directly iterating on models and post-training them.

And sometimes research and engineering even sit within the same team.

Similar to how product, design, and data science used to be grouped together, they have included research in that same team structure. They’ve found this approach to significantly increase development speed and iteration.

They have far more engineers than other roles

Sulman mentioned that the team is much smaller than most people would expect.

For example, in a typical company, the ratio of product managers to engineers might be around 1 to 8. For them, it’s closer to 1 to 30. And in the Codex team, the ratio is even higher: 1 to 40.

One reason for this is that the roles are starting to blur.

Engineers often own problems end-to-end, they help define the problem, write the specs, and build the solution. Product managers sometimes prototype with code, and designers don’t just work in Figma, they also write code.

Because of this, teams are becoming smaller and more unified, almost like a single group rather than many separate functions.

An interesting and unique aspect of how they work

One idea they strongly believe in is letting many ideas develop at the same time.

They hire very smart people, and smart people usually have good ideas. So they try to create an environment where they can experiment and build on those ideas.

They avoid bringing too much structure too early, because that can sometimes kill creativity.

A good example is Codex. When they were building the first version, there were about three to five teams working on the same problem in parallel.

Later, at the right time, they’ve brought those teams together and aligned on one direction. That process helped them move faster and ultimately build a better product.

Another important concept at OpenAI is DRI (Directly Responsible Individual).

At OpenAI, every project has a DRI. But unlike some companies where there’s a separate DRI for design, product, or engineering, their DRI owns the entire project.

They have full visibility, decision-making authority, and accountability for the final result. This structure helps teams move faster and work more efficiently.

They hire engineers in two extremes

They use what people often call a barbell approach.

On one side, they hire extreme generalists. These are engineers who can work across many areas: mobile, frontend, backend, and more. They might not specialize in just one domain, but they can do enough across all of them to build and ship products.

Tools like Codex have helped make this possible by lowering the barrier to working in different technical areas. As a result, someone who joins as a mobile engineer might quickly become a full-stack engineer and work across the entire product.

On the other side, they hire extreme specialists. These are people with very deep expertise in a specific domain and often many years of experience.

For example, one of their specialists in real-time communication is the person who wrote the WebRTC standard, while many of the engineers around him are extreme generalists.

This mix works really well for them. The specialists bring deep expertise, while the generalists move quickly across problems, and together they create a strong culture of mentorship and learning.

They try to create an environment where people can experiment, take risks, and explore ideas

That mindset comes from their origins as a research organization. Even their engineers often work the way researchers do, trying new ideas and seeing what works before formalizing anything.

Because of that, they usually add structure only after they see real potential in something.

Codex is a good example. At one point, three to five different teams were working on the same idea. Some were part of the organization building ChatGPT, and others were outside of it.

Once it became clear that there was a big opportunity, that’s when they created a team fully responsible for it.

This is a common pattern for them:

1. Ideas start informally

2. They get incubated inside existing teams

3. If they show strong potential, they create dedicated teams around them

This approach works well because building something from zero to one is very different from scaling it from one to a hundred.

Early on, you need flexibility and exploration. Later, you need focused teams and more structure.

Turning an idea into an actual product, how do they know when it makes sense?

At a high level, they usually look at two things: breadth and depth.

First, breadth: are enough people using the product? “Enough” is somewhat subjective, but Sulman mentioned that usually, they have a good sense ahead of time of what healthy adoption should look like.

Second, depth: are people actually benefiting from it, and do they love using it?

Beyond usage, they also ask whether the product aligns with their long-term mission and strategy. If it does, it’s something they will be more likely to keep investing in.

So the main questions are:

• Are people using it and finding real value in it?

• Does it align with our long-term goals?

Sometimes a project may not have strong adoption yet, but the idea is still promising, so they continue investing in it. Other times, something may have traction but doesn’t fit their long-term direction, and in that case, they may decide to move away from it.

Most of the ideas come from the bottom up

At the top level, everyone is aligned around one mission: building AGI that benefits humanity and is developed safely.

Leadership usually provides only high-level direction, often in the form of a few key metrics that represent progress toward that mission.

Beyond that, most ideas come from the bottom up.

People who are passionate about a problem area look at those goals and propose ways to move the mission forward. But instead of writing long documents or product specs, the most common way to propose an idea is simply to build something.

Often, someone quickly prototypes an idea and then shares a demo or a short video. That becomes the starting point for discussion:

• Are we excited about this?

• Why does it matter?

• If we go all in on this, what would it look like?

This approach can create some chaos. For interesting ideas, you might have three, four, or even five people exploring the same direction at once. But eventually the work converges, and teams align around the best version of the idea.

Sulman mentioned that around 80% of ideas come from the bottom up, which is a big part of what makes the environment so creative.

The scaling challenge when something becomes successful...