Brought to you by:

Eppo—Run reliable, impactful experiments

Persona—A global leader in digital identity verification

OneSchema—Import CSV data 10x faster

Kevin Weil is the chief product officer at OpenAI, where he oversees the development of ChatGPT, enterprise products, and the OpenAI API. Prior to OpenAI, Kevin was head of product at Twitter, Instagram, and Planet, and was instrumental in the development of the Libra (later Novi) cryptocurrency project at Facebook.

In this episode, you’ll learn:

1. How OpenAI structures its product teams and maintains agility while developing cutting-edge AI

2. The power of model ensembles—using multiple specialized models together like a company of humans with different skills

3. Why writing effective evals (AI evaluation tests) is becoming a critical skill for product managers

4. The surprisingly enduring value of chat as an interface for AI, despite predictions of its obsolescence

5. How “vibe coding” is changing how companies operate

6. What OpenAI looks for when hiring product managers (hint: high agency and comfort with ambiguity)

7. “Model maximalism” and why today’s AI is the worst you’ll ever use again

8. Practical prompting techniques that improve AI interactions, including example-based prompting

Some takeaways:

1. OpenAI’s philosophy of model maximalism—the idea that AI models will improve so quickly that it’s better to build for capabilities that are just emerging rather than extensively scaffolding around current limitations. This approach acknowledges that today’s AI models are “the worst you’ll ever use for the rest of your life,” with capabilities increasing exponentially while costs decrease by orders of magnitude. The pace of improvement is staggering—what once took 6-9 months between model iterations has accelerated to 3-4 months with the O-series models, each representing a substantial leap in capability.

2. OpenAI structures its AI systems as ensembles of specialized models—similar to how human organizations work. Rather than relying on a single general-purpose model, they deploy multiple specialized models (some fine-tuned for specific tasks, others chosen for speed or cost efficiency) working together to solve complex problems. This mirrors how companies function as collections of specialists with different skills and costs. OpenAI applies this approach internally to handle customer support for 400+ million users with just 30-40 staff members.

3. Writing effective evals is becoming a core skill for product managers and teams building AI products. These structured tests measure model performance on specific tasks, helping teams understand where models excel (99.95% accuracy) versus where they struggle (60% accuracy). This knowledge fundamentally shapes product design decisions. The quality of evals effectively caps the potential of AI products, as models can only be optimized for what you can measure well.

4. OpenAI embraces an “iterative deployment” approach, preferring to ship products early and refine them in public rather than perfecting them internally. This collaborative evolution with users acknowledges that everyone is learning about model capabilities together. This philosophy extends to how OpenAI approaches roadmapping—they set directional alignment but expect plans to change as technology evolves, focusing on the planning process rather than rigidly following the plan itself.

5. While many dismiss chat as a primitive interface that will be superseded, Kevin argues that it may be the ideal interaction model for AI. Chat’s unstructured, flexible nature maximizes communication bandwidth in a way that more structured interfaces cannot. It mirrors how humans naturally communicate and can adapt to any intelligence level—from basic to superintelligent systems. This flexibility makes it a “catchall for every possible thing you’d ever want to express to a model.”

6. As AI becomes ubiquitous, Kevin predicts that fine-tuned models will proliferate across industries. Consequently, product teams will increasingly include “quasi-researcher, machine-learning engineer types” to customize models for specific use cases. This integration is already happening at foundation model companies but will spread throughout the industry as organizations recognize that generic models can’t match the performance of those fine-tuned for specific domains.

7. OpenAI maintains velocity through a strongly bottom-up approach to product development, empowering teams to move quickly without extensive consensus-seeking. While they do quarterly roadmapping, they readily discard plans as they learn new information. This philosophy values planning over plans, and accepts that mistakes will happen when moving quickly. Their approach emphasizes ownership, autonomy, and the ability to learn and pivot rapidly.

8. Kevin describes vibe coding—a collaborative coding approach where developers work alongside AI models like Cursor or Windsurf, accepting most suggestions while providing guidance. Rather than meticulously writing every line, developers maintain a high-level direction while letting models handle implementation details. Kevin believes product teams should increasingly use this approach for prototyping and demos instead of static designs.

9. A counterintuitive insight is that designing AI experiences often works well when modeled after human behavior. When creating UIs for reasoning models that need time to “think,” OpenAI looked to how humans behave when pondering a difficult question—not going silent, not babbling every thought, but providing occasional updates to maintain engagement. This human-centered approach to AI design creates more intuitive and satisfying user experiences.

10. Kevin says personalized AI tutoring is potentially “the most important thing AI could do” and is surprised that there isn’t yet a solution serving billions of children. With studies consistently showing dramatic learning improvements from personalized tutoring, and chat interfaces now sophisticated enough to provide it at scale for free, this represents an enormous opportunity to transform education globally, particularly for underserved populations.

Where to find Kevin Weil:

• X: https://x.com/kevinweil

• LinkedIn: https://www.linkedin.com/in/kevinweil/

In this episode, we cover:

(00:00) Kevin’s background

(05:16) OpenAI’s new image model

(08:13) The role of chief product officer at OpenAI

(11:42) His recruitment story and joining OpenAI

(15:59) Working at OpenAI

(18:44) The importance of evals in AI

(24:40) Opportunities in the space

(26:34) Shipping quickly and consistently

(29:47) Product reviews and iterative deployment

(32:53) Winning consumer awareness

(36:03) Designing thoughtful experiences

(40:56) Chat as an interface for AI

(45:21) Collaboration between researchers and product teams

(48:05) Hiring product managers at OpenAI

(53:06) How OpenAI uses AI: vibe coding, AI prototyping, and more

(01:04:34) Raising kids in an increasingly intelligent AI world

(01:08:07) Why Kevin feels optimistic about our AI future

(01:14:20) The AI model you're using today is the worst AI model you'll ever use

(01:17:58) Reflections on the Libra project

(01:21:51) Lightning round and final thoughts

Referenced:

• OpenAI: https://openai.com/

• The AI-Generated Studio Ghibli Trend, Explained: https://www.forbes.com/sites/danidiplacido/2025/03/27/the-ai-generated-studio-ghibli-trend-explained/

• Introducing 4o Image Generation: https://openai.com/index/introducing-4o-image-generation/

• Waymo: https://waymo.com/

• X: https://x.com

• Facebook: https://www.facebook.com/

• Instagram: https://www.instagram.com/

• Planet: https://www.planet.com/

• Sam Altman on X: https://x.com/sama

• A conversation with OpenAI’s CPO Kevin Weil, Anthropic’s CPO Mike Krieger, and Sarah Guo: https://www.youtube.com/watch?v=IxkvVZua28k

• OpenAI evals: https://github.com/openai/evals

• Deep Research: https://openai.com/index/introducing-deep-research/

• Ev Williams on X: https://x.com/ev

• OpenAI API: https://platform.openai.com/docs/overview

• Dwight Eisenhower quote: https://www.brainyquote.com/quotes/dwight_d_eisenhower_164720

• Inside Bolt: From near-death to ~$40m ARR in 5 months—one of the fastest-growing products in history | Eric Simons (founder & CEO of StackBlitz): https://www.lennysnewsletter.com/p/inside-bolt-eric-simons

• StackBlitz: https://stackblitz.com/

• Claude 3.5 Sonnet: https://www.anthropic.com/news/claude-3-5-sonnet

• Anthropic: https://www.anthropic.com/

• Four-minute mile: https://en.wikipedia.org/wiki/Four-minute_mile

• Chad: https://chatgpt.com/g/g-3F100ZiIe-chad-open-a-i

• Dario Amodei on LinkedIn: https://www.linkedin.com/in/dario-amodei-3934934/

• Figma: https://www.figma.com/

• Julia Villagra on LinkedIn: https://www.linkedin.com/in/juliavillagra/

• Andrej Karpathy on X: https://x.com/karpathy

• Silicon Valley CEO says ‘vibe coding’ lets 10 engineers do the work of 100—here’s how to use it: https://fortune.com/2025/03/26/silicon-valley-ceo-says-vibe-coding-lets-10-engineers-do-the-work-of-100-heres-how-to-use-it/

• Cursor: https://www.cursor.com/

• Windsurf: https://codeium.com/windsurf

• GitHub Copilot: https://github.com/features/copilot

• Patrick Srail on X: https://x.com/patricksrail

• Khan Academy: https://www.khanacademy.org/

• CK-12 Education: https://www.ck12.org/

• Sora: https://openai.com/sora/

• Sam Altman’s post on X about creative writing: https://x.com/sama/status/1899535387435086115

• Diem (formerly known as Libra): https://en.wikipedia.org/wiki/Diem_(digital_currency)

• Novi: https://about.fb.com/news/2020/05/welcome-to-novi/

• David Marcus on LinkedIn: https://www.linkedin.com/in/dmarcus/

• Peter Zeihan on X: https://x.com/PeterZeihan

• The Wheel of Time on Prime Video: https://www.amazon.com/Wheel-Time-Season-1/dp/B09F59CZ7R

• Top Gun: Maverick on Prime Video: https://www.amazon.com/Top-Gun-Maverick-Joseph-Kosinski/dp/B0DM2LYL8G

• Thinking like a gardener not a builder, organizing teams like slime mold, the adjacent possible, and other unconventional product advice | Alex Komoroske (Stripe, Google): https://www.lennysnewsletter.com/p/unconventional-product-advice-alex-komoroske

• MySQL: https://www.mysql.com/

Recommended books:

• Co-Intelligence: Living and Working with AI: https://www.amazon.com/Co-Intelligence-Living-Working-Ethan-Mollick/dp/059371671X

• The Accidental Superpower: Ten Years On: https://www.amazon.com/Accidental-Superpower-Ten-Years/dp/1538767341

• Cable Cowboy: https://www.amazon.com/Cable-Cowboy-Malone-Modern-Business/dp/047170637X

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

Lenny may be an investor in the companies discussed.