Open models are often discussed as if they’re competing head-to-head with frontier systems. Are they catching up? Falling behind? Are they “good enough” yet?

Nathan Lambert doesn’t believe open models will ever catch up with closed ones, and he explains clearly why. But he also argues that this is the wrong framing.

Nathan is a research scientist at the Allen Institute for AI (AI2), the author of the RLHF Book, and the writer behind the Interconnects newsletter. He’s also one of the clearest voices on what open models are for, and just as importantly, what they are not.

We talk about how academic AI research lost influence as training scaled up, why open models became the main place where experimentation still happens, and why that role matters even when open models trail frontier systems. We also discuss why China’s open model ecosystem developed so differently from the US one, and what that tells us about incentives, talent, and access to resources.

From there, the conversation moves into the mechanics: post-training and reinforcement learning complexity, low hanging fruits, data availability, coding agents, hybrid architectures, and the very practical reasons most people continue to rely on closed models, even when they support openness in principle.

There were quite a few things that surprised me. You should definitely watch it.

We prepared a transcript for your convenience. But as always – watch the full video, subscribe, like and leave your feedback. It helps us grow on YouTube and bring you more insights ⬇️

Ksenia: Hello, everyone. I'm very happy to be hosting Nathan Lambert, research scientist at the Allen Institute for AI (AI2), one of the best RLHF specialists, open model advocate, and a very well-articulated educator. I think I was one of your first paying subscribers back in 2022 or 2023. So you can tell I'm a big fan.

Nathan:
Yeah, it's been fun. We've been crossing paths for years now on the web and in real life. It's fun to join the pod.

From Researcher to Celebrity

Ksenia: When you were starting your career, could you imagine – I was just watching your interview with Lex Fridman – could you imagine that you would become a celebrity?

Nathan:
No. I talk about this with my partner and family regularly. Largely due to the dynamics of the AI industry, things have evolved so fast. There are so many articulate educators who went to these labs to go all in on building AI. And with the stakes involved, people at these labs don't tweet publicly that much. OpenAI is a whole special thing, but a lot of these communicators can't talk. So there's this void that I've been launched into.

There's no way to actually prepare when you go through levels of influence so quickly. I haven't fully grappled with what it means to have this ability. I get invited to all sorts of fancy things – most serve no purpose, and it's easy to say no. But in the last year, I've become more conscious of trying to help AI policy go well. I think a lot of what we do – building open models that are so well documented and explaining the science – is most impactful on the policy side. Open models are so geopolitical, and they shape how AI will diffuse through the world.

That's a very different track than what's in vogue right now, which is that coding agents are becoming so good and so impactful. There are attempts to make it seem like these topics are linked – yes, Chinese model players are releasing agents too – but I really think they're not as linked as people think, and a lot of open models are overhyped in their abilities. It would be cool if open models were better, but it's very different to sit down and use Claude Code Opus 4.5 or Codex than to play with an open model.

Open models are a case study in emerging technology – understanding how it influences the world and creates new pockets of influence. But that's so new to me. I'm not trained in that at all, so I'm trying to learn.

Why Open Models Matter

Ksenia: That's exactly my question. If open models are not that good, why are you so passionate about them? Why are there so many conversations about open models for the US?

Nathan:
They're going to be the engine for the next ten years of AI research. Academia is in a lull of influence in terms of the evolution of the science. People say academic AI research doesn't matter right now, which I think is short-sighted because it's going to be an engine for exploration in a way that companies can't really nurture. Open models are the platform by which that innovation is happening.

For a country like the US that has such an excellent history of scientific projects and institutions, if they want to be the institution of AI research, they should consider it useful and imperative to have open model investment be intentional and understood – something they're in control of. Right now, that influence is shifting to China. We don't know what the future will hold, but China now has a very interesting ecosystem of open models and research. The US has the resources to own this and also great academic institutions that want to be more activated and involved.

China's Open Model Strategy

Ksenia: When DeepSeek and Chinese models started taking off one after another – was it pure geopolitical competition with the US? What's the reason behind it?

Nathan:
I think DeepSeek was fairly ideological – in the purest scientific way of wanting to create knowledge and share it with the world. They curated an industry standard in China where DeepSeek was the spark that made Chinese companies interested in participating in AI. They saw you could do this through open models, so lots of them considered that the default starting behavior.

If you talk to the companies, they're extremely reasonable. They know that tech companies and wealthy customers in the US won't sign up for an API where data is sent to China. They've told me: "Well, our only other option is open weights because then they could still most likely use it." They know there are other concerns – IT departments will ask, "Is the open weight safe?" – but it's at least a card they can play.

I don't think they have a business model figured out any better than Llama. In the next one to three years, we'll see how funding evolves for open models in the US and China. My hunch is the US ecosystem has more liquidity to fund model training. But then, why are the Chinese open models so legitimately close to the frontier? We don't really know. Opus 4.5 and GPT-5.2 are clearly better than the best open models, but the best open models from China have exceeded my expectations. There's a lot of real information that makes it seem like weird things might be going on that we can't quite tie a bow around.

The China AI Research Ecosystem

Ksenia: I just had a conversation with MiniMax researcher Olive Song. It seems they're shipping every month – research just keeps going, nights, days, weekends. People work in shifts. I don't see anything like this in the world of open models in the US, especially since Zuckerberg backed off with Llama. Who is the player on that level? Like DeepSeek, MiniMax, Qwen?

Nathan:
The absolute best talent in China is doing this at this level for open models. That vibe is very similar to Anthropic, OpenAI, Gemini right now. But the talent density there is definitely higher than in open models, which are often linked to academic projects. AI2 is heavily influenced by University of Washington, Percy Liang's efforts at Stanford. So some of this is different talent pools.

The closest thing the US has is NVIDIA's Nemotron efforts, which have made a lot of strides in the last six to twelve months. My read is they've figured out some internal team org and culture alignment, which has let them put out more in higher quality. But it hasn't had the top-end breakthrough that Qwen or Llama has. Breaking into the absolute cutting edge of AI takes something special. I don't think you can just brute-force that, but NVIDIA is close.

Ksenia: They also found the business model behind it – making it so people will use their hardware and software later. That makes total sense for NVIDIA.

Nathan:
Yeah, I asked one of the VPs who leads the Nemotron effort, "Why do you do this?" He said, "Because we're at the frontier of language modeling research and NVIDIA is going to sell more GPUs." I was like, damn straight. At least they have a much clearer business model than anybody else. For that reason, I'm optimistic about the longevity of it.

Cutting-Edge Research in the Open Ecosystem

Ksenia: What's the shift happening in the open ecosystem with all these new models? What's most interesting to you?

Nathan:
People are trying to figure out the right ways to make models and recipes that are extremely compelling – like tool-use agents. The closed labs have invested so much in training environments where they can do post-training in many different domains. The open ecosystem is in the early days of creating ecosystem-like systems for training. Prime Intellect's Verifiers is one; there are others, but they're so early.

I'm trying to figure out what research means for open models to be more cooperative in a coding agent setup in six to nine months. There's clearly going to be a multi-agent future where multiple models can be used. I think the absolute top end of academic work could still tap into things that are actually used, but the ecosystem doesn't operate that way yet. If I'm using Claude, I don't offload to a local model that can read all my files.

There are open-source coding agents like OpenCode trying to figure out how to make coding agents around open models. The area where there will be the most dynamism and excitement is this tool-use post-training, hopefully generalized across many environments. It's exciting because it's the path towards what they're doing at the frontier. But there's a chance the frontier becomes even more separated from academic and small-scale open models this year with the compute spend forecasted to go up.

The Challenge of Post-Training

Ksenia: Is post-training the hardest part to make open?

Nathan:
They all have different challenges. Pre-training data is the hardest legal part to get open because you want every corpus on the internet and human knowledge, and some have been fairly litigious. Post-training tends to be fairly complex at the frontier with a lot of models, sequencing, and hard decision-making about what you put together.

There was a big increase in infrastructure complexity for post-training with this RL and scaling reinforcement learning revolution. The open-source infrastructure for pre-training with things like Megatron-LM from NVIDIA is actually very strong. Labs raising hundreds of millions just take this open NVIDIA software and make it work.

Reinforcement learning is in the era of many libraries, but over the years it'll distill down to a few that work fairly well. In the meantime, the complexity of doing it very well at the cutting edge is pretty hard. Post-training data is potentially further behind, but high-quality released data is just super rare these days.

AI2's Approach to Open Models

Ksenia: Do you work on this at AI2?

Nathan:
Yeah, we touch on all these things. For example, we released Olmo 3 in the fall. One of the current things is trying to transition this to be more agentic and more tool-use. The workflow is finding existing open datasets, looking at evaluations you want to improve on, and trying them. For some evaluations, we'll find open data that makes it fairly easy to hill-climb, but for things super popular to closed labs, we'd potentially have to buy data for millions of dollars.

I expect there to be gaps because so few people release data. It would seem oddly convenient if academics happened to make all the datasets we need when you know the frontier model playbook is to buy a lot of this data to get the flywheel going. Once closed models are good at something, it's easier to create training data with those models and put human effort into the gaps. That's the ever-evolving dance.

People overhype open models as if they're going to cross closed models. I think the equilibrium will continue where the best open models are six to nine months behind the best closed models. That's fine. That's a pretty short timeline with how good the best closed models are now. I don't think the dynamic is changing. If anything, it might err slightly on the side of closed models being more ahead. But even nine months of a gap is crazy.

Will Open Models Catch Up?

Ksenia: You don't think they will catch up?

Nathan:
There's no reason to think open models will. They have fewer resources, and resources normally determine the outcome. Resources and talent determine outcomes. In Chinese labs, the talent is proportionally similar to OpenAI and Anthropic, but the resources in terms of compute and ability to buy data is much lower. At the end of the day, it's mostly compute that people need to make improvements – spent either on training or generating large amounts of synthetic data.

In Olmo 3, we spent millions of dollars on synthetic data, a lot through a federal government grant for this frontier supercomputer. But if even AI2 is spending millions, the compute expenditure at frontier labs is almost like billions. These are huge costs constantly that Western companies are way more capitalized to do.

The Practical Timeline for Open Models

Ksenia: But it's not infinite. If they're behind only six to eight months, and if businesses and individuals can already use them, we can imagine in six months you'll be able to use an open model for daily life as you use ChatGPT or Claude. Don't you think so?

Nathan:
It's a bit of a bet. On the chatbot side, open models will definitely be there. There's some robustness and tool-use where I haven't seen an open model quite as good as GPT-5.2 search. But in chat interface, I think it will be there. Coding agents, potentially not.

If they're still just getting obsessed with them, they'll decide, "We need to train the model that's really good at this." There's still catch-up time where they're going to start turning that crank, and coding agents will get much better. You could say these costs – the upside won't be proportionate and open models will catch up. It just seems like there's no clear indication the models will hit a wall. Everybody I talk to says there's a lot of low-hanging fruit, complex technical work in research and execution and cost, and we need to keep turning the cranks.

Obviously there's macroeconomic time gating on companies to show value. But this Claude Code moment at least buys them more time to spend.

Low-Hanging Fruit in AI Research

Ksenia: What low-hanging fruit do you see?

Nathan:
It's literally everywhere. Different flavors of "we put this dataset in and it helped a lot – how do we make it 10x bigger?" Or "we put this dataset in, it helped, and we didn't filter it well yet, so let's filter it more." Or "our training code only uses 60% GPU utilization, so we can make it 10% faster by writing better kernels." If you do that 40 times, you end up with a 4x faster codebase, and all your experiments are way faster.

From the most established – "we have our pre-training dataset we've been filtering for years, and we're still tweaking it to better serve current tasks" – to cutting-edge things like "we just spent $40 million on three training environments for coding agents because we have all these new Codex users. We plugged it in on our first run and some numbers go up – how do we pick the next thing?" Or complicated things like "we tried to make the model bigger and the numerical issues are too hard – how do we come up with a new RL algorithm that handles numerics better?"

A lot of open models are switching to hybrid architectures – a mix of linear attention and traditional attention. They're more complex numerically, but the upside on downstream RL and inference is so high. It's the industry collectively pushing through the next harder thing to do.

Qwen 3 has a linear model, Kimi has a linear model, we're working on a linear model. People found it scraping GitHub. Why does that happen? It's collective readiness of infrastructure and ideas tested at smaller scales so they work. I asked the leader of our project, "Do you think all models will be hybrid in the future? Why didn't this happen two years ago when Mamba hype was high?"

A few settings on Mamba needed to be figured out. Mamba models did well on pre-training benchmarks, but actual text generation wasn't nice. There are just a few things in the architecture that needed to be changed, balancing them better with traditional transformer architectures. With more tinkering, the models are more stable and people are trying to use them.

That's an example over two and a half years where it's like, "Remember when the hype for Mamba and state-space models was so high?" But now it seems to be really hitting a lot of places. NVIDIA had a model with a hybrid attention as well. How do you predict that timeline? I don't know. But I think there's definitely plenty of things like that that are going to continue to evolve.

From AI research to reality is just – the big question is, what does Sam Altman do on the fundraising engine? I don't think that it's going to be a bubble-popping thing, but there could be corrections on NVIDIA stock or stuff in this time. I don't know. People are rumoring OpenAI and Anthropic IPOs by the end of the year. I wouldn't be that surprised.

SpaceX and xAI

Ksenia: What do you think about SpaceX acquiring xAI?

Nathan:
Musk Industries, for better or worse, has villain vibes, but in a sci-fi movie. I'm not enough of a businessman to comment on what's actually happening. I'd guess there's a reason other than just greed and recouping losses. For whatever opinion on Elon you have, especially politically – I think he should spend less time there – he has such a track record in building businesses that I have to think there's a plan.

The recent Tesla comments about stopping Model S and Model X are fairly shocking to me, particularly because the timeline on robotics seems too soon in my intuitions. But when there are that serious of bets by somebody like Elon, there have to be reasons and things I don't know. As a fairly ignorant person about SpaceX, Tesla, and large-scale robotic manufacturing, I'm not informed here. We'll see.

Ksenia: Yeah, I think you can add a couple of years to what Elon Musk projects, but eventually he reaches the goal. That's the track record.

Beyond Transformers

Ksenia: Everything you described from the research point is mostly transformers and occasional hybrid models. Do you see serious research going into other areas?

Nathan:
There's this continual learning hype point. In coding agents, we have these Claude.md files and agents.md that are actually fairly good at learning from them. In the short term, that'll do a lot. But obviously there's such a big gain in something.

Continual learning is best thought of as an example for a research problem that will eventually be solved and is wonderfully motivated, but you'll never know when a real solution hits at-scale. The idea is the model needs to change based on experience personal to you or the model in the world. That's so reasonable when training is so expensive and models can seem so dumb in some situations. As a motivation, it has to be true.

The likelihood we're on something that looks like a transformer in 20 years – I don't know, it's not that high. We have the hybrid model thing. A couple more transformations down the line. Are people going to still call it a transformer? I don't know. Attention seems pretty good. It's just so hard to predict research evolution.

Advice for Academic Researchers

Ksenia: I'm asking because in the Lex Fridman podcast, you said if you were starting, you probably would not do transformers now.

Nathan:
The best research is further out. I think of deep learning and transformers as fundamental skills you have as a CS researcher. When I was starting, you just needed to learn about how deep learning and backprop libraries work. Now people need to learn how transformers work. But if you're doing work that's so blatantly just "improve on what we already have," it's hard to market it academically unless you're at the absolute top end of academia with more resources and industry connections to stay at the frontier. Some labs at UW, Stanford, and Berkeley do this, but most of academia is not like this.

You need something that's less busy or takes longer. Another thing could be the fine space between two popular ways of thinking or two subfields. Just go where there are not many people. Things by their nature are very likely to work in AI. The amount of things that just work is really high if you set the optimization up right. I don't know if I love my own advice because it's easy to say "go off somewhere," but it's also hard to make it work. Don't do transformers – okay, what do I do? I don't know.

Ksenia: I mean, you did reinforcement learning when it was kind of down.

Nathan:
Yeah. Working in robotics, I've seen many people be so happy. It's a bit more grounded, but you accumulate the benefits happening in this language model revolution while being more secondhand to it. You're probably not subject to the whiplash, but you're getting the upside of new types of things working and the ability to try new things in a specific domain.

Defining Open Source AI

Ksenia: If we talk about practical implementation of open models, let's start from defining what open model, open-source AI is. Do you think the definition matters?

Nathan:
The definition doesn't matter in the classical sense of the debate. It matters in what the community definition is. The community definition of open source is: weights are released and available openly so anyone can use them. That chimes into the license discussion. One of the better things Chinese labs have done is releasing their models with extremely permissive licenses, whereas a lot of US companies have non-commercial or extra terms and conditions cached.

In enterprise situations, lawyers don't like vague extra terms attached. A lot of people writing custom AI licenses do it with vague language and exposure of risk. Just having simple terms where it's clear people can use them is great. In the past, there's been debate – "You need the code and the data available" – which, yes, these are valuable resources and fit more with open-source software motivations. But the debate has died down when people just use open-weight models. This is the thing and the paradigm by which people are operating. I'm happy to not be in stupid debates on "Is this open source or not?"

Ksenia: And still, Olmo is fully transparent, fully open source. So it matters for AI2.

Nathan:
Yeah, there's a group. It's AI2, Percy Liang's Marin thing, Hugging Face, LLM360, which is affiliated with MBZUAI – the UAE thing. The Swiss had a project that was very open. There are actually more people, especially in the Western ecosystem, doing this fully open thing, which is very good appreciation for scientific standards and progress. But it's still niche. It's not the fact that these are fully open that's causing dramatically more uptake. It would be nice if some company said, "Oh, these are fully open so I can iterate faster." But just using a better model is way more benefit than having the model be fully open in real-world applications.

The Future of Open Model Adoption

Ksenia: My bet is 2026 will be the year when people and enterprises start using open models much more. How do you see it?

Nathan:
I think they are. It's been ongoing. A lot of companies' default position is they want to use open models for information security, more predictable costs, owning their stack. It takes a long time for companies to move into these new types of technology. I suspect it'll look like all the same things – surveys asking "Are companies getting benefit out of using AI tools?" It'll be noisy for a very long time. But then it'll be seen that a lot of companies are using open models and fine-tuning them for their use cases in ways that create value.

It's just going to take a long time, especially if any training is involved. To thoroughly test a model – to actually do the training yourselves – is months-long commitment.

Ksenia: And you need to have a team of dedicated people! I started to learn about economics of open-source AI. If you're a company and want to use an open model, it's sometimes prohibitively expensive. So free open-source model is not cheap.

Nathan:
Yeah, because you have to commit to a certain amount of compute to serve a model meaningfully. You can very easily buy the minimum compute to host said model and get nowhere near the inference load to frankly support that spend. That's why some disaggregated inference companies make sense. They will get enough load to make certain features worthwhile. It just takes time to build that up.

AI2's Use of Open Models

Ksenia: How is it organized in AI2? Do you guys use open models in the Institute?

Nathan:
Some people do, but not a lot. The core thing AI2 needs to get to for the models to take the next step is to actually dogfood the models and build development loops between using the model, improving it, and getting real feedback. Until then, it's somewhat of a toy project where there are users and things it's deployed into, but if you're not touching it, why do you expect other people to with serious intent? One thing at a time.

Ksenia: Do you actively use open models? Or do you use closed APIs?

Nathan:
I mostly use closed models. I try open models, especially around launches, and I'm just like, they're not good enough or cheap enough where it's obvious I want to switch my habits. A lot of this is classic product patterns in tech where users have strong habits. I see the slow evolution back and forth between ChatGPT and Claude for certain uses. It takes a long time for those habits to shift.

Even though I think Claude's language on normal conversation is much more succinct and palatable than a lot of GPT-5 things – Claude models have been like that for a while – it's just taken me months to re-shift the habit. This evolution has happened multiple times in AI's recent history.

Florian, who works with me at Interconnects to study open models, has a few things he uses them religiously for because only open models do this. He uses one of NVIDIA's Parakeet models, which is speech-to-text, as a much faster way to input comprehensively into Claude Code. He'll speak to his AI, Parakeet will transcribe, then Qwen 4B will rewrite it and pass it to Claude. That's a very real use case he swears by and gets a lot of value out of. But I haven't done it yet. I haven't used these substantially.

Ksenia: Yeah, it's surprisingly hard to change a habit with a model. I still struggle with ChatGPT 5.2. I think they changed something. No matter what character I choose, it sucks. I still use it for thinking, still like the deep research part. But when I read what it tells me – I just can’t. And it doesn't follow the rules I pit in the memory. I don't know how to make them follow the rules.

Nathan:
It's pretty funny. Everybody that's deep in it has these opinions. That's what I mean by low-hanging fruit. People just need to fix that. That's actually fixable. It just takes a lot of work.

Model Character and Post-Training

Ksenia: Isn't it post-training to make the model actually remember the constant prompts, like the same prompt I put in the memory?

Nathan:
Yeah, it's changing data and creating new data. The process for getting that type of potentially niche problem represented when you're optimizing for thousands of use cases at once – you probably need to build an evaluation so you can automatically measure it. Then you have one more evaluation. They probably have hundreds of evals. There's just a lot of moving pieces to get right. I think they're fairly conservative in what things are really the priority. They're probably willing to take regressions in some things like memory management to push the frontier in utility. If you're going to take a gigantic step in the things you were targeting and have some second-order effects you'll revisit later, you're probably going to do it.

Ksenia: Right, the focus now is on coding, and in coding you don't really need this beautiful language. You said you were interested in working on characters of the model. Is it the same with open models? Is it more complicated?

Nathan:
It reduces to pretty much being data work – you want all your data to look very similarly. Closed models are famous for this. We had this GPT-4o thing where people are still fiercely loyal to it. Claude's language is very different than GPT-5.2, and this is very intentional at every layer of the stack. The first intervention is at post-training because it's most direct. But eventually you need to align all of your – you'd probably align pre-training to make sure it helps achieve this.

It's largely an industrial habit because it fits so nicely into the model as the product. The model's language is the interface by which the user sees the information. The model's outputs are so literally the product. If the product is what keeps the person engaged and uses certain triggers to get people to come back to it, the output is so deeply entwined in how you'd approach the user experience. That is much harder to study in academia.

This is continuing with coding agents. What is Claude presenting to me as I use it is a very interesting problem because the CLI is such a sparse interface, but the few interactions I have with Claude are so impactful to the final outcome. That's also very much – literally the model's output in that harness is so existentially tied to this product. Due to its importance to the AI industry, I'm trying to help grow academic research around it. But I don't expect it to take off like other projects I've worked on. There's no evaluation for reward models – reward models are important for RLHF and AI research. Yeah, that blew up because obviously people want to do post-training research. But I'm not sure character training will blow up in the same way because it's just not as easy to study. It fits with my goals of making it so AI is, if not accessible to everybody at the frontier, at least you can understand what's happening and make your own bets about the direction.

Key Directions for 2026-2027

Ksenia: Where would you say are a few directions to follow for 2026 and 2027?

Nathan:
This agentic thing is very real.

There were rumors about Claude 5 being released today, so I was thinking about what the story would be. I remember last year with Claude 4, their releases were very muted on benchmarks. They had different benchmarks than OpenAI and Gemini. They didn't look as flashy. This fits with their bet on code, but also fits with this thing where evaluations for AI are much more about what some technical people call the harness or the product than just the model.

I think Anthropic was very early on this. The comparison is Gemini 3, which came out late in the year and was so hyped as "Google is back," but has almost zero really cutting-edge impact right now. They didn't have this agentic experience at launch and all these things they're still lagging behind on. So that gray area in between the lines of what's happening with releases is where the frontier is felt today. I expect Gemini to try to catch up, which explains OpenAI's efforts on Codex.

But it's just much harder to follow because it's not just numbers go up. It's much more – you need to try it and it might emerge into new domains. For that reason, following AI is a bit messier than it had been post-ChatGPT, where it was just smarter chatbot, turn the crank, GPT-4 or whatever. It was much easier. But the chat domain is so saturated, it's really hard to see the differences between the models.

Other Emerging Areas

Ksenia: We talk a lot about chatbots. What are other big things? Robotics, agentic systems?

Nathan:
Off the top of my head, I don't really know. The thing that goes with open models is sovereign AI will continue to be a term. Yes, the US and China obviously have a foothold in AI, but there's going to be a lot of continued international investment in compute and models. That's why France is so happy with Mistral. They found a company willing to accept that narrative early on, and that puts them in a really strong position in the EU and elsewhere in the world. I think that will continue, but I don't think that's as important on the scale of the other things you're talking about.

Important things are like power build-out and the sociological factors. Yes, GPUs themselves don't use a lot of water, but construction is deeply personal to communities, and construction materials do use a lot of water. I'm not super well-read on this issue, but that's a major blocker in how AI is viewed in the US right now and will still be a very important story.

Ksenia: That's true. A lot of kids when they talk about AI – the first thing they say is it uses a lot of water. And I don't even understand where it comes from because that's not exactly how it works.

Nathan:
The way it's described will evolve and pass, but the construction limiting factor is real in the US. I think that will continue to be a big story. I don't think it'll always be just water. The language people use to express the problem is often hard to predict.

But the water thing is mostly about the build-out of these wealthiest companies happening in ways that are not often that supportive of the environment and the communities where the build-out happens. Even if the wealth creation from large data centers is so high in terms of tech company bottom line, it's a microscale problem blocking tech companies in a way they haven't had to deal with in the past. But it makes sense. It's a big picture, a lot of money happening, but it happens in a very small part of the world where they don't get that benefit.

Open Source and the Resource Narrative

Ksenia: Can open source be the opposite of this narrative that it doesn't need so much resources?

Nathan:
Most of the resources I see as being for inference. Some inference you can obviously do on your computer, but things like coding agents – that inference is not going to be offloaded onto your computer anytime soon if you want to be at the frontier of performance. Gemini and OpenAI image generation – both of those transformer-based image gen or video gen models are very far away from being cheap.

Much of the build-out, to the extent the AI world succeeds, has to be for inference because inference is what pays back. You can get a version of this on open models on your local computers and some of it. And open models are good in ways where you don't have to retrain your model from scratch. But I think these are not defining pivot points. These are not things that are going to put us in a fork in the road between two different outcomes for AI. They're contributing and they can pile up, but they're not the central issues.

Ksenia: It likely will be hybrid, open and closed together.

Nathan:
Yeah, and we will see if Apple pulls things together. There are companies like that where they should be figuring out what open models mean to consumers in much more compelling, easy-to-use, and at-scale ways. Currently there's maybe no exploration to the general public in terms of real open model usage at scale.

Ksenia: There should be a stack, at least for developers they can use easily. So there are a lot of problems. What is clearly working in open source?

Nathan:
There's kind of insatiable demand for people who want to own their stack. That's in US tech companies – even in very large companies like banks and very large sectors of the US economy if it's a very financialized economy where these financial services are valuable. But internationally, there's so much value as well where new economies aren't traditionally spending a lot on paid internet services. So then open models become their only access to AI and whatever capacity fits in their current situation.

I see open models as being on the path to be disconnected from the absolute frontier of performance, which is these expensive agents with AI subscriptions getting even more expensive. I pay a lot for them. When these models can be pushed even further and people pay even more – that's not happening on an open model, and that's where the frontier of AI is going.

But it's kind of this background of everything else where the ecosystem is so focused on SF and so focused on the cutting edge. There's still a lot of other AI use in the world to be filled in and to be evolved with. I think there are a lot of unknowns that are going to happen in open models where things are just not as closely tracked and still very influential in the world scale.

Research is just slow. Research and establishing technologies are both very slow trends. That's where I see open models to be so important. If the tech sector is being fast, it's just kind of a different story. The timelines are just so different.

Open Models as a Field for Experimentation

Ksenia: So open models should be just a field for experimentation.

Nathan:
Yes, and it's harder to see this experimentation and harder to track, just because it's slow. I think it's still happening, whether or not the frontier succeeds to the best ability. The more the frontier succeeds, the more demand for open models will increase as well, based on certain use cases where they can't use closed models.

Ksenia: When you go to people to tell them about your project, Olmo, what do you tell them? Why is it important?

Nathan:
Owning the engine for AI innovation for decades to come and being the central source of influence in AI research. You can look at the media ecosystem around AI research where it's so obviously valuable to be seen as even associated with the cutting edge. And we need strong open models so that the pipeline from research in the US to new startups and established companies building on top of research – there's not this "but wait, that's a Chinese model. Can I use it?" friction. It's not so expensive compared to the build-out that's underway. Owning this innovation is so worthwhile to the US in terms of being – it also will give us a better ability to understand how open models are used because people will come to the people building the models and be like, "Make it better for this."

Ksenia: So the gap between academia and business and open models and business should be breached to get more feedback, more use cases. That's tough.

Nathan:
Whether or not you believe in research as an engine for innovation and value – I think much of the current tech ecosystem was built off of that. Some of that innovation is just downstream of basics like the internet, but a lot of it is other research and databases and fundamental deep learning stuff, which Big Tech has captured the value from. Realistically these companies would capture the value from AI research being here. That has been the pattern for the tech ecosystem's history, and letting Chinese companies take more ownership of that is just increasing the likelihood of those companies becoming the ones that captured the value. That's why NVIDIA is investing. NVIDIA sees the path.

Open Source, AGI, and the Future

Ksenia: NVIDIA knows how to make money, that’s for sure, and how to capture attention. If we throw in a very famous word, AGI, do you think open source is vital for reaching AGI, or fundamentally in tension with it?

Nathan:
I have two views on AGI:

Open models can contribute to this and can cross the thresholds at their own time. I think open models are mostly just for education and reducing the risk of concentration of power and bringing transparency to an ecosystem where AGI is obviously this important thing, and open models should help increase trust and awareness of the story that is evolving. I think there are different forms of AGI you can imagine with open models, where you have tons of different models from different use cases and other kind of dramatic ideas – what if an architecture changes where these MoE models, you can really swap in experts. These are mostly far out, but sometimes far-out ideas become reality, and people will keep exploring this because it's happening. It's happening in the open whether or not people are following it.

A Book That Influenced You

Ksenia: Thank you. To wrap it up, my last question is always about a book. What was the book that influenced you, maybe in your childhood, maybe recently, but that you remember?

Nathan:
There's one recently. I mentioned this on the Lex podcast as well. I read Season of the Witch, which is a history of San Francisco from the sixties, seventies, eighties, where there's multiple movements through the hippie movement, the Vietnam war, and then when the gay community came to the city. There's just so much turmoil and human challenge that transitioned San Francisco from almost a New England vibe of a traditional Irish, heavily Catholic city to this multicultural phenomenon of dynamic culture and people coming. That's just so recent to have spent over seven years of my life there and to not know most of this history and how it seems like so much of the tech culture is so separated from this and what is such a rich city's history.

I just think more people should know about this and think about the community and area they live in. So I recommend people that I know in SF to read it. I got the recommendation from multiple of my friends in my Bay Area circles. But I just think that this stuff still matters. There's currently a lot of friction between tech and society writ large, I think, largely due to this disconnect and lack of empathy towards very recent things that have happened.

Ksenia: Yeah, it was a very interesting time, full of hope at that moment in America. I really like that period because people had so many ideas and dreams. I think that's what we lack currently in the States.

Nathan:
Yeah, it was a very human era where now it seems like it's somewhat of a meme – big tech is dumping so much money into industry where big tech is defining the trajectory of the country's economy and things. And that is very dissociative to many people for good reasons.

Ksenia: And with open source, you want to prevent it a little bit.

Nathan:
Realistically, I think they're going on the path whether or not people build open models, but building open models is a good way for people that do not want to be partaking in that economy and have other options to use AI and understand the world.