In this episode, my goal was to record a conversation that was completely free of any “AI doom” content. There’s so much of it out there, I hope that this can be a nice timeline cleanser.

I had Casey Muratori on who is a video game developer and programming creator also known for several talks (ex1, ex2) on the history of computing. In putting together those talks he dug deep into computing history. Enough where he had interesting stories and tidbits to share.

For instance, Dijkstra wrote about his depression and there were also several famous flame wars that happened between legends in the industry. They didn’t have social media back then so they wrote letters to each other calling each other out which we discussed.

Lastly, we talked about his career and the video game industry. I was curious to compare it with big tech and hear more about his career path. He’s a great story teller so I hope you enjoy the episode!

Check out the episode wherever you get your podcasts: YouTube, Spotify, Apple Podcasts.

Timestamps

01:08 - Digging into computer science history

04:08 - What shocked him

16:23 - Dijkstra was depressed

26:17 - The personal side of goto considered harmful

35:09 - The anatomy of a 35 year mistake

50:12 - Clean code horrible performance

58:33 - How to write high performance code

01:01:18 - Where bad code comes from

01:06:37 - Why design docs before code is a bad idea

01:09:20 - The only unbreakable law in software engineering

01:15:57 - How he got into programming

01:21:16 - Why he didnt work in big tech

01:30:44 - Should you work at a startup early on

01:34:52 - What video game engineering is like

01:39:57 - Why preventing recursion is reasonable

01:43:24 - Is vibe coding bad for the industry

01:50:17 - Technical reading recommendation

01:54:27 - Advice for his younger self

Transcript

01:08 — Digging into computer science history

Ryan:

[01:08] There’s this famous quote by Donald Knuth. The big part of it is, “Premature optimization is the root of all evil.” And I remember, I’ve heard that before, but I don’t think I know the full history behind it. And why is that significant?

Casey:

[01:26] So I guess we can split that into two parts. What’s the history behind it and why is this significant? The significant part is probably the easiest to answer because, for whatever accident of history, it’s something that people really remembered. It stuck. And so the reason that it ended up being significant is people keep repeating it, and they keep interpreting it or reinterpreting it. So the effects of that phrase are far beyond what anyone may have done with it in context at the time or what they meant or any of those things, or the history. The effects of it are still felt to this day.

[02:02] Some people, I think probably correctly, you might say, interpret the term with a fair bit of nuance and say, oh, well, it can mean a number of different things or it’s trying to capture something subtle or whatever. Other people are very blunt about it and just think, oh, it just means I don’t have to think about the performance of my code to the end of the project. “Premature” just means any consideration of performance is not worth it.

[02:28] Eventually, we’ll see where the code is slow and we’ll fix it or something. It’s a significant phrase for that reason, at least to me. You still hear it to this day, and it’s at least, depending on how you want to account for it, it’s at least something that’s 50 years old now. And so that’s an incredibly lasting impact for a rule of thumb, or however you want to look at it. So on the history side, this was really.

[02:57] When I decided to do a talk on this, I knew I wanted to do another computer history talk because I had given one before. This is at something called the Better Software Conference. There’s been two of them so far. At the first one, I did a history talk, and I wanted to do another history talk. And this was going to be like, I’m probably only going to do two of these, and this is the second one. I wanted to do it on the history of this phrase because when I started looking into the history of this phrase a while back, I found that there was just a lot of stuff there that I didn’t know.

[03:27] And this is true. I mean, I guess I would say every time I’ve ever looked at computer history, every time I go, I refer to it as dumpster diving. But it’s not like that makes it sound derogatory. But I think of that as kind of what I’m doing. But it’s not really like, it’s actually a very enjoyable experience to go back through history and try to pick through what happened. So to do this talk, I basically went and tried to do a pretty detailed analysis of how did this phrase end up being put in print?

[03:58] What were the worked examples at the time? What were they thinking about? Why might it have been said?

Ryan:

[04:02] What did you learn in doing all that research about these famous computer science phrases?

04:08 — What shocked him

Casey:

[04:08] The thing that shocks me both times, even though I’ve now done it twice, is the amount of stuff that you learn about what was going on at that time that you just had no idea about. It’s just mind boggling. And I would point out that this is for a talk. I’m not a historian, I don’t do this for a living. I’m not at a university somewhere doing computer history research. So what I think of as a pretty thorough read for the history that I did for the talk is actually somewhat shallow.

[04:39] If you had time and the resources, you could go access some of these people’s personal papers, make appointments to see things that are not really in the public record so much, and so on. So even just with publicly available things that you can get, the amount of stuff that I found was just really kind of intriguing to me. And so I’ll give kind of a brief overview of sort of what I thought was the story of this kind of phrase. In a way, I think there are kind of two.

[05:10] I guess I would say there are sort of two threads that are good to understand that come together. And this is what I tried to portray in the talk as well. The first thread is this idea of a thing, and we didn’t really do an intro to this podcast, but I’ll say it now. I’m a big fan of your show. When you said, do you want to be on the podcast? You didn’t have to tell me what the show was. I’ve already watched it.

[05:34] I love it. And one of the people you had on, for example, was Barbara Liskov. And in that interview, she actually refers to something called the software crisis, right? And I think you prompted her a question about this as well, right? So a lot of people don’t really know about the software crisis. They don’t know that that was a term. They don’t know. This is something that happened because it’s kind of ancient history now.

[05:58] It was in the sort of late 60s, early 70s. This was the thing. What this was is originally computer hardware, in a sense that you or I can’t really conceive of it in practice because it was so far before our time that basically computer hardware originally was very simplistic. And so the things that you were going to do with it could be described very easily. And really all you were trying to do when you programmed a computer was just translate a fairly straightforward definition of some computation into machine instructions.

[06:36] The kind of thing that we would think about today as hand optimizing assembly code was really just all computer programming was. It was just like, okay, we’re going to take this problem description and translate it in. But during the 60s, computing power was advancing to the point where people could consider doing much more significant things with it. They could run payroll systems, they could do analysis of data sets, and they would want to be able to be flexible and change the sorts of computations they were doing and all these other sorts of things.

[07:06] They were starting to have graphical displays. Ivan Sutherland’s Sketchpad is in the early 1960s. What they found was that at that time, programmers were not really ready to make that transition. They didn’t think about things the way we all now take for granted, where it’s like, oh, yeah, I built this library for loading JPEG images, and now when I want to load a JPEG, I just call load JPEG and it’s done.

[07:32] All of these things that we take for granted all had to be a cultural shift. The idea that there were going to be these abstractions and these ways of sort of breaking down a much more complex system into parts that we could manage, that actually was a transition that had to happen. And so the software crisis was kind of this period where people were starting to say, the methods that we’re using for programming will not scale to these larger problems that we want to do.

[07:59] We’ve got to figure out something to do about it. So that’s one thread of history that’s coming. Another thread of history that’s coming. It’s a little bit more specific to Donald Knuth, who was the first person to really put this into a worked example. He’s kind of who I would credit with the origin of this phrase, even if there are people who might debate that fact. And to me, it doesn’t really matter who said the phrase.

[08:23] The phrase lives on its own. So I wouldn’t even argue with someone if they wanted to sort of make other claims about it. But just in general, he was going, like, at Stanford at this time, he was sort of learning about the fact that you could do these profiles, like taking actual execution profiles, figuring out where time was spent in the execution of a program and using that to guide your decision-making about where you’re going to spend time developing the software.

[08:55] Again, something that even today, performance is maybe not as much of an emphasis as it maybe should be in software. But even today, with that in mind, most people understand the concept of an execution profile. They understand that you could go into Chrome and see a network profile of how that’s working. You could see a flame graph of where time is being spent, or all these sorts of things. Again, you have to remember, if you rewind the clock back far enough, these were all brand new concepts, right?

[09:24] A sampling profiler, a block-based profiler, a binary instrumentation profiler. All these sorts of things were not just tools that everyone assumed that they could just go have and had seen at some point. Some people may never have even realized that was a thing. So that’s another thread that’s happening. And those two things converge effectively in 1974, right? Roughly. To get us this statement, Knuth is very much into this idea of structured programming, which is sort of a proposed solution to the software crisis.

[10:02] Maybe a proposed solution is the wrong way to say it. It’s a certain set of ideas that are trying to be employed to improve the situation. Let’s say he’s very much into this. We can talk about what that is as sort of a separate tangent if we want to later. He’s very much into that. He’s thinking about that. He’s thinking about this idea that programmers can’t just be about writing all of this very tightly optimized assembly language code.

[10:31] And that’s all we’re going to spend all our time doing. We have to be thinking bigger picture. We have to be picking and choosing our battles. We have to make concessions to maintainability. If this particular routine is not really accounting for hardly any of our runtime, we should not be obfuscating it with all of these hand-unrolling the loops and all these things. Remember again, underlying all this, they don’t have massive optimizing compiler passes like we have with LLVM now.

[10:57] They don’t have all of this extra stuff. So if you wanted something unrolled, you were going to unroll it yourself. If you wanted some kind of loop, hoist a loop invariant out, you probably had to do that yourself. A lot of that stuff that we take for granted today, again, not in there. So that sort of. He’s very unstructured programming. He’s starting to see these execution time profiles. They do a thing at Stanford in 1970, actually he does it with some students where they take a look at FORTRAN programs and they instrument them to see where the time is being spent.

[11:26] Not in exactly the most rigorous way that you might imagine. That’s another thing from my talk that I kind of go into. But he’s sort of saying, look, we as programmers need to start taking the decision about what to optimize more seriously. And he kind of has a fairly nuanced take on it. If you actually read the section of his 1974 paper Structured Programming with go to Statements where kind of the first time this was really put to print that I know of.

[11:54] He does say it in an ACM Turing Lecture a little earlier, but he would have submitted the manuscript for this publication prior to the lecture. So it was the first time that I know of that he actually committed it to paper. It would have been there. But he may have said this a lot in lectures. He kind of obliquely refers to that in the Turing Lecture that maybe he had said this around town, so to speak.

[12:20] So where this was first uttered, I don’t think anyone knows. But the point being, when he says that phrase, it’s in this context of saying, look, we need to be very concerned about efficiency. We shouldn’t be leaving performance on the table when it matters, but at the same time, we need to know that it matters. If we don’t know that it matters, then we’re going to apply this optimization. That’s a premature optimization, and it has real cost.

[12:48] It’s going to make our maintenance and debugging of this program worse for all the same reasons that it would make it worse today. If we go into something and we start applying these optimizations to it that are above and beyond the kind of structural things that we might be doing to that code otherwise, that’s creating a cognitive burden that other people are now going to have to deal with, and you’re going to have to deal with in the future.

[13:10] So I don’t want to put too many words in Knuth’s mouth, certainly, but in general, that’s sort of the way that this comes together is those two things, this idea that we need to be doing things to solve the software crisis. How have a more structured approach to programming, care about maintainability and debugability, composability, abstraction, all that stuff. But also at the same time, we do care about optimization.

[13:33] So how do you dovetail those things? “Premature optimization is the root of all evil” is kind of like this phrase he uses, I would say, if I had to imagine his way of sort of reminding himself, think about all this before you make a decision. Right. That would be kind of my take on it. Right. And where it comes from. So hopefully, I don’t know, that’s very long, but it’s a very expansive question. So hopefully I’ve kind of wrangled most of the stuff into there.

Ryan:

[14:00] It sounds like this is really an engineering tradeoff, where the software crisis is about the maintainability and the complexity for people to actually write and handle their code, and then the tradeoff between performance and, I guess, code that is maintainable. And it sounds like this Knuth quote is, be mindful of where you pay that cost.

Casey:

[14:29] Yes. And it’s always important to understand the context in which this was said historically. So I want it to be a historical talk, because who your audience is is a large determiner of what the root of all evil is. If you’re talking to a bunch of people who don’t think about optimization at all, then you don’t really need to say this phrase to them. If you fast forward to today, when a lot of people don’t really take optimization into account at all during their programming, or very, very little, this phrase doesn’t make very much sense to someone who does care about performance, because it’s like why would you ever say that?

[15:08] It just seems like it encourages people to not be mindful of the performance of their program. But if you rewind to that time and you understand that, well, a lot of the programs, you know, they had the opposite problem. A lot of the programs at that time, what programming was, was coming up with all these crazy little assembly tricks to shave a cycle off here or there, right? And if you’re talking to an auditorium full of people who are really into that sort of thing, then it really can be the root of all evil, right?

[15:41] It’s like you guys are spending all your time on this, but we have this really big other problem. And that is not the right tradeoff, right? That is, you’ve swung, you know, like calling it tradeoffs is a great way to say it. You’ve swung the pendulum way too far in one of those directions at some point. And so to some degree, while I would highly dispute the description “root of all evil” today as a good thing to say to somebody, or that it leaves a good impression, when we rewind to that time period, I think it makes a lot more sense.

[16:12] Because when you look at what people were actually spending their time doing, where their priorities were, I think that maybe that wasn’t so much of an exaggeration. I mean, obviously it’s hyperbole, but it’s less of one.

16:23 — Dijkstra was depressed

Ryan:

[16:23] In your slides, there were so many quotes and snippets of really kind of digging into the true history. Was there anything you found that surprised you when you were doing the research?

Casey:

[16:33] I mean, constantly, absolutely constantly. And first of all, I’ll just say that there’s, as a meta point, I’ll give you a specific thing that surprised me that’s kind of funny. That’s more about a technical thing. But first I want to talk about a broader picture, which is one of the things that really hits home for me whenever I do these, is the personal aspects of it. These people were friends and competitors and all these sorts of things.

[17:08] And it’s all kind of lost to us in a way when we think about computing history in a sterile way. When I read up on what Edsger W. Dijkstra wrote, Dijkstra was the person who kind of kicked off structured programming. No, I’ll call it a revolution. That’s a bit too probably grandiose, but he kicked off that train of thought. He wrote a thing called Notes on Structured Programming. It was a manuscript or a monograph, I don’t remember what they called it.

[17:39] Got a typewritten thing. It looks exactly like a typewritten thing. Right. When he wrote that, he was very depressed, like he was having a lot of trouble in his life at that time. He had gone through this situation where they had done what people today now recognize as pioneering work in distributed computing. He and these other people at Eindhoven University of Technology.

[18:07] I apologize, I can’t say the proper name for it; it’s well beyond my pronunciation capabilities. But point being, he had done this really foundational work in distributed computing and published it, and people today now recognize it as that. It is undisputedly a massive contribution to distributed computing. He did it with a part-time team of people at the university, who—this wasn’t their job; it was a mathematics department they were in.

[18:34] And as a result of that, basically they sort of just got dissolved, like the Department of Mathematics. I guess he wasn’t really specific about it, but in his notes the Department of Mathematics was just like, they disbanded the team. It was like, this isn’t math or whatever, I don’t know, they just had a pessimistic view of it. And he was really depressed about that, and he wasn’t sure what he should do with his life and, what’s the deal here, right?

[18:57] And he was having this sort of, I don’t want to call it an existential crisis because I don’t want to again put words in his mouth. These are historical figures and, you know, I can only. That’s why I try to use quotes to try to show you like what they said. But that’s, it’s just so relatable when you go through the history this way. It’s not just some random guy who wrote some math down in a paper.

[19:17] It’s like people were really struggling with this, and some of the most important aspects of computer history come out of these amazing human stories. And I found that absolutely fascinating. So I’ll just put that out there as one of the biggest rewarding things about looking into this history, if you ever do it, is if you can go find the actual writings of the people, like things outside of just their technical papers.

[19:41] It’s just fascinating. And it’s so much more relatable and fascinating because you feel the story. It’s not just this abstract computer science thing that happened. So that’s one thing. But the other thing I was going to say is when you read through this stuff, I just go through piles of documents and I’m just reading them and seeing, does this fit into this story? Should it be part of what I’m telling, or is it extraneous?

[20:07] Is it something that is interesting, perhaps, but not actually part of it. And one thing that I found, when the VOD of this lecture or talk goes up, I’m going to include this in the notes because I thought it was so fun, didn’t make it into the talk. I found a thing, like a thing from Tony Hoare. So C. A. R. Hoare, who again is another massive figure in computer science. Hoare, Dijkstra, and Knuth are like the three amigos.

[20:38] And they all write to each other. They’re very important people in computer science history. So I found a thing by him that was like a thing he did not decide to pursue. It’s like this note where he’s talking about—he’s talking about this thing that he’s thinking of doing, and there’s just a handwritten thing on him from later on in his life, when he was, I guess, categorizing these documents, and he just writes down, like, I decided not to pursue this because I talked about it at this conference that I was at.

[21:10] And Peter Naur, who’s again another famous figure in computer science history, if you’ve ever looked at context-free grammars and parsing, you’ve probably heard Backus–Naur Form. He’s the Naur in Backus–Naur Form. If you’ve ever heard of the language ALGOL, he was a major figure in standardizing that and writing up the standard and all this stuff. Anyway, Hoare. Yeah, I proposed this thing, I went through and said it, and Peter Naur was like, making multipass compilers is easy.

[21:40] I just wrote a nine-pass one. So I decided not to pursue this. So what’s the thing? I read through the thing and maybe I’m just overreading it with the benefit of hindsight, but he pretty much describes static single-assignment form, like SSA, which is a very standard compiler technique developed in the ‘80s. But this note is from the ‘60s. So it’s like, in my head, I’m like, did Peter Naur accidentally set back compiler science by 20 years by telling Tony Hoare?

[22:18] Well, 15, let’s say, by telling Tony Hoare this is not important, when actually it would have been very, very important. You find stuff like that all the time where you’re like, whoa, what is this? I saw another one too. Just one more I’ll mention. I can’t remember now, I’m sorry again. Your brain kind of turns to mush when you try to dump this many documents into it for a talk. But there was a pretty interesting thing, I thought, where Margaret Hamilton, so the person who managed the Apollo Guidance Computer, she was one of the core programmers originally on the project and then was the manager of the whole OS, like the whole real-time software that ran the Apollo 11.

[23:03] Well, all the Apollo Guidance Computer stuff, right? Which I mean, most people today herald as a very, very significant real-time systems achievement in that era, right? Everyone pretty much agrees with that. She had to write a defense. I think it was in Communications of the ACM, maybe it was in Datamation. They say—apologize. I can’t remember the venue because it had been pointed out as a failure because people didn’t understand the context that the reason that the Apollo computer had to trip those alarms was because that had been used improperly.

[23:40] It was used in a configuration that wasn’t supposed to. And it actually, rather than crashing, went into backup modes and successfully landed. It worked. It actually was a triumph of fault-tolerant engineering. And she had to write a defense of this because people at the time were saying, “Oh, they screwed up. This is an example of why you wouldn’t want to do engineering this way, or don’t write the code this way.”

[24:03] And so again, stuff like that always makes me think, the more things change, the more they say the same. That’s exactly what would happen on Twitter now, right? Some very successful thing that people did, right? They would just get slandered and say that it was done wrong, and there would be a flame war, and then they’d have to come out and have a thing. So anyway, those are just some examples.

[24:23] But there’s so much great stuff in there. I highly recommend it to anyone who likes this kind of thing. You won’t be disappointed if you go dumpster diving, as they call it.

Ryan:

[24:32] When you said that you saw the human stories and that Dijkstra was depressed at that time, what did you see that made you realize that?

Casey:

[24:42] One of the nice things about Dijkstra is that he’s kind of a pretty straight shooter. And so I didn’t actually have to do any interpretation. He literally talks about being depressed. There’s a paper, not a paper, like a thing that he did that’s like a retrospective that’s handwritten by him later on in life. And he literally says, in this period I was very depressed because of these reasons.

[25:12] Now, he could be wrong about the source of his depression. Right. So I don’t want to claim that just because he said it, it’s necessarily true or something like that. But one of the amazing things, I think one of the really cool things about academics, especially that era, is they just produced a voluminous amount of material, and so we don’t have to guess that much. In the private sector, I think it’d be a lot harder, like if you wanted to know what people at maybe Lockheed or something were thinking or doing at that time.

[25:47] I imagine it’s much more difficult just because things are classified or they’re not academic, so they’re not necessarily going to write them all up. So a lot of the internal stuff when you write them, academics don’t really have that. They don’t have an incentive or a prohibition on writing up everything they do. So they don’t just have to have a public-facing statement of what they did or a public-facing paper of what they did.

[26:07] And then, oh, there’s this extra secret stuff. The academics don’t have that restriction. They’re sort of... They benefit from being as forthcoming as possible about what they’ve accomplished, usually.

26:17 — The personal side of goto considered harmful

Ryan:

[26:17] I saw in the slides that you looked at Go To Statement Considered Harmful. That email, do you have any sense of the personal or people side of that?

Casey:

[26:28] So you’re talking about the original Dijkstra letter to the editor. That’s like Edsger W. Dijkstra’s Go To Statement Considered Harmful. The complete story is actually relatively simple according to him. He was at a conference in Tennessee where he was talking to Brian Randell, who’s another computer science guy. He doesn’t quite get the same level of name recognition as a Knuth or something like that, but he has a lot of papers at that time.

[27:04] You can go find him. He’s not obscure. He’s not an obscure figure. I wouldn’t say to anyone who reads the history. You’ll see him come up. He’s talking to Brian Randell and some other people outside. This is exactly like the same thing that happens at modern conferences. There’s the talks and then there’s all the stuff that goes down at the bar. Right. It’s very much that. They’re outside, they’re talking.

[27:25] And according to him, he’s basically giving the same sort of example of a problem with the Goto that he gives in the paper. Right. This idea of enumeration, we can talk about that later. But sort of a side note, he’s sort of saying, like, this is kind of a problem with Goto. And one of the reasons that maybe it’s not such a good idea. And the people who were listening to him were like, Brian Randell and the others were like, you should publish that.

[27:56] That would be helpful because there’s these arguments going on about whether GOTO is good or bad. That was kind of happening at the time already, since around 1959 even, I think there’d kind of been a little bit of that, had been kind of growing, this idea that maybe Go To Statement Considered Harmful wasn’t the best idea as a way to structure your programs. And so he does. He goes back to Eindhoven University of Technology and he writes up this same thing he was saying, basically, and he sends it to Communications of the ACM for publication.

[28:29] Niklaus Wirth, who is the creator of Pascal, a very prominent figure, also was very heavily involved in ALGOL and all this sort of stuff. Language designer guy, he is the editor who is in charge of getting this thing published. I guess I’m not exactly sure how things work at Communications of the ACM. He doesn’t want to wait to have it published. He doesn’t want to take the time to go through the referee process or whatever.

[28:58] I don’t know at that time what the requirements were for publishing an article in Communications of the ACM, but I’m sure that it involved a lot of procedure. It wasn’t just like, oh, hey, I’m the creator of Pascal at that time. He wouldn’t have been the creator of Pascal yet, because Pascal comes later, I think. But either way, he’s like, hey, I’m this important guy. I’m just going to put this whatever I want.

[29:19] Communications. That’s not how it worked. And so he decides that in order to get it published more quickly, he’s just going to publish it as a letter to the editor. Because then there’s no anything goes there. As long as the editors are fine with the content. I assume it doesn’t have to be refereed, it doesn’t have to have any kind of review. It doesn’t have to be, et cetera, et cetera. So he turns it into a letter to the editor and he takes the name of the thing that Dijkstra sent, which was A Case against the GO TO Statement.

[29:47] That was what Dijkstra wrote at the top of the thing as the title. He changes it for letters to the editor to Go To Statement Considered Harmful. Right. So it wasn’t even Dijkstra’s plan to have it maybe be that confrontational. But that’s what happens now. This is not well received, to say the least. According to Knuth, again, I’m just trying to say who said what here. According to Knuth, Dijkstra said he got letters, angry threatening letters, in much the same way you would today.

[30:34] Got people who were very abusive, telling him off. I’m sure part of that is because by all accounts, Dijkstra himself was also somebody who liked to push people’s buttons. That’s well acknowledged. I think Alan Kay is probably the best source for this. He talks about this. He liked Dijkstra, and they apparently got along well. But he’s often said that Dijkstra was someone who kind of leaned into being sort of brash about stating things about programming and kind of relished that position.

[31:07] So it probably didn’t help that he was already sort of known a little bit in that way. But in general, that’s how it went down. And, like I said, that wasn’t the initial foray against Go To Statement Considered Harmful by any stretch of the imagination. But it just kind of maybe, you call it the straw that broke the camel’s back. It was like a flashpoint, might be the way to say it. And then the title, of course, that Niklaus Wirth picked is quite the doozy and kind of clickbaited everyone, as I call it, into that soap.

Ryan:

[31:43] It’s funny because, I mean, social media, it’s similar patterns. People have very explosive first lines, and then the comments are full of all this hate and stuff. In their case, I’m guessing this is all papers. So you submit a letter to that. It’s a physical thing, and people are reading maybe a newspaper or something. I don’t know.

Casey:

[32:05] Yeah, it’s like a periodical. It comes as a bound. I mean, I guess there’s. All the things around us here are hardbound, but it’s like usually with soft. It was a maybe a perfect binding kind of bound thing that has you open it up, table of contents and letter to editor. Right. And yeah, like you can go find. There are a few. If you go look at people’s personal papers, which, like I said, if I did this full time, I’m sure I could find out way more stuff than I did.

[32:34] Right. But some people have gone and looked at the personal papers, and occasionally some of them have been able to digitize some of those and put them online that we can look at. And you can find, for example, online right now, if you search for it, there’s a back-and-forth letters, personal letters between Edsger W. Dijkstra and a guy who is kind of into functional programming and promoting that. And you can look at the correspondence, and it’s kind of.

[33:02] It’s a little, it’s a little flame war. Like, really, like that’s what they did. They didn’t have the ability to do, like, pithy Twitter replies, so it was just on paper. And Knuth did something similar. It wasn’t really a flame because he had, at least when reading comes through, a tremendous amount of respect for the authors of the book Structured Programming, which is Dijkstra, Hoare, and Dahl.

[33:32] One of the things that he published was a series of open letters to them reviewing the book and talking about the things that he didn’t find compelling in it. Very respectful. So it wasn’t. That one wasn’t a flame war. But that’s what they had to do because they didn’t have social media. So Edsger W. Dijkstra wrote all these things called EWD with a number. It was his initials, basically, and then a number.

[34:00] And that’s like this serialized list of all the things he wrote. And he labels them this way. It’s not like some historian characterizes it after the fact. It’s like, oh, I’m doing. He’s like, I’m doing EWD 937 now or whatever. Right. It’s hilarious. But anyway, several of those have been put online that are not necessarily technical. For example, his trip reports. You can go read those and you can read about, oh, I went and stayed at, like, such and such’s house and we went to dinner or whatever, right.

[34:31] So you can find some nice personal anecdotes in even the publicly available stuff. But in terms of a real heart-to-heart conversation, I didn’t have access to anything like that that wouldn’t have just been in a paper, more or less, normally. So it’s a shame. One of the problems with this stuff is it’s so interesting, but it’s not a job to do. If somehow it was a job, I probably would almost take that job.

[34:58] Being the person who crawls through and tries to redocument this whole thing. But, no, computer historian, no one’s hiring, no one’s interested in paying for that.

35:09 — The anatomy of a 35 year mistake

Ryan:

[35:09] You have this other talk, and the title was just so catchy. It was “The Big Oops: Anatomy of a 35-Year Mistake.” What is that? 35-year mistake.

Casey:

[35:21] So this was a kind of funny thing that happened to me that I then did a historical lecture on. I had worked on systems in the past that were basically like editors. You have 3D graphics editors. You have to multi-select things and move them around. There’s a bunch of architecture things you have to learn to be able to write that kind of code and certain kinds of problems you have to solve. A very basic one that I would point out that hopefully most people can relate to would be if I have a bunch of things on the screen, some of which have a color.

[36:00] So maybe this is a drawing program and I’ve got some text and I’ve got some shapes and I’ve got some strokes, some hand-drawn stuff, whatever, and I want to be able to select a bunch of them, and I want the user interface to present to me which things I could edit on these shapes. So I want to be able to edit the color, and I want it to apply to all of them. Right. This is just a basic architecture problem that you have to solve if you’re going to write one of these programs and you want it to be any good.

[36:26] Now, oftentimes, you will see programs where the person didn’t solve this problem, and you can’t do that. It’s very frustrating, right? So a good program, someone has solved that architectural problem in some way. And I was watching, I just happened to be watching because sometimes I do like to watch old videos of computer science pioneer stuff. And I was watching a demo of Sketchpad. I’d seen it before, but I was watching a demo of Ivan Sutherland’s Sketchpad.

[36:53] And Sketchpad, for those who don’t know, is a groundbreaking computer graphics program. It’s done with a light pen, and it was done in the early 60s. And you could draw things like you do in a modern CAD program, and you could do stuff that even a lot of modern CAD programs don’t really offer or only offered recently. I think in the talk I said in 2007 was the first time that AutoCAD got some of these features.

[37:19] You could do stuff like say these two shapes, this line has to be perpendicular to that line, constrain it, and you could just do whatever you want with the drawing and it would resolve for making that happen. And these are very rare in programs even to this day. That’s just not a common operation you would see in a drawing package today. I was looking at this and I was like, how the heck did he solve this architectural problem?

[37:46] How did he do this stuff in the early 1960s in assembly language, with no editing tools, no debugging tools? I mean, this would have been like either punch cards or one step away from punch cards. It would have been hard to develop this stuff. And when I went back and I went through his thesis and I looked at the code, it’s actually documented how he did it. And I realized, like, oh crap, this is actually like an entity-component system.

[38:18] Basically what we would now consider sort of state of the art of real-time architecture for doing these kinds of operations, meaning cross-cutting, operating on this kind of property across many things at once that are all kind of different. And this was something that object-oriented programming — I’m editorializing now, a bunch of people will complain that I’m saying this, but this is just my opinion.

[38:47] Object-oriented programming and the pedagogy around it really struggled with that problem for a long time because they were teaching, despite people who will now claim otherwise. I document it very completely in the talk, so I don’t think they have any leg to stand on. But the pedagogy at the time was all about build a domain model. The domain model looks like your hierarchy. Basically, like I’ve got employees and employees, I’ve got contractors and I’ve got derived, like is this a full-time employee or a part-time employee?

[39:15] That was how they suggested these things should be implemented. That kind of architecture doesn’t work for these kinds of operations that I’m talking about. Entity-component systems do. And generally, that kind of rotated architecture, I might call it, does work for these. What I call the 35-year mistake is the fact that I very ironically, in my opinion, Sketchpad sort of showed in its architecture how you could solve this problem and how you could solve it in arguably an object-oriented way.

[39:52] I think if you squint at it, a person who does object-oriented programming could easily make an argument that you can do an object-oriented version of this. You don’t have to violate any particular principles to do it. It’s just not the hierarchy domain model way that people were conceptualizing it through a lot of the 80s and 90s, let’s say, and even unfortunately to this day. But I think a lot of object-oriented programs don’t do that anymore.

[40:17] Ironically, that was in Sketchpad. The DNA was there, and we could have had it right away because Ivan Sutherland figured it out. But weirdly enough, the whole idea of this sort of inheritance hierarchy with lots of encapsulation around it grew in part out of Sketchpad because Alan Kay looked at it and said the interesting part of this is the fact that you could take a shape and not know what it was and just have this draw circle call on it or something.

[40:49] That was the part they took away from it, which is much less interesting in my opinion and not that architecturally useful in most cases. So in kind of this amusing way, Sketchpad contained, if I’m editorializing, it contained what I consider to be a good architectural idea, but the takeaway from it was a bad architectural idea, is how I would say it. And saying bad architectural idea is a little bit strong because, as I say in the talk, I think there are times when you do want to think very high level.

[41:17] You might want to think about things in kind of the way that Smalltalk thinks about them. So I don’t want to say bad idea in the absolute sense, but bad in the way it ended up being applied, let’s say.

Ryan:

[41:29] Why does that class hierarchy not work for this problem?

Casey:

[41:33] It’s pretty easy to understand, actually. So when you think through an inheritance hierarchy that’s based on sort of what you see in the real world or what you’re trying to model. So you say, the typical example, and it’s brought up tons of times by people who wrote the OOP literature, like Bjarne Stroustrup, like the Smalltalk people, the idea is keeping it with a Sketchpad example, I’m going to have a shape.

[41:58] From the shape, I’m going to derive triangle or circle. And the encapsulation is around that thing. So a circle has a radius, but shapes don’t have radiuses. Shapes don’t necessarily know what that is, right? So it tends to get deferred down and the encapsulation boundary is drawn around that sort of derived class. That’s, like, the most derived version of whatever this thing is.

[42:25] The problem is that creates a lot of headaches when now something that wants to work across a lot of shapes needs to actually do something intelligent with modifying, say, the scale of something or wanting to change its color because it doesn’t really have a way to know what’s going on in these highly encapsulated things. So typically what you end up having to do in those situations is you then have to take this inheritance hierarchy and kind of, in a sense, throw away the benefits of it.

[42:57] You end up having to make at the top all of these accessor iterator things that allow you to probe into that lower class and say, tell me all of the colors that you might be using and a name for them. In a sense, what you end up having to do when you want the architecture to work is you have to rebuild the Sketchpad version on top of the inheritance hierarchy version, which doesn’t really do anything for you.

[43:24] And it’s not to say that there isn’t some benefit that you can derive from having derived is kind of a pun here, I guess, that you can derive from having this idea of saying this thing is like this other thing. Please give me some of the implementation of it. That’s not necessarily a bad thing in practice. The problem is when you teach people that this is how it’s going to work, like you just make this hierarchy and then your architecture is just supposed to work, it really ill-prepares them for the reality that that’s actually not going to solve most of the problems that you have are not going to be solved that way.

[43:56] And again, it just comes from this fact that typically when we’re working with computer programs, most of the time what we need to do is create cross-cutting operations that work with a lot of the data that’s inside these derived classes. That’s actually the thing that we want to do. And it’s really cumbersome and typically a bad fit for the problem space to put that into some virtual functions that are sitting at the top of this very sort of tall hierarchy.

[44:23] It’s usually much better to do. Again, even if you’re doing object-oriented programming, you don’t have to stop doing object-oriented programming. You just have to think about it differently. You have to think about what the objects are differently, where you’re drawing the encapsulation boundaries. It just helps to think about it differently and say, actually, the more important things to model are what are the things that stuff is built out of?

[44:44] How do I build a circle out of objects, right? What are those things? A radius property, a center property. Those maybe should be the objects that I’m thinking of as more first-class citizens. And this inheritance hierarchy stuff about what the objects are that I see on the screen, like triangle and circle, that’s a distraction. That’s not the proper way to model the problem. And again, I don’t think I would be pissing off any object-oriented programmers by saying that today because I think a lot of them use architectures that do think about objects in that sort of rotated sense and not in the domain model hierarchy sense.

[45:20] Not that there aren’t still people who are doing that. But it’s not the only way to go nowadays. It’s not the only way that’s promulgated.

Ryan:

[45:28] because I guess my immediate thought would have been in that case, the base class has the notion of a color or some shared thing, and we just operate. If you’re a shape, then we assume you have a color. But you’re saying refactoring and putting up in there is suboptimal and you want all these subclasses to have some handle to some radius property or something like that.

Casey:

[45:54] Is that what you just described is what tends to happen in systems that are architected this way? You end up with your derived class not really being very much of anything because in order to edit stuff, you had to migrate it all up to the top. And there was actually terms for this in the old days. I don’t still use, like, fat base classes, right? Or things like that. It just ends up your base class has every type of property that any derived class might ever have.

[46:18] You have get primary color, get secondary color, get tertiary color, right? Because something needed three different colors or whatever. And most things don’t use those colors and all these other sorts of things, right? There’s certainly nothing wrong with running a program that way. It’s just, again, why did you bother with the hierarchy? You’re not getting any actual encapsulation benefits from it because you just forced everything in the base class anyway.

[46:41] So why didn’t you just make that be your thing? Just make one class called Shape and be done in terms of the ECS style of implementation. I don’t necessarily want to advocate for it or not because I don’t actually personally use ECS or anything like that, but they are very common. Now the idea there is to just turn it on its side and say, look, typically what we want to do is do things like, okay, there’s stuff that has physics on it.

[47:09] I’m doing this thing, and I want to have physical simulation of entities. Well, the physics system is the thing that probably knows how that should be stored. So rather than having that be data in my derived class that’s encapsulated around the derived class, instead, why don’t I just have the physics system know what physics is, and then if you want to make something that participates in physics, you just have a handle for whatever this thing is, like a handle for my shape.

[47:36] That handle is valid in all the systems. So I can look up the physics properties of the shape and gather it when I want to actually do something with it. But generally speaking, the encapsulation stays around the system. So physics properties are defined by the physics system because, hey, that’s who knows how physics should work. And I just ask about my physics when I need to do something with it.

[47:59] And this kind of gets you out of that problem. And then now you can just compose things. Oh, I want to make something that has several physics elements in it. Fine. I can just have multiple physics handles if that’s what I need, or something like this. I can very flexibly merge these things together. I want something to not have physics. I just don’t ever insert the. I don’t ever ask the physics system to have a valid piece of data for this handle in it.

[48:21] And it won’t simulate any physics for it. Right. Whether that’s the world’s best architecture or not, I’m not going to argue for it or against it. It’s just a lot better than domain model hierarchies, that I will say, right?

Ryan:

[49:31] Earlier we talked about the tradeoff between, I guess, performance code and maintainable code. And I saw you had this one video. It said in quotes, “Clean Code, Horrible Performance.”

[50:03] I feel like this is on the same topic. What’s your take on that? Because you also put Clean Code in quotes. What does that mean?

50:12 — Clean code horrible performance

Casey:

[50:12] So it’s a fairly subtle topic, right? And that’s why the quotes are there. Because the first thing that you have to point out is that clean code is clean code. Object-oriented programming. A lot of these phrases, they mean different things to different people. And so if you’re going to say that this code is clean, it’s pretty hard to find two programmers who will agree on exactly whether it is or not.

[50:39] They all have different ideas about what is clean and what is not. The reason I put that in quotes is I was talking about a very specific thing, which is literally the book Clean Code. The thing where there’s, here are the things that we would recommend that you do in that particular case. What I was talking about is there’s a certain set of ideas that is advocated in Clean Code as these are the things you should do.

[51:04] And they are like you should have everything should be kind of, I guess I would say dynamic dispatch, or at least code shouldn’t know the types it’s operating on, right? Saying it’s dynamic dispatch is maybe it depends on the language whether that’s going to be true or not. But when I write code, I don’t know what types I have. It’s just, I don’t know. Some base classes is what I’m operating on, and they could be anything.

[51:28] That’s thing one. Thing two is functions should be very small. In fact, the numbers are kind of weird. They’re like four lines or five lines. When you go look at the actual things that are claimed in some of this literature, it’s really strange. You’re just like, gosh, that’s very small, right? And I go through some of those things and I say, look, if we were to actually follow these, we get into a really bad state because a lot of the languages that people are going to be using to implement this stuff, like C, for example, which even would be a fairly good case in a lot of cases.

[52:01] At least it’s a compiled language and so on. In a lot of cases, these things are kind of a recipe for disaster. If you’re handing out types where you can’t know the type at compile time, or you can’t know the type, or it’s going to be in a different module, so unless you have really aggressive link-time code generation, it’s not going to be likely that you could figure this out. And you’re saying all the functions should be very small.

[52:24] And of course they’re going to be virtual because they’re on these sort of types that you don’t know what they are. This creates a really toxic combination for the compiler. Normally you can have— I mean, you can have small functions if you want them only four lines long. If the compiler can know that, it can just inline that, right? If it can see clearly what you’re doing, it can just merge those things in. We don’t have a problem if they’re behind a virtual function.

[52:50] So it doesn’t know, it can’t guarantee that it is that type. It can’t do that. And so when you talk about all this stuff, what you’re giving up... People, I think, also because the video—I mean, the video was a very short video as part of a series—I think people sometimes get the wrong idea that I think that virtual function calls cost a lot. They do cost something, but depending on how you want to look at it, they actually don’t cost that much because it depends on whether it’s predicted correctly or not.

[53:19] But in general, the cost of a virtual function is less than you would think a lot of times. The actual reason that it slows things down is because the compiler cannot merge the code together. It can’t inline stuff and remove and reduce the waste. That’s the actual problem. So if you just want to call a virtual function, yeah, if it was a really, really hardcore optimization scenario, then you don’t want anything, probably calling a virtual function in general, because there is some cost to that.

[53:45] Just calling a function has cost. But that wasn’t the really bad part of it, right? So it’s that. Plus there’s an additional thing, which is that it can’t unroll and go wide on things. If the compiler can see everything you’re doing, it can use SIMD, it can widen loops to operate on multiple things at once. It can unroll those loops. There’s all these things it can do. If it’s got these virtual functions, it’s just like, that’s it.

[54:08] I don’t know. I can’t make decisions about that. I have no idea what this thing on the other end is doing. That was my point on that. And again, it’s not meant to be assailing the idea that your code should be easy to read or easy to maintain. It’s just more like these guidelines seem really bad. And also, I don’t think we need them for code to be maintainable. I don’t have trouble maintaining functions that are 30 lines long or 50 lines long.

[54:33] I don’t find that five is a magic number or something, or that it has to be really short. And also I find that it’s usually pretty easy to write code where I know what the types are, that those can be determined at compile time. I don’t think that results in code that’s particularly hard to read.

Ryan:

[54:48] When I was in college and very early, I remember people would recommend that book, “Oh, you should read Clean Code.” So my understanding is that you wouldn’t recommend that to people who are software engineers to read that. I guess the way I would categorize it is a mixed bag.

Casey:

[55:00] So I wouldn’t actually say that I disagree with all of the things in the book, though. I mean, some of the things are things that I definitely do myself. Like giving variables easy-to-understand names is something that I think is good advice, and that’s in that book. Right. And so kind of more, I wouldn’t necessarily say do or don’t read the book. What I would say is, like, I think there’s some things in this book that don’t.

[55:31] That aren’t addressed properly. Like, I don’t think you want to tell people these rules of thumb and not tell them about these other problems. That was exactly what I was pointing out there. And so usually it’s more that it’s like I find that oftentimes there is this tendency, I guess I’ll say, to pretend that we don’t have to talk about performance and we can just say, in fact, premature optimization is the root of all evil.

[56:01] Bringing it back to there, I feel like there’s this temptation and very prevalent practice of sort of taking that idea to mean we don’t have to talk about performance when we’re teaching people things at all. Or like when you write the book Clean Code, you don’t have to talk about performance at all. You can just include a thing that’s basically like, hey, there’s performance issues, but most of the time it’s not a problem or something like that, which is, or if you look at Refactoring, the book Refactoring, very popular, it literally says exactly that in like the OOP thing.

[56:34] It’s like there might be performance concerns, but they’re usually okay or something, right? And that is not true. I just think that’s fundamentally not true. I think these books should include detailed discussions of the actual performance problems that you are very likely to hit if you take some of their advice, because it’s not that it means you can’t do those things.

[56:57] But I’ll quote you back to you. It’s about trade-offs. There is always a trade-off that you’re making, and sometimes if you understand the trade-off, you will make it. You’ll say, I know this is going to cost. This may be a serious performance problem for us, but I think that’s okay. I think it’s not like our performance won’t suffer to the point where it’s a problem for the product. I understand what that is.

[57:23] There’s a huge difference between that and just going like, eh, we don’t worry about performance till the end. It’s completely different. Those are two different engineering approaches. And so what I try to do is encourage people to put that analysis back in, understand the performance trade-offs you’re making, understand how much it might cost in the future to make these fixes. And I don’t really do any AI stuff, but my sort of feeling is more so now than ever.

[57:53] I feel like that’s got to be pretty important because you’re instructing these AIs to do what they’re going to do. And I feel like it would be a bad idea not to know about performance trade-offs because, especially if an AI is going to be doing your bidding and structuring the code the way you want to structure it or doing whatever, it seems like a fairly straightforward part of the process to include performance stuff in those instructions.

[58:17] Right? That would be a natural thing that we would want to understand and do. So I feel like really I’ve been just trying to get that more into the conversation, and I feel like it’s as relevant now as it ever was and arguably maybe more so. I don’t know, but I could be wrong about that.

58:33 — How to write high performance code

Ryan:

[58:33] What are some things that are in your mind? They’re those high-value performance things to consider that don’t cost a whole lot to think about.

Casey:

[58:43] So to me, awareness is the number one thing, right? Understanding roughly the performance characteristics of the hardware that you’re working on, which includes things like if there’s a network, what does that look like? And so on. It’s really about awareness more than anything else. Because a little bit of awareness can go a very long way if you understand the basic concept that network latency versus network throughput are different things.

[59:15] You can make upfront decisions about structuring code such that you batch things properly and so on and so forth. If you don’t understand those things, you could get very far down a project only to realize that everything you designed and the way it works is all very serial and really just cannot be accelerated over a network at all. It’s never going to run reasonably or something like that.

[59:36] Right. And so I think the lowest hanging fruit is just to get some education in performance, get some education in how to think about the way a machine works and what makes it fast or slow, what it struggles with and what it doesn’t. And just to keep that in the back of your head, because at the end of the day, if you have that knowledge, I think you’re very unlikely to make the kinds of architectural decisions that will be hard to undo later.

[01:00:06] Right. And so that’s really the majority of it, I think. And that is by far the highest impact, lowest cost thing you can do is just do that training once, because once you have it, it’s with you forever. Once you understand how to think about performance, it can always be there. And at any time you can sort of have that alarm bell of like, “I don’t see...” You’re always kind of looking like, “I don’t see the path towards this running well,” then that’s your cue to stop.

[01:00:38] Maybe we need to rethink how we were making some of these decisions. As long as you can see that path, you can delay most optimization work. As long as you can see the path, like here is how we will optimize this, you’re in pretty good shape because if you’re making those trade-offs correctly, you’re not going to paint yourself into a corner where there’s nothing that you can do other than scrap and rewrite.

Ryan:

[01:01:05] You had this one video title. It said, “Where does bad code come from?” I guess, yeah. My question to you is: what’s the answer to that? Where does bad code come from?

01:01:18 — Where bad code comes from

Casey:

[01:01:18] Yeah, that’s a good question. My feeling on where bad code comes from is that, for everything that I’ve seen on projects in the past, the bad code all seems to come from roughly the same source. And that is not dealing with the actual thing that’s happening. Right. There’s a lot of ideas in computer science about sort of doing upfront design and making a lot of decisions without ever really implementing anything.

[01:01:54] And I have found that universally that leads to the worst kind of code. And the reason for that is, I hate to be pessimistic, and I hate to be pessimistic about my own ability, but I’ll just say, for me personally, I am not able to correctly hold all of the details for most complex software in my head at once. There’s just a lot of stuff going on at, like when the actual instructions hit the CPU, there’s a lot going on down there that’s very hard to keep all in your head.

[01:02:34] And so when you approach an upfront design, typically, unless the problem is very stupidly simple, right? When you approach something with upfront design and you think you’re going to get all of this architecture worked out ahead of time, you forget some important things. You make decisions without realizing, oh, wait, there’s this thing that has to happen there that means that this isn’t really the right way for these pieces of code to interoperate.

[01:03:03] Then you end up seeing these hilarious APIs or something where it’s like you’re just like, how did this end up being the way that you do this? I have to create all these objects and I have to connect them together with these things and I have to create this weird filter graph thing and then I have to call compile on it. But only you end up with these insane things. And you’re like, all I wanted to do was call this one thing that said, like low pass filter this buffer.

[01:03:26] It could have been one function call, right? It’s like, how did you get there? And the answer is because you weren’t actually dealing with the actual problem and looking at what the actual code looks like when you actually want to solve the problem in practice. And so my idea where bad code comes from is usually just that. It’s that. It’s this failure to engage with the actual reality of the situation.

[01:03:52] Maybe there are people out there. I certainly have never met any, but maybe there are people out there whose brains are so expansive that they can actually hold all that in there and they can therefore just do it upfront. But other than for various sort of constrained problem spaces, I’ve never seen that work. And there are some times when that’s the case with, if you’re just doing, I’m trying to do this particular mathematical operation, that might be a case where you can work it all out on paper because it’s very specific what it is.

[01:04:21] It takes these N inputs, produces this output. And I can work, right? But when you’re talking about architecture, like, hey, it’s a web browser, right? Some complicated thing, the details are all that matter to me, right? It’s like, it’s where all the complexity lies. And if you try to do the design without reckoning with them, it just leads to bad results in my experience.

Ryan:

[01:04:41] So how do you avoid writing bad code in that case?

Casey:

[01:04:45] I try to start with the actual solutions to the problems and work up from there. So one way to say it would be instead of top-down programming, bottom-up programming as much as possible. And top-down programming isn’t really the same as designing up front. But I’m just saying as a contrast there, try to start with the things that you know you need. It’s like, okay, if I’m building this thing, it’s gotta have a rasterizer.

[01:05:10] All right, I’m gonna start making a rasterizer. I’m gonna see what kinds of stuff that does and how I would like, what’s the most straightforward way to call this and use it. Okay, let’s make an API out of that, right? Build it out of steps where the abstraction comes from actual working code that gets abstracted rather than pushing the abstraction down from the top, where I say, this is how I will abstract the rasterizer.

[01:05:36] And now I go write the rasterizer and the thing that uses the rasterizer, only to find that that is not a very good way to use a rasterizer. So to me, starting with that and working upwards is the best way to ensure that you will get an architecture that is at least good for one thing. Now, if you want it to be good for multiple things, you probably need a couple different. I want a few different varied things that would use this rasterizer that I will kind of work on together, and I’ll make abstraction decisions that work across all three of them.

[01:06:07] That’s a good way to make a more reusable API that works for a lot of things. But I never want to just sit down and say, “How will I design an API for lots of people to use a rasterizer without actually writing any of it?” It’s like, no, no, no. And I definitely can’t do it. And I would question someone who says that they can. Unless one caveat: they’ve written a lot of them before, right? If you’ve written tons before, then you’ve sort of done the thing that I’m talking about already.

[01:06:33] And you might remember, oh, it’s this way, right?

01:06:37 — Why design docs before code is a bad idea

Ryan:

[01:06:37] I think there’s a lot of common advice, especially at these larger companies where there’s this process to write a design doc and put together everything in advance, present it before you write any code. And it sounds like you’re saying that that’s a terrible idea.

Casey:

[01:06:57] I think that’s an absolutely terrible idea. Now, that doesn’t mean that I wouldn’t be okay with that process with a slight modification.

[01:07:05] If what you’re doing during that time is writing test code in the way that I’m saying. So we’re going to write a little experimental rasterizer. We’re going to do those things. We’re going to start, and then our document that we’re producing is, here is what we have determined is a good API based on these experiments. I have no problem with that. That’s following my procedure pretty much to a T. And I don’t tend to produce documentation because I work on smaller teams.

[01:07:32] I don’t have to have a 1,000-person org know exactly what I’m doing there or whatever. So I wouldn’t produce upfront documentation normally in a case like that. But if you wanted to, that seems totally reasonable. Right. And that is communicating our research into how this should be structured out to a wider audience. But we still did the due diligence of determining that it really does work in practice.

[01:07:55] It’s not just our guess about what it will be.

Ryan:

[01:07:59] I see. Okay, so you’re saying kind of this pair design, pair prototype, just build as you go, but it’s just so you get a sense of the reality of the design.

Casey:

[01:08:10] Yeah. And I think I would be very comfortable with the team that wanted to work that way. If they’re like, look, we want to produce, we want to document this thing before we start developing. That’s just how we feel more comfortable with how we’re going to do it. Maybe it’s a very large org, maybe we have some very good reasons for that. Then I would say totally fine. It’s just, I want to hear if I’m going to be comfortable.

[01:08:31] I want to hear that that process is not, we’re just typing on paper. We are actually testing these API design decisions, and they come from looking at actual usage code that we have made and that we have implemented at least experimental versions of, so that we know they really do account for all the details that we’re likely to encounter in practice. That’s what I want to hear. Right. I don’t want to hear, we thought it through.

[01:08:57] I’m like, did you think it all the way through? Because I’ve seen a lot of times where people thought they did and they didn’t, including myself. That is not. That is not. I always think it all the way. So it’s like, no, this comes from a personal place of I forgot the thing. I forgot this important thing and I made a really stupid design, right?

Ryan:

[01:09:14] There’s another video you had. It was titled The Only Unbreakable Law.

Casey:

[01:09:19] Oh, yeah.

01:09:20 — The only unbreakable law in software engineering

Ryan:

[01:09:20] What is the only unbreakable law in software engineering?

Casey:

[01:09:24] So, yeah, that was a lecture I did where I was trying to think of, if I was going to say one thing about architecture to people, right? If I was going to say one thing about architecture, what could I say that isn’t probably wrong? Because you think about a lot of our ideas about architecture, it’s like even the stuff I just said to you, who knows what we’re going to be thinking 10 years from now.

[01:09:47] It’s like, they’re not really laws, they’re just observations that we’ve had that maybe this is a good way to do things, but it’s not really a law, right? So the only thing I’ve seen that feels like a real law to me is the thing I cover in this lecture, which is Conway’s Law. And this is a. It’s not a real law in the sense of like the laws of gravity or so, like, it’s not a physics law.

[01:10:14] Like a physicist would laugh at calling it a law. It’s probably not even a theory; it’s more like a hypothesis at this point, right? But in terms of things that I would place money on being validated as a law at some point, if we ever had the means to do so, this would be one of the only software engineering things that I think would qualify. And it is that if you would like to organize a series, like a set of people to work on something, or in the modern parlance, let’s say agents.

[01:10:56] So it could be. It doesn’t have to be human. It’s any system that has sort of its own internal processing, like a human brain does or like a computer does. If you need to partition a problem into a set of workers in this way, where the communication between those workers is slower than their own internal computation ability, which we would all agree is true about humans, it’s true about two computers running an AI model as well, right.

[01:11:25] The amount of time it takes to send information back and forth to them is significantly slower than what the GPU can be processing directly on its own, right? In systems that look like that, when you start to tackle a problem, in order to get the benefits of that parallelization, you will have to make decisions about who is working on what, at least some decision. For example, if we are making a car and you and I are the two people who are going to make the car, I’m going to design the body of the car, you’re going to design the wheels.

[01:11:54] Right. Just a decision someone could make. Once you’ve made that decision, you have now locked in the fact that the iteration on the design will be slower across that boundary than it is interior to either of the two parts. So, for example, your ability to improve the design of the wheels on their own and my ability to design the body of the car and improve that on its own will be much faster than our ability to design the interface between the wheels and the tire.

[01:12:32] Or it’s not even just the interface, the degree of harmony between the designs. So the degree to which your wheels complement my body design and my body design complements your wheels, that they all are considering the trade-offs together. Right. And Melvin Conway’s paper, which lays this out, this is a very early paper. It’s in the 60s at least. Was it in the 50s? It’s way back. When that lays this out, points out the consequence of this is that products or any.

[01:13:06] And when I say product, I mean anything that comes out of one of these design processes or development processes. Products will have a similar structure to the organization that produced them. You will be able to see in the product evidence of this because the degree to which things are able to harmonize is restricted across that boundary. So when you look at the object, it will have that boundary visible in some way.

[01:13:33] Right. It won’t be quite as good at the interface between these two things or in the way that they work together as the things are themselves to themselves. It gets sort of flattened, like one way. The rule is pithily stated is products look like the org chart, but that’s not exactly what it says. But that’s kind of a downstream consequence, and boy, do you ever see it in the real world.

Ryan:

[01:14:03] So, I’ve heard that in the context of these big tech companies, like, almost like it’s a bad thing to quote unquote ship your org chart, or basically the product you put out there has these inorganic, yes, things. So it sounds similar to what you’re saying.

Casey:

[01:14:24] Yeah, it’s. And I think part of the reason I think it is kind of an unbreakable law is I think it’s somewhat unavoidable. It’s more something that you just have to be aware of and mitigate to the degree that you can. Like, you just have to understand, look, that lower communication time, whether it’s humans or computers or whoever is doing this work, that lower communication bandwidth is whatever.

[01:14:54] That just means that where we draw these lines has consequences for what we ship. And so we really want to try, over time, to align those boundary drawings to places where it will have the least bad impact on the result of the product. And that’s true for org charts, like I said. I suspect it will also become true for AIs and things like that, where it’s like, you will want to partition these problems in ways that respect that line drawing, because that will be less good than the thing that can be wholly solved by one unit, whatever that unit is.

[01:15:36] And so to me, it’s a very helpful construct for thinking about things. It’s also a very good explanation for why you see something somewhere. You’re like, why doesn’t this just integrate this? It’s like, because there were two different teams. It’s like, I’m sorry, that’s just the answer. And it costs a lot more for those two teams to have integrated this. That’s not how we work. Right.

01:15:57 — How he got into programming

Ryan:

[01:15:57] I’d love to hear more about your career and how you got into programming.

Casey:

[01:16:04] I started programming when I was really little because my father worked at Digital Equipment Corporation, which is actually a very important company in that era. No longer exists. Right. They would be sort of like almost like a case study in how not adapting to a change in technology leads to your demise. They were one of the most important companies in the world for computing. If you ever heard of a PDP-11 or a VAX in computing history, that’s them.

[01:16:35] Right. They made these things that were foundational computers in computing history, gone today. Right. Part of them got absorbed by Intel, part of them got absorbed by Compaq, but they don’t exist. So he worked for that company. And so we always had computers in the home, and I learned to program when I was little, and that’s just sort of always what I wanted to do. I really enjoyed it.

[01:16:59] And so I ended up getting an internship at Microsoft when I was pretty young, and I met some people there. I came out to — I should say, I grew up on the East Coast, so nowhere near Microsoft. I met some people there. I ended up coming out to the West Coast really early on. This would be in, like, ’95. So it didn’t feel early at the time. It felt late in computer history. But nowadays, like, oh, wait, ’95.

[01:17:26] Oh, my God. Before everything was on the web, right? There was a web, though it was nascent. I ended up coming out and working sort of in the game industry, and that’s what I did ever since. And I’ve mostly always worked on game technology. I’m famously absolutely terrible at understanding game design. People who watch my stuff know this about me. I’m very, very bad at it.

[01:17:54] I’ve tried to make games a couple times and I just cannot do the design side. I’m awful at it. But I really enjoy the engine stuff, and I feel like I’m okay at it, and I’ve been able to contribute to projects. So in general, most of the time, if you’ve used code that was written by me, you probably used it in the context of a video game that was using technology that I wrote. And an example would be I worked at RAD Game Tools on a character animation system.

[01:18:23] I wrote the entire thing myself. That is used. Well, that’s not entirely true. I wrote the entire thing myself, except for the texture compressor, which Jeff Roberts wrote. There was this texture compressor that you could use as part of the pipeline for the exporting and stuff like that. It was a pretty cool project at the time. I’m really proud of it. The first version was awful because I was pretty new at the time.

[01:18:44] The second version, I thought we did a really nice job. And it was made at a time when people didn’t think you could do licensable game technology of that kind because it was too hard to integrate into things, like you couldn’t get the performance or whatever. And we did a lot of things that I think were pretty innovative at the time. And we were able to make something that was very successful. And it’s.

[01:19:04] I mean, we released the first version of that in ‘99. It’s still in use today, largely unchanged from the architecture that I guess was the second version I did in 2001 or something. There are a few changes to the architecture that people have made over time, but it’s largely unchanged. I found out recently Baldur’s Gate 3, which was a big game from Larian, came out. I had no idea. It turns out they use it.

[01:19:30] It’s in their engine or whatever. So I’m very proud of that product because I think it was in tons of games and was a very hard problem to solve. And I think we solved it pretty well. It’s largely irrelevant today. I would say it’s still in some people’s engines over time, but it’s not the kind of product you would make today because nowadays engines are monolithic and you license them as a whole. Typically, you wouldn’t be getting a character animation library.

[01:19:57] It’s going to be something that’s built into Unreal Engine or built into Unity. It’s not the kind of product you would probably consider making today. That was mostly, in terms of things that I’ve done that people might actually have experienced or used, like folks at home. If you’ve played games, you may have played something that I had a hand in at some point, but only the technology, not the design.

[01:20:20] I also worked a little bit on The Witness, which was a game by Jonathan Blow and team that I thought was absolutely fantastic. And I did some work on the walk system there that I was pretty proud of. I thought it came out pretty well. There’s some parts of it that are really janky. I don’t think I did a very good job of the actual implementation of it, but the design was pretty good, let’s put it that way.

Ryan:

[01:20:43] Early in your career. So you worked at Microsoft and then went to go.

Casey:

[01:20:48] I was only ever an intern. I never actually, I mean, I guess that’s technically working there, but I didn’t ever work there as an employee. Employee, right.

Ryan:

[01:20:55] As someone who’s writing code, I mean, there’s a lot of different paths, but there’s one path I imagine: you go and you work at one of these big tech companies and create their tech products, I guess. And then video gaming seems like another path. And I think there’s other paths as well, of course.

01:21:16 — Why he didnt work in big tech

[01:21:16] What drew you to going towards video gaming and not continuing down the Microsoft path or something like that?

Casey:

[01:21:25] I think that there’s a bunch of things I could say about that, but they’re probably all BS. The truth is probably that I’m probably more affected by the people around me than I would like to admit. I mean, I guess I don’t have a problem admitting it now, but at the time I probably wouldn’t have said that, right. I could envision an alternate past where I did stay at Microsoft and try to get a job there and work there instead of being just an intern and then going off and working at a different place.

[01:22:00] The reason that didn’t happen was because of the people who I worked with there when I was an intern. I fundamentally really like. Nowadays I fundamentally really love doing things like analyzing assembly code or looking at exactly how a microarchitecture is working and these sorts of things. If I had gone to Microsoft and had just happened to be under some people who were doing that kind of work and had taught me how to write device drivers in assembly language or something like that.

[01:22:33] I might still be there today, right? That’s not what happened. The capsule summary is the group that I was supposed to be in, the way that they did internships at that time was that a set of interns, say three or four of them, would be underneath a particular manager who was going to be managing those interns. So there were a couple of us who were supposed to be reporting to this guy whose name I won’t mention, just in case for some reason it’s ancient history.

[01:23:02] He probably wouldn’t care at this point, but we were supposed to be reporting to this particular person. And literally the week before we arrived, he has this massive flame-out with upper management. Leaves, just walks out of the building, and has not been heard from since. So we show up, and there was me, a guy named Rudy, a guy named Rajeev, and we’re just interns. We show up, we’re like, hey, how’s it going?

[01:23:39] And we report to this test guy, this SDET guy named Scott Latham, really nice guy. And we’re like, we’re supposed to be reporting to a program, like a software, or what’s going on? This is one of the guys in the test org. He’s like, yeah, that guy’s gone, he’s out of here. And me and this other guy, Bruce Johnson, are going to take care of the interns because, eh. Right. We don’t really know what you’re going to do.

[01:24:13] The first thing I think Bruce had me do was like an ANI cursor loader. There’s this format, I don’t know, ancient history now, but there’s this format for animated cursors on Windows. If you ever see the stupid little walking dinosaur or the, like, no one uses these anymore, I don’t think. But they were this thing that was in there. He had me write a parser for loading ANI files, right? Like it’s just meaningless stuff.

[01:24:36] So my experience there was pretty lame. I was like, this is kind of dumb. I don’t really want to work here. But it could have been totally different if I had been on some team that had really inspired me to learn the stuff that I didn’t know. I didn’t know assembly language at that time, really, at all. I think the only thing I’d ever written assembly language was a joystick polling routine for DOS because it was the only way you could poll the joystick in DOS.

[01:25:00] I think that’s really it, if I’m honest about it. I think most of it is that I think even if you’re a very brash youngster, which I was, and even if you think that you’re very independent, I think you definitely gravitate toward people who you see doing things you think are impressive or technologically interesting. And that is just not the experience I had at Microsoft. Now, there were some people who I like, the people I went out to go work with were people in other parts of Microsoft who were leaving Microsoft to do a startup company.

[01:25:34] And those were the people that I was really impressed with. Those are the people that I wanted to hang out with. So, if I just look at it in arrears, if those people had just been staying at Microsoft and doing something, maybe I would have gone.

[01:25:47] And I think that’s the truth of it, as best I can determine at this point anyway.

Ryan:

[01:25:51] So you moved across the country to go work with those people at a startup?

Casey:

[01:25:56] Yes.

Ryan:

[01:25:56] That feels pretty risky to do.

Casey:

[01:26:00] It was especially because at that time, a startup is not really what you think of as a startup today. Like, a startup was just some scrappy people in a crappy office doing stuff. Nowadays, you think startup, it’s like, well, we’ve got $10 million in VC funding and we have free sodas and whatever, like in the break room, and a masseuse comes in periodically or something like this.

[01:26:22] Right. I don’t know what the now version of a startup is, but it’s very different, right? So, yeah, it was pretty risky. And I don’t really know why I did it. It was mostly just because I hadn’t had really good experiences with education. I didn’t really want to do college, or I didn’t want to do any more education. I wanted to actually work, for whatever reason. And this seemed like a.

[01:26:52] But this was just the easiest way to do that. And I don’t really regret it, honestly. I’ve been able to learn most of the things that I would have needed to, if I was gonna go do a PhD or something. I’ve ended up doing work that is the sorts of stuff, like I’ve been lucky enough to be in positions where I could go spend a few months writing this particular linear least squares solver or something like that.

[01:27:19] The kinds of things you might have done as a thesis or something, or that walks on somebody done The Witness. Those sorts of things are the kinds of things you would have done had you decided to get a master’s degree or something like that. So I’m fortunate enough that I’ve kind of been able to have that part of the education because not everyone gets that opportunity. You may never get, if you don’t do a master’s thesis or you don’t do a PhD thesis, you may never really get the chance to really do deep work on one problem and learn a lot about it, read a lot of papers, maybe make some novel contribution in some very small way usually.

[01:27:59] Right. And so I was just lucky enough to do that. So I don’t really regret not doing something like that, but I think had I not had those opportunities subsequently, I could see being regretful about that because that is something that I do enjoy doing. And if I’d never had the chance, I would have been sad.

Ryan:

[01:28:18] So you went and you traveled to this startup because there’s no funding, it sounds like.

Casey:

[01:28:25] So, yeah.

Ryan:

[01:28:26] What, you? It’s just a group of guys. Was there pay or...

Casey:

[01:28:30] It’s just not really. No, it was. It was pretty rough, and it didn’t last very long. Right. I ended up going to work at an actual game company shortly after that called Gas Powered Games. And then shortly after that I started working at RAD Game Tools, which I stayed at for quite some time and where I did that character animation system and stuff. So it was pretty quick into not doing that, but it was still a pretty educational experience for me.

[01:28:53] And also what I will say is at the time, I worked with a guy named Chris Hecker, who I have to give basically complete credit for teaching me basically about reading technical papers. Before that time, I just thought math was kind of stupid and an annoying thing you had to do in class. Right. And I definitely didn’t know anything about reading a SIGGRAPH proceedings, really.

[01:29:26] And I may have been aware. I mean, I was aware of SIGGRAPH. I knew what it was. But I don’t think if you’d handed me that binder, I would have known what that was or what to do with it. I think the biggest takeaway that I got from that, other than startups are hard and maybe don’t do them unless you have a lot of people and a lot of funding and aren’t really that much on the line, maybe.

[01:29:51] But the biggest takeaway that I got from that that was positive was just a much deeper appreciation for math, a much deeper appreciation for research, how to read it. That’s where I learned to use CiteSeer, which would be kind of the precursor to Google Scholar or whatever. Crawling references and all that stuff. In a lot of ways you could say if I hadn’t had that experience, I bet I wouldn’t have given the two talks that we’ve talked about for most of the interview.

[01:30:17] Because what are those talks? They’re me crawling every reference. They’re just me going back and back and back and looking at everything that I can find. And that’s something that if no one ever conveys to you the importance of reading, the scholarship and being aware of what’s being done, and also teaches you how to read a tech paper and how to parse through it, I don’t know that you get that.

[01:30:42] I don’t know that you get that.

01:30:44 — Should you work at a startup early on

Ryan:

[01:30:44] I remember early in my career, I think there’s this common path of these big companies, and then there’s this thought of, oh, me and a couple buddies, let’s go build something. Now, with your experience looking back, if someone out there is young and thinking about starting, would you say go that common path or would you say take the chance?

Casey:

[01:31:10] It’s a really tough question. And I think the advice is that you have to think about what it is that you want to do every day. So in my mind, the things I regret are the times when I’ve had to do things or chosen to do things that I wasn’t really that happy doing every day. Right? And so I think you just have to optimize for the experience you want to have. Because a startup might work out, it might not.

[01:31:45] It’s always a risk, right? You may go to the big company and it may be cool. There may be interesting things there, or there might not be. You don’t really know. You’re making a kind of blind decision. And so I think you kind of have to optimize for what do you want your day to day to be? What do you want the experience to be? And you want to keep a running talent, you want to be aware of it.

[01:32:12] If you make a decision and six months in, you are not liking what you’re doing every day, then you need to get out of that, right? That’s my opinion. So I don’t know. I think most people probably, if they sit down and actually clear their mind and aren’t engaging in too motivated a reasoning just for themselves, go, what do I envision working at the startup will be like? What will I actually be doing every day?

[01:32:40] Am I gonna like that? Is this gonna be thrilling? Trying to make this thing work and being kind of on the edge, and do I want the kind of war stories of the things we had to do to pull off the demo or whatever? Right. If all that sounds exciting to you and you wanna have that experience, then you should do that thing. And I would say, really, I really mean that.

[01:33:05] I don’t even think you should take into account, even, like, let’s say, the startup’s gonna fail. If you want to have that experience, then you need to do it, right? It’s just like anything else. It’s like going and backpacking across Europe or playing guitar at a nightclub. It’s like, you may know that these things are not profitable. Is that the experience you want to have?

[01:33:26] Are you going to look back on that six months or a year or five years or whatever, the amount of time you’re thinking of committing to it? Are you going to look back and say, I’m glad I did that, or are you gonna be like, that was time that I would have rather spent some other way? Right. And so I think most people can probably, if they’re honest with themselves, at least make a pretty good guess.

[01:33:45] And that guess, if they’re honest with themselves, is gonna be better than any advice I’m gonna give them. My guess about what you should do is gonna be worse than yours. So you should just make that determination. Because another way to look at it is, you know, the big company side, maybe you just wanna have some security. Maybe when you’re starting out, I mean, I’ll just give some examples.

[01:34:04] Maybe you want to spend a lot of time dating. You want to find a partner, and you want to have a relationship, and you want to have a family. Maybe that’s more important to you. Maybe the startup, yeah, it might be fun. Maybe it would be more money if it worked out or whatever. But maybe the stability is actually going to be something that you would really benefit from because the sorts of things that you imagine being fulfilling in your life are not all around what you’re going to be doing in computing.

[01:34:34] I think you can have some guess about those things if you just let yourself have some space to think about it and don’t engage in too much motivated reasoning about it. And I think that would be the best way to make a decision, if you’re going to make one. That’s my feeling about it. Anyway.

01:34:52 — What video game engineering is like

Ryan:

[01:34:52] We talked a lot about video game engineering, and I have no idea what goes into video games. If you were to just boil it down to the big pieces that you need for a video game, what are those software components?

Casey:

[01:35:07] So the biggest thing that’s different about a video game is that the sort of original mental model that you’re taught in programming, which is the standard I O, is not how it works, right? So the biggest mental shift is just like, oh. The way that a video game or a simulation of any kind works is I need to have some world state.

[01:35:37] And I’m constantly updating that world state on a regular interval. Everything is always happening. There’s no “I wait for input and then I do something,” right? Everything is always going because it is real time. The time is going forward. So the components tend to be built around this idea, and it depends on what level of sophistication you end up getting into. But in general, you need a way of storing that world state.

[01:36:03] So some kind of. We usually call these entities, like the things that make up a world.

[01:36:08] So some way of modeling what is in the world. Where are things in the world, what is their state, what are they doing right now? So you need something that does that, something that is in charge of advancing that state. This is a mixture of several things. It could involve physical simulation, it could involve AI, not necessarily large language model like the modern notion of AI, but pathfinding, making a decision between whether I should attack the player or not, that sort of AI.

[01:36:42] So we have the world. Some way of showing the world state, some way of advancing the world state, like an update step that may involve lots of things like that. Then some way of presenting the world state. So a renderer. Right. And this is something that typically has, I guess I would say it’s traditionally been a very important part of a game because the visuals are often something that is a selling point for games.

[01:37:12] People produce trailers, certainly in the AAA space. We’re trying to show you how cool the new lighting looks and the whatever. So that is a pretty big component of games. Oftentimes you go look at an indie pixel art game, it might be much smaller because that’s a much smaller part of the problem. Now that’s generally what the shape looks like. There’s a lot of little pieces there. Typically nowadays we would have some way of asset streaming. The renderer needs to have things like textures loaded, models loaded, things like that.

[01:37:45] We need world data, physics, collision models, these sorts of things. They might be too big to fit in memory, or we don’t want to spend a lot of load time to load them up front. We want to stream them in as they’re necessary. So there’s typically this thing that’s sitting there constantly grabbing things off of disk, caching things, pulling them in and out of a cache that’s being used by the renderer, by the physics, and so on.

[01:38:04] So that’s another common component that’s new, didn’t used to be there. There’s going to be an audio and music system. Obviously, that’s in charge of when sounds are triggered in the world, ambient sound effects that are happening in the world, music that’s playing, continuity. Again, all of that’s interacting with the world state to know which ones of those things are happening. The update step, which will be triggering things in that sound system.

[01:38:28] And of course, the asset streaming to load what sounds are being played or load the music. So we typically have that. And then I’m trying to think, I’m trying not to leave out any major components. In a modern context, there’s often networking. So we want to have a way for multiple of these game clients to communicate with a server. So typically what that means is that world, that state of the world might be provisional. It might be sort of a predicted model of the world that’s not the real model of the world.

[01:39:01] The simulator is actually running. The authoritative simulator is actually running on a server somewhere. And I am merely communicating with it to find out what the world state is. And then, because I don’t want to wait, I don’t want the latency of going all the way around, I am predicting the motion of things forward in time based on the last information I have. So that’s another kind of way that things tie in.

[01:39:27] If you want to start doing things like competitive multiplayer and all these sorts of things. Right.

Ryan:

[01:39:31] That makes a lot of sense because I used to play video games, and then these MMO online games, when I start to lag, everyone continues forward.

Casey:

[01:39:43] Yes.

Ryan:

[01:39:44] And then my Internet catches up, and then everyone jumps to where they’re actually supposed to be. Okay, that makes a lot of sense. I pulled some of your top tweets. I thought that might be interesting to discuss.

Casey:

[01:39:57] I’m sorry.

01:39:57 — Why preventing recursion is reasonable

Ryan:

[01:39:57] So someone said, NASA does not allow recursion in their code. How crazy is that? And then you said, not even slightly crazy, and it went really viral. Can you explain why that’s not even slightly crazy?

Casey:

[01:40:12] When you’re writing functions in a procedural programming language like we have, and that NASA is probably using, you are able to use the program stack for storage. I mean, that’s what it’s there for. It’s what local variables are. I call a function, I get some storage space on the stack. The compiler did that for me. It also saves the return address when I make a function call. That program stack is keeping track of where in my code I was, so that when the thing that I called is finished, I get back there, not some other point.

[01:40:50] Neither of those two things are things you couldn’t implement yourself. They’re both things you could do. You could break the thing up into pieces. You could have ways of remembering what they were, like a state machine. You can have your own stack that you put data on and access it. Right? So the only thing recursion really does is it allows you to leverage the fact that someone already wrote that code for you and maybe more convenient to use because it’s built into the language.

[01:41:18] Now, there are some things, if you really want to get super technical about it, there are some things that happen at a CPU level when functions are called, but we can put those aside for now if you want to talk about them after we wrote the code. So the reason that I don’t think it’s crazy to go, we’re not going to use recursion to implement an algorithm, is because if you’re doing that, you’re kind of just YOLO swagging that there’s room on the stack for whatever it was that you were doing, right?

[01:41:44] And you can’t even really check unless you’re going to do some kind of weird thing. We could sort of do a thing where we go like, okay, let’s try to determine how many more iterations, how many more recursion depths we have before we hit the end of our stack. We can do those calculations, but it’s like, eh. And also if the compiler changed something about the layout, it wouldn’t be true anymore, and so on and so forth.

[01:42:07] So it’s like, so if I’m NASA and I’m like, hey, I don’t want my astronauts to crash into the moon, it seems much more logical to say, don’t use recursion. Just figure out whatever this thing was you’re going to do, turn it into a loop, keep a stack, and make a state machine for it. We can reason about that much more clearly. We know exactly how far it’s going to take. It doesn’t matter what compiler, it’s going to work the same way every time, and it will know the bounds.

[01:42:31] Precisely.

[01:42:33] Very sensible to me.

[01:42:35] And again, you assume at NASA that they’re not doing it for their health. They’re doing it because they have hard constraints on the problem domain where someone’s life is at risk, or at a minimum, many millions of dollars in equipment is at risk. If you screw up, if your stack falls, if you get something where you’d recurse too many times and hit the end of the stack, that is not just a, oh, I rebooted the computer or restarted the software.

[01:43:08] So I don’t know. Hopefully that makes sense.

Ryan:

[01:43:11] Andrej Karpathy had this famous tweet that kind of coined the phrase vibe coding.

Casey:

[01:43:16] Yes.

Ryan:

[01:43:17] And then you said, if you thought software was bad today, buckle up, because it’s about to get a whole lot worse.

Casey:

[01:43:23] Yes.

01:43:24 — Is vibe coding bad for the industry

Ryan:

[01:43:24] It’s been about a year and a half since that tweet came out. The tweet came out in February of 2025. Would you say that that was an accurate prediction?

Casey:

[01:43:35] To be clear, I think the whole lot worse part is predicated on something that may not happen. And that is that the idea that we just kind of type some stuff into a computer and ship it to prod, right? Basically like, hey, could you make me a thing and then publish it, becomes a common way of doing things, right? For example, also done by people who maybe don’t have a computer science background. I think it’s fair to say, and that I wouldn’t be being sort of overly dismissive of AI at this point, to say that a person who is well trained in computer science using an AI to make code right now can make substantially better code than someone who doesn’t know anything about computer science, who is just given Fable and types some stuff in.

[01:44:27] The difference is rather dramatic, I would say, from everything that I’ve seen. So part of my concern that I was trying to express in that tweet was like, if the idea is like, we’re just going to type stuff in and we’re not going to really be checking the code, someone who knows computer science is not really going to be looking at it. Worst-case scenario, it’s literally just like some random person in marketing somewhere who has no idea what programming is, just types of stuff in and crosses their fingers.

[01:44:57] Right. I think we’re in for a world of hurt right now. It’s a race, so it’s hard to say because it’s basically a race of how good can you make the AI versus how much adoption does it get? It’s a curve. If you can make the AI good enough that the people who are adopting it at a particular rate are always using an AI that’s good enough for what they’re adopting it for, we wouldn’t expect software to get significantly worse.

[01:45:24] If those curves go the other way, we’re in a lot of trouble. Right. So I feel like right now we’re almost kind of teetering on this knife’s edge. It’s really, to me, it feels like a footrace of improving AI so it can be more autonomous and make better decisions without you needing to make them for it, versus the capability level of people who are using it and the degree to which they’re paying attention to its output.

[01:45:49] It’s like these two curves that are just, what’s going to happen? I don’t have a prediction. I don’t know where we’ll be in a year. Obviously, for all of our sake, I’m hoping that the AI curve wins because I agree. I understand certainly the perspective of people who maybe just don’t like AI and don’t want there to be AI. I can understand wanting it to fail. I understand that.

[01:46:21] Right. And I’m no fan of AI myself, so it’s not like I’m going to criticize someone for taking that position, but at the end of the day, if you’re talking about something that tons of people are using, you’re going to kind of want it to be good. I think at this point, given the level of adoption of AI, I really don’t think it would be great if it stopped getting any better right now.

[01:46:46] Like, if this was as good as it was going to get, I think that might be bad. Certainly six months ago, I think that was true, and I think it’s probably still true today. So I think ideally, if you want software to not be terrible, you have to kind of still be hoping that six months from now the AIs are again significantly better than they were. Right. Like, that is the only way out of the current situation, as I see it.

[01:47:15] Right. I don’t know if that’s fair, but that’s my sort of feeling on that.

Ryan:

[01:47:21] One of your other top tweets, it was Shopify put out this internal memo and you just replied, Slopify.

Casey:

[01:47:30] Yes.

Ryan:

[01:47:30] So I guess it’s because in this tweet it’s leadership pushing the adoption curve, maybe harder than the capabilities of the AI in this case.

Casey:

[01:47:39] Yeah. Although I also just like the pun. One of the things that I think is most unfortunate about the AI adoption, as I’ve seen it, is just the, because people think that it’s going to be this major. I guess if I had to categorize the way it appears that companies are reasoning about it, they’re assuming that if they don’t get in early it will be a big disaster for them, right? Like there’s a tremendous.

[01:48:07] They don’t just think, “Oh, well, we can just wait until the AI does what we need it to do and then start using it.” They’re like, “No, we have to do it now. Even before we know whether it can really do the thing that we want it to do or whether we know whether the outcomes are good, everyone has to do it right now. Let’s do this, right?” And I understand why they want, why they’re going about it that way, because they think that that’s critical, right?

[01:48:27] They obviously believe that’s very important. And to me, that’s just kind of terrifying because, as with any technology, the sane way to do it is to measure its capabilities, see how well it is able to solve problems that you have, see if it solves them faster than the way that you were doing it, and put it into a workflow at such a time as you’ve determined that it is a net positive. That’s just the same; that’s what you would do with any technology.

[01:48:57] And you’d probably have your team of people whose job it is to assess this thing, and they’re out there YOLO swagging it. They’ve got 3,000 agents working on this cluster, talking to each other and doing God knows what, right? So there’s going to be that, and someone’s going to do that. But that should not be every org. You wouldn’t just be like, everyone needs to use a ton of tokens, right? So yeah, I do have concerns about that.

[01:49:24] I don’t think that the way AI adoption was done was the best way for quality in software. But I would temper that statement with just the obvious fact that we were not exactly a five nines industry to start out with. Software quality was really pretty low rolling into the AI era. So I always try to just also caveat most of the things that I have to say that might be critical of a particular thing happening with AI with just the fact that, look, it wasn’t particularly great beforehand either.

[01:50:01] A lot of this software was pretty low quality, and so you can’t. Some AI things may make things worse, but it’s not like software was amazing and the AI showed up and ruined everything. That is a completely ridiculous narrative. That is not true at all.

01:50:17 — Technical reading recommendation

Ryan:

[01:50:17] Do you have any top technical book recommendations for engineers that might be listening?

Casey:

[01:50:25] No. I would probably use this opportunity to try to pitch reading technical papers. I think something that really the industry could use more of is people being more aware of what’s happening, both the historical papers, but also just current papers. It’s daunting at first, to be sure. If you don’t tend to read technical papers in your field, you will probably find it confusing. You won’t know where to find them, you won’t know how to approach them.

[01:51:00] It will seem to take too long to read them because you don’t know how to skim them properly and determine whether it’s worth your time to investigate a particular section of a paper and all that stuff. But if you’re willing to spend a few months just, at night I look at a paper, or on my lunch break I look at a paper, if you’re willing to spend a few months just doing that, you will get your bearings and you will start to know where the good papers are in your field.

[01:51:25] You’ll know how to find them, you’ll know where they tend to be published. You’ll be able to read them much more effectively, you’ll be able to know how to spend your time on them. And I think in all but probably a few small fields that probably exist somewhere that maybe people don’t tend to write many papers in or something like that, it’s tremendously valuable, and I think it’s so valuable that I read papers in disciplines I don’t even do.

[01:51:50] And I find it extremely rewarding. I read security research papers all the time. I don’t even work in a field where there are security research implications. Games don’t really do much of that. They tend to be run very sandboxed, and they don’t. Sometimes there are. But they’re not that kind of thing. They’re not like a web server authentication protocol or something like this. And I just find it incredibly rewarding because there’s just so much good stuff out there that you can learn.

[01:52:18] So I would say that would be it. Instead of a book rec, I would say find a paper, try reading a paper.

Ryan:

[01:52:24] How do I go and find that first paper or someplace to get started?

Casey:

[01:52:29] That will be valuable for most people who are watching because they might be generalists or working in web dev or in just a general tech org at a big tech company or something like that. I would say pull up the proceedings of USENIX, look through it for a paper that sounds interesting to you and try reading that paper. Or oftentimes there’s awards. I think USENIX has them where it’s like best paper of the conference.

[01:53:00] Try reading the best paper of the conference, see what you think, because that’s a collection of papers that’s usually about operating system stuff and systems design stuff. So it’s going to be something that most people can relate to. It’s not going to be really esoteric. If you were to open up the proceedings of SIGGRAPH, for example, it’d be like, oh, neural networks for cloth simulation or something, and you’re like, okay, this is not.

[01:53:28] I can’t relate to this because I don’t do graphics or whatever. Right. So USENIX might be a good place to start. But in general, yeah, if you had another option would be to go to scholar.google.com, which is their search that just specializes in papers, and type in a topic description that you find interesting. So if you wanted to learn, maybe you were very interested in consensus algorithm Paxos or something, and I don’t know what that is, or it came up at work and I haven’t really ever looked at it.

[01:54:02] You can just type that in, Paxos, and they’ll just be a list of papers, and they’ll say a thing like cited by. You can see how many citations they have. That’s usually how influential that paper was, to a certain extent. So those are some ways you could get started and find something that might interest you. It won’t be for everyone. You may bounce off it, but that’d be my recommendation. It’s something to try. You might find it interesting.

01:54:27 — Advice for his younger self

Ryan:

[01:54:27] And then, yeah, last question for you is if you could go back to the beginning of your career, when you just entered the industry, and give yourself some advice, what would you say?

Casey:

[01:54:38] I think the advice I would have given to myself was to get into low-level programming earlier. I didn’t really learn how to properly analyze and even really write assembly language code until probably 2010 or 2015 or something. I mean, it’s recent because I’m pretty old, and it’s like the last 10 years or so. Right. And I feel like I always wanted to know how to do it and just never seemed to.

[01:55:21] And the advice I would have given to myself was, just go down the hall. What I was interested in, go down the hall and be like, grab somebody. Show me. How the heck do you write this? Just show it to me. I think one of the problems is a lot of people, especially when they’re young, they are afraid of appearing like they don’t know things. And I think that can be a real impediment to learning.

[01:55:48] I’m sure it was for me. And I probably just didn’t want to literally say, “I don’t understand any of this stuff. Can you explain it to me?”

Ryan:

[01:55:55] To me?

Casey:

[01:55:58] And I think that’s one of the most useful things you can do. I don’t understand this. Please explain it to me. It’s very useful, and most people will be very happy to do that, if they’re not a dick. Most people will be like, oh, sure. Now, they might not be good at explaining it. There are plenty of engineers who are really good at something and suck at telling you how they do what they do. So you won’t always get a great explanation, but sometimes you will.

[01:56:26] Sometimes you’ll find the type of person who can explain very clearly how it is they do what they do. So if you just keep asking, you’ll get the good explanation. Nowadays, I wouldn’t really need to give myself exactly that advice because the Internet has so much great information on it. I could have taught myself that did not exist at the time.

Ryan:

[01:56:46] Awesome. Well, thank you so much for your time, Casey. I really appreciate it.

Casey:

[01:56:49] Thank you so much for having me. I love the show. It was an honor to be invited on, so thank you very much.