Critiquing Existential Risk Assessments

Speaker 1
The biggest thing that I learned was the value of this thing called culture and stuff. I used to be, you know, I'm like, I'm a zero to one guy. And like, I didn't sort of, I thought like culture and all that is like jargon that big companies use to sound good, to look good. And I didn't see the value in it. And then what I sort of actually saw was, you know, the first prince, so I was able to experience it from first principles when you start something new. You know, you sort of experience, you get to run the A.B. test for what happens if you if there's, if you don't have it. And, and I, you know, the thing that I learned was like, as you hire more and more smart people, like when you're growing really fast, you don't have a ton of time to bring everyone on and train them. Right. And you know, you're growing, like we were growing on orange teams, like two x three x a year. Right. So the average age of an engineer in the company is like three months or four months. Right. So what did they know about any of this stuff? So you don't have a ton of bandwidths to train them. Right. And often what you're doing as a result, as a coping mechanism, is you're hiring more senior and higher sort of experienced people who don't need that much training, they come in and they can perform. Right. And then what happens if you do that is you're sort of hiring people to drive your car for you. And you're hiring like pretty good drivers by default, they'll make by default, like very rational choices. Those are the right hires to make. Right. And once you do that, you do that a few times, then you notice that your org is becoming really slow. And you notice that you're sort of starting to operate more and more and more like a big company. Like I call it mean reversion. Like any outlier startup is founded on some contrarian insight. Right. And a startup sort of starts at like several standard deviations of the mean. Right. And when you get like normal people in the company, right, who are like who make like really good normal decisions, the company tends towards the normal
Speaker 2
identity or that intuition
Speaker 1
starts getting lost. That's that's lost. That's lost. So it starts behaving more and more like a really big company like Google or something. Yeah. Because those are the safe defaults for a reason. They just work in more circumstances. Right. And then there's nothing differentiated about yourself, the company. And that is why it was so frustrating. I was fighting that mean reversion by saying, you know, if you just cared more, yeah, maybe this wouldn't have happened. Right. And I realized that's not the way to fight it. That is super blunt weapon. You know, so what you what in retrospect, I realized the job of someone like the leader who's scaling is actually or you know, the founder who's scaling actually is actually to look inside his brain, run a debug PDB trace on my own brain of like, hey, why do I think we should have we should have an opinion in its stack? Why should other people not develop their own technology? Why is that? And why are we willing to pay a big cost for this contrarian opinion? Right. Because anytime you have a contrarian opinion, there are some real costs. Right. And to avoid those costs is why like reasonably smart people mean revert. Right. Right. So you have to like trace through your brain and ask like, why is this the right decision for us? Right. And why are we willing to pay these obvious costs in front of us? Like engineers won't join you. Right. Because you know, they can't build cool architecture. Right. They need to sort of work on business or something. Right. And that's still worth it. Right. You know, you stand for something and you're willing to die for something. Yeah. And that's still worth it for our business. Why is that? So you need to sort of like make a case, not just a case, but also like, why you personally think that? Yeah. So you have to, and there's not a lot you can communicate. In the culture. So culture is like few nuggets of statements that can penetrate from, you know, penetrate a large org, where they can just like, that is something that they can remember, right? And use as frameworks to make decisions when they're alone. And you can't communicate a lot. They won't remember a lot. True. So you have to distill it down. Right. To a few things that we do differently here. Right. And why that matters. Right. So a lot of this is about just like tracing what goes on looking internally, first of all. Right. You know, and tracing what goes on inside and exposing the software. Yeah.

#200 – Ezra Karger on what superforecasters and experts think about existential risks

80,000 Hours Podcast

Remember Everything You Learn from Podcasts