I was inspired to write this post right after talking to what felt like 3,000 super enthusiastic undergrads at EAGX Boston a couple months ago. Who wanted to get into AGI safety slash alignment? Among the people I talked to there seemed to be an unspoken assumption that if they wanted to go to grad school at all, they should try to maneuver into doing AGI safety-related projects with an AGIsafety-concerned PI. Let's not pretend that this isn't a compromise. You will wind up in a project which is simultaneously optimising for both, advancing AGI safety. It's no coincidence that the weird sci-fi stuff side of AGI safety