The Core Hypothesis Behind AGI Safety Concerns Is the Quote Orthogonality Thesis

I think that people have a lot of intuitions related to things like justice being good and compassion being good. And I think all of those intuitions are sort of rooted in our human innate drives. When we make future AGIs, we get to put whatever innate drives into them that we want. So it could have an innate drive to maximize stock prices. If you can program it, you can put it in. But we don't actually know how to write the code for compassion and friendship innate drive right now. That's one of the things that I would like to figure out.