You are no longer creating a code that you model inside your own minds. You are blindly exploring a space of possibilities where you don't understand the possibilities. This itself is not enough to create things with strange and scutable desires, but it's step one. When you take a giant inscrutable set of arrays of floating point numbers and differentiate them with respect to a loss function,. And repeatedly, notice the giantinscrutable rays to drive the loss function lower and lower. The right peach was better for you than the rock, than the heart is a rock peach that had no nutrients because it was not ripened. So you develop the sweet tooth and now it leads you runs amok
Eliezer Yudkowsky insists that once artificial intelligence becomes smarter than people, everyone on earth will die. Listen as Yudkowsky speaks with EconTalk's Russ Roberts on why we should be very, very afraid, and why we're not prepared or able to manage the terrifiying risks of artificial intelligence.