Values and the Alignment Problem in AI

The alignment problem in AI revolves around ensuring that the values of highly powerful AI systems align with human values. Merely instructing an AI to perform a task, like making paper clips, does not equate to instilling values in it, as values are not instructions but are rooted in the evolutionary history of human beings. Moral constructivism suggests that morals are constructed by humans based on their biological motivations, feelings, and preferences. Unlike humans, AI models like LLMs lack underlying moral intuitions, and providing them with instructions does not suffice to instill values. Thus, in the context of AI, the term 'values' does not hold the same meaning as it does for human beings, necessitating careful philosophical consideration in addressing the alignment problem.