80,000 Hours Podcast
#163 – Toby Ord on the perils of maximising the good that you do
Episode guests
Podcast summary created with Snipd AI
Quick takeaways
- Maximizing the good we do can sometimes lead to harmful actions.
- Effective altruism combines the beneficial aspects of utilitarianism with caution.
- Joining initiatives like Giving What We Can demonstrates commitment to effective altruism.
- Distinguishing between naive utilitarianism and global consequentialism improves decision-making.
- Character and integrity amplify the impact of our actions in doing good.
Deep dives
The Importance of Doing Good and Saving Lives
The moral theory of effective altruism emphasizes the significance of helping others and saving lives. It recognizes that making a positive impact on the world is a key aspect of living a moral life. This principle is supported by the idea that saving multiple lives is proportionally more valuable than saving just one life. Effective altruism encourages individuals to consider the scale of their impact and prioritize actions that can create the greatest amount of good.
The Role of Utilitarianism in Effective Altruism
Although not fully embracing utilitarianism, effective altruism acknowledges the valuable aspects of this moral theory. It accepts the idea that positive outcomes and the well-being of others are important moral considerations. However, effective altruism diverges from utilitarianism by not endorsing controversial aspects such as the disregard for constraints on actions or the exclusive focus on consequences. Instead, effective altruism seeks to take the beneficial aspects of utilitarianism while remaining cautious and considering other moral principles as well.
The Creation of Giving What We Can
In light of the recognition that helping others is crucial, the organization Giving What We Can was established. Giving What We Can aims to incentivize and facilitate charitable donations by creating a public register for individuals committed to donating a percentage of their income. This initiative recognizes the immense value of financial contributions in saving lives and making a significant positive impact on society. By joining Giving What We Can, individuals publicly demonstrate their commitment to effective altruism and the importance of charitable giving.
Naive Utilitarianism and the Failure to Evaluate Decision Procedures
One of the main insights in the podcast episode is the distinction between naive utilitarianism and the evaluation of decision procedures. Naive utilitarianism treats the maximization of happiness as a direct decision-making process, calculating the consequences of each individual action. However, this approach can lead to problems, such as disregarding social conventions or biases. Global consequentialism, on the other hand, evaluates everything, including rules, character traits, and principles of behavior, based on their overall consequences for well-being. This perspective allows for a unified view of different moral traditions, including deontology and virtue ethics. By understanding the distinction between criterion of rightness and decision procedure, individuals can make more informed and effective decisions to promote positive outcomes.
The Danger of Falling into the Trap of Naive Utilitarianism
The collapse of FDX and the actions taken by Sam Bankman-Fried highlight the potential danger of falling into the trap of naive utilitarianism. Naive utilitarianism advocates for constantly calculating the consequences of each individual action to maximize happiness. However, this approach can lead to uncooperative behavior, disregard for social norms, and potential ethical breaches. It is crucial to recognize that naive utilitarianism is not a comprehensive or accurate representation of utilitarianism. Instead, a more nuanced and informed understanding of global consequentialism, which evaluates rules, character traits, decision procedures, and other factors based on their overall consequences, can lead to more ethical and effective decision-making.
The Importance of Evaluating Decision Procedures and Cultivating Virtuous Behavior
In practical decision-making, it is essential to evaluate decision procedures and cultivate virtuous behavior. Relying solely on naive utilitarianism can result in biased, self-serving decision-making and a failure to consider the broader consequences of one's actions. By adopting decision procedures that consider the overall consequences for well-being, individuals can make more cooperative, ethical, and effective choices. Additionally, cultivating virtuous behavior, such as integrity, prudence, and fairness, can further enhance the positive impact of one's actions on the world. Effective altruism is not just about maximizing outcomes, but also about adhering to principles and promoting virtuous behavior that aligns with the goal of doing good in the world.
Lessons from Utilitarian and Consequentialist Moral Philosophers
Utilitarian and consequentialist moral philosophers have provided valuable insights into the importance of doing good and thinking carefully about the potential traps and constraints on our actions. While we may not agree with all aspects of their theories, we can still learn from their emphasis on the significance of doing good and considering the impact of our actions.
The Importance of Personal Character and Integrity
Personal character and integrity play a crucial role in our impact on the world. While utilitarianism and effective altruism may prioritize impact and effectiveness, they may undervalue character and virtue. However, character acts as a multiplier, influencing the outcomes we produce. Being virtuous and having integrity can contribute to positive impacts, while lacking these qualities can lead to negative effects and harm in our endeavors.
Importance of Including AI Labs in Policy Discussions
It is crucial to involve AI labs in policy discussions regarding regulation and implementation. While they possess technical expertise, it is important to be mindful that their motivations might not align with the broader interests of society. Including them in discussions allows for a balanced approach where their input can be considered, but they should not hold substantial decision-making power.
Concerns with AI Labs Racing in Development
The rapid pace at which AI labs develop and release new technologies is a cause of concern. It raises worries about potential risks and vulnerabilities that might arise. Efforts should be made to encourage AI labs to slow down and prioritize safety, as well as preventing a dangerous competition where the focus is on being the first to release rather than ensuring responsible deployment.
Effective altruism is associated with the slogan "do the most good." On one level, this has to be unobjectionable: What could be bad about helping people more and more?
But in today's interview, Toby Ord — moral philosopher at the University of Oxford and one of the founding figures of effective altruism — lays out three reasons to be cautious about the idea of maximising the good that you do. He suggests that rather than “doing the most good that we can,” perhaps we should be happy with a more modest and manageable goal: “doing most of the good that we can.”
Links to learn more, summary and full transcript.
Toby was inspired to revisit these ideas by the possibility that Sam Bankman-Fried, who stands accused of committing severe fraud as CEO of the cryptocurrency exchange FTX, was motivated to break the law by a desire to give away as much money as possible to worthy causes.
Toby's top reason not to fully maximise is the following: if the goal you're aiming at is subtly wrong or incomplete, then going all the way towards maximising it will usually cause you to start doing some very harmful things.
This result can be shown mathematically, but can also be made intuitive, and may explain why we feel instinctively wary of going “all-in” on any idea, or goal, or way of living — even something as benign as helping other people as much as possible.
Toby gives the example of someone pursuing a career as a professional swimmer. Initially, as our swimmer takes their training and performance more seriously, they adjust their diet, hire a better trainer, and pay more attention to their technique. While swimming is the main focus of their life, they feel fit and healthy and also enjoy other aspects of their life as well — family, friends, and personal projects.
But if they decide to increase their commitment further and really go all-in on their swimming career, holding back nothing back, then this picture can radically change. Their effort was already substantial, so how can they shave those final few seconds off their racing time? The only remaining options are those which were so costly they were loath to consider them before.
To eke out those final gains — and go from 80% effort to 100% — our swimmer must sacrifice other hobbies, deprioritise their relationships, neglect their career, ignore food preferences, accept a higher risk of injury, and maybe even consider using steroids.
Now, if maximising one's speed at swimming really were the only goal they ought to be pursuing, there'd be no problem with this. But if it's the wrong goal, or only one of many things they should be aiming for, then the outcome is disastrous. In going from 80% to 100% effort, their swimming speed was only increased by a tiny amount, while everything else they were accomplishing dropped off a cliff.
The bottom line is simple: a dash of moderation makes you much more robust to uncertainty and error.
As Toby notes, this is similar to the observation that a sufficiently capable superintelligent AI, given any one goal, would ruin the world if it maximised it to the exclusion of everything else. And it follows a similar pattern to performance falling off a cliff when a statistical model is 'overfit' to its data.
In the full interview, Toby also explains the “moral trade” argument against pursuing narrow goals at the expense of everything else, and how consequentialism changes if you judge not just outcomes or acts, but everything according to its impacts on the world.
Toby and Rob also discuss:
- The rise and fall of FTX and some of its impacts
- What Toby hoped effective altruism would and wouldn't become when he helped to get it off the ground
- What utilitarianism has going for it, and what's wrong with it in Toby's view
- How to mathematically model the importance of personal integrity
- Which AI labs Toby thinks have been acting more responsibly than others
- How having a young child affects Toby’s feelings about AI risk
- Whether infinities present a fundamental problem for any theory of ethics that aspire to be fully impartial
- How Toby ended up being the source of the highest quality images of the Earth from space
Get this episode by subscribing to our podcast on the world’s most pressing problems and how to solve them: type ‘80,000 Hours’ into your podcasting app. Or read the transcript.
Producer and editor: Keiran Harris
Audio Engineering Lead: Ben Cordell
Technical editing: Simon Monsour
Transcriptions: Katy Moore
Remember Everything You Learn from Podcasts
