David Thorstad: Bounded Rationality and the Case Against Longtermism
The Gradient: Perspectives on AI
Balancing Constraint and Ambition in Long-Term Funding
Exploring the implications of unrestrained spending by effective altruists on long-term programs, this chapter challenges traditional beliefs around financial constraints and decision-making processes. It delves into the complexity of evaluating long-term versus short-term values and the inherent difficulties of long-term forecasting in a dynamic global context.
Episode 122
I spoke with Professor David Thorstad about:
* The practical difficulties of doing interdisciplinary work
* Why theories of human rationality should account for boundedness, heuristics, and other cognitive limitations
* why EA epistemics suck (ok, it’s a little more nuanced than that)
Professor Thorstad is an Assistant Professor of Philosophy at Vanderbilt University, a Senior Research Affiliate at the Global Priorities Institute at Oxford, and a Research Affiliate at the MINT Lab at Australian National University. One strand of his research asks how cognitively limited agents should decide what to do and believe. A second strand asks how altruists should use limited funds to do good effectively.
Reach me at editor@thegradient.pub for feedback, ideas, guest suggestions.
Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter
Outline:
* (00:00) Intro
* (01:15) David’s interest in rationality
* (02:45) David’s crisis of confidence, models abstracted from psychology
* (05:00) Blending formal models with studies of the mind
* (06:25) Interaction between academic communities
* (08:24) Recognition of and incentives for interdisciplinary work
* (09:40) Movement towards interdisciplinary work
* (12:10) The Standard Picture of rationality
* (14:11) Why the Standard Picture was attractive
* (16:30) Violations of and rebellion against the Standard Picture
* (19:32) Mistakes made by critics of the Standard Picture
* (22:35) Other competing programs vs Standard Picture
* (26:27) Characterizing Bounded Rationality
* (27:00) A worry: faculties criticizing themselves
* (29:28) Self-improving critique and longtermism
* (30:25) Central claims in bounded rationality and controversies
* (32:33) Heuristics and formal theorizing
* (35:02) Violations of Standard Picture, vindicatory epistemology
* (37:03) The Reason Responsive Consequentialist View (RRCV)
* (38:30) Objective and subjective pictures
* (41:35) Reason responsiveness
* (43:37) There are no epistemic norms for inquiry
* (44:00) Norms vs reasons
* (45:15) Arguments against epistemic nihilism for belief
* (47:30) Norms and self-delusion
* (49:55) Difficulty of holding beliefs for pragmatic reasons
* (50:50) The Gibbardian picture, inquiry as an action
* (52:15) Thinking how to act and thinking how to live — the power of inquiry
* (53:55) Overthinking and conducting inquiry
* (56:30) Is thinking how to inquire as an all-things-considered matter?
* (58:00) Arguments for the RRCV
* (1:00:40) Deciding on minimal criteria for the view, stereotyping
* (1:02:15) Eliminating stereotypes from the theory
* (1:04:20) Theory construction in epistemology and moral intuition
* (1:08:20) Refusing theories for moral reasons and disciplinary boundaries
* (1:10:30) The argument from minimal criteria, evaluating against competing views
* (1:13:45) Comparing to other theories
* (1:15:00) The explanatory argument
* (1:17:53) Parfit and Railton, norms of friendship vs utility
* (1:20:00) Should you call out your friend for being a womanizer
* (1:22:00) Vindicatory Epistemology
* (1:23:05) Panglossianism and meliorative epistemology
* (1:24:42) Heuristics and recognition-driven investigation
* (1:26:33) Rational inquiry leading to irrational beliefs — metacognitive processing
* (1:29:08) Stakes of inquiry and costs of metacognitive processing
* (1:30:00) When agents are incoherent, focuses on inquiry
* (1:32:05) Indirect normative assessment and its consequences
* (1:37:47) Against the Singularity Hypothesis
* (1:39:00) Superintelligence and the ontological argument
* (1:41:50) Hardware growth and general intelligence growth, AGI definitions
* (1:43:55) Difficulties in arguing for hyperbolic growth
* (1:46:07) Chalmers and the proportionality argument
* (1:47:53) Arguments for/against diminishing growth, research productivity, Moore’s Law
* (1:50:08) On progress studies
* (1:52:40) Improving research productivity and technology growth
* (1:54:00) Mistakes in the moral mathematics of existential risk, longtermist epistemics
* (1:55:30) Cumulative and per-unit risk
* (1:57:37) Back and forth with longtermists, time of perils
* (1:59:05) Background risk — risks we can and can’t intervene on, total existential risk
* (2:00:56) The case for longtermism is inflated
* (2:01:40) Epistemic humility and longtermism
* (2:03:15) Knowledge production — reliable sources, blog posts vs peer review
* (2:04:50) Compounding potential errors in knowledge
* (2:06:38) Group deliberation dynamics, academic consensus
* (2:08:30) The scope of longtermism
* (2:08:30) Money in effective altruism and processes of inquiry
* (2:10:15) Swamping longtermist options
* (2:12:00) Washing out arguments and justified belief
* (2:13:50) The difficulty of long-term forecasting and interventions
* (2:15:50) Theory of change in the bounded rationality program
* (2:18:45) Outro
Links:
* David’s homepage and Twitter and blog
* Papers mentioned/read
* Bounded rationality and inquiry
* Why bounded rationality (in epistemology)?
* Against the newer evidentialists
* The accuracy-coherence tradeoff in cognition
* There are no epistemic norms of inquiry
* Global priorities and effective altruism
* Against the singularity hypothesis (+ blog posts)
* Three mistakes in the moral mathematics of existential risk (+ blog posts)
Get full access to The Gradient at thegradientpub.substack.com/subscribe
Remember Everything You Learn from Podcasts
