The Nonlinear Library EA - A Research Agenda for Psychology and AI by carter allen
Jun 28, 2024
29:36
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Research Agenda for Psychology and AI, published by carter allen on June 28, 2024 on The Effective Altruism Forum.
I think very few people have thought rigorously about how psychology research could inform the trajectory of AI or humanity's response to it. Despite this, there seem to be many important contributions psychology could make to AI safety. For instance, a few broad paths-to-impact that psychology research might have are:
1. Helping people anticipate the societal response to possible developments in AI. In which areas is public opinion likely to be the bottleneck to greater AI safety?
2. Improving forecasting/prediction techniques more broadly and applying this to forecasts of AI trajectories (the
Forecasting Research Institute's Existential Persuasion Tournament is a good example).
3. Describing human values and traits more rigorously to inform AI alignment, or to inform decisions about who to put behind the wheel in an AI takeoff scenario.
4. Doing cognitive/behavioral science on AI models. For instance, developing diagnostic tools that can be used to assess how susceptible an AI decision-maker is to various biases.
5. Modeling various risks related to institutional stability. For instance, arms races, risks posed by malevolent actors, various parties' incentives in AI development, and decision-making within/across top AI companies.
I spent several weeks thinking about specific project ideas in these topic areas as part of my final project for BlueDot Impact's
AI Safety Fundamentals course.
I'm sharing my ideas here because a) there are probably large topic areas I'm missing, and I'd like for people to point them out to me, b) I'm starting my PhD in a few months, and I want to do some of these ideas, but I haven't thought much about which of them are more/less valuable, and c) I would love for anyone else to adopt any of the ideas here or reach out to me about collaborating! I also hope to write a future version of this post that incorporates more existing research (I haven't
thoroughly checked which of these project ideas have already been done).
Maybe another way this post could be valuable is that I've consolidated a lot of different resources, ideas, and links to other agendas in one place. I'd especially like for people to send me more things in this category, so that I or others can use this post as a resource for connecting people in psychology to ideas and opportunities in AI safety.
In the rest of this post, I list various topic areas I identified in no particular order, as well as any particular project ideas I had which struck me as potentially especially neglected & valuable in that area. Any feedback is appreciated!
Topic Areas & Project Ideas
1. Human-AI Interaction
1.1 AI Persuasiveness
A lot of people believe that future AI systems might be extremely persuasive, and perhaps we should prepare for a world where interacting with AI models carries a risk of manipulation/brainwashing. How realistic is this concern? (Note, although I don't think this sort of research is capable of answering whether AI will ever be extremely persuasive, I think it could still be very usefully informative.) For instance:
How good are people at detecting AI-generated misinformation in current models, or inferring ulterior motives in current AI advisors? How has this ability changed in line with compute trends?
Are people better or worse at detecting lies in current AI models, compared to humans? How has this ability changed in line with compute trends?
How does increasing quantity of misinformation affect people's susceptibility to misinformation? Which effect dominates between "humans get more skeptical of information as less of it is true" and "humans believe more false information as more false information is available"?
In which domains is AI most likely to b...
I think very few people have thought rigorously about how psychology research could inform the trajectory of AI or humanity's response to it. Despite this, there seem to be many important contributions psychology could make to AI safety. For instance, a few broad paths-to-impact that psychology research might have are:
1. Helping people anticipate the societal response to possible developments in AI. In which areas is public opinion likely to be the bottleneck to greater AI safety?
2. Improving forecasting/prediction techniques more broadly and applying this to forecasts of AI trajectories (the
Forecasting Research Institute's Existential Persuasion Tournament is a good example).
3. Describing human values and traits more rigorously to inform AI alignment, or to inform decisions about who to put behind the wheel in an AI takeoff scenario.
4. Doing cognitive/behavioral science on AI models. For instance, developing diagnostic tools that can be used to assess how susceptible an AI decision-maker is to various biases.
5. Modeling various risks related to institutional stability. For instance, arms races, risks posed by malevolent actors, various parties' incentives in AI development, and decision-making within/across top AI companies.
I spent several weeks thinking about specific project ideas in these topic areas as part of my final project for BlueDot Impact's
AI Safety Fundamentals course.
I'm sharing my ideas here because a) there are probably large topic areas I'm missing, and I'd like for people to point them out to me, b) I'm starting my PhD in a few months, and I want to do some of these ideas, but I haven't thought much about which of them are more/less valuable, and c) I would love for anyone else to adopt any of the ideas here or reach out to me about collaborating! I also hope to write a future version of this post that incorporates more existing research (I haven't
thoroughly checked which of these project ideas have already been done).
Maybe another way this post could be valuable is that I've consolidated a lot of different resources, ideas, and links to other agendas in one place. I'd especially like for people to send me more things in this category, so that I or others can use this post as a resource for connecting people in psychology to ideas and opportunities in AI safety.
In the rest of this post, I list various topic areas I identified in no particular order, as well as any particular project ideas I had which struck me as potentially especially neglected & valuable in that area. Any feedback is appreciated!
Topic Areas & Project Ideas
1. Human-AI Interaction
1.1 AI Persuasiveness
A lot of people believe that future AI systems might be extremely persuasive, and perhaps we should prepare for a world where interacting with AI models carries a risk of manipulation/brainwashing. How realistic is this concern? (Note, although I don't think this sort of research is capable of answering whether AI will ever be extremely persuasive, I think it could still be very usefully informative.) For instance:
How good are people at detecting AI-generated misinformation in current models, or inferring ulterior motives in current AI advisors? How has this ability changed in line with compute trends?
Are people better or worse at detecting lies in current AI models, compared to humans? How has this ability changed in line with compute trends?
How does increasing quantity of misinformation affect people's susceptibility to misinformation? Which effect dominates between "humans get more skeptical of information as less of it is true" and "humans believe more false information as more false information is available"?
In which domains is AI most likely to b...