The Nonlinear Library: LessWrong LW - Advice to junior AI governance researchers by Akash
Jul 9, 2024
08:22
Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Advice to junior AI governance researchers, published by Akash on July 9, 2024 on LessWrong.
This summer, I'm supervising some research fellows through
Cambridge's ERA AI Fellowship. The program started last week, and I've had conversations with about 6 fellows about their research projects & summer goals.
In this post, I'll highlight a few pieces of advice I've found myself regularly giving to research fellows. This post reflects my own opinions and does not necessarily reflect the views of others at ERA.
Prioritize projects that have a clear target audience
Problem: One of the most common reasons why research products fail to add value is that they do not have a target audience. I think it can be easy to find a topic that is interesting/important, spend several months working on it, produce a 20-50 page paper, and then realize that you have no particular stakeholder(s) who find the work action-relevant.
Advice: Try to brainstorm what specific individuals you would want to have affected by your piece. This might be some folks in the AI safety community. This might be government officials at a relevant agency in the US or the UK. Prioritize projects that have a clear target audience and prioritize projects in which you have a way of actually getting your paper/product to that target audience.
Ideally, see if you can talk to representative members of your target audience in advance to see if you have a good understanding of what they might find useful.
Caveat #1: Gaining expertise can be a valid reason to do research. Sometimes, the most important target audience is yourself. It may be worthwhile to take on a research project because you want to develop your expertise in a certain area. Even if the end product is not action-relevant for anyone, you might have reason to believe that your expertise will be valuable in the present or future.
Caveat #2: Consider target audiences in the future. Some pieces do not have a target audience in the present, but they could be important in the future. This is particularly relevant when considering Overton Window shifts. It's quite plausible to me that we get at least one more major Overton Window shift in which governments become much more concerned about AI risks.
There may even be critical periods (lasting only a few weeks or a few months) in which policymakers are trying to understand what to do. You probably won't have time to come up with a good plan in those weeks or months. Therefore, it seems like it could be valuable to do the kind of research now that helps us prepare for such future scenarios.
Be specific about your end products
Problem: A lot of junior researchers find tons of ideas exciting. You might have a junior researcher who is interested in a topic like "compute governance", "evals", or "open-sourcing." That's a good start.
But if the research proposal is to "come up with gaps in the evals space" or "figure out what to do about open-source risks", there's a potential to spend several months thinking about high-level ideas and not actually producing anything concrete/specific It's common for junior researchers to overestimate the feasibility of tackling big/broad research questions.
Advice: Try to be more specific about what you want your final products to look like. If it's important for you to have a finished research product (either because it would be directly useful or because of the educational/professional benefits of having the experience of completing a project), make sure you prioritize finishing something.
If you're interested in lots of different projects, prioritize. For example, "I want to spend time on X, Y, and Z. X is the most important end product. I'll try to focus on finishing X, and I'll try not to spend much time on Y until X is finished or on track to be finished."
Caveat #1: You don't need...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Advice to junior AI governance researchers, published by Akash on July 9, 2024 on LessWrong.
This summer, I'm supervising some research fellows through
Cambridge's ERA AI Fellowship. The program started last week, and I've had conversations with about 6 fellows about their research projects & summer goals.
In this post, I'll highlight a few pieces of advice I've found myself regularly giving to research fellows. This post reflects my own opinions and does not necessarily reflect the views of others at ERA.
Prioritize projects that have a clear target audience
Problem: One of the most common reasons why research products fail to add value is that they do not have a target audience. I think it can be easy to find a topic that is interesting/important, spend several months working on it, produce a 20-50 page paper, and then realize that you have no particular stakeholder(s) who find the work action-relevant.
Advice: Try to brainstorm what specific individuals you would want to have affected by your piece. This might be some folks in the AI safety community. This might be government officials at a relevant agency in the US or the UK. Prioritize projects that have a clear target audience and prioritize projects in which you have a way of actually getting your paper/product to that target audience.
Ideally, see if you can talk to representative members of your target audience in advance to see if you have a good understanding of what they might find useful.
Caveat #1: Gaining expertise can be a valid reason to do research. Sometimes, the most important target audience is yourself. It may be worthwhile to take on a research project because you want to develop your expertise in a certain area. Even if the end product is not action-relevant for anyone, you might have reason to believe that your expertise will be valuable in the present or future.
Caveat #2: Consider target audiences in the future. Some pieces do not have a target audience in the present, but they could be important in the future. This is particularly relevant when considering Overton Window shifts. It's quite plausible to me that we get at least one more major Overton Window shift in which governments become much more concerned about AI risks.
There may even be critical periods (lasting only a few weeks or a few months) in which policymakers are trying to understand what to do. You probably won't have time to come up with a good plan in those weeks or months. Therefore, it seems like it could be valuable to do the kind of research now that helps us prepare for such future scenarios.
Be specific about your end products
Problem: A lot of junior researchers find tons of ideas exciting. You might have a junior researcher who is interested in a topic like "compute governance", "evals", or "open-sourcing." That's a good start.
But if the research proposal is to "come up with gaps in the evals space" or "figure out what to do about open-source risks", there's a potential to spend several months thinking about high-level ideas and not actually producing anything concrete/specific It's common for junior researchers to overestimate the feasibility of tackling big/broad research questions.
Advice: Try to be more specific about what you want your final products to look like. If it's important for you to have a finished research product (either because it would be directly useful or because of the educational/professional benefits of having the experience of completing a project), make sure you prioritize finishing something.
If you're interested in lots of different projects, prioritize. For example, "I want to spend time on X, Y, and Z. X is the most important end product. I'll try to focus on finishing X, and I'll try not to spend much time on Y until X is finished or on track to be finished."
Caveat #1: You don't need...