The Nonlinear Library EA - Conditional Trees: Generating Informative Forecasting Questions (FRI) -- AI Risk Case Study by Forecasting Research Institute
Aug 13, 2024
19:06
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Conditional Trees: Generating Informative Forecasting Questions (FRI) -- AI Risk Case Study, published by Forecasting Research Institute on August 13, 2024 on The Effective Altruism Forum.
Authors of linked report: Tegan McCaslin, Josh Rosenberg, Ezra Karger, Avital Morris, Molly Hickman, Otto Kuusela, Sam Glover, Zach Jacobs, Phil Tetlock[1]
Today, the Forecasting Research Institute (FRI), released "Conditional Trees: A Method for Generating Informative Questions about Complex Topics," which discusses the results of a case study in using conditional trees to generate informative questions about AI risk. In this post, we provide a brief overview of the methods, findings, and directions for further research. For much more analysis and discussion, see the full report: https://forecastingresearch.org/s/AIConditionalTrees.pdf
Abstract
We test a new process for generating high-value forecasting questions: asking experts to produce "conditional trees," simplified Bayesian networks of quantifiably informative forecasting questions. We test this technique in the context of the current debate about risks from AI.
We conduct structured interviews with 21 AI domain experts and 3 highly skilled generalist forecasters ("superforecasters") to generate 75 forecasting questions that would cause participants to significantly update their views about AI risk.
We elicit the "Value of Information" (VOI) each question provides for a far-future outcome - whether AI will cause human extinction by 2100 - by collecting conditional forecasts from superforecasters (n=8).[2] In a comparison with the highest-engagement AI questions on two forecasting platforms, the average conditional trees-generated question resolving in 2030 was nine times more informative than the comparison AI-related platform questions (p = .025).
This report provides initial evidence that structured interviews of experts focused on generating informative cruxes can produce higher-VOI questions than status quo methods.
Executive Summary
From May 2022 to October 2023, the Forecasting Research Institute (FRI) (a)[3] experimented with a new method of question generation ("conditional trees"). While the questions elicited in this case study focus on potential risks from advanced AI, the processes we present can be used to generate valuable questions across fields where forecasting can help decision-makers navigate complex, long-term uncertainties.
Methods
Researchers interviewed 24 participants, including 21 AI and existential risk experts and three highly skilled generalist forecasters ("superforecasters"). We first asked participants to provide their personal forecast of the probability of AI-related extinction by 2100 (the "ultimate question" for this exercise).[4] We then asked participants to identify plausible[5] indicator events that would significantly shift their estimates of the probability of the ultimate question.
Following the interviews, we converted these indicators into 75 objectively resolvable forecasting questions. We asked superforecasters (n=8) to provide forecasts on each of these 75 questions (the "AICT" questions), and forecasts on how their beliefs about AI risk would update if each of these questions resolved positively or negatively.
We quantitatively ranked the resulting indicators by Value of Information (VOI), a measure of how much each indicator caused superforecasters to update their beliefs about long-run AI risk.
To evaluate the informativeness of the conditional trees method relative to widely discussed indicators, we assess a subset of these questions using a standardized version of VOI, comparing them to popular AI questions on existing forecasting platforms (the "status quo" questions). The status quo questions were selected from two popular forecasting platforms by identifying the highest-...
Authors of linked report: Tegan McCaslin, Josh Rosenberg, Ezra Karger, Avital Morris, Molly Hickman, Otto Kuusela, Sam Glover, Zach Jacobs, Phil Tetlock[1]
Today, the Forecasting Research Institute (FRI), released "Conditional Trees: A Method for Generating Informative Questions about Complex Topics," which discusses the results of a case study in using conditional trees to generate informative questions about AI risk. In this post, we provide a brief overview of the methods, findings, and directions for further research. For much more analysis and discussion, see the full report: https://forecastingresearch.org/s/AIConditionalTrees.pdf
Abstract
We test a new process for generating high-value forecasting questions: asking experts to produce "conditional trees," simplified Bayesian networks of quantifiably informative forecasting questions. We test this technique in the context of the current debate about risks from AI.
We conduct structured interviews with 21 AI domain experts and 3 highly skilled generalist forecasters ("superforecasters") to generate 75 forecasting questions that would cause participants to significantly update their views about AI risk.
We elicit the "Value of Information" (VOI) each question provides for a far-future outcome - whether AI will cause human extinction by 2100 - by collecting conditional forecasts from superforecasters (n=8).[2] In a comparison with the highest-engagement AI questions on two forecasting platforms, the average conditional trees-generated question resolving in 2030 was nine times more informative than the comparison AI-related platform questions (p = .025).
This report provides initial evidence that structured interviews of experts focused on generating informative cruxes can produce higher-VOI questions than status quo methods.
Executive Summary
From May 2022 to October 2023, the Forecasting Research Institute (FRI) (a)[3] experimented with a new method of question generation ("conditional trees"). While the questions elicited in this case study focus on potential risks from advanced AI, the processes we present can be used to generate valuable questions across fields where forecasting can help decision-makers navigate complex, long-term uncertainties.
Methods
Researchers interviewed 24 participants, including 21 AI and existential risk experts and three highly skilled generalist forecasters ("superforecasters"). We first asked participants to provide their personal forecast of the probability of AI-related extinction by 2100 (the "ultimate question" for this exercise).[4] We then asked participants to identify plausible[5] indicator events that would significantly shift their estimates of the probability of the ultimate question.
Following the interviews, we converted these indicators into 75 objectively resolvable forecasting questions. We asked superforecasters (n=8) to provide forecasts on each of these 75 questions (the "AICT" questions), and forecasts on how their beliefs about AI risk would update if each of these questions resolved positively or negatively.
We quantitatively ranked the resulting indicators by Value of Information (VOI), a measure of how much each indicator caused superforecasters to update their beliefs about long-run AI risk.
To evaluate the informativeness of the conditional trees method relative to widely discussed indicators, we assess a subset of these questions using a standardized version of VOI, comparing them to popular AI questions on existing forecasting platforms (the "status quo" questions). The status quo questions were selected from two popular forecasting platforms by identifying the highest-...