The Nonlinear Library EA - Rethink Priorities' Digital Consciousness Project Announcement by Bob Fischer
Jul 5, 2024
04:24
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rethink Priorities' Digital Consciousness Project Announcement, published by Bob Fischer on July 5, 2024 on The Effective Altruism Forum.
One of the core questions regarding the moral status of AI concerns their consciousness. Is there anything it's like to be them? Contemporary AI systems are widely regarded as clearly not conscious, but there seems to be growing concern among experts that we may see conscious AI systems in the not-too-distant future.
Understanding our duties to the AI systems we create will involve assessing the nature of their minds, and thus their moral status. There are many important questions about AI minds that bear on their moral status, but whether they are consciousness has a clear and widely recognized role. In addition, it may be important in securing or denying AIs the public's moral consideration.
Existing consciousness research revolves first and foremost around human beings. The physical bases (or neural correlates) of consciousness in humans remain uncertain. Leading proposals are both vague and highly controversial. Extending theories of consciousness to AIs will require careful thought about how to generalize beyond the human case.
Alternatively, we might look to identify behavioral indicators of consciousness. Behavior has a much more salient role in swaying our attitudes than abstract considerations of architecture. But modern AIs are carefully trained to behave like us, and so it is not easy to tell whether their behaviors indicate anything beyond mimicry.
Therefore, we see a variety of kinds of uncertainty at play: there is methodological uncertainty, uncertainty regarding the underpinnings of human consciousness, uncertainty regarding the significance of behavioral evidence, uncertainty about how AIs work, etc. Coming up with any concrete estimate of the probability of consciousness in AI systems will require mapping, measuring, and aggregating these uncertainties.
Rethink Priorities has overcome similar challenges before. Our
Moral Weight Project wrangled patchy evidence about behavioral traits and cognitive capacities across the animal kingdom through a Monte Carlo framework that output probabilistic estimates of welfare ranges for different species. We learned a lot from this work and we are eager to apply those lessons to a new challenge.
We are now turning to the question of how best to assess the probability of AI consciousness. Over the coming months, we plan to carry out a project encompassing the following tasks:
1. Evaluating different modeling approaches to AI consciousness estimation. What different paradigms are worth exploring? What are the pros and cons of each?
2. Identifying some plausible proxies for consciousness to feed into these models. What are the challenges in pinning down values for these proxies? Where might future technical work be most fruitful?
3. Producing a prototype model that translates uncertainty about different sources of evidence into probability ranges for contemporary and hypothetical future AI models. Given our uncertainties, what should we conclude about the overall probability of consciousness?
Having such a model is valuable in a few different ways. First, we can produce an overall estimate of the probability that a given system is conscious - an estimate that's informed by, rather than undermined by, our uncertainty about the correct theory of consciousness. Second, because the inputs to the process can be updated with new information as, say, new capabilities come online, we can readily update our overall estimate of the probability of consciousness.
Third, because we can repeat this process based on the capabilities that were present at earlier dates, we can also model the historical rate of change in the probability of digital consciousness. In principle, we can use that to make...
One of the core questions regarding the moral status of AI concerns their consciousness. Is there anything it's like to be them? Contemporary AI systems are widely regarded as clearly not conscious, but there seems to be growing concern among experts that we may see conscious AI systems in the not-too-distant future.
Understanding our duties to the AI systems we create will involve assessing the nature of their minds, and thus their moral status. There are many important questions about AI minds that bear on their moral status, but whether they are consciousness has a clear and widely recognized role. In addition, it may be important in securing or denying AIs the public's moral consideration.
Existing consciousness research revolves first and foremost around human beings. The physical bases (or neural correlates) of consciousness in humans remain uncertain. Leading proposals are both vague and highly controversial. Extending theories of consciousness to AIs will require careful thought about how to generalize beyond the human case.
Alternatively, we might look to identify behavioral indicators of consciousness. Behavior has a much more salient role in swaying our attitudes than abstract considerations of architecture. But modern AIs are carefully trained to behave like us, and so it is not easy to tell whether their behaviors indicate anything beyond mimicry.
Therefore, we see a variety of kinds of uncertainty at play: there is methodological uncertainty, uncertainty regarding the underpinnings of human consciousness, uncertainty regarding the significance of behavioral evidence, uncertainty about how AIs work, etc. Coming up with any concrete estimate of the probability of consciousness in AI systems will require mapping, measuring, and aggregating these uncertainties.
Rethink Priorities has overcome similar challenges before. Our
Moral Weight Project wrangled patchy evidence about behavioral traits and cognitive capacities across the animal kingdom through a Monte Carlo framework that output probabilistic estimates of welfare ranges for different species. We learned a lot from this work and we are eager to apply those lessons to a new challenge.
We are now turning to the question of how best to assess the probability of AI consciousness. Over the coming months, we plan to carry out a project encompassing the following tasks:
1. Evaluating different modeling approaches to AI consciousness estimation. What different paradigms are worth exploring? What are the pros and cons of each?
2. Identifying some plausible proxies for consciousness to feed into these models. What are the challenges in pinning down values for these proxies? Where might future technical work be most fruitful?
3. Producing a prototype model that translates uncertainty about different sources of evidence into probability ranges for contemporary and hypothetical future AI models. Given our uncertainties, what should we conclude about the overall probability of consciousness?
Having such a model is valuable in a few different ways. First, we can produce an overall estimate of the probability that a given system is conscious - an estimate that's informed by, rather than undermined by, our uncertainty about the correct theory of consciousness. Second, because the inputs to the process can be updated with new information as, say, new capabilities come online, we can readily update our overall estimate of the probability of consciousness.
Third, because we can repeat this process based on the capabilities that were present at earlier dates, we can also model the historical rate of change in the probability of digital consciousness. In principle, we can use that to make...