The Nonlinear Library EA - Project idea: AI for epistemics by Benjamin Todd
May 20, 2024
05:31
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Project idea: AI for epistemics, published by Benjamin Todd on May 20, 2024 on The Effective Altruism Forum.
If transformative AI
might come soon and you want to help that go well, one strategy you might adopt is building something that will improve as AI gets more capable.
That way if AI accelerates, your ability to help accelerates too.
Here's an example: organisations that use AI to improve epistemics - our ability to know what's true -- and make better decisions on that basis.
This was the most interesting impact-oriented entrepreneurial idea I came across when I visited the Bay area in February. (Thank you to Carl Shulman who first suggested it.)
Navigating the deployment of AI is going to involve successfully making many crazy hard judgement calls, such as "what's the probability this system isn't aligned" and "what might the economic effects of deployment be?"
Some of these judgement calls will need to be made under a lot of time pressure - especially if we're seeing
100 years of technological progress in under 5.
Being able to make these kinds of decisions a little bit better could therefore be worth a huge amount. And that's true given almost any future scenario.
Better decision-making can also potentially help with all other cause areas, which is why
80,000 Hours recommends it as a cause area independent from AI.
So the idea is to set up organisations that use AI to improve forecasting and decision-making in ways that can be eventually applied to these kinds of questions.
In the short term, you can apply these systems to conventional problems, potentially in the for-profit sector, like finance. We seem to be just approaching the point where AI systems might be able to help (e.g.
a recent paper found GPT-4 was pretty good at forecasting if fine-tuned). Starting here allows you to gain scale, credibility and resources.
But unlike what a purely profit-motivated entrepreneur would do, you can also try to design your tools such that in an AI crunch moment they're able to help.
For example, you could develop a free-to-use version for political leaders, so that if a huge decision about AI regulation suddenly needs to be made, they're already using the tool for other questions.
There are already a handful of projects in this space, but it could eventually be a huge area, so it still seems like very early days.
These projects could have many forms:
One example of a concrete proposal is using AI to make forecasts, or otherwise better at truthfinding in important domains. On the more qualitative side, we could imagine an AI "decision coach" or consultant that aims to augment human decision-making. Any techniques to make it easier to extract the truth from AI systems could also count, such as relevant kinds of interpretability research and the
AI debate or
weak-to-strong generalisation approaches to AI alignment.
I could imagine projects in this area starting in many ways, including a research service within a hedge fund, a research group within an AI company (e.g. focused on optimising systems for truth telling and accuracy), an AI-enabled consultancy (trying to undercut the Big 3), or as a non-profit focused on policy makers.
Most likely you'd try to fine tune and build scaffolding around existing leading LLMs, though there are also proposals to build LLMs from the bottom-up for forecasting. For example, you could create an LLM that only has data up to 2023, and then train it to predict what happens in 2024.
There's a trade-off to be managed between maintaining independence and trustworthiness, vs. having access to leading models and decision-makers in AI companies and making money.
Some ideas could
advance frontier capabilities, so you'd want to think carefully about either how to avoid that, stick to ideas that differentially boost more safety-enhancing aspects of ...
If transformative AI
might come soon and you want to help that go well, one strategy you might adopt is building something that will improve as AI gets more capable.
That way if AI accelerates, your ability to help accelerates too.
Here's an example: organisations that use AI to improve epistemics - our ability to know what's true -- and make better decisions on that basis.
This was the most interesting impact-oriented entrepreneurial idea I came across when I visited the Bay area in February. (Thank you to Carl Shulman who first suggested it.)
Navigating the deployment of AI is going to involve successfully making many crazy hard judgement calls, such as "what's the probability this system isn't aligned" and "what might the economic effects of deployment be?"
Some of these judgement calls will need to be made under a lot of time pressure - especially if we're seeing
100 years of technological progress in under 5.
Being able to make these kinds of decisions a little bit better could therefore be worth a huge amount. And that's true given almost any future scenario.
Better decision-making can also potentially help with all other cause areas, which is why
80,000 Hours recommends it as a cause area independent from AI.
So the idea is to set up organisations that use AI to improve forecasting and decision-making in ways that can be eventually applied to these kinds of questions.
In the short term, you can apply these systems to conventional problems, potentially in the for-profit sector, like finance. We seem to be just approaching the point where AI systems might be able to help (e.g.
a recent paper found GPT-4 was pretty good at forecasting if fine-tuned). Starting here allows you to gain scale, credibility and resources.
But unlike what a purely profit-motivated entrepreneur would do, you can also try to design your tools such that in an AI crunch moment they're able to help.
For example, you could develop a free-to-use version for political leaders, so that if a huge decision about AI regulation suddenly needs to be made, they're already using the tool for other questions.
There are already a handful of projects in this space, but it could eventually be a huge area, so it still seems like very early days.
These projects could have many forms:
One example of a concrete proposal is using AI to make forecasts, or otherwise better at truthfinding in important domains. On the more qualitative side, we could imagine an AI "decision coach" or consultant that aims to augment human decision-making. Any techniques to make it easier to extract the truth from AI systems could also count, such as relevant kinds of interpretability research and the
AI debate or
weak-to-strong generalisation approaches to AI alignment.
I could imagine projects in this area starting in many ways, including a research service within a hedge fund, a research group within an AI company (e.g. focused on optimising systems for truth telling and accuracy), an AI-enabled consultancy (trying to undercut the Big 3), or as a non-profit focused on policy makers.
Most likely you'd try to fine tune and build scaffolding around existing leading LLMs, though there are also proposals to build LLMs from the bottom-up for forecasting. For example, you could create an LLM that only has data up to 2023, and then train it to predict what happens in 2024.
There's a trade-off to be managed between maintaining independence and trustworthiness, vs. having access to leading models and decision-makers in AI companies and making money.
Some ideas could
advance frontier capabilities, so you'd want to think carefully about either how to avoid that, stick to ideas that differentially boost more safety-enhancing aspects of ...