The Nonlinear Library LW - AI #62: Too Soon to Tell by Zvi
May 3, 2024
49:12
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #62: Too Soon to Tell, published by Zvi on May 3, 2024 on LessWrong.
What is the mysterious impressive new 'gpt2-chatbot' from the Arena? Is it GPT-4.5? A refinement of GPT-4? A variation on GPT-2 somehow? A new architecture? Q-star? Someone else's model? Could be anything. It is so weird that this is how someone chose to present that model.
There was also a lot of additional talk this week about California's proposed SB 1047.
I wrote an additional post extensively breaking that bill down, explaining how it would work in practice, addressing misconceptions about it and suggesting fixes for its biggest problems along with other improvements. For those interested, I recommend reading at least the sections 'What Do I Think The Law Would Actually Do?' and 'What are the Biggest Misconceptions?'
As usual, lots of other things happened as well.
Table of Contents
1. Introduction.
2. Table of Contents.
3. Language Models Offer Mundane Utility. Do your paperwork for you. Sweet.
4. Language Models Don't Offer Mundane Utility. Because it is not yet good at it.
5. GPT-2 Soon to Tell. What is this mysterious new model?
6. Fun With Image Generation. Certified made by humans.
7. Deepfaketown and Botpocalypse Soon. A located picture is a real picture.
8. They Took Our Jobs. Because we wouldn't let other humans take them first?
9. Get Involved. It's protest time. Against AI that is.
10. In Other AI News. Incremental upgrades, benchmark concerns.
11. Quiet Speculations. Misconceptions cause warnings of AI winter.
12. The Quest for Sane Regulation. Big tech lobbies to avoid regulations, who knew?
13. The Week in Audio. Lots of Sam Altman, plus some others.
14. Rhetorical Innovation. The few people who weren't focused on SB 1047.
15. Open Weights Are Unsafe And Nothing Can Fix This. Tech for this got cheaper.
16. Aligning a Smarter Than Human Intelligence is Difficult. Dot by dot thinking.
17. The Lighter Side. There must be some mistake.
Language Models Offer Mundane Utility
Write automatic police reports based on body camera footage. It seems it only uses the audio? Not using the video seems to be giving up a lot of information. Even so, law enforcement seems impressed, one notes an 82% reduction in time writing reports, even with proofreading requirements.
Axon says it did a double-blind study to compare its AI reports with ones from regular offers.
And it says that Draft One results were "equal to or better than" regular police reports.
As with self-driving cars, that is not obviously sufficient.
Eliminate 2.2 million unnecessary words in the Ohio administrative code, out of a total of 17.4 million. The AI identified candidate language, which humans reviewed. Sounds great, but let's make sure we keep that human in the loop.
Diagnose your medical condition? Link has a one-minute video of a doctor asking questions and correctly diagnosing a patient.
Ate-a-Pi: This is why AI will replace doctor.
Sherjil Ozair: diagnosis any%.
Akhil Bagaria: This it the entire premise of the TV show house.
The first AI attempt listed only does 'the easy part' of putting all the final information together. Kiaran Ritchie then shows that yes, ChatGPT can figure out what questions to ask, solving the problem with eight requests over two steps, followed by a solution.
There are still steps where the AI is getting extra information, but they do not seem like the 'hard steps' to me.
Is Sam Altman subtweeting me?
Sam Altman: Learning how to say something in 30 seconds that takes most people 5 minutes is a big unlock.
(and imo a surprisingly learnable skill.
If you struggle with this, consider asking a friend who is good at it to listen to you say something and then rephrase it back to you as concisely as they can a few dozen times.
I have seen this work really well!)
Interesting DM: "For what it's worth this...
What is the mysterious impressive new 'gpt2-chatbot' from the Arena? Is it GPT-4.5? A refinement of GPT-4? A variation on GPT-2 somehow? A new architecture? Q-star? Someone else's model? Could be anything. It is so weird that this is how someone chose to present that model.
There was also a lot of additional talk this week about California's proposed SB 1047.
I wrote an additional post extensively breaking that bill down, explaining how it would work in practice, addressing misconceptions about it and suggesting fixes for its biggest problems along with other improvements. For those interested, I recommend reading at least the sections 'What Do I Think The Law Would Actually Do?' and 'What are the Biggest Misconceptions?'
As usual, lots of other things happened as well.
Table of Contents
1. Introduction.
2. Table of Contents.
3. Language Models Offer Mundane Utility. Do your paperwork for you. Sweet.
4. Language Models Don't Offer Mundane Utility. Because it is not yet good at it.
5. GPT-2 Soon to Tell. What is this mysterious new model?
6. Fun With Image Generation. Certified made by humans.
7. Deepfaketown and Botpocalypse Soon. A located picture is a real picture.
8. They Took Our Jobs. Because we wouldn't let other humans take them first?
9. Get Involved. It's protest time. Against AI that is.
10. In Other AI News. Incremental upgrades, benchmark concerns.
11. Quiet Speculations. Misconceptions cause warnings of AI winter.
12. The Quest for Sane Regulation. Big tech lobbies to avoid regulations, who knew?
13. The Week in Audio. Lots of Sam Altman, plus some others.
14. Rhetorical Innovation. The few people who weren't focused on SB 1047.
15. Open Weights Are Unsafe And Nothing Can Fix This. Tech for this got cheaper.
16. Aligning a Smarter Than Human Intelligence is Difficult. Dot by dot thinking.
17. The Lighter Side. There must be some mistake.
Language Models Offer Mundane Utility
Write automatic police reports based on body camera footage. It seems it only uses the audio? Not using the video seems to be giving up a lot of information. Even so, law enforcement seems impressed, one notes an 82% reduction in time writing reports, even with proofreading requirements.
Axon says it did a double-blind study to compare its AI reports with ones from regular offers.
And it says that Draft One results were "equal to or better than" regular police reports.
As with self-driving cars, that is not obviously sufficient.
Eliminate 2.2 million unnecessary words in the Ohio administrative code, out of a total of 17.4 million. The AI identified candidate language, which humans reviewed. Sounds great, but let's make sure we keep that human in the loop.
Diagnose your medical condition? Link has a one-minute video of a doctor asking questions and correctly diagnosing a patient.
Ate-a-Pi: This is why AI will replace doctor.
Sherjil Ozair: diagnosis any%.
Akhil Bagaria: This it the entire premise of the TV show house.
The first AI attempt listed only does 'the easy part' of putting all the final information together. Kiaran Ritchie then shows that yes, ChatGPT can figure out what questions to ask, solving the problem with eight requests over two steps, followed by a solution.
There are still steps where the AI is getting extra information, but they do not seem like the 'hard steps' to me.
Is Sam Altman subtweeting me?
Sam Altman: Learning how to say something in 30 seconds that takes most people 5 minutes is a big unlock.
(and imo a surprisingly learnable skill.
If you struggle with this, consider asking a friend who is good at it to listen to you say something and then rephrase it back to you as concisely as they can a few dozen times.
I have seen this work really well!)
Interesting DM: "For what it's worth this...