The Nonlinear Library: LessWrong LW - We ran an AI safety conference in Tokyo. It went really well. Come next year! by Blaine
Jul 18, 2024
10:09
Link to original article
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We ran an AI safety conference in Tokyo. It went really well. Come next year!, published by Blaine on July 18, 2024 on LessWrong.
Abstract
Technical AI Safety 2024 (TAIS 2024) was a conference organised by
AI Safety 東京 and
Noeon Research, in collaboration with
Reaktor Japan,
AI Alignment Network and
AI Industry Foundation. You may have heard of us through ACX.
The goals of the conference were
1. demonstrate the practice of technical safety research to Japanese researchers new to the field
2. share ideas among established technical safety researchers
3. establish a good international reputation for AI Safety 東京 and Noeon Research
4. establish a Schelling conference for people working in technical safety
We sent out a survey after the conference to get feedback from attendees on whether or not we achieved those goals. We certainly achieved goals 1, 2 and 3; goal 4 remains to be seen. In this post we give more details about the conference, share results from the feedback survey, and announce our intentions to run another conference next year.
Okay but like, what was TAIS 2024?
Technical AI Safety 2024 (TAIS 2024) was a small non-archival open academic conference structured as a lecture series. It ran over the course of 2 days from April 5th-6th 2024 at the International Conference Hall of the Plaza Heisei in Odaiba, Tokyo.
We had
18 talks covering 6 research agendas in technical AI safety:
Mechanistic Interpretability
Developmental Interpretability
Scaleable Oversight
Agent Foundations
Causal Incentives
ALIFE
…including talks from Hoagy Cunningham (Anthropic), Noah Y. Siegel (DeepMind), Manuel Baltieri (Araya), Dan Hendrycks (CAIS), Scott Emmons (CHAI), Ryan Kidd (MATS), James Fox (LISA), and Jesse Hoogland and Stan van Wingerden (Timaeus).
In addition to our invited talks, we had 25
submissions, of which 19 were deemed relevant for presentation. 5 were offered talk slots, and we arranged a poster session to accommodate the remaining 14. In the end, 7 people presented posters, 5 in person and 2 in absentia. Our best poster award was won jointly by Fazl Berez for
Large Language Models Relearn Removed Concepts and Alex Spies for
Structured Representations in Maze-Solving Transformers.
We had 105 in-person attendees (including the speakers). Our live streams had around 400 unique viewers, and maxed out at 18 concurrent viewers.
Recordings of the conference talks are hosted on
our youtube channel.
How did it go?
Very well, thanks for asking!
We sent out a feedback survey after the event, and got 68 responses from in-person attendees (58% response rate). With the usual caveats that survey respondents are not necessarily a representative sample of the population:
Looking good! Let's dig deeper.
How useful was TAIS 2024 for those new to the field?
Event satisfaction was high across the board, which makes it hard to tell how relatively satisfied population subgroups were. Only those who identified themselves as "new to AI safety" were neutrally satisfied, but the newbies were also the most likely to be highly satisfied.
It seems that people new to AI safety had no more or less trouble understanding the talks than those who work for AI safety organisations or have published AI safety research:
They were also no more or less likely to make new research collaborations:
Note that there is substantial overlap between some of these categories, especially for categories that imply a strong existing relationship to AI safety, so take the above charts with a pinch of salt:
Total
New to AI safety
Part of the AI safety community
Employed by an AI safety org
Has published AI safety research
New to AI safety
26
100%
19%
12%
4%
Part of the AI safety community
28
18%
100%
36%
32%
Employed by an AI safety org
20
15%
50%
100%
35%
Has published AIS research
13
8%
69%
54%
100%
Subjectively, it fe...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We ran an AI safety conference in Tokyo. It went really well. Come next year!, published by Blaine on July 18, 2024 on LessWrong.
Abstract
Technical AI Safety 2024 (TAIS 2024) was a conference organised by
AI Safety 東京 and
Noeon Research, in collaboration with
Reaktor Japan,
AI Alignment Network and
AI Industry Foundation. You may have heard of us through ACX.
The goals of the conference were
1. demonstrate the practice of technical safety research to Japanese researchers new to the field
2. share ideas among established technical safety researchers
3. establish a good international reputation for AI Safety 東京 and Noeon Research
4. establish a Schelling conference for people working in technical safety
We sent out a survey after the conference to get feedback from attendees on whether or not we achieved those goals. We certainly achieved goals 1, 2 and 3; goal 4 remains to be seen. In this post we give more details about the conference, share results from the feedback survey, and announce our intentions to run another conference next year.
Okay but like, what was TAIS 2024?
Technical AI Safety 2024 (TAIS 2024) was a small non-archival open academic conference structured as a lecture series. It ran over the course of 2 days from April 5th-6th 2024 at the International Conference Hall of the Plaza Heisei in Odaiba, Tokyo.
We had
18 talks covering 6 research agendas in technical AI safety:
Mechanistic Interpretability
Developmental Interpretability
Scaleable Oversight
Agent Foundations
Causal Incentives
ALIFE
…including talks from Hoagy Cunningham (Anthropic), Noah Y. Siegel (DeepMind), Manuel Baltieri (Araya), Dan Hendrycks (CAIS), Scott Emmons (CHAI), Ryan Kidd (MATS), James Fox (LISA), and Jesse Hoogland and Stan van Wingerden (Timaeus).
In addition to our invited talks, we had 25
submissions, of which 19 were deemed relevant for presentation. 5 were offered talk slots, and we arranged a poster session to accommodate the remaining 14. In the end, 7 people presented posters, 5 in person and 2 in absentia. Our best poster award was won jointly by Fazl Berez for
Large Language Models Relearn Removed Concepts and Alex Spies for
Structured Representations in Maze-Solving Transformers.
We had 105 in-person attendees (including the speakers). Our live streams had around 400 unique viewers, and maxed out at 18 concurrent viewers.
Recordings of the conference talks are hosted on
our youtube channel.
How did it go?
Very well, thanks for asking!
We sent out a feedback survey after the event, and got 68 responses from in-person attendees (58% response rate). With the usual caveats that survey respondents are not necessarily a representative sample of the population:
Looking good! Let's dig deeper.
How useful was TAIS 2024 for those new to the field?
Event satisfaction was high across the board, which makes it hard to tell how relatively satisfied population subgroups were. Only those who identified themselves as "new to AI safety" were neutrally satisfied, but the newbies were also the most likely to be highly satisfied.
It seems that people new to AI safety had no more or less trouble understanding the talks than those who work for AI safety organisations or have published AI safety research:
They were also no more or less likely to make new research collaborations:
Note that there is substantial overlap between some of these categories, especially for categories that imply a strong existing relationship to AI safety, so take the above charts with a pinch of salt:
Total
New to AI safety
Part of the AI safety community
Employed by an AI safety org
Has published AI safety research
New to AI safety
26
100%
19%
12%
4%
Part of the AI safety community
28
18%
100%
36%
32%
Employed by an AI safety org
20
15%
50%
100%
35%
Has published AIS research
13
8%
69%
54%
100%
Subjectively, it fe...