The Nonlinear Library EA - Case-control survey of EAGx attendees finds no behavioural or attitudinal changes after six months by Fods12
Jul 27, 2024
25:39
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Case-control survey of EAGx attendees finds no behavioural or attitudinal changes after six months, published by Fods12 on July 27, 2024 on The Effective Altruism Forum.
Prepared by James Fodor and Miles Tidmarsh
EAGxAustralia 2023 Committee
Abstract
EAGx conferences are an important component of the effective altruism community, and have proven a popular method for engaging EAs and spreading EA ideas around the world. However, to date relatively little publicly available empirical evidence has been collected regarding the long term impact of such conferences on attendees. In this observational study we aimed to assess the extent to which EAGx conferences bring about change by altering EA attitudes or behaviours.
To this end, we collected survey responses from attendees of the EAGxAustralia 2023 conference both before and six months after the conference, providing a measure of changes in EA-related attitudes and behaviours over this time. As a control, we also collected responses to the same survey questions from individuals on the EA Australia mailing list who did not attend the 2023 conference.
Across 20 numerical measures we collected, we did not find any statistically significant differences in the six-month changes across the two groups. Specifically, we are able to rule out effect sizes of larger than about 20% for most measures. In general, we found self-reported EA attitudes and behaviours were remarkably consistent across most individuals over this time period.
We provide a discussion of these results in the context of developing better measures of the impact of EAGx conferences, and conclude with some specific recommendations for future conference organisers.
Background
'EAGx' is the branding used by the Centre for Effective Altruism (CEA) for centrally-supported but independently-organised conferences held around the world each year. The aim of these events is to communicate EA ideas, foster community growth and participation, and facilitate the formation of beneficial connections for EA projects. EAGx conferences have been organised in Australia every year
since 2016 (with a hiatus in 2020 and 2021 due to the COVID-19pandemic), with the most recent event taking place in Melbourne in September 2023.
While EAGx conferences have proved popular with attendees, relatively little publicly available evidence has been collected regarding their impact or effectiveness. The main source of information can be found in the
forum sequence by Ollie Base. Most conference retrospective reports give details about attendance and self-reported attendee satisfaction, but do not attempt to measure the impact of the conference in achieving any concrete goals. The limited range of publicly-available evaluations is surprising given the importance of these events to the EA community, and has prompted comment on the EA forum regarding the relative
lack of evaluation of EA projects generally, and of
EAGx conferences specifically.
For the past few years, the main form of EAGx evaluation has been a post-conference survey, along with a six-month follow-up, administered by CEA, in which attendees are asked to report the beneficial outcomes of the conference for them personally, including making new connections, starting new projects, or learning key information that informed major decisions.
Of these, the number of new connections made is typically regarded as the most important, with the number of connections per dollar spent being
used as a key metric by CEA in assessing effectiveness. These methods have a number of advantages, including ease of collection, ability to compare across locations and over time, and relative ease of interpretation.
A major limitation of these existing measures is that they require survey respondents to explicitly make value judgements about their experienc...
Prepared by James Fodor and Miles Tidmarsh
EAGxAustralia 2023 Committee
Abstract
EAGx conferences are an important component of the effective altruism community, and have proven a popular method for engaging EAs and spreading EA ideas around the world. However, to date relatively little publicly available empirical evidence has been collected regarding the long term impact of such conferences on attendees. In this observational study we aimed to assess the extent to which EAGx conferences bring about change by altering EA attitudes or behaviours.
To this end, we collected survey responses from attendees of the EAGxAustralia 2023 conference both before and six months after the conference, providing a measure of changes in EA-related attitudes and behaviours over this time. As a control, we also collected responses to the same survey questions from individuals on the EA Australia mailing list who did not attend the 2023 conference.
Across 20 numerical measures we collected, we did not find any statistically significant differences in the six-month changes across the two groups. Specifically, we are able to rule out effect sizes of larger than about 20% for most measures. In general, we found self-reported EA attitudes and behaviours were remarkably consistent across most individuals over this time period.
We provide a discussion of these results in the context of developing better measures of the impact of EAGx conferences, and conclude with some specific recommendations for future conference organisers.
Background
'EAGx' is the branding used by the Centre for Effective Altruism (CEA) for centrally-supported but independently-organised conferences held around the world each year. The aim of these events is to communicate EA ideas, foster community growth and participation, and facilitate the formation of beneficial connections for EA projects. EAGx conferences have been organised in Australia every year
since 2016 (with a hiatus in 2020 and 2021 due to the COVID-19pandemic), with the most recent event taking place in Melbourne in September 2023.
While EAGx conferences have proved popular with attendees, relatively little publicly available evidence has been collected regarding their impact or effectiveness. The main source of information can be found in the
forum sequence by Ollie Base. Most conference retrospective reports give details about attendance and self-reported attendee satisfaction, but do not attempt to measure the impact of the conference in achieving any concrete goals. The limited range of publicly-available evaluations is surprising given the importance of these events to the EA community, and has prompted comment on the EA forum regarding the relative
lack of evaluation of EA projects generally, and of
EAGx conferences specifically.
For the past few years, the main form of EAGx evaluation has been a post-conference survey, along with a six-month follow-up, administered by CEA, in which attendees are asked to report the beneficial outcomes of the conference for them personally, including making new connections, starting new projects, or learning key information that informed major decisions.
Of these, the number of new connections made is typically regarded as the most important, with the number of connections per dollar spent being
used as a key metric by CEA in assessing effectiveness. These methods have a number of advantages, including ease of collection, ability to compare across locations and over time, and relative ease of interpretation.
A major limitation of these existing measures is that they require survey respondents to explicitly make value judgements about their experienc...