The Nonlinear Library LW - Computational Mechanics Hackathon (June 1 & 2) by Adam Shai
May 27, 2024
02:48
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Computational Mechanics Hackathon (June 1 & 2), published by Adam Shai on May 27, 2024 on LessWrong.
Join our
Computational Mechanics Hackathon, organized with the support of APART, PIBBSS and Simplex.
This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects!
Read more and sign up for the event
here.
We're excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In
,
Transformers Represent Belief State Geometry in their Residual Stream
, we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers.
This opens up a large number of potential projects in interpretability. There's a lot of work to do!
Key things to know:
Dates: Weekend of June 1st & 2nd, starting with an opening talk on Friday May 31st
Format: Hybrid - join either online or in person in Berkeley! If you are interested in joining in person please contact Adam.
Program:
Keynote Opening by @Adam Shai and @Paul Riechers - Friday 10:30 AM PDT
Online Office Hours with Adam and Paul on Discord - Saturday and Sunday 10:30 PDT
Ending session - Sunday at 17:30 PDT
Project presentations - Wednesday at 10:30 PDT
Projects:
After that, you will form teams of 1-5 people and submit a project on the entry submission page. By the end of the hackathon, you will submit: 1) The PDF report, 2) a maximum 10-minute video overview, 3) title, summary, and descriptions. You will present your work on the following Wednesday.
Sign up: You can sign up on this
website. After signing up, you will receive a link to the discord where we will be coordinating over the course of the weekend. Feel free to introduce yourself on the discord and begin brainstorming ideas and interests.
Resources:
You're welcome to engage with this selection of
resources before the hackathon starts.
Check out our (living)
Open Problems in Comp Mech document, and in particular the section with Shovel Ready Problems.
If you are starting a project or just want to express interest in it, fill out a row in
this spreadsheet
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
Join our
Computational Mechanics Hackathon, organized with the support of APART, PIBBSS and Simplex.
This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects!
Read more and sign up for the event
here.
We're excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In
,
Transformers Represent Belief State Geometry in their Residual Stream
, we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers.
This opens up a large number of potential projects in interpretability. There's a lot of work to do!
Key things to know:
Dates: Weekend of June 1st & 2nd, starting with an opening talk on Friday May 31st
Format: Hybrid - join either online or in person in Berkeley! If you are interested in joining in person please contact Adam.
Program:
Keynote Opening by @Adam Shai and @Paul Riechers - Friday 10:30 AM PDT
Online Office Hours with Adam and Paul on Discord - Saturday and Sunday 10:30 PDT
Ending session - Sunday at 17:30 PDT
Project presentations - Wednesday at 10:30 PDT
Projects:
After that, you will form teams of 1-5 people and submit a project on the entry submission page. By the end of the hackathon, you will submit: 1) The PDF report, 2) a maximum 10-minute video overview, 3) title, summary, and descriptions. You will present your work on the following Wednesday.
Sign up: You can sign up on this
website. After signing up, you will receive a link to the discord where we will be coordinating over the course of the weekend. Feel free to introduce yourself on the discord and begin brainstorming ideas and interests.
Resources:
You're welcome to engage with this selection of
resources before the hackathon starts.
Check out our (living)
Open Problems in Comp Mech document, and in particular the section with Shovel Ready Problems.
If you are starting a project or just want to express interest in it, fill out a row in
this spreadsheet
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org