The Nonlinear Library LW - Forecasting One-Shot Games by Raemon
Sep 1, 2024
11:28
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Forecasting One-Shot Games, published by Raemon on September 1, 2024 on LessWrong.
Cliff notes:
You can practice forecasting on videogames you've never played before, to develop the muscles for "decision-relevant forecasting."
Turn based videogames work best. I recommend "Luck Be a Landlord", "Battle for Polytopia", or "Into the Breach."
Each turn, make as many Fatebook predictions as you can in 5 minutes, then actually make your decision(s) for the turn.
After 3 turns, instead of making "as many predictions as possible", switch to trying to come up with at least two mutually exclusive actions you might take this turn, and come up with predictions that would inform which action to take.
Don't forget to follow this up with practicing forecasting for decisions you're making in "real life", to improve transfer learning. And, watch out for accidentally just getting yourself addicted to videogames, if you weren't already in that boat.
This is pretty fun to do in groups and makes for a good meetup, if you're into that.
Recently I published Exercise: Planmaking, Surprise Anticipation, and "Baba is You". In that exercise, you try to make a complete plan for solving a puzzle-level in a videogame, without interacting with the world (on levels where you don't know what all the objects in the environment do), and solve it on your first try.
Several people reported it pretty valuable (it was highest rated activity at my metastrategy workshop). But, it's fairly complicated as an exercise, and a single run of the exercise typically takes at least an hour (and maybe several hours) before you get feedback on whether you're "doing it right."
It'd be nice to have a way to practice decision-relevant forecasting with a faster feedback loop. I've been exploring the space of games that are interesting to "one-shot". (i.e. " try to win on your first playthrough"), and also exploring the space of exercises that take advantage of your first playthrough of a game.
So, an additional, much simpler exercise that I also like, is:
Play a turn-based game you haven't played before.
Each turn, set a 5 minute timer for making as many predictions as you can about how the game works, what new rules or considerations you might learn later. Then, a 1 minute timer for actually making your choices for what action(s) to take during the turn.
And... that's it. (to start with, anyway).
Rather that particularly focusing on "trying really hard to win", start with just making lots of predictions, about a situation where you're at least trying to win a little, so you can develop the muscles of noticing what sort of predictions you can make while you're in the process of strategically orienting. And, notice what sorts of implicit knowledge you have, even though you don't technically "know" how the game would work.
Some of the predictions might resolve the very next turn. Some might resolve before the very next turn, depending on how many choices you get each turn. And, some might take a few turns, or even pretty deep into the game. Making a mix of forecasts of different resolution-times is encouraged.
I think there are a lot of interesting skills you can layer on top of this, after you've gotten the basic rhythm of it. But "just make a ton of predictions about a domain where you're trying to achieve something, and get quick feedback on it" seems like a good start.
Choosing Games
Not all games are a good fit for this exercise. I've found a few specific games I tend to recommend, and some principles for which games to pick.
The ideal game has:
Minimal (or skippable) tutorial. A major point of the exercise is to make prediction about the game mechanics and features. Good games for this exercise a) don't spoonfeed you all the information about the game, but also b) are self-explanatory enough to figure out without a tut...
Cliff notes:
You can practice forecasting on videogames you've never played before, to develop the muscles for "decision-relevant forecasting."
Turn based videogames work best. I recommend "Luck Be a Landlord", "Battle for Polytopia", or "Into the Breach."
Each turn, make as many Fatebook predictions as you can in 5 minutes, then actually make your decision(s) for the turn.
After 3 turns, instead of making "as many predictions as possible", switch to trying to come up with at least two mutually exclusive actions you might take this turn, and come up with predictions that would inform which action to take.
Don't forget to follow this up with practicing forecasting for decisions you're making in "real life", to improve transfer learning. And, watch out for accidentally just getting yourself addicted to videogames, if you weren't already in that boat.
This is pretty fun to do in groups and makes for a good meetup, if you're into that.
Recently I published Exercise: Planmaking, Surprise Anticipation, and "Baba is You". In that exercise, you try to make a complete plan for solving a puzzle-level in a videogame, without interacting with the world (on levels where you don't know what all the objects in the environment do), and solve it on your first try.
Several people reported it pretty valuable (it was highest rated activity at my metastrategy workshop). But, it's fairly complicated as an exercise, and a single run of the exercise typically takes at least an hour (and maybe several hours) before you get feedback on whether you're "doing it right."
It'd be nice to have a way to practice decision-relevant forecasting with a faster feedback loop. I've been exploring the space of games that are interesting to "one-shot". (i.e. " try to win on your first playthrough"), and also exploring the space of exercises that take advantage of your first playthrough of a game.
So, an additional, much simpler exercise that I also like, is:
Play a turn-based game you haven't played before.
Each turn, set a 5 minute timer for making as many predictions as you can about how the game works, what new rules or considerations you might learn later. Then, a 1 minute timer for actually making your choices for what action(s) to take during the turn.
And... that's it. (to start with, anyway).
Rather that particularly focusing on "trying really hard to win", start with just making lots of predictions, about a situation where you're at least trying to win a little, so you can develop the muscles of noticing what sort of predictions you can make while you're in the process of strategically orienting. And, notice what sorts of implicit knowledge you have, even though you don't technically "know" how the game would work.
Some of the predictions might resolve the very next turn. Some might resolve before the very next turn, depending on how many choices you get each turn. And, some might take a few turns, or even pretty deep into the game. Making a mix of forecasts of different resolution-times is encouraged.
I think there are a lot of interesting skills you can layer on top of this, after you've gotten the basic rhythm of it. But "just make a ton of predictions about a domain where you're trying to achieve something, and get quick feedback on it" seems like a good start.
Choosing Games
Not all games are a good fit for this exercise. I've found a few specific games I tend to recommend, and some principles for which games to pick.
The ideal game has:
Minimal (or skippable) tutorial. A major point of the exercise is to make prediction about the game mechanics and features. Good games for this exercise a) don't spoonfeed you all the information about the game, but also b) are self-explanatory enough to figure out without a tut...