How Much World Building Is Enough Without Being Too Little?

It kind of straddles the line between historical fiction and creative nonfiction. It's wonderful in particular because of the portrait that it paints of a writer's daily life. Not everyone enjoys world building so I wanted to go the bare minimum. How much world building is enough without being too little? Well, it depends on the kind of story you're telling.