The speaker relies on building context rather than having a great memory to remember things. Re-reading alone doesn't help them understand better, but investing in more context helps pieces to fall into place. They believe in investing in context more than having a good memory and think that the deepest learning from books comes from understanding context and gaining insights.