AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Just because concepts, theories, and opinions are useful and make people feel comfortable, doesn’t mean they are correct. No one so far has come up with an answer in the field of safety that proves, “this is the way we should do it,” and in the work of safety, we must constantly evaluate and update our practices, rules, and recommendations. This of course means we can never feel completely comfortable – and humans don’t like that feeling. We’ll dig into why we should be careful about feeling a sense of “clarity” and mental ease when we think that we understand things completely- because what happens if someone is deliberately making us feel that a problem is “solved”...?
The paper we’re discussing deals with a number of interesting psychological constructs and theories. The abstract reads:
The feeling of clarity can be dangerously seductive. It is the feeling associated with understanding things. And we use that feeling, in the rough-and-tumble of daily life, as a signal that we have investigated a matter sufficiently. The sense of clarity functions as a thought-terminating heuristic. In that case, our use of clarity creates significant cognitive vulnerability, which hostile forces can try to exploit. If an epistemic manipulator can imbue a belief system with an exaggerated sense of clarity, then they can induce us to terminate our inquiries too early — before we spot the flaws in the system. How might the sense of clarity be faked? Let’s first consider the object of imitation: genuine understanding. Genuine understanding grants cognitive facility. When we understand something, we categorize its aspects more easily; we see more connections between its disparate elements; we can generate new explanations; and we can communicate our understanding. In order to encourage us to accept a system of thought, then, an epistemic manipulator will want the system to provide its users with an exaggerated sensation of cognitive facility. The system should provide its users with the feeling that they can easily and powerfully create categorizations, generate explanations, and communicate their understanding. And manipulators have a significant advantage in imbuing their systems with a pleasurable sense of clarity, since they are freed from the burdens of accuracy and reliability. I offer two case studies of seductively clear systems: conspiracy theories; and the standardized, quantified value systems of bureaucracies.
Discussion Points:
Resources: