Understanding Doxastic Involuntarism and Epistemic Responsibility

It's easy to say that people who hold extreme antisocial beliefs should be held responsible for those beliefs. But in fact, many extremists operate within what philosophers call impoverished epistemic environments - epistemic "bubbles" and echo chambers whose inhabitants might be ignorant of the truth, or subject to manipulation. But does that mean responsibility for extreme beliefs therefore lies with the wider public? And if so, what are we to do about it?