Is There a Case for Skepticism of Psychedelic Therapy?

There's been an explosion of interest in the use of psychedelics in psychiatry. Like everyone else, I hope this works out. But recent discussion has been so overwhelmingly positive that it's worth reviewing whether there's a case for skepticism. I think it would look something like this:

1. Psychedelics have mostly been investigated in small studies run by true believers. These are the conditions that produce a field made of unreplicable results, like the effects of 5-HTTLPR. Some of the most exciting psychedelic findings have already failed to replicate; for example, a study two years ago found that psilocybin did not permanently increase the Openness personality trait. This was one of the most exciting studies and had shaped a lot of my thinking around the issue. Now it's gone.

2. Some of the most impressive stories involve psychedelic-assisted psychotherapy, where people who talk with a therapist, while on a substance, obtain true insight and get real closure. But every psychotherapy has amazing success stories floating out there. Back when psychoanalysis was new, the whole world was full of people telling their amazing success stories about how Dr. Freud helped them obtain true insight and get real closure. I think of psychotherapy as a domain where people can get as many amazing success stories as they want whether or not they're really doing anything right, for unclear reasons.