The average level of power in these studies was 18%. Most listeners won't know what that means. What we try to ask is, what is the power of a study to be able to get a result that would cross that level of statistical significance at the.05 level? If the true effect out there is X, and now the question is, how do you know the true effect? Nobody really knows the true effect. There's different ways to approximate it,. One way to approximate it plausibly is to say that if you consider all the evidence, then the true effect is best represented or best approximated by the result. That's the best shot that we can have.
John Ioannidis of Stanford University talks with EconTalk host Russ Roberts about his research on the reliability of published research findings. They discuss Ioannidis's recent study on bias in economics research, meta-analysis, the challenge of small sample analysis, and the reliability of statistical significance as a measure of success in empirical research.