Twenty-nine research teams analyzed the same data, and they all reached different results.
Twenty-nine research teams analyzed the same data, and they all reached different results.
https://fivethirtyeight.com/features/science-isnt-broken/
Another study with the same goal of comparing the results from different research teams found similar disparities, though the graphs aren't quite as pretty.
You're viewing a single thread.
Obligatory link to Statistics Done Wrong: The Woefully Complete Guide, a book on how statistics can and has been abused in subtle and insidious ways, sometimes recklessly. Specifically, the chapters on the consequences of underpowered statistics and comparing statistical significance between studies.
I'm no expert on statistics, but I know enough that repeated experiments should not yield wildly different results unless: 1) the phenomenon under observation is extremely subtle so results are getting lost in noise, 2) the experiments were performed incorrectly, or 3) the results aren't wildly divergent after all.
- the whole point of statistics is to extract subtle signals from noise, if you're getting wildly different results, the problem is you're under-powered.
Thanks for taking the time to post these links, just letting you know you're efforts have benefited at least one person who's gonna enjoy reading this.