To Err is Science
Until now, discovery was often considered the main goal of medical science. But nowadays discovery is almost too easy. Anyone with a little funding and a few biological specimens in a refrigerator can make thousands of postulated “discoveries.”
Indeed, the number of research questions that we can pose is increasing exponentially. Medical kits the size of a thumbnail can measure a million different biological factors on an individual with an infinitesimal amount of blood. A million research questions can be asked on the spot. But even with proper statistical testing, many tens of thousands of these biological factors may seem to be important due to mere chance. Only a handful of them really will be. The vast majority of these initial research claims would yield only spurious findings.
So the main issue nowadays is to validate “discoveries” by replicating them under different settings. Several different teams of researchers need to see them “work” again and again using common rules. Moreover, all the teams should agree not to select and report only the data that seem most impressive. With selective reporting, we would end up with a long list of all the false discoveries made across all research teams, with only a few true findings buried among this pile of non-replicated waste.