On Saturday, The Globe and Mail ran an opinion piece by Susan Pinker, a psychologist, about a replication crisis in science. The past year and a half has brought hundreds of thousands of studies about COVID-19, along with an unusual level of enthrallment in science by the media and the public.
As Pinker points out, this wave of research highlights the importance of the principle that scientific advances are not based on single studies, but rather on the rigourous replication of findings. After sending Pinker's article to a colleague, he forwarded me a pre-COVID article on the same topic, which like Pinker's, shares interesting examples from psychology.
This replication principle of course holds true in mental health and addictions research. However, studies that startle us or findings that support our worldview are very tempting. I'm sure I've fallen for more than a few in my life.
In the end, it's the convergence of evidence from a number of settings that validates a finding. As well, the ways that the evidence does not converge on a finding can describe its limitations — for instance, the individuals who don't respond as theorized to a mental health treatment. These limitations can point the direction towards important future research.
As consumers of the science literature we need to be careful not to fall hard for that first study.