Feb 27, 2015
Why null results rarely see the light of day
The fact that experiments don’t always come to an end due to journals’ requirements is actually an important object of study. This problem, called ‘the file drawer effect’, is being carefully analyzed in order to provide a real insight into the issue.
A recent study started by a group of scientists from the Stanford University reveals that the majority of researchers reject the negative results, focusing on the positive ones, as the below graphic shows.
As we can see, more than 60% of the studies that gave null results were stored, while nearly all the studies providing strong results (96% approximately) were written up, which is the reason why literature is altered and we can find duplicated research.
To solve this problem, authors proposed gathering all data and studying designs in a public registry. This is a solution that has divided opinions. Most scientists agree and think that it could be useful, but there are others that are against this idea because it may introduces new biases.
One of the critical points in solving the problem of unacceptable results is the debate about the experiences related to medical studies because of their possible influence on people.
In this study, Neil Malhotra, a Stanford political economist, and two of his students have been investigating each subsidized research project since 2002, focusing on unpublished experiments and the granted ones. They have found that null results garner no interest from journals at all. The All Results Journals are the exception.
These detections suggested to Malhotra that it is all a matter of changing our thoughts on unsuccessful experiences. The statistics they considered show that only 42% of studies produce significant results and 62% of those are actually published. On the other side, 21% of studies are useless and about 65% of them are filed away.
Some scientists consider that publishing the results of these null experiments could avoid the waste of time and money and ultimately it could be a good way to prove that an experiment is as good as it seems, as well as discouraging researchers from altering their results in order to make them more publishable. They totally agree with the idea of a registry and they also approve of enclosing a preanalysis of the study to make the objective of their studies clear. There are also cautious opinions about the idea of publishing a plan, since it could show a lack of confidence in the scientist’s work. Others complain about the imposition of a registry, while others think it would be good to promote each study and lead the scientific community to adopt worthy rules.
Written by Dr. David Alcantara and Paula Ruíz for The All Results Journals.