Aug 15, 2013

Significance Bias in Animal Studies

John Ioannidis of Stanford and colleagues have published the paper “Evaluation of Excess Significance Bias in Animal Studies of Neurological Diseases” where they talk about results of animal biomedical experiments. These experiments generate valuable hypotheses that lead to the conduct of preventive or therapeutic clinical trials.

Studies have shown that the results of animal biomedical experiments fail to translate into human clinical trials; this could be attributed either to real differences in the underlying biology between humans and animals, to shortcomings in the experimental design, or to bias in the reporting of results from the animal studies.

They used a statistical technique to evaluate whether the number of published animal studies with “positive” (statistically significant) results is too large to be true. They assessed 4.445 animal studies for 160 candidate treatments of neurological disorders, and observed that 1.719 of them have a “positive” result, whereas only 919 studies would a priori be expected to have such a result. According to their methodology, only eight of the 160 evaluated treatments should have been subsequently tested in humans.

Ignoring the hypothesis that the "positive" results may be totally faked , this could be due to there are many analyses that can be performed, but only the analysis with the “best” results is presented resulting in potentially misleading findings or due to bias against publication of “negative” results (publication bias).

Ioanndis et al. showed (ref) a new concern about journals’ positive publication bias (non-publication of neutral/negative results) and points to problems elsewhere in the system – involving research paper authors, their institutions, funders and the journals.

They are convinced that publication and selective reporting biases may be diminished by pre-registering experimental animal studies. In this regard, access to the study protocol and also to raw data and analyses would allow verification of their results, and make their integration with other parallel or future efforts easier.

And why not? could be a beginning…

Written by Dr. Belén Suarez  for The All Results Journals

No comments:

Post a Comment