One of the most iconic examples of this research reform is the introduction of the Registered Report, which is a type of article published in a journal to represent their commitment to publish studies with both positive and negative outcomes. Although it may seem logical to share all information gathered despite the outcome, competition in the science industry and academic world has thus far made it an irregular practice.
Clinical Trial Negatives
Any scientific pursuit is affected by human error. Among all types of studies, clinical trials are those most often believed to be the most reliable. These occur near the end of the research process and often have life or death consequences in the medical community. A clinical trial has always been the last test of a new pharmaceutical or treatment method before it is dubbed effective and safe for human use. Trials are conducted with two groups of patients: one the control group who receives a facsimile treatment or a placebo and one who is given the real thing. No participants know which group they are in. The people conducting the study are also ignorant of each participant's group. Before the research starts, the study must be registered publicly and the data analysis method put into place. This transparent reporting is meant to prevent the scientists from seeing what they want to see instead of what is actually happening. Even with these safeguards, clinical trials suffer from the pressure put on researchers to "publish or perish." What should be the ultimate example of scientific truth may hide hidden agendas.
Are modern science practices fixing the problems of the past?
Although scientific journals do not trouble themselves with entertainment as much as a commercial magazine, the people who read them still want a good story. This publication bias makes certain research results more publishable than others, even if the study was conducted similarly and the outcome good. Take, for example, an imaginary study attempting to prove that a new treatment is more effective against a certain disease than the existing protocol. After careful blind research, the outcome is proven true. The news of a brand-new successful treatment is very exciting and medical journals will be overjoyed to publish this news. The study orchestrators may even find they can place the work in a highly prestigious publication, and, in turn, propel their academic career skyward.
On the other hand, if the research proves that the new treatment did not make any noticeable difference to those afflicted by the disease when compared with current treatments, the study is said to have a negative result. This does not make a good story and therefore may not be published anywhere. It certainly would not attract the attention of international medical journals. For the scientist or group that did the study, funding may be cut off and career trajectory may falter. The negative result studies end up languishing in a file cabinet somewhere and are never shared with the public or scientific community at large.
With this type of outcome for research that does not produce stellar results, scientists are unlikely to make the attempt to get the information out. Instead, they quickly move on to new research that may have a better outcome. The problem with this habit is that negative results are highly useful for the scientific community and necessary to advance the creation of new drugs and medical treatment. It is estimated that anywhere from 16 to 66% of clinical trials are never reported. These high numbers should definitely give pause to those who are interested in a complete and accurate picture.
In the 1980s, a rule about clinical trials was put into place to stop a form of outcome cherry picking. The rule states that all trials should be subject to an advanced registry procedure in which the study design, data collection measures, and plan for analyzation are recorded before any tests are done. The cherry picking that went on prior to this role being put in place may have been done purposefully or as an automatic side effect of simply wanting a positive outcome to a study. How was this accomplished? With a large research study which tests a plethora of variables, scientists could tweak the data collection measures and accepted outcome to achieve a certain result when one did not naturally occur. In layman's terms, they shifted the goalposts in order to achieve a positive outcome. Gotten in this way, an outcome may look positive when it is, in fact, directly opposite.
A researcher's career in the scientific or medical community or the academic world stems from the quality of their research and publications so much so that the bias toward positive outcomes affects their ability to perform a 100% honest study. As mentioned above, this is not always done maliciously. Corners can be cut and expected outcomes tweaked simply to keep their jobs. According to Ben Goldacre's COMPARE project, 58 out of 67 clinical trials reported different outcome measures than the preregistered protocols they had planned or promised to use. Another study in 2014 showed that a full 33% made the same change after study completion.
Registered Reports Offer Double Checks
The old way of doing things does not have the best interest of the public or industry at heart. Luckily, there is a straightforward solution to counteract these hidden changes and the publication bias that makes them so attractive for researchers. The first thing that needs to change is to keep scientific journals ignorant of the outcome of the study until after they deem it publishable or not. In other words, keep them as blind as the clinical trial participants and researchers themselves. Under this system, the study itself must be deemed worthy or not and both negative and positive outcomes will be reported. The second thing that must be enforced is a strict adherence to the outcome data measures that are outlined at the beginning of the research study after the data is collected. If study architects register a certain protocol, they must stick to it or give a thorough explanation why and how they were changed.How can these two things the put into place successfully? The answer is peer review. The introduction of a Registered Report journalistic article covers this admirably. Instead of a one-off like regular science articles, these have a two-step process. People who will conduct studies first submit the information and process description to the journal for feedback, revise it as necessary to satisfy the scientific community at large, and then publish the outcomes after. Every bit of data collected over the course of the clinical trial or another study must be available to the public for this to work.
There are other specific journals conceived as a way to combat the inability to reproduce outcomes in scientific fields (as The All Results Journals). No longer are researchers controlled by the publication bias and silently encouraged to cherry pick protocol and data. When positive outcomes are no longer the most valuable information, more data is accessible and can help fuel advancements that much quicker. The All Results Journals make negative outcomes as valuable as positive.
Written by Dr. David Alcantara for The All Results Journals
No comments:
Post a Comment