Is the “Bilingual Advantage” Skewed by Publication Bias?

In the past, research about bilingual speakers as an advantage for cognitive functions and executive control would be subjected to Second Opinion columns (such as this one).

Recently, a group of scientists from the University of Edinburgh and the University of Sassari- who themselves have published research that supports the idea that bilinguals have a cognitive advantage — have revealed evidence that bibliography on this topic may be skewed due to a bias toward publishing positive results.

Nevertheless, this study does not consider that the idea of a bilingual cognitive advantage is false but it teaches to always be skeptical about medical research.

The “File Drawer Effect” 

The authors of the Psychological Science paper say that at first, they thought that considering a bilingual cognitive advantage was a correct reflection of all the research they had submitted for publication enclosing just the positive results of the experiments.

This inclination of scientists to not publish their negative-result studies for publication is a well-known phenomenon known as the “file drawer effect.”

The researchers decided find evidence of the effect in the research literature. Firstly, they identified all the unpublished works about bilingualism and executive control presented in 169 important conferences between 1999 and 2012. They found 104. Of those, 38% had results that supported a bilingual advantage, 13% had mixed results that appeared to support a bilingual advantage, 32% had showed no executive control differences between monolinguals and bilinguals. Finally, 16% found no advantage at all. Fifty-two of those studies were finally included in 50 articles, which represents 68% of positive studies about bilingualism, 50% of the articles with mixed opinions but a positive tendency about bilingual advantage, 39% of the studies with mixed results that partly challenged the bilingual-advantage and 29% of research that found no differences between monolinguals and bilinguals or that reported a bilingual disadvantage.

According to detailed analyses, the size of the study or cognitive tasks are not the reasons why the studies are not published, a fact that the authors of the Psychological Science paper were unable to explain. They think researchers avoid publishing studies with mixed or negative results (the “file drawer effect”) or peer reviewers and editors may be rejecting studies that report those results more often than ones reporting positive results.

Staying Open to the Evidence 

Although all the scientific studies about bilingualism and executive control are not published, a cognitive advantage couldn’t be discarded. “We agree that bilingualism should be conceived, a priori, as a positive and desirable achievement,” they write, “[but] we are also convinced that educational and political debates addressing the relevance of bilingualism should not be promoted by ignoring null or negative results.”

Their findings also emphasize the ongoing need for the scientific community to “be more open to studies that challenge the existing theories, especially when these are not yet fully established.

All data, not just selected data that supports a particular theory, should be shared,” the researchers conclude, “and this is especially true when it comes to data regarding issues that have enormous societal relevance and implications, such as bilingualism.”

The abstract of the study is available on the Psychological Science website, but the study itself is behind a paywall.

Written by Dr. David Alcantara and Paula Ruíz for The All Results Journals.