Most health research publications may seem like legitimate science; filled with experiment design descriptions, esoteric terms and a statistic here and there. However, with the recent retraction of the paper on weight loss and green coffee bean – a product that Dr. Oz touted on his show – there is a haze of skepticism and mistrust of some scientific studies among both the public and some health care professionals. How could something like this be published in a field as rigorous and objective as science? Most of the blame is on bias. In this case, bias is defined as a systematic error that was introduced into testing or sampling by encouraging or selecting one answer or outcome over others. While there may be hundreds of different types of research bias, some of them are more common than others. Understanding research bias better can help the general public combat “bad science” and pseudoscience if they know what to look for when reading health articles and studies.
Sampling Bias: If a cola company wants to prove its soda is more popular than root beer or any other type of non-cola sodas, it would most likely pick a sample population of people who already like and drink colas regularly and get them involved in a study. This is an example of sample bias, when a group or number of people are favored over other groups. Oftentimes, sample bias is unintentional, yet its outcome is very similar to the situation when sample populations are cherry-picked to support or refute a hypothesis. An example of unintentional sample bias is telephone marketing, where a group of pollsters call people about their opinions on a topic. This system may be biased toward those who do not have a phone or those with cell phones that have a different area code than their area of residence.
Other characteristics of sampling bias include a very small number of test subjects, favoritism (e.g. race, gender, age, education), geography and language barriers. Thus, a weight-loss study among 20 overweight, male college sophomores in UCSD who speak only English does not reflect the rest of the population in the U.S. or the world.
Experiment Design Bias: In genuine scientific research, the object of the game is to refute a given hypothesis, not to “prove” that it is true. Therefore, the experimental setup would be designed to see if the hypothesis is false. Even if the sample population has the least amount of bias, the research can still be heavily biased with its design. Dr. Kevin Mullane and Dr. Mike Williams, two of the editors of Elsevier Journal, wrote in an editorial that design bias includes attempting to prove a hypothesis is true, the “lack of the consideration of the null hypothesis,” reliance on a single piece of data, and a lack of a double-blind and randomized selection.
A common example are studies that lack a control group which does not receive an intervention or receives a dummy treatment (commonly known as a placebo). Without a control, it would be difficult to determine if confounding factors affected the outcome or not.
WEIRD Science: Most of the sample population of many research papers tend to be WEIRD people: Western, Educated, Industrialized, Rich, and Democratic (the last word refers to democracy in general, not the Democratic Party). In a paper published in Behavioral and Brain Sciences in 2010, Canadian psychologists Joseph Henrich, Steven Hein and Ara Norazayan from the University of British Columbia stated that the WEIRD population that is common in published research does not represent the majority of the human species. When psychological and sociological factors, such as spatial perception, moral reasoning, cooperation, fairness, and visual perception are examined cross-culturally, the WEIRD population tends to be composed of outliers. According to the authors, there are no visible a priori grounds for making a claim that a specific behavioral phenomenon is universal based on samples from a single subpopulation of individuals.
Taking one or more research papers with a predominant WEIRD population, which is also a type of sampling bias, do not always justify that a claim or hypothesis to be true or false for everyone. Of course, this can also be said about research that take a small sample population from a non-WEIRD population.
Publication Bias: Perhaps the king of all research bias may be publication bias, a vital consideration in what to look for when reading health studies and articles. No matter how least biased the research is, even if it is Carl-Sagan-approved, publication bias acts as a gatekeeper on what gets published or what does not. This makes even some scientists scratch their heads as they sometimes have trouble figuring out which are junk sciences or not.
To combat publication bias, Jeffrey Beall, a research librarian at the University of Colorado in Denver, created and launched a website (Scholarly Open Access) that lists what he called “predatory open-access journals.” So far, there are 477 predatory journals listed in 2014. Beall estimates that there could be as many as 4,000 bogus journals.
“Honestly, it is very difficult for a non-scientist to determine whether a given study suffers from any sort of bias,” said research communication specialist Molly Gregas, Ph.D., in an online interview with Guardian Liberty Voice. “And researchers and scientists themselves face this problem when looking at research in an area or subfield different from their own.”
Non-scientists still have some tools in their belt to combat bias-heavy research. “Looking at the journal where the study is published can be helpful,” Gregas explained. “How long has the journal been publishing? What is its impact factor? What other similar studies or papers by other researchers in this field have been published in this journal? Looking at Beall’s list is a good place to eliminate the most suspect sources, and clearly there is a difference in the quality of research and review between the New England Journal of Medicine and Joe’s Heart Disease blog, but the vast area in between can be difficult for the non-expert to navigate.”
Other questions that Gregas suggest readers ask are: Who is citing the study? Is it a marketing company or a medical news article? Do they Does the study or article provide links to the actual study or enough information to find the actual study (name of journal, title of article, author names, etc) or do they just say “a study shows that…”?
“A marketing scheme, supplement company, or MLM seller will claim ‘clinical studies have proven’ but then will not be able to produce those studies or provide additional details and/or they might claim that the studies are proprietary which can’t be reviewed because of ‘trade secrets’,” Gregas added.
It is almost impossible to eliminate any kind of bias in scientific research, but this does not mean that science is less reliable or more faulty. However, recognizing research bias when reading any health study or article can help everyone make better decisions. Science still has a lot to clean up, and in order to make any progress in health research, such biases need to be reduced almost to a crumb on a plate.
By Nick Ng
Sources:
Elsevier
Plastic Reconstruction Surgery
University of Texas
Interview with Molly Gregas, Ph.D.
Behavioral and Brain Sciences
Scholarly Open Access 1
Scholarly Open Access 2
Merriam-Webster
New York Times
