Exposing Publication Bias

The Unflattering Data that Gets Lost in Action

In his interesting (and infuriating) Ted Talk “What doctors don’t know about the drugs they prescribe”, Ben Goldacre talks about Publication Bias which he defines as,

“….the technical term for the phenomenon where unflattering data gets lost, gets unpublished, or is left missing in action.”

He continues to explain about an anti-depressant called Reboxetine that he had previously prescribed to his patients. But being the self confessed “nerdy doctor” he read all the studies that he could find on this drug.

Firstly he read the 1 study that was published that showed that the drug was better than placebo. BUT, as he says, he was misled.

There were actually 7 trials that tested the drug against a placebo. He read the one positive trial that was published. But this meant that 6 were left unpublished. Why? Well, maybe the fact that they showed negative results could have had something to do with it!?

He also read the studies that compared Reboxetine against other antidepressants. 3 studies showed that the drug was’ just as good’ as any other anti-depressants and they were published. BUT 3 times as many studies showed that Reboxetine was worse than other drugs and guess what? They were left unpublished!

Goldacre goes on to explain how researchers went and found all the trials on all antidepressants that were presented to the FDA. ¹ There were 38 trials with positive results. And 36 with negative results.

But when they searched to see which of these were then published in the academic literature (where the majority of people find their research), it was a very different story. 

37 of the positive trials were published in full within the academic literature. However, only 3 of negative trials were ever published.

The fact is, positive findings are more likely to be published. This, ladies and gentlemen, is publication bias! And something we all need to be aware exists.