• It is OK to be wrong

    As researchers, we are in the business of uncovering the truth, and truth is one of our greatest motivators. Truth is what gets us to work late into the night and what gets us up the next morning. The thought of being wrong terrifies us, to the point where our writing can become an impenetrable mix of jargon and qualification. And when we see others – politicians, news media, and colleagues – get things wrong, it really pisses us off. This is a good thing. As a community, scientists are trustworthy in large part because of their dedication to the truth.

    But I think we also need to allow ourselves – and each other – to be wrong once in a while. It’s great that as scientists we publish papers the measure the nth decimal point on a known quantity, that apply known techniques and paradigms to new environments and conditions, and that verify things we already know to be true. But we must also be willing to accept that our paradigms might be wrong… and if we have good data to present that are contrary to those paradigms, we must be willing to take the gamble of publishing them even if our conclusions and proposed paradigm shifts may end up being incorrect.

    You’re probably thinking that I’m talking about the “Arsenic life” paper or the “faster than light neutrinos” work. I am. But I’m also talking about an impact event being causally linked to the extinction of the dinosaurs. I’m also talking about shotgun sequencing. I’m also talking about plate tectonics. I’m also talking about the discovery of the first exoplanets, and the techniques currently used to find thousands of them. All of these discoveries were soundly criticized at the time they were first discussed or proposed because they went against the prevailing wisdom in their respective fields, but all of these ideas have survived to become the new prevailing wisdom.

    For that matter, I’m also talking about the “Martian life in a meteorite” work (amongst astrobiologists, “ALH84001″). The current prevailing opinion on that within the field is that the data presented in the paper does not constitute evidence for ancient Martian life, and that the main hypothesis presented in the paper was not validated. And yet… the subsequent work refuting the claims in that paper have significantly advanced our understanding of biosignatures, the size/volume limits to life, the chemistry of hematite precipitation, the large-scale carbon cycle of Mars, and even the definition of life.

    Through journal clubs and the like, we’re trained as scientists to reply to such works by gathering in a room and playing a game of “who can slam this paper the harshest?” The person that comes up with the most basic, damning, and snarky critique seems to be the “victor” of this game. I actually think this is a reasonable and healthy exercise, as it can help teach students and remind senior scientists of good lab techniques and sound argument construction.

    My concerns arise when we do this in public and the critiques grow personal. I fear that by publicly vilifying those that were wrong, we isntill in ourselves a major disincentive to do “big science” that challenges conventions and has the potential to set us up for major advances. And while we should always be afraid of being wrong, I am concerned that this extra step of public vindictiveness will petrify many young scientists and thereby prevent major and correct discoveries in the future.

    Further, the pressure to never be wrong encourages defensiveness and doubling down on things that been demonstrated to be incorrect. This is the fatal flaw in most work done by global warming deniers and creationists, and on a smaller scale it’s something that has happened in the wake of papers published in high-impact journals that were subsequently refuted. Our instinct is to toss those individuals into the “unresonable” bin (after all, this action is often unreasonable)… but seldom do we question the contextual forces set up by our academic culture and by confirmation bias that pressures us to act in this manner when we’re wrong.

    Criticism – even public criticism – is a good thing. I’m a big proponent of the ability of the internet to provide rapid, detailed analyses of work… and believe it could dramatically speed-up the cycle of drafting, reviewing, correcting, and verifying work without significant loss in quality. But we must – we MUST – avoid the temptations of public snark, vindictiveness, and personal attacks. These are all provided by the ability to post anonymously, are encouraged by academic chest-beating, and fit into a greater internet culture. It’s OK to tell someone that they are wrong. Just be professional about it. And when you see someone else being a jerk when criticizing someone else’s work, don’t let it discourage you from taking a chance on making the “next big discovery.”

    Because it’s OK to be wrong.

    • Lucas Mix

      Have you read  >Being Wrong< by Kathryn Schulz?  Great book.  Thanks for the wonderful post.
      Lucas

      • http://paleblueblog.org/ Shawn Domagal-Goldman

        No… but it sounds wonderful. I assume you enjoyed it?

    • http://www.facebook.com/profile.php?id=14102172 Amanda Marie Cook

      I agree, Shawn.  I was really surprised in particular with the reaction I saw to the Arsenic paper.  People calling for this woman (a POST-DOC, good grief!) to be fired.  It’s a bit extreme.  I think you’re right — while it seems the paper’s conclusions are not valid, the great news is that her paper will spawn 10’s or 100’s of others, and make that branch of astrobiology even stronger than it was before.  We will be better at answering the questions she poses. 

      I also totally agree that this villification only serves to scare us into pursuing only the least provocative questions.  That’s a very bad thing.  It’s not what we’re about.  In fact, I think it’s the opposite.

    • http://www.facebook.com/profile.php?id=14102172 Amanda Marie Cook

      I
      agree, Shawn.  I was really surprised in particular with the reaction I
      saw to the Arsenic paper.  People calling for this woman (a POST-DOC,
      good grief!) to be fired.  It’s a bit extreme.  I think you’re right —
      while it seems the paper’s conclusions are not valid, the great news is
      that her paper will spawn 10’s or 100’s of others, and make that branch
      of astrobiology even stronger than it was before.  We will be better at
      answering the questions she poses. 

      I also totally agree that this villification only serves to scare us
      into pursuing only the least provocative questions.  That’s a very bad
      thing.  It’s not what we’re about.  In fact, I think it’s the opposite.