The Moscow Rules - Science Edition: Part 2

May 01 2012 Published by under Uncategorized

Today, my second SciFund project - “Beach of the Goliath Crabs” - launched at RocketHub. I’m excited to be taking part in round 2 of this crowdfunding endeavor, because it is the future of science.

It’s fun, but not easy. Part of putting together a project required making a video. There are a lot of decisions to make about exactly what to say, show, and what music to use. (Speaking of music, here’s today’s vintage Cold War spy music to set the mood.)

I did a full two weekends before #SciFund started. I ended up redoing almost all the sound, and shooting a new section, the next weekend. I did all this extra work because I was following Moscow Rules number 2.

Never go against your gut.

I watched my draft video, and I just knew there were places that were weak. Sometimes, it was as small as how I said one word in a sentence. But could I tell you why it was wrong in an analytic way? Nope. This was gut instinct.

This is a hard lesson for scientists. We are supposed to be analytical. Ken Robinson jokes that professor live in their heads so much that their bodies are just a way to get their heads to meetings.

Four organs of decision-making

Randy Olson talks about this in Don't Be Such a Scientist. In his “four organs” model, the head is your intellect, the heart is the emotion, gut is intuition and humour, and the sex organs, libido and survival instinct. Randy was talking about communication, but the model is also about decision making. What do you decide to do? What are your motivations? Scientists are unusually head-centric. We’re all about facts and data. We go through a long period of training that forces us to think that way. Consequently, we tend to underestimate the pull of the other organs on our decisions.

For all that we pretend that we are proceeding logically in generating our hypotheses and testing them, there's still a huge role for gut instinct in the life of a scientist. If that freaks you out, just think of them as ideas that are insufficiently articulated for immediate verbalization.

For example: When is a project done?

We all talk about doing that one beautiful experiment. But it's rare that you get the “smoking gun” experiment: the one with crystal clear results that definitively tells a single, unified, publishable story. More often, you have a set of interrelated experiments that all get at slightly different angles of the same problem. When do you stop running experiments and write it up for publication?

There’s no simple test to decide when you have “enough” for a paper.

I often think about my projects as having “confidence intervals.” (I use the phrase informally here, not in it’s proper mathematical definition.) When I am examining data, I’ll often notice something that seems to be a pattern. But I’m good at guessing wrong, and so in the early stages, my confidence may be only 50%: I’m imagining things, or it’s real, but it could go either way. I keep running more tests, and if I’m lucky, my confidence interval creeps up past 70%, even to 90%.

I have a few projects where I tell my students, “I’m 90% sure that this is what’s going on, but I don’t want to submit it until I’m 95 or 98% sure this is the case.” I never want to be 100% confidence of a result, because I always want to be ready to change my mind given good data. I always tell students that any scientist who says there is nothing that can change their mind on an issue should have their “Scientist” card taken away from them. (If only we gave them out!)

At what point do you write it up and try to publish? You can’t get infinite data. There is always a point of diminishing returns, where new data leads to less and less new understanding for me. I only have my own gut instinct that what I have will be persuasive to reviewers, editors, and readers.

The few times I’ve thought, “I’m only 90% sure this is the case, but getting that last 5% confidence is going to take a long time, so I’ll have a go and submit the paper anyway,” guess what? Rejected. If something doesn’t convince you, even if you can’t spell out why, it’s probably not going to convince other people.

There are lots of other examples of where you sometimes have to listen to your gut in science. I admit that the “Never” in Moscow Rule #2 does makes me nervous - “never” is is a long time. But sometimes, if something feels wrong, it’s because it is wrong.

External link

Does trusting your gut make you unscientific?

One response so far

  • Erin says:

    I think our long period of training to trust our heads and not our guts explains why most scientists (in my experience) love being in the lab, but hate writing up their results. Writing pushes us into a domain where 'gut-feeling' is about all we have to decide if a manuscript is finished or 'not quite right' in an intangible sort of way. We have to learn to trust both to effectively communicate our work.