Thursday, February 07, 2008

Falsifying Data I: Some of Our Favorite Fakes

L., a former member of Dr. S's lab, published a paper- several years ago- with un-reproduceable data. She confided to my spousal unit, when he tried to repeat the experiment, that she got the result once and ignored the other ten times.

Somehow, faked data came up in conversation. 'Well, L. falsified data,' I said.

'No she didn't!' protested Dr. S. 'She reported a part of it. But she really got the data.'

'She published a result she knew wasn't true,' I said. 'How is that not falsifying data?'

L. now has a faculty job at Snooty U. Virtue rewarded.

Our favorite kinds of fake data:

1. Someone makes up a result out of whole cloth, or Photoshop, as it were. Or republishes with a different caption.

These people often get caught. Some bright-eyed person notices that the cells look disturbingly similar, or that the graph appeared in another paper. In some ways, this is morally the worst- after all, there is no ambiguity: DO NOT MAKE UP DATA. Doubtless a fair bit does get through, especially with less high-profile papers. But it is not a subtle lie. This one bothers me less than subtle faking, somehow.

2. Someone makes up part of a result, or fills in a form that was partly blank- on purpose, knowing it's wrong- and so on.

I imagine this gets through a lot more often. It seems to happen frequently in clinical trials, possibly because their data sets are often re-analyzed by more than one group (and so it gets caught in these circumstances). There was that one breast cancer trial (whoops- apparently a lot of breast cancer trials! Money does lead to corruption!)- at Dartmouth maybe? where a lab member discovered the PI had massaged the data interpretation. The PI was fired.

Partial fakery, I find very objectionable. It seems malicious in a way that outright lying doesn't, although the intent is no different: to report a false result. Perhaps it's squidgier because it may have a lower chance of detection: after all, some of the data is real.

Which one do you think is worse, full-on or partial making-it-up? Any juicy stories to tell?

Next: Mistakes, White Lies, and Retractions