Once again this week the process of peer review, how it is viewed and possibly abused has been under scrutiny. Nic Lewis and Matt Ridley say in their Spectator article story of the Steig/O’Donnell affair (the full article is now available free on line here):
“Nature’s original peer-review process had let through an obviously flawed paper, and no professional climate scientist then disputed it – perhaps because of fear that doing so might harm their careers.”
Let me take you back to the Wegman Report. Here is the first paragraph of Section 5:
“One of the interesting questions associated with the ‘hockey stick controversy’ are the relationships among the authors and consequently how confident one can be in the peer review process. In particular, if there is a tight relationship among the authors and there are not a large number of individuals engaged in a particular topic area, then one may suspect that the peer review process does not fully vet papers before they are published. Indeed, a common practice among associate editors for scholarly journals is to look in the list of references for a submitted paper to see who else is writing in a given area and thus who might legitimately be called on to provide knowledgeable peer review. Of course, if a given discipline area is small and the authors in the area are tightly coupled, then this process is likely to turn up very sympathetic referees. These referees may have coauthored other papers with a given author. They may believe they know that author’s other writings well enough that errors can continue to propagate and indeed be reinforced.” (Bold mine)
In a conversation with Kevin the last week he urged me to read the Wegman Report (which I’ve since done); it was published while I lived happily fearing global warming and believing there was nothing but ‘a consensus’.
At the time it was published I read it back to front and thought it’s social-network analysis was quite irrelevant to the issue at hand, namely Mann’s inappropriate (I’m being kind using that word rather than another) use of de-centred PCA. In the fullness of time given Climategate and other incidents like this recent one, his analysis has been fully vindicated and has been shown to be very relevant.
It seems it is still very relevant. Of course the problem with O’Donnell et al. (2010) was rather the opposite and it is credit to their persistence that the S09 errors are now corrected.
I find it hard to read some RealClimate posts on the odd occasion I stray over there. It is as if they forget that tone betrays much in writing. When I read Steig’s posts on O’Donnell’s paper the feelings that his words suggest to me are: negativity, defensiveness, self-protection, passive aggression. Where he does praise or show magnanimity, it reads to me like a forced attempt at balance, lest he be thought less of for being so totally negative. But as I say, that is just the impression I get from reading his words.
Anonymous peer review assumes that reviewers will act only in the best interests of the paper, the journal and the field itself, however scientists are only human and can be as corrupt and self-serving as in any profession. If the ‘closed shop’ described in the climategate emails (with ‘Team’ scientists manipulating who and what should be published) is indeed in operation, this does the field and the science itself a real injustice.
“A man should look for what is, and not for what he thinks should be.” (Albert Einstein)