Thoughts on economics and liberty

Now John Quiggin says that Donna Laframboise is lying. I’ll ask her about it.

Either John Quiggin has particular insights into IPCC's data that I don't, or he knows something about Donna Laframboise that I don't. Either way, this matter needs further consideration.

I tend to not question the INTEGRITY of anyone till I've got substantial proof. But in this case the events have been thus.

I dealt with John Quiggin's issues re: IPCC in two blog posts. One of these related to the question of the extent to which IPCC uses peer-reviewed literature. I cited the work of Donna Laframboise, whose analysis showed that 30.1 per cent of the citations in IPCC's fourth report were not peer reviewed.

Here's what John first wrote:

"Pick a chapter of the IPCC Reports at random. Look at the first 30 references – that's enough for a reliable sample – and report the proportiion that aren't in peer-reviewed journals or conference proceedings (some may have been reviewed in other ways). I picked WGI, Chapter 4, which happened to be on snow, ice and frozen ground (as I'm sure you're aware, there was a mistake in the WGII report on glaciers).    I found 25 peer-reviewed journal articles, 2 proceedings, 2 book chapters and 1 article from the encyclopedia of hydrological sciences. Conclusion: 90 per cent of work cited is peer-reviewed, and most of the rest is from authoritative secondary sources. I invite others to replicate (or not) my results."

He was impatient that I wasn't spending the five minutes to reproduce this. I was not going to reproduce this, since I BELIEVED John (I don't question anyone's integrity without proof), but had something else to say.

This is what he wrote:

It will only take you 5 minutes to replicate (or not) my work here. To make ti even easier, here’s the link to the IPCC (AR4, WG1) report

So, how about a response?

I then wrote a more detailed piece, pointing out a comprehensive analysis that had examined all 18000+ citations. In particular, I noted that Donna Laframboise has apparently:

described a collaborative project involving a worldwide team of helpers who checked all the cited references in the 44 chapters of the 2007 report, counting how many were peer-reviewed and how many came from the “gray” literature. Her suspicions were aroused by reports from IPCC expert reviewers (not insiders to the writing) that some items were being submitted which did not have scientific status. These even included some press releases, however their concerns were dismissed and the reports were listed as input to the final report (p46).

The final score for 18,531 references in the 2007 report was 5,587 (one third) not peer reviewed. In 21 of the 44 chapters the score for peer reviewed references did not reach 60%.

Indeed, Rajendra Pachauri himself confirmed at least PART of this data when he wrote:

"AR4 cited approximately 18,000 peer-reviewed publications. It also included a limited amount of gray (or non-peer-reviewed) literature in cases where peer-reviewed literature was unavailable." [Source]

But now John Quiggin is suggesting that Donna Laframboise is a liar. In his comment a few minutes ago on this blog he has written thus:

I assume by now you realise that Laframboise is lying.

The dilemma before me now: whom do I believe?

I'm happy to believe that 90 per cent of IPCC's citations are peer reviewed provided John can prove to me that Chapter 4 is sufficiently representative. I'm a trained statistician and econometrist (only a doctorate, though, with a person of the stature of Cheng Hsiao on my dissertation committee; I didn't get time to publish any work in peer reviewed journals), and like to see the assumptions underpinning such a claim, offset against which is a comprehensive census.

So here I have one well qualified person who has widely published in top peer reviewed journals. His authority is close to 100 (out of 100) in terms of recognised competence. This person checks 30 (THIRTY) of the 18000+ citations in IPCC's report and passes a comprehensive judgement that the figure of 90 per cent that he arrives at from analysis of these THIRTY citations, is the ONLY truth.

He not only denies that the comprehensive analysis by Donna Laframboise, a mere feminist with no credentials and no peer-reviewed publications, is correct, but AFFIRMS CONCLUSIVELY that Donna Laframboise is a liar.

On the one hand I have one of Australia's most cited and well-known economists. On the other hand I have a non-credentialed feminist.

Whom do I believe?

As a professional economist I am clearly inclined to prefer the comprehensive census. Even though Donna is not trained in statistics and econometrics (which John is), she has surely got the capacity to count.

But I'm happy for John to explain why the comprehensive census is not appropriate in this case.

In the meanwhile, I'll attempt to get in touch with Donna Laframboise and ask her if she is a liar. She should know.


In addition to comments below, please read my next blog post:John (Quiggin), Donna’s methodology is totally transparent. Please PROVE she is a liar.

(Donna, if you do read this, please respond on the next post (i.e. the one cited in the addendum).


The complete discussion

Sanjeev Sabhlok

View more posts from this author
8 thoughts on “Now John Quiggin says that Donna Laframboise is lying. I’ll ask her about it.
  1. Tim Curtin

    Actually Donna did not herself do the evaluation of the peer review status of the citations in ALL 44 chapters of AR4, she obtained independent reviewers, three per chapter, and took the highest count for pr from each to get her total. So for John to accuse her of lying sounds a bit risky to me!

  2. Sanjeev Sabhlok

    Thanks for this, Tim. I’ve tried to contact Donna, and she can further clarify if she gets my message. Prima facie, the methodology you’ve cited (that she has used) is extremely conservative, and sensible.


  3. Sanjeev Sabhlok

    I’ve received an update on this from someone who knows more:

    “The point about her methodology is that she recruited about 35 volunteers who sifted through the IPCC data to examine the literature and the cvs of the writers. There has been no refutation of her work.”

    So far, it is John Quiggin who is looking compromised, not Donna. I gather Donna is currently travelling and unable to attend to her emails, but look forward to her comment/s (if any) once she gets time to check my messages.

    Once I’ll firm up a view on this, I’ll let people know who’s right and who’s wrong. The evidence to date is going against John Quiggin, but let me not be in haste.


  4. James Sandry

    Dear Sanjeeve
    As you yourself said “Science is about precision, and proof”. You expect John Quiggin to prove that Chapter 4 (which had a 90% peer reviewed rate) was representative of the whole document. Meanwhile, you are happy to take Donna Laframboise at her word when she claims that only 70% were peer reviewed.

    John Quiggan has invited you to replicate his results by checking the citations yourself. He didn’t suggest that you check the same chapter he did but to sample a few chapters at random. This is something that you could easily do.

    Instead you assume that Donna Laframboise is telling the truth because her analysis is “comprehensive’ and because she can count. Both of those two facts are true but neither of these facts PROVE that the figures she has presented are true or accurate. If I were to make up figures, I could make them as comprehensive as I wanted.

    You can PROVE her figures are correct (or not) by checking the citations yourself. I respectfully suggest that you do so.

  5. Sanjeev Sabhlok

    Dear James

    I trust you’ll read recent comments. Donna’s methodology is extremely rigorous. I have no basis to question her findings.

    I need strong substantiated evidence from John that Donna is a liar. So far, that’s not been forthcoming. I hope John will shortly provide a persuasive argument to so attack someone’s character and person. Else, I trust John is aware he is committing the sin of libel and exposes himself to serious ridicule and possibly worse outcomes.


  6. James Sandry

    Dear Sanjeev

    I think you misunderstood my point. How do you know that Donna’s methodology is extremely rigorous? Because she or someone else told you that it was?. If Donna’s honesty is the issue then the things that she says about her research aren’t proof either way. She will give you the same answer whether she is lying or telling the truth.

    You feel you have no basis to question Donna’s findings, John Quiggan feels that he does have a basis to question them because what Donna is saying doesn’t match what John has discovered through independant observation.

    If one person says something and then independant observation shows that it isn’t true that is usualy seen as either evidence of dishonesty or of poor methodology.

    But my point isn’t to accuse Donna of dishonesty. My point is that YOU should independantly check the facts. That YOU are able to check the citations (as John Quiggan invited you to do). If you want to defend Donna, the best way to do that is to check the citations and publish your results on this blog.


Leave a Reply

Your email address will not be published. Required fields are marked *