It's been suggested that I give an overview of the process of writing and reviewing the IPCC 5th Assessment Report, since I'm involved as a lead author (and was also an author on AR4).

There are 3 Working Groups, each preparing its own volume of the assessment.

WG1: The Physical Science Basis
WG2: Impacts, Adaptation and Vulnerability
WG3: Mitigation

The 3 volumes are being prepared roughly in parallel, but the schedules for WG2 and WG3 are approximately 6 months behind that for WG1. They follow the same routine for drafting and reviewing.

Each volume consists of a number of chapters (ranging from 14 for WG1 to 30 for WG2), and each chapter has approximately 10 lead authors although the exact number varies a lot.

The lists of all authors for AR5 are here:

WG1 authors
WG2 authors
WG3 authors

The drafting and reviewing process for each chapter in each volume is as follows:

Zero Order Draft
Informal peer review
First Order Draft
Formal Expert Review
Second Order Draft
Formal Expert and Government Review
Final Draft
Preparation of Summary for Policymakers (SPM) for each volume
Government Review of SPM
Revision fo SPM
Final approval of SPM

The provisional schedules for the WG1 and WG2 volumes are here:

WG1 AR5 schedule
WG2 AR5 schedule
(schedule not available for WG3 but I imagine it is similar to WG2)

The Zero Order Drafts are very informal and allow the authors to give each other a preliminary indication of the topics and material to be covered, and begin the process of discussion and debate between the authors. The Zero Order Draft of each chapter will not necessarily be endorsed by all the chapter's authors, since each author will have written their text in parallel but may not have been able to fully discuss the contributions of other authors. They are informally reviewed by a small number of peers nominated by the authors. The ZOD serves to identify gaps and overlaps in the topics of the different chapters, and also acts as a "dress rehearsal" for meeting a hard deadline for submission of a major piece of multi-authored writing (so an important exercise for the more critical First, Second and Final drafts later).

The First Order Draft is the first proper attempt at the draft of each chapter. It is put out for review by anybody who self-nominates as an expert. This is an important point - anybody can self-nominate as an expert reviewer, and will be provided with the draft of the relevant chapter(s) as long as they are prepared to abide by the conditions (eg: respecting confidentiality of the draft) [NB This is my current understanding anyway - but I am not an official representative of IPCC]. So the review process is open to anybody who wishes to participate. I imagine that details of how to self-nominate as expert reviewers will probably be posted on the respective AR5 websites of each Working Group:

WG1 AR5 website
WG2 AR5 website
WG3 AR5 website

Review comments will need to be sent to the Technical Support Unit (TSU) for the appropriate Working Group by a specific deadline. All comments will then need to be addressed by the authors.

Tips for writing effective review comments:

1. Be very specific about any changes you are suggesting. Don't just write something vague like "this is wrong" as this will probably be rejected as not suggesting a specific change. Say which sentence you would like to see changed, and suggest a new version.
2. Provide evidence to back up the suggested change. Don't just write things like "insert 'not' here" without saying why - preferably by referring to a paper in the peer-reviewed literature, or grey literature if that can be provided to the authors for their assessment.

The Second Order Draft is then prepared in the light of the review comments, and again sent out to review by self-nominated experts and also government representatives (since these are the stakeholders for the reports).

After this second round of review, the final draft of each chapter is prepared, and the SPM which must be based on material in the chapters. The SPM is reviewed by government stakeholders, and finally approved line-by-line in a plenary session with the coordinating lead authors of each chapter and government representatives.

The First and Second Order Drafts and all review comments plus the authors specific responses to them will be published online as soon as possible after publication of the report, so the evolution of the reports will be auditable.

Richard:

Thanks for that description, but it leaves me totally confused as to what will be produced and when.

The process seems rather opaque. What is the name of the final version of the report (or is that what is called the final draft - surely not, a draft is still a draft), when is it published and how soon after that is the SPM produced? The link to the schedule for WGI just adds more confusion. Is it the intent that the whole process appears like a confusing mess?

It seems like the great unwashed have no say in the SPM, which can be produced to say whatever the undemocratic bureaucrats wish it to say (despite you saying that it must be based on the material in the chapters). Is that a fair summary?

Thanks Richard

Perhaps as an insider it means you can't see what a strange way of doing things this is..

Seriously, in areas of considerable uncertainty, would not minority and majority reports be more beneficial, rather than attempting a consensus. Much discussion atClimate Etc about this approach.

When the next IPCC report is available I wonder if it will receive as muchattention from politicians and the media as previous reports ?

From the Ecclesiastical Uncle, an old retired bureaucrat in a field only remotely related to climate, with minimal qualifications and only half a mind.

Is what follows fair?

Mr Betts sets out the reality of the process that will surely result in another biased report. His words draw attention to the significant fact that, in exactly the same way as with previous versions, the report is to be submitted to governments ‘as stakeholders’, as part of the review process. This must surely result in a report that will be acceptable to them.

Many governments, certainly including ours here in the UK, already have policies that the public interest is served by costly action to ameliorate climate change and alleviate its effects. They have sought to avoid embarrassment caused by the resentment of consumers, taxpayers and business at the resulting high costs by perpetrating the idea that the policies are based on a consensus that their scientific foundations are sound.

Governments actually have alternative consensuses (? consensi) to choose from. Their favourite is that the science is settled, because, of course, all doubt is then dispelled and extreme action is in the public interest almost regardless of cost. However, it has proved difficult to hold this true and simultaneously fund further work on climate science. In situations where there is awareness of the continuing funding of climate science, the alternative consensus, that the overwhelming weight of scientific evidence supports the severity of climate change and the need for drastic action to alleviate its effects, suffices. Different choices have been made about what scientific evidence is required to provide the overwhelming support without causing governments apparent difficulty.

It has not proved difficult to sustain either or both of these consensus positions by political or financial mechanisms.

To be acceptable, therefore, the upcoming Assessment Report must support climate change policy and provide it with the required scientific basis. It is not surprising that the view that the science is settled has not been advanced in past reports because most of the authors, etc., have been employed as climate scientists in ‘civilian’ life. Various versions of the case that the overwhelming weight of science supports the expectation that climate change will be extreme and that drastic ameliorative actions are required have appeared in previous assessment reports.

The upcoming report will be no different. It will be a political document masquerading as a scientific one.

If the foregoing is free of critical errors, do I not have reason to be surprised that the scientific cognoscenti on this and other blogs wax so indignant about IPCC reports?

Thank you Richard, this is useful.
Another related web page on the IPCC AR5 process is here.

I very much doubt that the review process will be as you suggest (anybody can be a reviewer). In fact I would wager a pint on it!

Phillip, in fact the SPM is released before the main report - at least it was last time. That way, the political message and the headlines get out but the details can't be checked.

And in fact it is even more confusing - as well as the three working groups each with its SPM, there is a 'synthesis report', and that has its SPM as well!

PaulM:

Yes I appreciate that the SPM is written before the final technical reports. That way, the science can be backfitted to fit the policy. But I don't see where the final technical report is in the schedule.

Glad this was useful.

I realise it is commonly claimed that the SPM is written by the policymakers and then the technical volumes are then back-fitted, but I saw no evidence of this in AR4. In fact this can be easily checked because the drafts of all the chapters and the SPM and technical summary are all online here so can be compared with the final published version (I've only provided links to WG1 but the other WG chapters are also available)

The review comments, and the responses of the authors, are all there too.

The reason for involving government reps in the SPM is because the document is supposed to be meaningful to them, so the document is not supposed to be merely a summary of the scientific points that the authors found interesting - it is intended to explain the science in a way which is informative for policy (without being policy prescriptive) and is clearly understood. The government review comments, like the expert comments, need to be backed up by evidence.

I find it quite strange that a lot of people think the world's governments are somehow speaking with one voice in their supposed manipulation of the SPM and then the underlying chapters. In reality, the debates between govt reps at the SPM plenary are deep and protracted, and it takes ages to reach agreement on the wording - but at all times the chapter Coordinating Lead Authors need to be happy that any alterations are backed up by the original chapters. The chapters are edited a bit afterwards to make their executive summaries consistent with the SPM, but only in a word-smithing way and not altering the actual scientific conclusions - at least, I saw no evidence of this, but if anyone can find any by comparing the drafts with the final versions, I'd be interested to see.

And if the world's governments were really speaking with one voice in order to support a particular climate policy, surely the UNFCCC process would not be proceeding so slowly would it?

Yes it is extremely frustrating that the main reports come out after the SPM (which is released immediately at the end of week-long plenary session where it is approved line-by-line). As PaulM says, not only does it make it impossible for readers to check the technical detail behind the SPM statements, it also makes it impossible for the authors to point to the evidence when requested. I agree that there ought to be a better way of doing it, as it doesn't look good and allows people to think the whole thing is being fiddled!

PaulM, I checked just a few weeks ago about whether the review process is open to anyone, and was told (by the WG1 TSU, via colleagues who are WG1 authors) that yes this is the case (like in AR4) as long as the reviewers are prepared to put their expertise on record and agree to the terms of being a reviewer (like respecting confidentiality of the draft). However, again I'd be interested to hear if anyone does find themselves being turned down as a reviewer despite agreeing to the terms - so we can see who owes who a pint :-)

Thanks for reminding me about the Synthesis Report, I had forgotten to mention that. Preparing that is a bit of a nightmare as it is supposed to bring together conclusions from the 3 working groups, but sometimes they can be inconsistent especially in the case of future projections, because they've been written in parallel not in series. In AR4 this was a particular problem because WG1 were assessing projections from the new set of climate models but WG2 were assessing impacts studies that had been carried out using projections from the older set of models, because there had not been sufficient time for the new models to be used for impacts studies and the papers to filter through the literature. Hopefully it may not be quite so bad in AR5 since the different WG volumes are being prepared to more staggered schedules.

Having said that, the AR4 Synthesis Report writing team (which I was not on) still did a great job in bringing it all together as well as they possible could.

Barry, your suggestion to have minority and majority reports is quite interesting, and John Christy suggested something along these lines at an IPCC pre-AR5 meeting a couple of years ago. (It didn't go down very well I must admit!)

So yes the whole process is a long way from being perfect, there's lots of things I'd like to see done differently, but like anything else to do with the UN there is lots of complexity and compromise. But overall I remain sufficiently happy with it to want to be a part of it. In particular, as a scientist, I find the interaction with the other authors (leading experts from across the world) extremely exciting and rewarding. Having moved from WG1 last time to WG2 this time, AR5 presents new challenges for me as there is a much more diverse range of disciplines involved.

From the Ecclesiastical Uncle, an old retired bureaucrat in a field only remotely related to climate, with minimal qualifications and only half a mind.

Mr Betts - thank you.

(1) While I am not mentioned by name I deduce that your remarks about governments were provoked by my earlier contribution. However, I am aware, of course, that there in no unanimity among governments about climate change and tried, apparently unsuccessfully, to word my contribution accordingly. However, I venture that, in this context, governments fall into two camps: those with a policy that do care, and those without a policy that don’t. Very few marks indeed for spotting that the policies that the governments with policies have embrace the global warming idea with various degrees of catastrophe baggage attached. So do not your intergovernmental debates boil down to arguments about scientific detail and the catastrophic baggage within the governments that have policies, while those without a policy, or therefore much interest, watch them slug it out? So, do not the minutes more or less reflect the views of governments with alarmist policies?

(2) I note your explanation of the purposes of the governmental review but wonder if it works that way within the governments that have policies. Notwithstanding some skepticism, I admire the sure touch of your explanation and would like to think that I demonstrated such abilities in my working days. However, a story from my past: I recall an instance when a bureaucratic acquaintance suffered a severe public accusation and gave an assured rebuttal in much the same way as you have given your explanation. This closed the case to his satisfaction. The accuser, however, quite justifiably in my view, pointed out that the accusations were not answered. Both sides were left feeling victorious! And I fear it may be the same with what actually happens in and in connection with reviews by governments with policies. As well as performing the duties you describe, do not some of the individuals involved, maybe the more ambitious, also attempt career leg-ups by distorting review processes to support policy? And, things being what they are, is this not routine? So, in practice, do not government reviews actually come to require that their policies be endorsed?

(3) My impression is that previous Assessment Reports have scandalized a considerable body of opinion. Is this not inevitable for the upcoming report? As I think is implied by your descriptions, IPCC processes are tortured and labyrinthine. Personally, I do not doubt they were developed in this way so that the IPCC could demonstrate that it had taken steps to preclude bias. However, I observe that those who have been scandalized claim that these efforts have failed and, to some extent, that the processes themselves have presented opportunities to introduce bias. The problem, of course, is the people appointed to author teams. Governments have some sort of influence, not necessarily direct, on their selection, I think. Again, the influence of governments with alarmist policies is likely to dominate. So the people within the author teams actually get to come from organizations that work on or in sympathy with alarmist policies. They approach their tasks in the author team etc on the basis of their day-to-day work and the assumptions they have made about personal responsibility and humankind’s role vis a vis the planet necessary to sustain that employment. They are biased and have influenced past reports accordingly. I do not think that such changes that have been made to IPCC procedures are anything more than cosmetic. They have worked perfectly satisfactorily for the governments with policies and they will not allow changes that endanger continuation. It therefore puzzles me why you were put to the trouble of formulating the description of IPCC processes. The thrust of the next report is precictable and not greatly dependent upon the detail you provide.

Enough!

My observations and no doubt jaundiced views are informed by perusals of this and other blogs, detailed reading of the HSI, and a description of the formation of the IPCC I found at www.spiked-online.com/index.phph/site/3540/. This is probably old hat to the cognoscenti here. However, for the new boys, it is 12 pages long and covers the formation of the IPCC in some detail, thereby repeating much of the first chapter of the Bishop’s book. Its author is one Tony Gillard, a top man of an ex-Marxist organization, no less, so it must be true!

Ecclesiastical Uncle

Thanks for your further thoughts. Yes, my comments were partly in response to you, and also Philip Bratby - sorry to not make that clear!

In response to your itemised points:

1) No I don't think that is the case. In one of the SPM plenary sessions, the representatives of one particular country (which definitely does not sanction an "alarmist" view, and indeed has a critical dependence of their economy on fossil fuel production) fought very very hard indeed to get a draft statement on climate change attribution downgraded or even removed. The discussion went on for a very long time and was quite forceful. Many different alternative wordings were tried until eventually a form of words was found that all parties found acceptable. So no, I don't think one side or the other dominates.

2) Again, no, since policies vary a lot.

3) This is a reasonable point, and more subtle than my initial reading of yours and Philip's comments, which I took to imply actual corruption (although apologies if that was a misinterpretation). You seem to be saying that it's more about a risk of an inherent internal bias in the attitudes of the authors themselves. I have to admit that this is a possibility (in some cases, but certainly not all). The InterAcademies Council review found that there was a tendency for the WG2 AR4 report to focus more on the negative than the positive, and for uncertainties to not always be recognised as much as they should.

I really don't think that governments put forward authors specifically because of their political views (I'm certain the UK doesn't, at least), but since the author selection process starts off with self-nomination, then there is of course the possibility that some people whose interest goes beyond mere academia put themselves forward because they want to "make a difference" - possibly more so in the more applied Working Groups (2 and 3) but not exclusively. HOWEVER they will need to have a strong scientific record to actually get nominated by their governments and also to be selected by IPCC (by which I mean the panel itself - the govt reps). And moreover, again because of the differing positions of the various governments, and because of the genuine recognition that the report needs to be scientifically robust (yes, they really do value that!) then it ought to be the case (in theory anyway!) that a reasonable balance is achieved.

There are a number of authors this time with a well-known "no-nonsense" reputation (eg: Hans Von Storch) and also I know of several who have stepped forward with the specific intention of trying to avoid the embarrassments of AR4 like the Himalayan Glaciers episode (and the wider criticisms of the IAC).

So in summary, yes there is of course something of a risk of internal bias, but my experience so far in AR5 leads me to be optimistic about there being enough checks and balances to allow the process to be credible.

Of course, whether I am right in this remains to be seen!

Incidentally, one of my motivations for starting this discussion was to encourage expert reviews from a wider cross-section of people than in AR4, specifically to act as one of the further checks and balances. You can help make sure we don't introduce personal bias, intentionally or not, by challenging what we write in the drafts!

@Richard Betts

Richard, like others here I appreciate your willingness to engage and your sharing with us your insider view of the IPCC process. That being said, you wrote:

Incidentally, one of my motivations for starting this discussion was to encourage expert reviews from a wider cross-section of people than in AR4, specifically to act as one of the further checks and balances. You can help make sure we don't introduce personal bias, intentionally or not, by challenging what we write in the drafts.

While one might hope it was more the exception than the rule, the responses to the reviews i.e. "challenges" by "outsiders" in AR4 that I've looked at were considerably less than exemplary (or encouraging!). For example, pls see

The climate change game ... Monopoly: the IPCC version

I don't know if you're aware of AccessIPCC. This is a computerized tagging of all 44 chapters of AR4. This is still a work in progress, btw; but we also attempted to quantify the "chapter team" responses to the reviews of the Second Order Draft. Because of the way in which the document containing this data was structured, the best that we could do was to quantify those responses which could be unambiguously identified as "Accepted" ... and it's not a pretty picture!

Earlier in this thread, you mentioned the harvard.lib collections. The experience of others may be different, but I have found that the choice of formatting there is considerably less than optimal if one is attempting to trace the "history" of a particular paragraph of text - or even to quantify and/or compare the reviews (and responses thereto) of a particular paragraph. From where I'm sitting, I don't think they could have chosen a more cumbersome technology if they tried :-)

hro001

Thanks for the feedback. Yes, I'd seen AccessIPCC - interesting to see some of those numbers. Are you going to do it for AR5 too?

May I make a couple of suggestions? It would be useful to present the numbers as a percentage of the totals, as well as the absolute numbers. So for example, the "Self Reference Concern" figure would be more useful (IMHO) as a percentage of the total number of references / citations, to give a better indication of how much weight these carried in relation to other papers. Same goes for "Non Peer Reviewed".

Also, for your "2007" tag, I may be wrong, but I think you are looking at the WG1 timing and protocol and assuming it applies to WG2 and WG3 as well - these ran to different schedules and had different protocols, so I think you should check you are showing relevant info there (the other WGs would have had different literature cut-off dates).

Joint a point about the "accepted" tag. If I remember correctly, a fair few comments were not tagged "accepted" simply because they were not well-posed in the way I described in my post above. Many of these would have been categorised as "noted" because it was not clear that the reviewer was suggesting a specific change - or because they were providing information without suggesting a specific change, such as recommending a paper that the authors should read (many of these were of course self-citations by reviewers, but that's fine because it's one way of bringing papers to authors' attention).

Also just because a comment is not "accepted" it doesn't mean it wasn't worth making - it's still a way of making sure the authors can demonstrate that the text is sound.

Richard, thanks for your response.

Are you going to do it for AR5 too?

After viewing (one of several unadvertised) new additions I've recently stumbled across on the IPCC site, all I can say is, "it depends"!

It would be useful to present the numbers as a percentage of the totals, as well as the absolute numbers.[...]

We actually do show these as a separate table in the "big picture" for each WG e.g. http://accessipcc.com/AR4-WG1-Index.html.

Peter and I had a "great debate" about including this percentage table at the chapter level; however, (rightly or wrongly!) I felt it would be overkill - and considering requests we'd had to move all summary tables off the chapter pages, adding another one did not seem to me to be a good idea. And I won :-)

Also, for your "2007" tag, I may be wrong, but I think you are looking at the WG1 timing and protocol and assuming it applies to WG2 and WG3 as well

My recollection is that AR 4 was released (with much fanfare from some quarters) in Feb. 2007. This makes the inclusion of 2007 publications somewhat suspect (or at least worth examining). But I did check out the schedules prior to highlighting this "feature" in a post I had made in March 2010:

IPCC’s 4th Assessment Report: 354 leaps back to the future

Joint a point about the "accepted" tag. If I remember correctly, a fair few comments were not tagged "accepted" simply because they were not well-posed in the way I described in my post above. [...]Many of these would have been categorised as "noted" because it was not clear that the reviewer was suggesting a specific change - or because they were providing information without suggesting a specific change

[As a digression I won't even ask what constitutes "a fair few"! But ( full disclosure) I'm a pre-post-modernist Englsih major, so this phrasing (framing?!) greatly offends my sensibilities!] And now to your point ...

That may well be the case in a number of instances; however, as I demonstrated in the "Monopoly" post to which I'd referred in my observations, it certainly was not the case wrt the specific paragraph of text I had examined.

many of these were of course self-citations by reviewers, but that's fine because it's one way of bringing papers to authors' attention

Quite so. But why should self-citations by reviewers carry less weight (or be reason for exclusion/rejection as seemed to be the case in the Monopoly post I mentioned) than self-citations by CLAs, LAs, or CAs?

And I won't even mention citations of publications by "Review Editors" in chapters under their bailiwick. Oooops, sorry, I just did!

Also just because a comment is not "accepted" it doesn't mean it wasn't worth making - it's still a way of making sure the authors can demonstrate that the text is sound.

I don't disagree, Richard. However - and please correct me if I'm wrong - there was no indication in the documentation pertaining to AR4 that anyone (let alone the Review Editors whose responsibility it was) had verified that the authors had "demonstrated that any [contested] text [was] sound".

But, as I concluded during the course of my recent dialogue [well, at this point it's only a monologue] with another CLA ...

A Helpful Hint from Hilary™ that you might pass on to the powers that be for AR5: Add another column to the Word doc sent to the “chapter teams” which contains a dropdown list from which the responder (“on behalf of the chapter team”) will select the category of response (Accepted, Rejected etc). This will make life easier for the Review Editors to carry out their responsibilities, lessen the appearance of being a “knowledge monopoly” – and make the process slightly more transparent when the review comments are (finally!) made public!

From the Ecclesiastical Uncle, an old retired bureaucrat in a field only remotely related to climate, with minimal qualifications and only half a mind.

Again, Mr Betts, thank you very much for addressing the paranoid skepticism I probably should do more to control.

Before the serious stuff, the timing of your posts suggests that you are either on hols or that you do your blogging in your employer’s time. Which prompts me to ask if the Hadley centre (if I have your employer correct) has a public relations function and if so, are you specifically tasked/encouraged to help with it?

Now, using my previous number system,

(1) The inter-governmental debates. (Not really serious this.) Interesting. Of course, as a result of your revelations, somewhat idly I speculate about the identity of the countries involved and observe that it’s nice that there is a country on the side of the angels.

Don’t enlarge on this unless you feel a pressing need to do so.

(2) I am not sure I can agree that there is a multiplicity of policies. That part of government that leads public opinion and manages public affairs has adopted the policy (the ‘fundemental’ policy) that climate change will be calamitous and that the behaviour of the public can be altered to ameliorate upcoming difficulties, albeit at some sacrifice of immediate well-being. This is singular. A multiplicity of policies can exist, however, about subsidiary areas: the extent of the seriousness of the problem (not, however, extending to questioning whether the word should be used at all) , the ameliorative measures and the scientific basis which the government would like there to be to underpin the policy.. It seems likely that ministers watch the arguments that rage over these subsidiary matters without much interest or perhaps mild approval, irrespective of results, because they rest on the assumption that the fundamental policy is right.

The policy I referred to is the fundamental policy of the government. I guess the host of policies you refer to concern the subsidiary areas (probably the extent of seriousness and the scientific basis).

My expectation is that the sheer volume of unrelated independent research under way forces those involved in the review process to make selections. There could be arguments, which would be interminable, about the merits of this or that selection, but deadlines have to be met, and, in the end the reviewer makes his selection and writes his review.

My suggestion is that the reviewer cherrypicks, (he has to) with perfect honesty, the selection that more closely supports his employer’s objectives. Mostly, the ultimate employer is the government. Accordingly the reviews become biased towards the government’s fundamental policy.

I do not suggest, in any way, that there is corruption or that unsound science is knowingly misrepresented as sound science.

(3) The same arguments percolate into this section.

I was surprised to see your remark that the UK does not select authors and the like on the basis of politics. Good heavens, no! Yes, of course, the science should be sound but has this always been achieved?

But I think your confidence that IPCC’s author selection process, which you advise is implemented by government representatives, leads to a balanced selection is misplaced. The process is a zero sum game. For every country that succeeds in getting its man into such-and-such a slot on the team, there is at least one country that fails. So the team will inevitably reflect the views of the countries that put the most energy into their climate change policies.

And I note that this seems to be entirely consistent with the purposes for which governments actually (not ostensibly) established the IPCC in the first place – to provide them with tools to help them pursue the already settled fundamental policy (see the previous section).

Thank you again.

Ecclesiastical Uncle

Thanks for your response. I don't think I have anything further to add, other than to answer your question about the basis for my engagement here. Many (I think probably most) of my posts are actually in my own time (see my original post on this discussion, and my most recent one above) but some are indeed in work time. The Met Office Hadley Centre does have a role in communicating its science (NB this does not mean promoting govt policy!!) and I am involved in that from time to time, eg: talking to the media about my work, or that of colleagues, when appropriate. Occasional blogging along these lines is approved, but the initiative and interest came from myself (simply as part of trying to make sure that what I do is understood beyond the confines of the scientific community). However of course I can only devote a very limited time to this, as it's not the main part of my job, which is why my presence here is somewhat sporadic and sometimes rushed!

Incidentally, I'll probably go quiet over the next week or so now, due to (a) a major writing deadline and (b) family time.

hro001

(One last comment before I go offline for a while.)

Thanks for the follow-up and for pointing me to the percentage tables, that's useful.

Enjoyed your comment about my use of language, especially your (deliberate?) mis-spelling of "Englsih" :-)

On the self-citations by reviewers, I didn't mean to imply these carry any less weight than those of authors, or be a reason for rejection in themselves - I just meant that they may just be "noted" rather than "accepted" because they are a point of information rather than a specific request for change.

Citations of papers by Review Editors, like those of authors, isn't entirely surprising as everyone involved is there precisely because they are an expert in the field - yes it is crucial that this doesn't lead to bias in the assessment, but equally it would be worrying if the papers of those involved in the IPCC process were not significant enough to be included in the assessment.

But hopefully you may be (slightly?) reassured to know that it has been specifically pointed out on a number of occasions (including the invitation to self-nominate as an author) that our role is not to promote our own work. But by all means check for this in AR5 just as you have in AR4.

Finally, I think the documentation process in general will be further improved in AR5 over AR4, so hopefully again that will help to allay some more of your concerns.

Thanks for the discussion!

Richard Betts,

I was going to have a look the AR5 process, for interest, but my browser is advising me not to. Is this just a Firefox thing or do you have an incorrect link:

This Connection is Untrusted

You have asked Firefox to connect
securely to www.ipcc-wg1.unibe.ch, but we can't confirm that your connection is secure.
Normally, when you try to connect securely,
sites will present trusted identification to prove that you are
going to the right place. However, this site's identity can't be verified.

Lord Beaverbrook

Thanks for taking a look. That's a rather unfortunate and ironic error message, especially to someone coming from this blog's audience!

That particular link to the WG1 homepage which is hosted by the University of Bern (where the Technical Support Unit is based, along with one of the WG1 co-chairs Thomas Stocker).

The IPCC homepage itself is here, you could go into that first and then go to "Working Groups / Task Force" on the left hand side, and try the other Working Group pages from there - eg: WG2 is at Carnegie in the US. That may at least let you see what that WG has to say.

Here are the journal deadlines that apply for papers appearing in AR5 WG1 (HT Tamsin)

Submission to journal: 31st Jul 2012
Accepted by journal: 15th Mar 2013

[These are the deadlines that were broken in order to get certain papers into AR4]

The invitation to self-nominate for the Expert Review of the First Order Draft of the Working Group 1 contribution to the IPCC Fifth Assessment Report is here

Reviewers are asked to self-declare on their expertise. They will also be asked to agree to the terms of the review (eg: only use it for the purpose of the review, and not cite, quote or distribute).

As in my original post on this discussion, here are a couple of tips for making effective review comments:

1. Be very specific about any changes you are suggesting. Don't just write something vague like "this is wrong" as this will probably be rejected as not suggesting a specific change. Say which sentence you would like to see changed, and suggest a new version.

2. Provide evidence to back up the suggested change. Don't just write things like "insert 'not' here" without saying why - preferably by referring to a paper in the peer-reviewed literature, or grey literature if that can be provided to the authors for their assessment.

The Expert Review processes for the WG2 and WG3 FODs will start in a few months' time (these WGs run to a different schedule).

Aug 12, 2011 at 6:22 AM Ecclesiastical Uncle

I see this thread has sprung back to life yesterday, so I popped in for a look and I see that nobody reacted to the Ecclesiastical Uncle's last contribution (Aug 12, 2011 at 6:22 AM).

But I think your (RB) confidence that IPCC’s author selection process, which you advise is implemented by government representatives, leads to a balanced selection is misplaced.

I agree with this thought, in fact I tried to generate some interest in the matter at the end of July 2011 in "Unthreaded".

I had time some on my hands and I discovered that there are 3 Coordinating Lead Authors, 14 Lead Authors and 4 Review Editors (total 21) working in AR5 WG1 who had signed the Met Office 2009 Statement:

We, members of the UK science community, have the utmost confidence in the observational evidence for global warming and the scientific basis for concluding that it is due primarily to human activities.

..... we uphold the findings of the IPCC Fourth Assessment Report, which concludes that 'Warming of the climate system is unequivocal' and that 'Most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations'.

Now, is it fair to think that individuals who would sign such a document have a "strongly held view" that AGW is real?

If it is "fair", then surely these individuals have a conflict of interest, in that their jobs depend on the continuance of funding from governments who then require those individuals, through the auspices of the IPCC:

"to provide them with tools to help them pursue the already settled fundamental policy"

The InterAcademy Council (IAC) has already considered this "strongly held view/conflict of interest" problem, see page 3 of:

http://tinyurl.com/66xzpzl

In July, I asked "is it OK for this state of affairs to exist" and I can only find two responses to the question - one "yes" and one "no".

In my opinion it is not OK.

Hi Brownedoff

Actually I did react to the Uncle's contribution - see post immediately following it.

But anyway, I don't see why having confidence in a particular aspect of the science (whether warming is occurring and whether it is mostly due to increased GHG concentrations) constitutes a conflict of interest. The IAC was referring to vested interests such as political or commercial stances.

You'll notice that the Met Office statement said nothing about the severity of future impacts. They are much more uncertain. Also the statement said nothing about policy responses - it didn't say "emissions need to be cut immediately" or anything like that.

Sep 21, 2011 at 11:54 AM Richard Betts

Actually I did react to the Uncle's contribution - see post immediately following it.

No, you did not react to the bit I am talking about, you just dealt with Uncle's concern that you might be using up valuable holiday time.

But anyway, I don't see why having confidence in a particular aspect of the science (whether warming is occurring and whether it is mostly due to increased GHG concentrations) constitutes a conflict of interest.

By now, we all know that you see nothing wrong or even suspicious in the fact that a small coterie of individuals, who have a vested interest in the continuation of AGW, are able to continue, for ever it appears, to ensure that uncertainties are suppressed and that the "science is settled" assertion is refreshed every few years.

You'll notice that the Met Office statement said nothing about the severity of future impacts. They are much more uncertain. Also the statement said nothing about policy responses - it didn't say "emissions need to be cut immediately" or anything like that.

For the purpose the statement was intended (i.e. deflect attention from the e-mails) it did not have to do anything other than scream "look, here are 1,700 or so highly qualified individuals at various seats of learning in the UK who have a strongly held view that AGW is real, so it must be true, therefore please ignore the disastrous e-mails which tend show that the so-called "evidence" for AGW is a bit iffy".

Let us return to the original point, that is, Uncle's message:

"But I think your (i.e. RB) confidence that IPCC’s author selection process, which you advise is implemented by government representatives, leads to a balanced selection is misplaced."

I agree with that view, do any other readers also agree?

@Brownedoff

Let us return to the original point, that is, Uncle's message:

"But I think your (i.e. RB) confidence that IPCC’s author selection process, which you advise is implemented by government representatives, leads to a balanced selection is misplaced."

I agree with that view, do any other readers also agree?

Based on all the evidence I've seen, "government representatives" may give their blessings to the author selections (as they do to much of the work of the IPCC), but the actual selection process seems to be concerned more with a need to ensure gender and geographical "balance".

If you take a look at the list of authors and review editors for AR5, all we are told about each person is a name, an institution (with which the person is presumably affiliated) and a country.

There is no indication of the person's expertise, or area(s) of interest. How is one to know whether an individual is a bona fide scientist - or a janitor who happens to be a Greenpeace activist?

One IPCC insider who responded to the IAC's questionnaire indicated (in response to a question on the Selection of lead authors):

A list of all potential lead authors (with CVs) is sent to the secretariat for selection by National IPCC Focal Points. The secretariat then selects candidates with the requisite qualifications.

Interestingly, while a number of other IPCC insiders responded to the same question with replies along the lines of:

"I'm not clear how this actually happens"

another such insider wrote:

IPCC works hard for geographic diversity. This is one valuable criterion, but it is not sufficient to choose a lead author. The result is that some of the lead authors (generally although not always from developing countries) are clearly not qualified to be lead authors and are unable to contribute in a meaningful way to the writing of the chapter. [...]

In short, Brownedoff, I'm inclined to agree with you and Uncle that RB's confidence may well be misplaced.

Sep 22, 2011 at 6:03 AM | hro001

Please, for as long as you are able, continue with the mammoth task of probing the inner workings of the IPCC and then letting us know what is actually going on, rather than the stuff that the Chairman would have us believe.

hro001

Thanks for that - yes, that is indeed a slightly thorny issue! The relevant annex to IPCC's procedures says this:

Coordinating Lead Authors and Lead Authors are selected by the relevant Working Group/Task Force Bureau, under general guidance and review provided by the Session of the Working Group or, in case of reports prepared by the Task Force on National Greenhouse Gas Inventories, the Panel, from those experts cited in the lists provided by governments and participating organisations, and other experts as appropriate, known through their publications and works. The composition of the group of Coordinating Lead Authors and Lead Authors for a section or chapter of a Report shall reflect the need to aim for a range of views, expertise and geographical representation (ensuring appropriate representation of experts from developing and developed countries and countries with economies in transition). There should be at least one and normally two or more from developing countries. The Coordinating Lead Authors and Lead Authors selected by the Working Group/Task Force Bureau may enlist other experts as Contributing Authors to assist with the work.

Since it's a document that has international influence then it needs to be as internationally representative as possible. It makes sense to me to have a wide range of geographical representation as long as the people are also good scientists - if I write stuff about the Amazon (which I do) then, even though I've been working on the subject for years and been there several times, I'd still much rather be working alongside someone with more direct, ongoing personal experience there.

Of course this does create the potential for difficulty in author selection if all the top experts are from a few countries, and the IPCC have to make a judgement on where the right balance lies. Maybe they haven't got this right in the past, I don't really know. (NB. It definitely should not be assumed that developing countries do not have top scientists - far from it! There are excellent scientists from many countries in all regions of the world.)

However, my personal experience from WG2 this time suggests that the balance is, this time, more towards getting the right expertise rather than diversity for the sake of it. In the case of my chapter (WG2 ch 4) we (the lead author team) did actually note that we are not a very diverse group - although every inhabited continent except Asia is represented in the chapter, we are still all middle-aged men of European descent, including 3 Americans, and all except one having English as a first language. Clearly, political correctness did not influence the author selection for my chapter!

Also, I know somebody who is a national of a country not renowned for being a "big player" in IPCC who put themselves forward to be an author even though (whilst they are good) they do not have the very long publication record normally required. Their nomination was supported by their government, but they were not selected by IPCC to be an author.

So, while you are right that factors other than the pure scientific expertise of the authors do come into play, I think the other factors are secondary - the primary selection criteria are on expertise (at least this is my personal impression for AR5 anyway).

BTW Hilary I have a question for you regarding the "self-citation alert" metric of yours. If I were to cite a paper which challenged a major conclusion of AR4 (eg: this one which argues that the "30% of species at risk of extinction" statement is unreliable but I was an author on the paper myself (as is indeed the case with that paper), in your eyes is that "good" or "bad"? ie: is it "good" because I've been an objective scientist and spoken up when I think something cited in AR4 was not well-founded, or "bad" because I've cited my own paper? Do I still get a black mark even for doing the right thing?