Saturday
Jan312015
by 
Josh
Josh It was the best of times - Josh 312
Jan 31, 2015 Josh Temperatures 
There has been much discussion recently about the adjustments made to past temperatures: see Paul Homewood's excellent posts on Paraguay, Bolivia and around the world; also from Shub; Brandon at WUWT and on his own blog; and a very readable summary by James Delingpole. All very interesting.
Update on Jan 31, 2015 by
Josh
Josh
Jimmy Haigh, SandyS, Stephen Richards and Stewgreen allude to this topic's substantial history and quite rightly so - there are many bloggers and posts that could be mentioned (see the James Delingpole article). Though this particular cartoon was inspired by Paul Homewood's recent post I think there is lot more to come on this story.
Reader Comments (86)
J Abbot, is it well commented code that BEST has for download?
Rog,
I'm afraid I have no idea. Life's too short, you know? But Mosher's always jumping up onto his high horse about it, calling sceptics lazy for not doing their own analysis on it. Maybe he's right. I seem to remember Brandon saying that it wasn't possible to work out which version of code and/or raw data goes with which set of results BEST have published. But then, he and Mosher are usually at each other like starving dogs over a bone.
A number of people have referenced Steven Goddard in a positive light, with Stephen Richards even saying:
I can't speak for anyone else, but I won't apologize for anything I've said about Goddard. Goddard has presented a great deal of false information. Many of his analyses are inexcuably bad. The general tone of his blog is completely unjustified.
Which is also relevant to this post. The repeated accusations of "fraud" are unwarranted. There are certainly problems with the temperature record, but they're nothing like what people are making them out to be. Anyone who thinks the surface temperature record is being intentionally manipulated to exaggerate global warming is wrong. Anyone who thinks the warming in the temperature record is purely an artifact of manipulation is a fool.
There are problems, but they are mostly problems of incompetence and confirmation bias. Aside from that, the worst there is is just a lack of openness about problems with the current results.
Brandon Shollenberger says "There are problems, but they are mostly problems of incompetence and confirmation bias".
At the end of the day the result is the same- $£billions is been spent as a result of this incompetence and confirmation bias.
Take your head out of the sand, Brandon.
Jonathan Abbott, you say:
I don't think your caveat is fair given Steven Mosher is incredibly hostile toward me, an attitude I barely return. Regardless, BEST doesn't archive previous versions of data or code. One can maybe figure out which code and data goes with most of the papers BEST has published since there was a collected code/data dump at one point, but nobody can find out which code and data goes with results published on the BEST website or in BEST's various reports.
But that only deals with what is possible. If you're looking for what is practical, things are far worse. There were several changes to the BEST methodology made as things evolved. Those changes were never disclosed. For the longest time, they weren't even acknowledged as the BEST website still described the old methodology. If was only after I repeatedly pressed the point, suffering a great deal of abuse and misrepresentation from Steven Mosher, that BEST changed the website. And of course, when it did, it didn't make any note of the change so people could be aware of it.
If you read BEST's published papers, you can find at least one case where the methodology changed. That change isn't noted in the papers. It isn't noted anywhere else. There are just two different approaches described in their papers, both being listed as what BEST does. The only way a person could realize there was a change is to read both papers, note the difference in description then guess there was an undocumented change.
Don Keiller:
What exactly should I be doing? Should I go around crying "FRAUD!" about matters I have no understanding of? Should I call for people to be arrested because they did things I couldn't explain in any detail? Should I tell everyone to just ignore the temperature record because it has problems whose effect I couldn't possibly estimate?
I don't think so. I think I'll keep doing what I've been doing: Calmly discussing issues and trying to figure out what effect they have.
About six years ago, I wasted a lot of time on Hansen's "two legged" adjustments and successfully figured out much of what was going on. As a caveat to readers, the adjustment examples at issue in the present conversation all are from Hansen GISS, which applies idiosyncratic adjustments, but these often bizarre adjustments are not used in other indices.
I did a number of posts on adjustments to temperature data, including close examination of some South American stations in Peru and Bolivia. See for example:
http://climateaudit.org/2008/02/24/the-adjusters-visit-peru/
http://climateaudit.org/2008/02/25/hansens-rural-peru/
http://climateaudit.org/2008/02/27/cobija-then-and-now/
http://climateaudit.org/2008/03/01/positive-and-negative-urban-adjustments/
http://climateaudit.org/2008/03/01/hansen-and-false-local-adjustments/
In http://climateaudit.org/2008/02/24/the-adjusters-visit-peru/, I drew attention to NASA adjustments of up to 3 deg C at sites where annual variability was only a few degrees, pointing out that such adjustments warranted a very clear explanation:
I subsequently and laboriously managed to emulate Hansen's adjustment method - described in a CA post:
http://climateaudit.org/2008/06/23/nasa-step-2-some-benchmarks/
Readers who are working on the topic should familiarize themselves with this analysis, as most of the present discussion is treading on paths that I had previously looked at, but without yet looking closely at how Hansen's adjustments actually worked.
At the time, I compared Hansen's adjustments to a pointless toy. In my opinion, it remains possible that Hansen's adjustment are a pointless permutation of data, as opposed to a substantial bias. The similarity of Hansen's overall results to results from other indices indicates this possibility. It's a possibility that critics have to be alert for.
Brandon,
I'm sorry I was flippant, but it's Saturday night and I've had a glass of wine. I know Mosher does his best to wind you up. If I were you I would ignore it more often, but I don't know all the ins and outs.
Thanks for explaining the problems with the BEST code in more detail.
By the way, I agree completely that anyone who thinks BEST or any of the other series are outright frauds or hoaxes is a fool. And Goddard is a crank with a very nasty streak when so inclined. If he sometimes hits on something interesting well, throw enough mud and some will stick.
No conspiracy needs to be invoked then ridiculed.
It merely takes an assembly of like minded people.
Think trade unions.
Union members act, sometimes forcibly, in unison to push belief in union causes. The causes can be different to public expectation and are often seen by many of the public to be extremist and wrong.
But not a conspiracy.
Brandon Shollenberger, thank you for appraisal, and summary.
I have never known the bank to make an error, in my favour.
I have never known the IPCC, or science intended to support it, to underestimate anything, that might support the belief, in AGW. No doubt I am not the only person to have noticed a decreased bladder capacity over time, but whilst this is exacerbated reading the latest AGW scare stories, I am not convinced of a causal link.
If you pay people to find any evidence, of gold in them thar' hills, they will return with evidence, and ask for more pay, to find more evidence, irrespective of whether there ever was any gold. Perhaps if people were not being paid, to find evidence of something, that may exist, but in scarcity, there would be fewer people charging around shouting nonsense, including me!
Brandon
Look at the two charts in Josh's cartoon.
Go and look at the blink charts in Delingpole's (for him, very restrained and thoughtful) piece at Breitbart.
Then come back here and explain and justify.
To extend Delingpole's argument ...
For the best part of 30 years I was well acquainted with a man of excellent repute, an acknowledged expert in animal nutrition, a published author, and the local weather recorder. I presume that his daily readings formed part of some data set or other somewhere in the UK. I also assume his records were accurate, or as accurate as he could be.
By what right does anyone, anywhere in the world, have the right to say to this man, "your readings are total bollocks; we have decided that they ought to be this."
Because that is exactly what you are trying to justify. It may well (just conceivably) be that his readings were total bollocks (though I very much doubt it) but they were readings; they were data, they were fact. By what authority does GISS or NOAA or CRU or anyone say that their plucked-out-of-the-ether figures are more accurate or more reliable? And on what evidence precisely do they — or you — make that claim?
As I have said time and again here and elsewhere: If you don't have data then you don't have data. If you do have data then no matter how much you may not like it or "feel" that it isn't right then tough shit! Because that is what you have. Learn to live with it.
During my red pill or blue pill moment there were three things that opened my mind to Mann Made Global Warming (tm) bollovks, the first was an article on WUWT about temperature homoginisations across Africa and how temps hundreds and sometimes thousands of miles away were being used to influence and create temperatures where data didn't exist. The second was Steve McIntyres post about the "Teams" Jesus paper and the lengths so called climate scientists went to influence the science and of course the third item was the discovery of this blog!
On another note what a devistating documentary the sordid history of Mann Made Global Warming (tm) would make and if it was honest and no holds barred none of the pro-Mann Made Global Warming (tm) scientists would come out the other end smelling of anything other than dung!
Regards
Mailman
Credit where due, geologist Warwick Hughes in Australia was among the first to question observations, around 1992. Recall that he received the infamous Phil Jones response along the lines of "Why should I send you my data when you just want to find faults with it."
And even before Warwick, what about John Daly?
The Australian BOM method of homogenisation cannot be applied without altering a trend in the time series. Simple math outcome. So Mosh, if you did say that raw = adjusted, that is incorrect as night is different to day. I do not think you said precisely what is alleged above
When was the first frost of the year, when was the last?
Surely in the UK there are going to be stately homes, sportsgrounds etc with paper records compiled by gardeners/groundsmen, for whom the first and last frost will have been significant enough, to be worth recording? No dodgy temperature measurements, just, was there a frost?
Even village cricket clubs, must have dusty tomes, propping up piles of broken deck chairs.
Couldn't the Central English Temperature record be updated with records from village halls?
Does Sweden maintain records, about when "snow tyres" must be fitted?
Should be more accurate than drilling random holes in bristlecone pines
Geoff,
I suggested Mosher says that the difference in overall trend between raw and adjusted is negligible and/or within the bounds of uncertainty. Not that raw = adjusted. It seems to crop up at Climate Etc about once a week.
So in conclusion what I tapped here and elsewhere between 2-5 years ago is correct:
"We know f**k all about what is going on in the climate".
And it follows that the idea we can make models to predict the future, or use proxies matched to records to describe the what has happened in the past is equally as futile - no matter who does it and how.
Local records must not be tampered with. If you do, it is fraud. What people do with local records in order to produce a global average is a different matter. That is already a manufactured product, I can take it with the required grain of salt or disregard it. As a scientist, if you tell me, that a local station's history cannot be trusted because of poor exposure, siting or improper observation, I can disregard it. But if you tell me you successfully estimated the magnitude of the effect for a local station - from the information present in the altered record itself - and mess with it - it is fraud. There are different kinds of fraud and people like to begin medieval theological exercises at this point about what 'fraud' means etc but that is not how I define and use fraud. Fraud, in this instance, is committed via flawed circular reasoning, which is an act of commission, and one carried out in the face of viable alternatives that are present but not adopted. People can legitimately differ on methodology but circular reasoning is not a viable method.
@Steve McIntyre, a difference now is that the GOP has gained control of the US Senate EPW committee, with Jim Inhofe taking the chair (from Barbara Boxer), so myself and others would very much hope that you and others such as Paul Homewood, Steven Goddard, Marc Morano, Tallbloke, etc., etc. get together to compile evidence you have so far and also input into what information/individuals should be subpoenaed.
As an aside I am taking to heart very much what James Delingpole has been saying about the 'lukewarmer' stance which previously appeared wise for political expediency in getting a foothold in the debate, particularly in terms of the dominant voice WUWT has and whether it is a good or bad thing in getting in the way of individuals with fresher momentum and insights getting a voice that they deserve. More than anything I think sites such as WUWT have rather tried to create a monopoly by broadening the meaning of lukewarmer to encompass too wide a range of climate sensitivity in particular. To be fair there was a little blip with this line of attack on datasets due to Richard Muller's 'BEST' circus for a while. To be clear though I am more in line with Prof. Lindzen than the sky dragon slayers although I do believe once the normal scientific method has run its course further we will find climate sensitivity is below 0.7 degC, quite possibly significantly below due to much more to be learned regarding negative feedback and self-regulation via. cloud cover.
For me there is one overriding concern, it is the solace the major surface data suppliers apparently gain from mutual confirmation.
From competition comes insight and development. There was a time, not that long ago when Phil Jones championed his global series over all others and for global GISS even agreed with him! Over the last few years the compilers of HadCRUT appear to have put more emphasis on compliance with the competition.
I have long yearned for an out and out fight between the data set suppliers. I want to see GISS take apart HadCRUT methodology and vice, both tear apart NCDC?
All suppliers agreeing with each other? Well many moons and numerous scars from painful past experience makes me twitch and until I witness true competition I will continue to twitch.
Greensand, it is what they agreed, NOT to disagree on, that would be most interesting!
Golf Charlie
It is the incorporation of the following statements in the HadCRUT4 data set that give rise to my double bogie feeling!
I am right cos I agree with Fred and Fred is right cos Fred agrees with me! You put your left leg in....
Green Sand in other words, their models are so accurate, that they ARE used to verify each other, and therefore the fact they are all wrong indicates the possibility of a common "X" factor, ie add, on average, an extra 0.1 degrees for every so many years the model runs.
I could have done that on a ZX Spectrum, in BASIC over 30 years ago, without the complexity of inputting, let alone gathering and adjusting, any data at all. A bit cheaper too.
Chicken or egg? Which is the more likely?
A consensus leading to a cartel?
or
A cartel leading to a consensus?
Irrespective of the route (mind games) the outcome is the same, and in this case it is one of the most obvious, self declared cartels we have ever decreed acceptable. But then again we have very short memories....
I note that the Australian BOM and NZ's NIWA both have credibility problems with their adusted records. I am a mere layperson, but the 'show me' gene is healthy in me. As a Kiwi with an Aussie-born father, I am aware that various things impinge upon accurate data collection in both countrys, but interfering with the life's work of dedicated individuals is despicable anywhere.
Further, to my mind, BEST is a bit of a methodological 'flash harry' that I would not trust too far.
@GS very interesting interpretation of the C word
climate Consensus ⇔climate Cartel
The western world Jo and Joanna public are fed up to the back teeth of it. Plus, with the hiatus in man made warming going back to almost 1850. What 'they' needed more than anything was a hook, and GADZOOKS it didn't matter a hoot if it was a great big lie - any bloody lie would do.
I don't quite think people [in the realist community] really make the effort to understand the heavy lifting going on behind the scenes. The USA propaganda machine is a formidable Leviathan and POTUS - Obama is cracking the whip. The corporate world particularly the investment banks [Goldman Sachs] are right behind and cheerleaders nonpareil of the great scam.
Thus, the warmists of the Whitehouse in cahoots with the alarmist lunatics in Brussels mainly Germany and Britain, like manic spiders on crack cocaine are spinning the lies funnelling these gossamer strands binding the world in miasma of fixed/spliced/diced/tampering Temp data BS - and the media it is no surprise to report supinely laid down and had their tums tickled.
A lot of people, from Soros to Merkel to Obama to most of the civil servants [DECC/Defra/quangocracy] dependent for their livelihoods promoting the great scam, in hospitals, schools, offices and all over Britain - need some sort of piece of paper, an agreement, a treaty KYOTO II coming out of Paris 2015.
Paris COP. We are entering the end game, it's all or nothing this time round.
I pray, I earnestly pray that, Paris UNFCCC COP implodes, is such an unmitigated disaster, the whole man made CO2 leads to global warming BS and scam is finally laid bare and exposed for what it is and that to ever meet up again will have been made impossible.
CAGW, all it is, is agenda 21 which is a UN tax redistribution and welfare scheme from North to South, the Pope and Potus and the illiberal apocalyptic doom mongers of Socialism deem it right - so that's OK then and bugger democracy and the taxpayer and the commandments and gospels too:
/For even when we were with you, this we commanded you, that if any would not work, neither should he eat./
Thessalonians 2 verse 3:1.
@Jonathan Abbott.
I suggested Mosher says that the difference in overall trend between raw and adjusted is negligible and/or within the bounds of uncertainty. Not that raw = adjusted.
If that is the case why does he not publish both the raw (actual readings) and the adjusted (fiddled to fit the required output) side by side for others to check the work? Or would doing that show up the bias?
Until the climate scientists publish the raw unfudged data that they rely on and the methods, with relevant programs, they use to produce their results so that outside observers can audit their work anything they produce can only be considered wishful thinking on their part.
Anyone who thinks warming is an artifact is a fool. OK.
What about people who wonder about the impact of upward adjusted local stations on "hottest year ever" claims? Are they fools?
That's where this started. Jo Nova has the graphic: http://jo.nova.s3.amazonaws.com/graph/temp/america/sth/paraguay-temperatures.gif
You see the enormous, continental scale blob of warmth over a large region that includes Paraguay, Argentina and Eastern Brazil over a region that has poor instrumental coverage. What is the effect such 'adjustments' and smearing - all methodologically complex and not 'fraud' no doubt - have one the origin of such blobs?
Talk about revising Climate history, Delingpole says:
" Among the first to spot the problem was Steve McIntyre who back in 2007 observed the curious fact that where NASA’s James Hansen had once acknowledged that the 1930s was the hottest decade in the US, he subsequently amended it – with the help of some conveniently adjusted records – to the 1990s."
One of the first? As much as I admire Steve McIntyre et al and their statistical microscopes, it was John Daly who first noted on his web site that temperature records were being adjusted about a decade before 2007. At the time the 1930s were still relatively warm for the continental US record and the 1960s were relatively cool. The GISS were adjusting the 30s down and the 60s up to "correct" the apparent variability not related to Greenhouse gases.
Singer was also noting these changes at SEPP, not to mention Michaels and of course Lindzen.
Not all that new.
Does anyone know where I can access the temperature records IPCC used for 2014 ?
I have access to the raw Irish temp data and wish to do a comparison.
Jan 31, 2015 at 5:45 PM TonyB said:
"Can I claim the record for the oldest instrumental temperature change?
It is from the very first year of CET which in the 2007 version is listed as 8.83c for the year. In this years version it has been upgraded to 8.87C
Many others in the historic record have been made higher and lower"
I can independently confirm that the temperature value for the very first year of CET in a download (obtained from the Multitaper R package) is 8.83 degrees C (derived from taking an average of the first 12 months worth of data, from the CETmonthly dataset in that R package, obtained by the authors from the Met Office on October 1st 2011). Strange if it has indeed been upgraded to 8.87 degrees C (something I haven't checked myself).
Regarding Pilar, here is the Berkeley Earth record for the station:
http://berkeleyearth.lbl.gov/stations/157441
There is little to no metadata available for the station (an unfortunately common occurrence outside the U.S. and Europe), but Berkeley identifies a number empirical breaks by comparing the station to neighbors and looking for localized changes that are not seen in nearby stations. Its rather unlikely that Pilar is actually cooling at 2 C per century while all the nearby stations have been warming over the same period, especially given that most stations have moved at least a few times and likely had instrument changes over the period in question.
Overall, the net effect of adjustments is relatively small globally. Ironically, if you include SST adjustments in your definition, the net effect of adjustments is to slightly dampen global temperature change, give that oceans are most of the world.
If you only look at land, adjustments slightly increase the trend. You can see raw and adjusted data compared here: https://curryja.files.wordpress.com/2014/07/slide1.jpg
If folks want to play around with raw data themselves, you can download it from the links below.
NCDC - ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/ (tavg qcu file)
Berkeley - http://berkeleyearth.org/source-files (single qced file here: http://berkeleyearth.lbl.gov/downloads/TAVG/LATEST%20-%20Quality%20Controlled.zip )
"while all the nearby stations have been warming over the same period,"
This is completely untrue.
In the GHCN record there are 15 Paraguay stations.
Eight of them show cooling in the raw data. See for example (graphs at top right)
ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/products/stnplots/3/30886068000.gif
ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/products/stnplots/3/30886260000.gif
"looking for localized changes that are not seen in nearby stations."
This is also completely untrue.
BEST puts a break in Pilar in the late 1960s, see Berkeley graph linked by Zeke.
In the raw data for Pilar you can see a steady cooling from about 1965 to 1975
ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/products/stnplots/3/30886255000.gif
(and you can see that GHCN also put a break in there).
Now look at the two stations I linked in the previous comment.
What do we see? Cooling from about 1965 to 1975.
And here's another one showing the same thing
ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/products/stnplots/3/30886297000.gif
So, South America, region with extremely sparse actual data and the result of the processing is a strong warming signal which seems to be absent from the raw data.
Antarctica, region with extremely sparse actual data and the result of the processing is a strong warming signal which has been disputed.
Hmm, could there be something amiss in the processing? Has Harry read it?
Have Antarctic temperatures been warming? Not according to Hadcrut4 area weighted 70-90S.
Source page: http://notalotofpeopleknowthat.wordpress.com/2013/09/01/is-antarctica-getting-warmer/
I long since gave up any trust in any of the global datasets, as they have clearly been adjusted and homogenised to merry hell and back. Raw data from individual (preferably rural) stations only please.