Bishop Hill New Zealand's temperature record
Oct 31, 2014 Climate: Surface Richard Treadgold emails with details of an interesting new paper, with quite an eye-opening end to the abstract.
Yesterday a paper on the New Zealand temperature record (NZTR) was accepted by Environmental Modeling & Assessment. Submitted in 2013, we can only imagine the colossal peer-review hurdles that had to be overcome in gaining acceptance for a paper that refutes the national temperature record in a developed country. The mere fact of acceptance attests to a fundamental shift in scientific attitudes to climate change, but expect strident opposition to this paper.
The authors present first a concise observational history of the NZTR, remarking that the established national record was a product of early methodology, then reconstruct an homogenised dataset using the peer-reviewed adjustment standards of Rhoades & Salinger, 1993 (RS93).
A Reanalysis of Long-Term Surface Air Temperature Trends in New Zealand was produced by principal author C.R. de Freitas with M.O. Dedekind and B.E. Brill.
Abstract
Detecting trends in climate is important in assessments of global change based on regional long-term data. Equally important is the reliability of the results that are widely used as a major input for a large number of societal design and planning purposes. New Zealand provides a rare long temperature time series in the Southern Hemisphere, and it is one of the longest continuous climate series available in the Southern Hemisphere Pacific. It is therefore important that this temperature dataset meets the highest quality control standards. New Zealand’s national record for the period 1909 to 2009 is analysed and the data homogenized. Current New Zealand century-long climatology based on 1981 methods produces a trend of 0.91 °C per century. Our analysis, which uses updated measurement techniques and corrects for shelter-contaminated data, produces a trend of 0.28 °C per century.
Reader Comments (52)
Given the strenuous defense of the official temperature 'manipulations' by NIWA in court, this must be particularly galling to be splashed around showing them up publicly.
Ahhhhh, so in 714 years we'll hit the catastrophic 2 degrees.
>:-0
Another 500 years before we hit the "Dangerous" level of warming?
This paper's going to be either attacked vigorously or studiously ignored by the great & good.
A prediction. The reaction will be based on defending the conclusions of established expert opinion against some mavericks, rather than demonstrations that the temperature dataset already meets the highest quality control standards
"Our analysis, which uses updated measurement techniques and corrects for shelter-contaminated data, produces a trend of 0.28 °C per century."
In line with the CET at 0.23 Deg C
The truth behind the "political-science of climate change" will out eventually, and the sooner the better.
Paper is here.
From the paper:
"The aim here is to apply the method set out by [17] (i.e. Rhoades and Salinger, 1993) exactly as they describe, without adjusting it in any way."
So the climate obsessed rent seekers have been deceiving the people of New Zealand.
Once again the climate skeptics are right.
"corrects for shelter-contaminated data"
But, but - contamination is fine if it provides the required result!
Some years back, 3, maybe 4, I followed the efforts of the NZCSC on their site as they tried time after time to obtain from NIWA office the 9 series of temp records. They fought in vain as NZ Met refused with various excuses and ultimately with a court case which the Met won. All of this will be on their site still (NZCSC).
Have a look at NZCSC now.
PM
Is this the revenge of Chris De Freitas over the Climate Research debacle. Revenge is a dish best served cold.
Chris de Freitas “noble cause corruption”, memories of Climategate and now this... Pass the Popcorn!
This is what happens when put the fox in charge of the hen house , those most motivated to 'prove' AGW were those in the best position to 'adjust' past temperatures, so guess what ?
Full paper here:
https://dl.dropboxusercontent.com/u/75831381/de%20Freitas%20NZ%20temp.pdf
wow
I've only skimmed the full paper so might have interpreted things wrongly, but it seems that J. Salinger published his thesis work in 1981, then collaborated with a statistician to improve the method (Rhoads and Salinger, 1993). However, the improved methodology was apparently not used by the New Zealand government agency NIWA, which employed the earlier 1981 approach in their 1999 and later publications. So this work by de Freitas is the first to employ the improved method? If so, it's a pretty clear case of government agency malfeasance.
Can someone indicate the status of this journal? Has de Freitas said anything about the nature and scope of reviewer feedback? Given the legal case (and the loss there of) I think it is important to understand how rigorously the article has been reviewed.
A little over a quarter of a degree over a century. And even that insignificant upward trend has ended. Definitely worth bombing with $1.7 thousand billion of taxpayers' money. I don't think.
"I think it is important to understand how rigorously the article has been reviewed."
There is a link just above your post, so you can review it yourself.
Rob: I read the article before posting. It looks very solid. I particularly like the way they have isolated the outlier that contributed to the inflated trend. However, that is not the same thing as determining potential weaknesses or counter arguments. I could find no SI for the article.
bernie185, you know the answer to how well reviewed this is. The answer is, look at how well reviewed it is going to be.
This paper is going to get a forensic review all right - with a claymore. It will be torn to shreds for every weakness. The very colour of the ink will be analysed for potential imperfection. It will get as good a kicking a kicking can be got.
If this paper survives a month you can be pretty sure it is far more trustworthy than 90% of papers published in the most prestigious of journals.
I performed a similar exercise a couple of years ago and got 0.45C/century, still a lot less than NIWA
http://tallbloke.wordpress.com/2012/11/01/roger-andrews-so-how-much-warming-was-there-in-new-zealand-anyway/
And so it begins: http://hot-topic.co.nz/nz-cranks-finally-publish-an-nz-temperature-series-but-their-papers-stuffed-with-errors/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+co%2FRbRF+%28Hot+Topic%29
Oh, and a response from Treadgold: http://www.climateconversation.wordshine.co.nz/2014/10/renowden-on-the-reanalysis/
mcraig: Thanks for the link. I have to say that I am immediately skeptical of a climate site that offers only one side of the debate: There appear to be no links to this site, ClimateAudit or WUWT. That said, Gareth,the writer of the post referencing this article - though given to <I>ad hominem</I> comments and hyperbole - raises some substantive points that need to be checked. The absence of an SI or link to the data set used in the article is not helpful. It also allows Gareth to score an easy point.
The mischief surrounding the adjusted New Zealand temperature series must have come to light at least a decade ago? It was one of the first alarm bells to ring for me. There is a summary about it here:
http://quadrant.org.au/opinion/doomed-planet/2010/05/crisis-in-new-zealand-climatology/
The story of NIWA, Salinger and De Freitas is like a soap opera
Some of the background about Jim Salinger that may not be widely know is:
1. Everyone is presumably aware that Salinger was on the edges of the Climategate group ... not in the middle but definitely in the group and at the edges.
2. When he worked at NIWA his thesis and subsequent article seemed to totally dominate the way in which the climate "adjustments" (which provided most but not all of the warming for NZ) were handled. Despite being so critical in an internationally important field, it is one of the few theses that Victoria University does not have as a downloadable file, and instead has it restricted to staff and students.
http://victoria.lconz.ac.nz/vwebv/holdingsInfo?bibId=186796
3. However, eventually NIWA sacked Salinger over him going to the press without authority. http://www.stuff.co.nz/science/2361900/Niwa-sacks-Jim-Salinger If I remember correctly he took them to the Employment court but lost. As NIWA is a Crown Research Institute ie. a government run and funded agency it was really something to get sacked in the first place.
bernie
>"The absence of an SI or link to the data set used in the article is not helpful. It also allows Gareth to score an easy point"
Not really. The raw data is housed in NIWA's Cliflo databse here: http://cliflo.niwa.co.nz/pls/niwp/wgenf.genform1
That goes without saying in this neck of the woods. And SI isn't necessary, all that's needed for replication is in the paper, viz:
de Freitas et al (2014)
Page 3 pdf: 3 The New Zealand Temperature Record
Page 4 pdf: 4 Rhoades and Salinger—RS93
Page 5 pdf: 5 Method 5.1 Description
Page 6 pdf: 5.2 Gradual Inhomogeneities
>"The absence of an SI or link to the data set used in the article is not helpful. It also allows Gareth to score an easy point"
I'd also like to add that we took pains to publish all the interim steps and results in section 6, to aid replication.
When we asked for this information from NIWA years back regarding their original analysis, we received no answers or help at all.
Who would of thought the NZ NIWA, stacked full of radical greens, would have produced a corrupted national temperature record!
Now for the Australian Bureau of Meteorology, stacked full of radical greens, who also may have produced a corrupted national temperature record (two actually).
Who would have thought!
Richard C, Bob D:
Many thanks. I am new to this particular issue and data set. I am going to have my hands full looking through these. If this is sufficient, then it would have helped me at least to have made this clear in the article.
Bob D:
Excellent article. You seem to have set the cat among the pigeons. One clean, transparently adjusted data set could set the model for the rest.
>Excellent article. You seem to have set the cat among the pigeons. One clean, transparently adjusted data set could set the model for the rest.
Thanks bernie, I think the time has come for each country to start challenging their own national records. As we have found, the science behind some of the adjustments is hardly sound.
The reviewed NIWA NZ temperature series which replaced the one by Salinger was done while a court case against NIWA for mishandling the NZ temperature records was in progress. There must have been pressure on NIWA at the time to come up with results similar to Salinger's numbers as to do otherwise would have been embarrassing in court. To what extent that pressure influenced the results is unclear. Personally I am strongly opposed to taking these kinds of issues to the courts as I think it is a very counterproductive approach. Judges have no place in a scientific dispute and legal threats just encourage defensive behavior.
Ian H.
Normally you would be right for a scientific debate. But there was far more going on here, and far more at stake. I think the court case was a good move, even though it didn't succeed in changing anything.
http://wattsupwiththat.com/2011/11/27/the-tribalistic-corruption-of-peer-review-the-chris-de-freitas-incident/www The Climategate crew worked hard to get de Freitus sacked. They also plotted to have his Phd cancelled.
Didn't yo Mama ever teach you that there are no such things as free easy points?
=========================
Bob D has put together a nice point by point rebuttal to Gareth's premature and immature response to the paper. Hopefully it will move the discussion forward.
http://www.climateconversation.wordshine.co.nz/2014/11/analysis-of-renowdens-analysis-of-our-reanalysis/#more-19623
This is not the only case of malfeisance involving global temperature. In my 2010 book "What Warming?" I reported that official temperature from HadCRUT3 in the eighties and nineties was showing a temperature rise when actual temperature was flat from 1979 to 1997. I even put a note in the book warning about it but nothing was done. Today they are still showing that bogus temperature rise in the eighties and nineties, in company with GISTEMP and NCDC. All three of them have computer processed their data but did it so incompetently that traces of it are present in their published temperature curves. You might call them cross-oceanic temperature improvement club. The processing marks consist of sharp upward spikes near the beginnings of years, all in exactly the same locations in all three data-sets. The fact that their temperature rise is phony is easy to check out simply by comparing their data to satellite temperatures which they so far have not been able to control. They used to call the eighties and nineties the "Late twentieth century warming." No one could explain what caused it so they jumped to the conclusion that its existence was proof of human-caused warming. Looking at the present day temperatures it is obvious that they have even more outrageously raised twenty-first century temperature records. Until someting is done about this outrage my suggestion is to use satellite temperatures exclusively starting in 1979.
When Salinger was principal scientist at NIWA in 1999, he began using his unpublished 1981 thesis as an authoritative source for the New Zealand temperature record. The thesis contained 35 adjustments (most of them pre-WW2) which increased the historical data trend from 0.2° to 1.0°C.
After Salinger was fired in 2009, the NZ Climate Science Coalition challenged the 7-station Series, posed Ministerial Questions, raised FOI questions, etc. It turned out that Salinger's calculations had been lost in a computer mishap in 1983 and nobody could reproduce them. NIWA reverse-engineered the series to find out the dates and values of the adjustments.
The Minister then announced a 3-pronged project involving (a) a NIWA Review which would supply and update the missing calculations; (b) an external peer-review by BOM of Australia; (c) an independently peer-reviewed journal paper on the methodology.
The NIWA review (Mullan et al) was hoisted on NIWA's website in December 2010 – it's still there. It transparently uses the same calculation method as the 1981 thesis, but doesn't say so. It obtained exactly the same result too.
The BOM peer-review was kept secret so an FOI complaint was made to the Ombudsman. That's still running nearly four years later.
The promised journal paper was postponed for a year. Then another year. Then it was said that the Court case had taken too much time. I presume its 30-year-old techniques could never get past peer review with any respectable journal.
Now the other point of view has survived lengthy and rigorous peer review after being submitted last year. It finds the adjustments pretty well balance out, so the historical trend of about 0.2°C is restored. Surprise!
BTW the journal paper finds the trend was 0.28 ± 0.29°C, so no statistical warming at all.
I strongly recommend that the authors provide turnkey code showing their results.
In response to an earlier comment that "The absence of an SI or link to the data set used in the article is not helpful. It also allows Gareth to score an easy point", Richard C says:
Some readers, if not most readers, are only semi-interested in the controversy, but insufficiently interested to try to code the results and figure out how to access the data from NIWA. The authors should place the NIWA versions as used in their own FTP location and provide the code by which they obtained their results. The advantage of placing the code online is that interested readers can see exactly what was done without having to parse and interpret the methodology in the article - though clear methodology is equally important in seeing what was done.
Nor is it a sufficient reply for the authors to complain about their own prior mistreatment by NIWA. Most of the climate community will be sympathetic to NIWA and unsympathetic to the authors. So they need to go the extra mile.
Steve:
The Rutherford/Mann data fiasco strongly supports the need for all authors of empirical research to provide exact copies of the data/code/documentation that they used. It surely saves considerable and removes the opportunity for ambiguity and Mannian type games. It also means that reviewers of journal articles can be more thorough in their reviews if they feel there is a need.
Your efforts to encourage transparency and to encourage and enable replication in Climate Science are significant contributions. Adhering to such norms makes it all the more difficult for opponents to misstate findings and claim non-existent errors.
Steve
?"Some readers, if not most readers, are only semi-interested in the controversy, but insufficiently interested to try to code the results and figure out how to access the data from NIWA."
Yes, "semi-interested" is automatically "insufficiently interested to try".
But the vitally interested are already immersed. Even so Bob D comments upthread "we took pains to publish all the interim steps and results in section 6, to aid replication" And at CCG:
Workings or SI
According to Gareth:
“The paper as published contains no workings or supplemental material that would allow reproduction of their results…”
Again, surely Gareth hasn’t even read the paper. It contains extensive workings and references in Section 5 and even a worked example in Section 6 to step the reader through the process. All interim results are recorded in detail in Table 3. How is that not reproducible?
http://www.climateconversation.wordshine.co.nz/2014/11/analysis-of-renowdens-analysis-of-our-reanalysis/#comment-1224251
Steve
>"provide the code by which they obtained their results"
The code is the method that Rhoades and Salinger (1993) provided, there was no additional code provided by RS93. From de Freitas et al:
5 Method
5.1 Description
Our method follows the RS93 neighbour comparison techniques
for estimating the effect of known site changes but
extends that approach to comparisons between well-correlated
distant stations. We de-trend the inhomogeneous section
where necessary by using the slope calculated from a reference
time series. Note that we have not modified the RS93
method at all, and we follow the same process as laid out in the
worked example (section 2.4) in that paper.
They then go on with the details of the RS93 method as provided by RS93 of which there was no SI or additional code.
Again, the fact that Rhoades and Salinger didn't show code is no reason for you not to show your code.
I don't understand why you are arguing about this. It is possible to provide turnkey code in R (for example) for analyses like this so that readers can use the script to access the data, watch the analysis and produce the graphs and stop and do their own variations if they want.
I've never heard a good reason why people can't do this. I think that you should as well.
Steve
>"It is possible to provide turnkey code in R (for example) for analyses like this"
Yes it is but I don't think there was a code implementation of any sort. I'm not a co-author but I'm reasonably sure there was no code used. I get the impression that the method was simply translated to spreadsheet functions. But whether there's VBA I don't know. I doubt NIWA have code for their 7SS version. They may do but the question has never been raised until now to my knowledge. I expect they just use a spreadsheet too but as far as i know, NZCSC has never requested code or spreadsheets from NIWA that I know of, it's not necessary for replication..NIWA produced their adjustments but NZCSC could not replicatethose adjustments using the established method (RS93) - that's the issue, not code.
I'm not arguing, I'm not a co-author, I'm just expressing opinion from the point of view of someone who has accessed the raw data from CliFlo and attempted a rudimentary replication myself prior to the 'Statistical Audit' but that was before it was realized that RS93 was the statistical method to use. Not being prepared to commit to the project (I'm not a member of NZCSC) and not able to assist statistically (skill not up to it), I didn't participate in the Audit. But with some effort I'm sure I could replicate de Freitas et al's adjustment for at least one site change from the method supplied without code.
The authors may make some statement re code - up to them. I think that rather than code, questions will eventually move to the respective adjustment methodologies e.g. RS93 (NZCSC) vs PM-95 (BOM). BOM do have code for ACORN-SAT implementing PM-95 (Percentile Matching), but they have not released it even though they promised to do so (they've only just released their adjustments). The issue then becomes - does PM-95 replicate RS93 and vice versa? Same for BEST's "scalpel" and whatever method GISS uses (what is that?).
If the respective methods don't replicate each other then code is the last thing we'll be asking about. It might be more productive, if you are interested enough, to acquire BOM's code (if they'll release it) and apply that to both ACORN-SAT Max/Min and eventually NZ 7SS Mean and Max/Min in the manner you have laid out. That would be interesting.
Steve, re NIWA vs NZCSC vs BOM vs BEST vs GISS
Apart from breakpoint statistical analysis techniques, the respective homogenization methods are vastly different too. But there are common breakpoints. Then there’s non pre-identified site change break adjustments by the others extra to those in NIWA/NZCSC for which NIWA/NZCSC do not adjust for – messy.
The respective breeakpoint identification approaches, NZ vs AU, are:
NIWA/NZCSC: site change identification => breakpoint analysis => adjustment criteria
BOM: breakpoint analysis => adjustment criteria => site change identification as an afterthought,
In Anthony’s post on de Freitas et al he featured Rutherglen Research, Victoria, Australia (for some reason) but BEST has a much shorter series for Rutherglen Research than BOM does. Their earlier Rutherglen data is not homogenized to Rutherglen Research, they go to Rutherglen Post Office.
RUTHERGLEN RESEARCH
BEST Mean http://berkeleyearth.lbl.gov/stations/151882
BOM Min https://wattsupwiththat.files.wordpress.com/2014/11/rutherglen1.png?w=720
Earliest Observations: 1965 vs 1912ish
BEST makes an adjustment at the 1980 Record Gap but as far as I know BOM doesn’t. In the immediate vicinity of Rutherglen, BOM has a continuous series, BEST has a gap 1921 – 1965.
RUTHERGLEN POST OFFICE
http://berkeleyearth.lbl.gov/stations/3816
Earliest Observation: Jan 1903
Most Recent Observation: Nov 1921
The NZ 7SS is clean by comparison to Rutherglen, BOM vs BEST.
Back to NZ. BEST runs Albert Park and Auckland Aero in parallel:
AUCKLAND, ALBERT PARK http://berkeleyearth.lbl.gov/stations/157062
AUCKLAND AERO AWS http://berkeleyearth.lbl.gov/stations/157061
NIWA/NZCSC drops Albert Park when Aero starts because Albert Park is contaminated by UHI'sheltering, homogenizing Albert Park to Aero instead. Neither NIWA nor BEST adjust Albert Park for UHI/sheltering.
By eye, in terms of trend, BEST Albert Park is above NZCSC Auckland but below NIWA Auckland. If BEST were to adjust for UHI/sheltering, their Albert Park series would corroborate de Freitas et al (2014)/’Statistical Audit’ Auckland but nowhere near NIWA Auckland.
It is really only necessary to consider a few specific breakpoints in order to compare NIWA vs NZCSC vs BOM vs BEST vs GISS. There is no need to reconstruct each of the entire Australian and New Zealand multi-location series. It is not even necessary to compile location series e.g. case studies of Auckland, Masterton (an easy one) in NZ or Rutherglen, Amberley, Bourke in AU. Just breakpoints within a homogenized location is a start, then a location series, then multiple locations.
Steve FYI. BOM do not adjust for breaks of less than 0.3 as per Trewin TR-049. Of the 20 adjustments de Freitas et al make to the NZ 7SS, 8 are for less than 0.3, smallest is +0.02, largest is −1.00.
Steve McIntyre,
You said:
You may be under the impression that Richard C speaks for the authors. I've given their response to the question of code at my blog (briefly, there is no code to show), but if you have any further technical queries they would welcome them. If you wish, email me at richard at wordshine.co.nz and I'll put you in touch with them.
Steve, re this in my comment upthread:
>"BOM do have code for ACORN-SAT implementing PM-95 (Percentile Matching), but they have not released it even though they promised to do so (they’ve only just released their adjustments)."
The agreement is here (page 7 pdf):
Bureau of Meteorology response to recommendations of the Independent
Peer Review Panel
15 February 2012
C2. The computer codes underpinning the ACORNSAT
data-set, including the algorithms and protocols
used by the Bureau for data quality control,
homogeneity testing and calculating adjustments to
homogenise the ACORN-SAT data, should be made
publicly available. An important preparatory step
could be for key personnel to conduct code walkthroughs
to members of the ACORN-SAT team.
Agreed. The computer codes underpinning the
ACORN-SAT data-sets will be made publicly available
once they are adequately documented. The Bureau
will invest effort in improving documentation on the
code so that others can more readily understand it.
http://www.bom.gov.au/climate/change/acorn-sat/documents/ACORN-SAT_Bureau_Response_WEB.pdf
If you're chasing code Steve, this is the code to chase.
I would point out however, that BOM have automated the process, made adjustments for “statistical” reason only (no recourse to station histories), and human input has been neglected. It has only been after the release of the list of adjustments that BOM have had to rush around retroactively to try to find local reasons for the automated statistical adjustments they’ve made when sceptics pointed out the glaring problems resulting from a lack of human input.
No similar agreement re code in NZ because it's not necessary. I get the impression that you are oriented only towards an automated process at this stage, but as above and as Richard Treadgold points out, there is considerable human input required for 7SS, and that was missing in ACORN-SAT. It does make sense that BOM automated ACORN because there's 118 locations (I think) times 2 for Max/Min to NZ's 7 for Mean only so hardly worthwhile for NZ given the human input requirement.
>"BEST makes an adjustment at the 1980 Record Gap but as far as I know BOM doesn’t."
BOM doesn't for sure:
ACORN-SAT station adjustment summary—Rutherglen (as at 24 September 2014)
http://www.bom.gov.au/climate/change/acorn-sat/documents/station-adjustment-summary-Rutherglen.pdf
No adjustments after 1/1/1974
Rutherglen 82039 Min 1/1/1974 Statistical Station documents suggest likely move c. 1974
Rutherglen 82039 Min 1/1/1966 Statistical Station documents suggest likely move c. 1966
Rutherglen 82039 Max 1/1/1965 Statistical Station documents suggest likely move c. 1966
Rutherglen 82039 Max 1/1/1950 Statistical Site reported as overgrown at 1949 inspection
Rutherglen 82039 Max 1/1/1938 Statistical
This table is linked from this page: http://www.bom.gov.au/climate/change/acorn-sat/#tabs=Adjustments
Also linked on that page is this:
A summary of the impact of adjustments on annual mean temperatures at each ACORN-SAT site
http://www.bom.gov.au/climate/change/acorn-sat/documents/ACORN-SAT-Station-adjustment-summary.pdf
Which differs from the updated Rutherglen summary above:
Rutherglen 82039 Min 01/01/1974 Statistical* Site moved at least once between 1958 and 1975
Rutherglen 82039 Min 01/01/1966 Statistical* 1955-59 used as reference
Rutherglen 82039 Max 01/01/1950 Statistical* inspection report said site was overgrown, possibly cleared?
Rutherglen 82039 Max 01/01/1938 Statistical
Rutherglen 82039 Min 01/01/1928 Statistical